You can tweak the behaviour of the janitor for a package by editing the configuration file (debian/lintian-brush.conf) in the package. For example, to prevent any changes that are incompatible with oldstable, add the following contents:
compat-release = oldstable
(the default compat-release is stable)
For a full list of supported options, see the lintian-brush.conf(5) manual page.
These configuration options can also be set on a per-maintainer basis; please send me a message if you'd like me to do that.