Issue #375: kallithea 0.6.0 under python3: alembic configparser crashes when smtp_password contains % symbols
| Reported by: | Maximilian Kerst |
| State: | resolved |
| Created on: | 2020-05-21 14:30 |
| Updated on: | 2020-06-05 07:38 |
Description
As the title suggests, running alembic -c my.ini current on Python 3.7.3 fails with a configparser.InterpolationSyntaxError if it encounters a % sign:
(kallithea-venv) xxx@xxx:/usb/kallithea/kallithea $ alembic -c my.ini current
Traceback (most recent call last):
File "/usb/kallithea/kallithea-venv/bin/alembic", line 8, in <module>
sys.exit(main())
File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/alembic/config.py", line 577, in main
CommandLine(prog=prog).main(argv=argv)
File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/alembic/config.py", line 571, in main
self.run_cmd(cfg, options)
File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/alembic/config.py", line 551, in run_cmd
**dict((k, getattr(options, k, None)) for k in kwarg)
File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/alembic/command.py", line 515, in current
script.run_env()
File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/alembic/script/base.py", line 489, in run_env
util.load_python_file(self.dir, "env.py")
File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/alembic/util/pyfiles.py", line 98, in load_python_file
module = load_module_py(module_id, path)
File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/alembic/util/compat.py", line 184, in load_module_py
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/kallithea/alembic/env.py", line 108, in <module>
run_migrations_online()
File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/kallithea/alembic/env.py", line 83, in run_migrations_online
cfg = config.get_section(config.config_ini_section)
File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/alembic/config.py", line 225, in get_section
return dict(self.file_config.items(name))
File "/usr/lib/python3.7/configparser.py", line 857, in items
return [(option, value_getter(option)) for option in d.keys()]
File "/usr/lib/python3.7/configparser.py", line 857, in <listcomp>
return [(option, value_getter(option)) for option in d.keys()]
File "/usr/lib/python3.7/configparser.py", line 854, in <lambda>
section, option, d[option], d)
File "/usr/lib/python3.7/configparser.py", line 394, in before_get
self._interpolate_some(parser, option, L, value, section, defaults, 1)
File "/usr/lib/python3.7/configparser.py", line 444, in _interpolate_some
"found: %r" % (rest,))
configparser.InterpolationSyntaxError: '%' must be followed by '%' or '(', found: '%&CENSORED'
The affected config was smtp_password, which is why i unfortunately cannot provide the complete field.
Fortunately, there is a workaround: the % sign needs to be escaped, e.g. replace ‘%' with '%%’.
After applying the workaround, alembic could successfully upgrade my database.
The bug was found while upgrading from 0.5.2.
System information:
Host: Raspberry Pi 4 (armv71)
OS: Raspbian 10, Kernel 4.19.97
Python: 3.7.3
Kallithea installed via Python venv and pip.
Please excuse my unprofessional report here, for I am still a student and not a professional.
Attachments
Comments
Comment by Mads Kiilerich, on 2020-05-22 13:08
Thanks for the fine report. Don’t think less of being a student.
I assume you get same failure with gearbox serve? In which case the stack trace shows that it happens in gearbox / paste / configparser, before hitting any Kallithea code.
True, % is special, as hinted by how %(here)s is special. https://docs.python.org/3/library/configparser.html hints towards that with a single %% example.
Would it help if the .ini header said something like
## The %(here)s variable will generally be replaced with the parent directory of ## ## this file. Other use of % must be written as %% . ##
?
Comment by Maximilian Kerst, on 2020-05-24 09:15
Yes, your assumption is right - without escaping %, gearbox also crashes and the stacktrace is similar:
(kallithea-venv) xxx@xxx:/usb/kallithea $ gearbox serve -c kallithea/my.ini
15:05:15,842 ERROR [gearbox] Failed to load server
15:05:15,843 ERROR [gearbox] Error in file /usb/kallithea/kallithea/my.ini: '%' must be followed by '%' or '(', found: '%&CENSORED'
Traceback (most recent call last):
File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/gearbox/main.py", line 172, in _run_subcommand
return cmd.run(parsed_args)
File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/gearbox/command.py", line 31, in run
self.take_action(parsed_args)
File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/gearbox/commands/serve.py", line 269, in take_action
relative_to=base, global_conf=parsed_vars)
File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/gearbox/commands/serve.py", line 308, in loadserver
server_spec, name=name, relative_to=relative_to, **kw)
File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/paste/deploy/loadwsgi.py", line 261, in loadserver
return loadobj(SERVER, uri, name=name, **kw)
File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/paste/deploy/loadwsgi.py", line 277, in loadobj
global_conf=global_conf)
File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/paste/deploy/loadwsgi.py", line 302, in loadcontext
global_conf=global_conf)
File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/paste/deploy/loadwsgi.py", line 326, in _loadconfig
return loader.get_context(object_type, name, global_conf)
File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/paste/deploy/loadwsgi.py", line 415, in get_context
defaults = self.parser.defaults()
File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/paste/deploy/loadwsgi.py", line 74, in defaults
defaults[key] = self.get('DEFAULT', key) or val
File "/usr/lib/python3.7/configparser.py", line 799, in get
d)
File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/paste/deploy/loadwsgi.py", line 101, in before_get
value, defaults)
File "/usr/lib/python3.7/configparser.py", line 394, in before_get
self._interpolate_some(parser, option, L, value, section, defaults, 1)
File "/usr/lib/python3.7/configparser.py", line 444, in _interpolate_some
"found: %r" % (rest,))
configparser.InterpolationSyntaxError: Error in file /usb/kallithea/kallithea/my.ini: '%' must be followed by '%' or '(', found: '%&CENSORED'
I agree with documenting it in the ini header since it is not Kallithea’s code. I also thought about a note in the documentation.