Issue #217: upgrade-db fails to fix auth modules after rebranddb.py

Reported by:	Matthias Langhammer
State:	resolved
Created on:	2016-05-08 09:59
Updated on:	2016-06-20 22:24

Description

While upgrading from RhodeCode 1.7.1 to Kallithea 0.3.2 I faced the following problem during the database migration:

***********************************
*** FIXING DEFAULT AUTH MODULES ***
***********************************
Traceback (most recent call last):
  File "/tmp/rhodecodetest/system/bin/paster", line 11, in <module>
    sys.exit(run())
  File "/tmp/rhodecodetest/system/local/lib/python2.7/site-packages/paste/script/command.py", line 102, in run
    invoke(command, command_name, options, args[1:])
  File "/tmp/rhodecodetest/system/local/lib/python2.7/site-packages/paste/script/command.py", line 141, in invoke
    exit_code = runner.run(args)
  File "/tmp/rhodecodetest/system/local/lib/python2.7/site-packages/kallithea/lib/utils.py", line 752, in run
    return super(BasePasterCommand, self).run(args[1:])
  File "/tmp/rhodecodetest/system/local/lib/python2.7/site-packages/paste/script/command.py", line 236, in run
    result = self.command()
  File "/tmp/rhodecodetest/system/local/lib/python2.7/site-packages/kallithea/lib/dbmigrate/__init__.py", line 57, in command
    dbmanage.upgrade()
  File "/tmp/rhodecodetest/system/local/lib/python2.7/site-packages/kallithea/lib/db_manage.py", line 175, in upgrade
    api.upgrade(db_uri, repository_path, step)
  File "/tmp/rhodecodetest/system/local/lib/python2.7/site-packages/kallithea/lib/dbmigrate/migrate/versioning/api.py", line 186, in upgrade
    return _migrate(url, repository, version, upgrade=True, err=err, **opts)
  File "<decorator-gen-16>", line 2, in _migrate
  File "/tmp/rhodecodetest/system/local/lib/python2.7/site-packages/kallithea/lib/dbmigrate/migrate/versioning/util/__init__.py", line 159, in with_engine
    return f(*a, **kw)
  File "/tmp/rhodecodetest/system/local/lib/python2.7/site-packages/kallithea/lib/dbmigrate/migrate/versioning/api.py", line 366, in _migrate
    schema.runchange(ver, change, changeset.step)
  File "/tmp/rhodecodetest/system/local/lib/python2.7/site-packages/kallithea/lib/dbmigrate/migrate/versioning/schema.py", line 92, in runchange
    change.run(self.engine, step)
  File "/tmp/rhodecodetest/system/local/lib/python2.7/site-packages/kallithea/lib/dbmigrate/migrate/versioning/script/py.py", line 145, in run
    script_func(engine)
  File "/tmp/rhodecodetest/system/lib/python2.7/site-packages/kallithea/lib/dbmigrate/versions/018_version_2_0_0.py", line 24, in upgrade
    fixups(db_2_0_0, meta.Session)
  File "/tmp/rhodecodetest/system/lib/python2.7/site-packages/kallithea/lib/dbmigrate/versions/018_version_2_0_0.py", line 69, in fixups
    setting = models.Setting(name, old_setting.app_settings_value, t)
AttributeError: 'NoneType' object has no attribute 'app_settings_value'

The problem appears while LDAP settings (which did not exist in the old database) are converted to the new format with 'auth_' as prefix. The script crashes when trying to access the attribute 'app_settings_value' from an old_setting which does not exist. So instead of using the non-existing old value, the default value 'v' should be used.

I fixed this by editing the file lib/dbmigrate/versions/018_version_2_0_0.py .

I replaced

    for k, v, t in old_ldap:
        old_setting = models.Setting.get_by_name(k)
        name = 'auth_%s' % k
        setting = models.Setting.get_by_name(name)
        if not setting:
            # if we don't have this option create it
            setting = models.Setting(name, old_setting.app_settings_value, t)

with

    for k, v, t in old_ldap:
        old_setting = models.Setting.get_by_name(k)
        name = 'auth_%s' % k
        setting = models.Setting.get_by_name(name)
        if not setting:
            # if we don't have this option create it
            setting = models.Setting(name, v, t)

With this patch I was able to compelte the migration.

Attachments

Comments

Comment by Mads Kiilerich, on 2016-05-25 01:10

Do you think this issue is related to rebranddb.py ? Why?

As far as I can see, the default value in v should only be used of there is no old setting to convert:

    for k, v, t in old_ldap:
        old_setting = models.Setting.get_by_name(k)
        name = 'auth_%s' % k
        setting = models.Setting.get_by_name(name)
        if setting is None:
            # if we don't have this option create it
            if old_setting is not None:
                v = old_setting.app_settings_value
            setting = models.Setting(name, v, t)

Do you agree?

Comment by Matthias Langhammer, on 2016-06-20 17:09

No, I don't think that this is related to rebranddb.py . I just mentioned rebranddb.py to explain that I faced this problem while switching from RhodeCode to Kallithea.

I totally agree, that the default/fallback value should only be used if there is no old setting to convert.

Comment by Mads Kiilerich, on 2016-06-20 22:24

This was fixed on the stable branch in 34ef164e0635 ... but not released yet. Do you have an old database dump and can verify that it works?

Comment by Mads Kiilerich, on 2016-06-20 22:24

Fixed in 34ef164e0635

Kallithea issues archive