Alembic

-------

Kallithea incorporates an [Alembic](https://alembic.readthedocs.org/en/latest/)

environment in `kallithea/alembic`, portions of which is:

Copyright © 2009-2016 by Michael Bayer.

Alembic is a trademark of Michael Bayer.

and licensed under the MIT-permissive license, which is

[included in this distribution](MIT-Permissive-License.txt).

recursive-include kallithea/alembic *

Migrating from RhodeCode

------------------------

Kallithea 0.3.x and earlier supports migrating from an existing RhodeCode

installation. To migrate, install the latest Kallithea 0.3.x and follow

the instructions in the 0.3.x README to perform a one-time conversion of

the database from RhodeCode to Kallithea, before upgrading to this

version of Kallithea.

### ALEMBIC CONFIGURATION   ####

################################

[alembic]

script_location = kallithea:alembic

################################

keys = root, routes, kallithea, sqlalchemy, alembic, beaker, templates, whoosh_indexer

[logger_alembic]

level = INFO

handlers =

qualname = alembic

=======================

Database schema changes

=======================

Kallithea uses Alembic for :ref:`database upgrades and downgrades <dbmigrations>`.

If you are developing a Kallithea feature that requires database schema

changes, you should make a matching Alembic database migration script:

1. Make your code changes (including database schema changes in ``db.py``).

2. Using a database that has *not* been upgraded, you can auto-generate

   a draft Alembic script that will upgrade a database to match the code::

    alembic -c my.ini revision -m "area: add cool feature" --autogenerate

3. Edit the script to clean it up and fix any problems (e.g. remove

   ``sqlite_sequence`` changes).

4. Run ``alembic upgrade`` to apply changes to the database.

The Alembic migration script should be committed in the same revision as

the database schema (``db.py``) changes.

Assuming your changes are eventually included in the Kallithea mainline,

the script's ``down_revision`` ("Revises") will at that point need to be

updated to match the linear mainline history.

See the `Alembic documentation`__ for more information, in particular

the tutorial and the section about auto-generating migration scripts.

.. __: https://alembic.readthedocs.io/en/latest/

   upgrade

   dev/dbmigrations

.. _upgrade:

===================

Upgrading Kallithea

===================

The following describes the upgrade steps needed, independently of the

Kallithea installation manner.

1. Stop the Kallithea web application

-------------------------------------

This step depends entirely on the web server software used to serve

Kallithea, but in any case, Kallithea should not be running during

the upgrade.

2. Create a backup of both database and configuration

-----------------------------------------------------

You are of course strongly recommended to make backups regularly, but it

is **especially** important to make a full database and configuration

backup before performing a Kallithea upgrade.

Back up your configuration

~~~~~~~~~~~~~~~~~~~~~~~~~~

Make a copy of your Kallithea configuration (``.ini``) file.

Back up your database

~~~~~~~~~~~~~~~~~~~~~

If using SQLite, simply make a copy of the Kallithea database (``.db``)

file.

If using PostgreSQL, please consult the documentation for the ``pg_dump``

utility.

If using MySQL, please consult the documentation for the ``mysqldump``

utility.

Look for ``sqlalchemy.db1.url`` in your configuration file to determine

database type, settings, location, etc.

3. Activate the Kallithea virtual environment (if any)

------------------------------------------------------

Ensure that you are using the Python environment that you originally

installed Kallithea in by running::

    pip freeze

This will list all packages installed in the current environment. If

Kallithea isn't listed, activate the correct virtual environment.

See the appropriate installation page for details.

4. Install new version of Kallithea

-----------------------------------

Please refer to the instructions for the installation method you

originally used to install Kallithea.

If you originally installed using pip, it is as simple as::

    pip install --upgrade kallithea

If you originally installed from version control, it is as simple as::

    cd my-kallithea-clone

    hg pull -u

    pip install -e .

5. Upgrade your configuration

-----------------------------

Run the following command to upgrade your configuration (``.ini``) file::

    paster make-config Kallithea my.ini

This will display any changes made by the new version of Kallithea to your

current configuration, and attempt an automatic merge. It is recommended

that you recheck the content after the merge.

.. note::

    Please always make sure your ``.ini`` files are up to date. Errors

    can often be caused by missing parameters added in new versions.

.. _dbmigrations:

6. Upgrade your database

------------------------

.. note::

    If you are *downgrading* Kallithea, you should perform the database

    migration step *before* installing the older version. (That is,

    always perform migrations using the most recent of the two versions

    you're migrating between.)

First, run the following command to see your current database version::

    alembic -c my.ini current

Typical output will be something like "9358dc3d6828 (head)", which is

the current Alembic database "revision ID". Write down the entire output

for troubleshooting purposes (in case anything goes wrong), or in case

you later want to revert to this Kallithea version.

The output will be empty if you're upgrading from Kallithea 0.3.x or

older. That's expected. If you get an error that the config file was not

found or has no ``[alembic]`` section, see the next section.

Next, if you are performing an *upgrade*: Run the following command to

upgrade your database to the current Kallithea version::

    alembic -c my.ini upgrade head

If you are performing a *downgrade*: Determine the target Alembic

"revision ID" (which, as noted above, you should have written down

before upgrading). Then run the following command to downgrade your

database to a given version (example shown here will downgrade to

Kallithea 0.4)::

    alembic -c my.ini downgrade 9358dc3d6828

Alembic will show the necessary migrations (if any) as it executes

them. If no "ERROR" is displayed, the command was successful.

Should an error occur, the database may be "stranded" half-way

through the migration, and you should restore it from backup.

Enabling old Kallithea config files for Alembic use

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Kallithea configuration files created before the introduction of Alembic

(i.e. predating Kallithea 0.4) needs to be updated for use with Alembic.

Without this, Alembic will fail with an error along the following lines::

    FAILED: No config file 'my.ini' found, or file has no '[alembic]' section

If Alembic complains specifically about a missing ``alembic.ini``, it is

likely because you did not specify a config file using the ``-c`` option.

On the other hand, if the mentioned config file actually exists, you

need to append the following lines to it::

    [alembic]

    script_location = kallithea:alembic

    [logger_alembic]

    level = INFO

    handlers =

    qualname = alembic

Next, find the existing ``loggers.keys`` entry in the file and append the

item ``alembic`` to it, like this::

    [loggers]

    keys = root, routes, kallithea, sqlalchemy, beaker, templates, whoosh_indexer, alembic

Your config file should now work with Alembic.

7. Rebuild the Whoosh full-text index

-------------------------------------

It is recommended that you rebuild the Whoosh index after upgrading since

new Whoosh versions can introduce incompatible index changes.

8. Start the Kallithea web application

--------------------------------------

This step once again depends entirely on the web server software used to

serve Kallithea.

Before starting the new version of Kallithea, you may find it helpful to

clear out your log file so that new errors are readily apparent.

.. note::

    If you're using Celery, make sure you restart all instances of it after

    upgrade.

.. _virtualenv: http://pypi.python.org/pypi/virtualenv

    import kallithea.brand

else:

    assert False, 'Database rebranding is no longer supported; see README.'

EXTERN_TYPE_INTERNAL = 'internal'

# Alembic environment (configuration), based on file generated using

# "alembic init" and modified for Kallithea.

#

# Portions copyright (C) 2009-2016 by Michael Bayer.

from logging.config import fileConfig

from alembic import context

from sqlalchemy import engine_from_config, pool

# this is the Alembic Config object, which provides

# access to the values within the .ini file in use.

config = context.config

# Setup Python loggers based on the config file provided to the alembic

# command. If we're being invoked via the Alembic API (presumably for

# stamping during "paster setup-db"), config_file_name is not available,

# and loggers are presumed to already have been configured.

if config.config_file_name:

    fileConfig(config.config_file_name)

# Support autogeneration of migration scripts based on "diff" between

# current database schema and kallithea.model.db schema.

from kallithea.model import db

target_metadata = db.Base.metadata

# other values from the config, defined by the needs of env.py,

# can be acquired:

# my_important_option = config.get_main_option("my_important_option")

# ... etc.

def run_migrations_offline():

    """Run migrations in 'offline' mode.

    This configures the context with just a URL

    and not an Engine, though an Engine is acceptable

    here as well.  By skipping the Engine creation

    we don't even need a DBAPI to be available.

    Calls to context.execute() here emit the given string to the

    script output.

    """

    # Default to use the main Kallithea database string in [app:main].

    # For advanced uses, this can be overriden by specifying an explicit

    # [alembic] sqlalchemy.url.

    url = (

        config.get_main_option('sqlalchemy.url') or

        config.get_section_option('app:main', 'sqlalchemy.db1.url')

    )

    context.configure(

        url=url, target_metadata=target_metadata, literal_binds=True)

    with context.begin_transaction():

        context.run_migrations()

def run_migrations_online():

    """Run migrations in 'online' mode.

    In this scenario we need to create an Engine

    and associate a connection with the context.

    """

    cfg = config.get_section(config.config_ini_section)

    # Default to use the main Kallithea database string in [app:main].

    # For advanced uses, this can be overriden by specifying an explicit

    # [alembic] sqlalchemy.url.

    if 'sqlalchemy.url' not in cfg:

        cfg['sqlalchemy.url'] = config.get_section_option('app:main', 'sqlalchemy.db1.url')

    connectable = engine_from_config(

        cfg,

        prefix='sqlalchemy.',

        poolclass=pool.NullPool)

    with connectable.connect() as connection:

        context.configure(

            connection=connection,

            target_metadata=target_metadata

        )

        with context.begin_transaction():

            context.run_migrations()

if context.is_offline_mode():

    run_migrations_offline()

else:

    run_migrations_online()

## Template for creating new Alembic migration scripts, based on file

## generated using "alembic init" and modified for Kallithea.

##

## Portions copyright (C) 2009-2016 by Michael Bayer.

"""${message}

Revision ID: ${up_revision}

Revises: ${down_revision | comma,n}

Create Date: ${create_date}

"""

# The following opaque hexadecimal identifiers ("revisions") are used

# by Alembic to track this migration script and its relations to others.

revision = ${repr(up_revision)}

down_revision = ${repr(down_revision)}

branch_labels = ${repr(branch_labels)}

depends_on = ${repr(depends_on)}

from alembic import op

import sqlalchemy as sa

${imports if imports else ""}

def upgrade():

    ${upgrades if upgrades else "pass"}

def downgrade():

    ${downgrades if downgrades else "pass"}

"""Drop SQLAlchemy Migrate support

Revision ID: 9358dc3d6828

Revises:

Create Date: 2016-03-01 15:21:30.896585

"""