Kallithea 0.7.0 released

We are pleased to announce Kallithea 0.7.0, with a potential security fix, improvements and fixes in many areas and some changes that might require configuration changes when upgrading.

Please note: this release of Kallithea fixes an issue that would be a security issue for Kallithea instances that have configured a specific list of allowed IP addresses. The issue is one of "automatic" trust in HTTP headers that only should be trusted in setups where a trusted proxy server is known to have set these headers.
This issue could not be fixed in a minor update without configuration changes. In Kallithea 0.7, these headers are now by default ignored. If necessary, Kallithea must be explicitly configured with the headers to use. Please refer to the security page for more details about the security implications, and the upgrade notes below for more details about the configuration aspect.

The installation instructions in the documentation have been improved and extended, for example in following areas:

  • Horizontal scaling and web/wsgi servers
  • Hooks
  • Overview of the various relevant locations in the file system
  • Apache mod_wsgi dispatch script and configuration
  • Configuration in combination with proxy servers

The API method get_pullrequest has been extended and a new API method edit_reviewers added.

In case Kallithea is placed behind another (proxy) server, you will need to configure two additional settings: url_scheme_header and remote_addr_header.

The https_fixup configuration setting has been removed and is to be replaced by a suitable value of url_scheme_header.

The extension system rcextensions has now been renamed to extensions. User-provided customizations are now expected in extensions.py rather than rcextensions/__init__.py. Compatibility with the old approach is still retained, but will be removed in a future major release.

A new create-pullrequest hook in this extension mechanism has been added, which makes it possible to e.g. trigger jobs in a CI server when pull requests are created. Such CI jobs could then use the comment_pullrequest API to post the result of the CI job as a comment.

Users of Git repositories can now add their own post-receive hooks by creating .git/hooks/post-receive-custom which will automatically be called by the Kallithea post-receive hook.

Autocompletion of users in pull requests and comments now also search for the combinations 'firstname lastname' and 'lastname firstname', which is interesting in cases where there are many users with the same firstname or lastname and where the autocomplete display limit would normally be reached. A specific example is with common Chinese surnames like 'Cheng' or 'Wang'. Additionally, user autocompletion based on email address has been added.

The rendering of diffs has been improved in several ways. For example, trailing tabs are now showed just like trailing spaces already were. It is now also possible to comment on 'meta' lines, e.g. chunk headers. In particular, this makes it possible to comment on empty added files.

MySQL/MariaDB handling has been improved to get full UTF-8 support. Note that this requires a change in your configuration file.

SSH keys of type ed448 are now supported.

Celery has been updated to version 5.

Support for Mercurial 5.8 has been added.

Support for hgsubversion has been removed. It was no longer working in Python 3 and seems no longer maintained upstream.

Please note: this version of Kallithea works with Python version up to (and including) 3.9.x, but will probably not work with 3.10.x.

See 'Changes' below for a more detailed list of changes most relevant to users.

Upgrading from 0.6.x

Please refer to the upgrade instructions in the documentation for detailed steps: https://kallithea.readthedocs.io/en/stable/upgrade.html. Below are just some specific notes.

As mentioned in the upgrade instructions, generate a new configuration file and apply the necessary changes to fit your installation. There have been changes in the configuration file; starting from the old one would be a source of errors or confusion. Specifically, pay attention to:

  • a number of changes in the celery settings

  • the charset value for the MySQL/MariaDB sqlalchemy.url setting

  • new settings remote_addr_variable and url_scheme_variable which you may need to modify in setups with other servers in front of Kallithea. See https://kallithea.readthedocs.io/en/stable/setup.html#proxy-setups.

  • the setting https_fixup has been removed. If you had this set to true in your configuration, you'll need to set url_scheme_variable correctly. See https://kallithea.readthedocs.io/en/stable/setup.html#proxy-setups.

  • if you are using the Paste Deploy Prefix Middleware (egg:PasteDeploy#prefix), make sure to set translate_forwarded_server = False in the same section.

There are also some minor database migration steps.

If you had a file rcextensions/__init__.py (at the same level as your configuration file), then move and rename it as extensions.py next to the configuration file (without subdirectory as before). So basically:

mv rcextensions/__init__.py extensions.py

The old file path rcextensions/__init__.py is currently still respected if no extensions.py file is found, but this compatibility will be removed in a future major release.

If you have git repositories, make sure to reinstall the hooks (refer to the upgrade documentation for instructions).

If you are using mod_wsgi, you may want to compare your WSGI dispatch script with the current example, which has been simplified. See https://kallithea.readthedocs.io/en/stable/setup.html#apache-with-mod-wsgi.

Changes

Below are the most relevant changes between 0.6.3 and 0.7.0. Note that it is not a complete list: many changes are purely internal refactoring. Please refer to the source repository if you are interested in full details.

API

  • extend get_pullrequest data with created_on and updated_on fields cs
  • new method: edit_reviewers cs
  • fix repo creation when using Celery cs
  • fix repo creation inside repo groups when using Celery cs
  • fix repo group permission check for repo creation cs
  • fix update_repo check for permission to create top level repos cs

Authentication

  • don't trust clients too much - only trust the last IP in the X-Forwarded-For header cs
  • only use X- headers instead of REMOTE_ADDR if explicitly told so in remote_addr_header cs
  • only use X- headers instead of wsgi.url_scheme if explicitly told so in url_scheme_header - drop https_fixup setting cs

Celery

  • drop pre-Celery 4 compatibility cs
  • upgrade to Celery 5.0 cs
  • celery-run should only initialize app and sqlalchemy after workers have been forked cs

Command-line interface (kallithea-cli)

  • db-create: introduce --reuse option cs
  • repo-scan: introduce options --install-git-hooks and --overwrite-git-hooks cs
  • repo-scan: log error and continue after failure creating a scanned repo cs

Configuration file (ini)

  • move archive_cache_dir to the data directory cs
  • celery: default to more than 1 task per worker cs
  • mysql: bump sqlalchemy.url for MariaDB/MySQL charset to utf8mb4 to get full UTF-8 support cs
  • suggest translate_forwarded_server = False when using Paste Deploy Prefix Middleware cs

Database

  • better support for databases with "odd" characters in the name, such as - cs
  • mysql: bump charset to utf8mb4 to get full UTF-8 support cs
  • mysql: create database with explicit UTF-8 character set and collation cs

Documentation

  • various improvements to the installation overview, setup and upgrade instructions
  • refer to EXTRA_MAPPINGS in extensions instead of hacking LANGUAGES_EXTENSIONS_MAP directly cs
  • describe, visualize, and verify internal code structure and layering cs

Email notifications

  • explicitly add author of pull request in invitation mail cs
  • add changeset author in subject of comment emails cs

Extensions (rcextensions)

  • drop unused (and thus misleading) return values cs
  • rename default hook stubs cs
  • rename rcextensions into extensions but provide compatibility cs

Middleware

  • drop sentry support cs
  • drop unused appenlight integration cs

SSH repository access support

  • update authorized_keys after deleting a user with SSH keys cs
  • support ssh-ed448 keys cs
  • verify SSH keys haven't been truncated cs

User interface

Changelog

  • fix crash on display of empty commit messages cs

User autocompletion

  • also query 'firstname lastname' and 'lastname firstname' combinations cs
  • also match on email address cs

Diff rendering

  • mark trailing tabs similar to trailing spaces cs cs2
  • fix per file display of "Raw diff" and others cs
  • consistently clarify in title text which icon links are for "this file" and not global cs
  • fix "Show inline comments" cs
  • drop per file "Ignore whitespace" and "Increase context lines" buttons - it didn't work and had conceptual issue (Issue #344) cs
  • allow commenting on meta lines (Issue #329) cs
  • slightly better error message for "diff not recognized as valid" cs
  • improved handling of Git diffs with " quoting cs
  • handle some escaped characters in Git diffs cs

Style

  • fix comment bubble appearance upon zoom cs
  • mark failed comment submissions with red panel heading cs

Version control support

Hooks

  • add extensible create-pullrequest hook cs
  • reserve .kallithea_ hooks namespace for internal use cs
  • drop UI display of internal hooks - these hooks are just an implementation detail cs
  • when overwriting non-Kallithea hooks, move the old hook to .bak cs
  • clarify in UI and documentation how the hook control essentially is Mercurial only cs
  • git: drop using Git pre-receive hooks - we don't need them cs
  • git: invoke hooks/post-receive-custom from hooks/post-receive cs
  • git: detect existing symlink hooks before overwriting - only update plain files cs
  • git: don't force exact mode on Git hooks when making them executable - use umask cs

Mercurial

  • support Mercurial 5.8 cs
  • restore make_ui reading of .hg/hgrc cs
  • read repo-specific .hg/hgrc file in more places cs cs2

Subversion

  • drop it - hgsubversion doesn't support Python 3 and seems dead upstream cs

Other changes

  • setup: bump version numbers of external dependencies cs
  • setup: make chardet a mandatory dependency cs
  • i18n: update translations for French, German, Greek, Polish, Portuguese, Russian
  • introduce configuration for running pytype cs

Thanks

We would like to thank everyone that contributed to the Kallithea repository (directly or by updating translations in Weblate) since release 0.6.3 (the numbers are the amount of commits)...

309 Mads Kiilerich
 45 Thomas De Schampheleire
  2 ssantos
  2 J. Lavoie
  2 fresh
  2 Michalis
  1 Tim Ooms
  1 Artem
  1 robertus
  1 Александр
  1 Private
  1 vs
  1 Eugenia Russell
  1 Étienne Gilli

... as well as everyone contributing in other ways, e.g. by testing, reporting issues, discussing via mail or IRC, etc.