Issue #325: Upgrade to SHA hashing
| Reported by: | James Shewey |
| State: | new |
| Created on: | 2018-06-27 05:29 |
| Updated on: | 2018-06-30 20:31 |
Description
The FIPS 140-2 standard, published by NIST disables support in the kernel and in OpenSSL. Basically, this breaks md5 hashing support through OpenSSL libraries. Thus, if a system sets the kernel flag fips=1, it breaks Kallithea. The service will start, but upon accessing the page, the stack trace below is recieved:
2018-06-26 18:47:59.806 ERROR [waitress] Exception when serving /favicon.ico Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/waitress-0.8.8-py2.7.egg/waitress/channel.py", line 337, in service task.service() File "/usr/lib/python2.7/site-packages/waitress-0.8.8-py2.7.egg/waitress/task.py", line 173, in service self.execute() File "/usr/lib/python2.7/site-packages/waitress-0.8.8-py2.7.egg/waitress/task.py", line 392, in execute app_iter = self.channel.server.application(env, start_response) File "/usr/lib/python2.7/site-packages/paste/gzipper.py", line 38, in __call__ response.gzip_start_response) File "/usr/lib/python2.7/site-packages/paste/cascade.py", line 130, in __call__ return self.apps[-1](environ, start_response) File "/usr/lib/python2.7/site-packages/paste/registry.py", line 379, in __call__ app_iter = self.application(environ, start_response) File "/usr/lib/python2.7/site-packages/Kallithea-0.3.5-py2.7.egg/kallithea/lib/middleware/wrapper.py", line 43, in __call__ return self.application(environ, start_response) File "/usr/lib/python2.7/site-packages/Kallithea-0.3.5-py2.7.egg/kallithea/lib/base.py", line 312, in __call__ return self._handle_request(environ, start_response) File "/usr/lib/python2.7/site-packages/Kallithea-0.3.5-py2.7.egg/kallithea/lib/middleware/simplegit.py", line 68, in _handle_request return self.application(environ, start_response) File "/usr/lib/python2.7/site-packages/Kallithea-0.3.5-py2.7.egg/kallithea/lib/base.py", line 312, in __call__ return self._handle_request(environ, start_response) File "/usr/lib/python2.7/site-packages/Kallithea-0.3.5-py2.7.egg/kallithea/lib/middleware/simplehg.py", line 73, in _handle_request return self.application(environ, start_response) File "/usr/lib/python2.7/site-packages/pylons/middleware.py", line 177, in __call__ self.app, new_environ, catch_exc_info=True) File "/usr/lib/python2.7/site-packages/pylons/util.py", line 50, in call_wsgi_application app_iter = application(environ, start_response) File "/usr/lib/python2.7/site-packages/weberror/errormiddleware.py", line 156, in __call__ return self.application(environ, start_response) File "/usr/lib/python2.7/site-packages/Kallithea-0.3.5-py2.7.egg/kallithea/lib/middleware/sessionmiddleware.py", line 62, in __call__ return self.wrap_app(environ, session_start_response) File "/usr/lib/python2.7/site-packages/routes/middleware.py", line 131, in __call__ response = self.app(environ, start_response) File "/usr/lib/python2.7/site-packages/pylons/wsgiapp.py", line 103, in __call__ response = self.dispatch(controller, environ, start_response) File "/usr/lib/python2.7/site-packages/pylons/wsgiapp.py", line 313, in dispatch return controller(environ, start_response) File "/usr/lib/python2.7/site-packages/Kallithea-0.3.5-py2.7.egg/kallithea/lib/base.py", line 439, in __call__ session.get('authuser'), File "/usr/lib/python2.7/site-packages/Kallithea-0.3.5-py2.7.egg/kallithea/lib/base.py", line 424, in _determine_auth_user return AuthUser() File "/usr/lib/python2.7/site-packages/Kallithea-0.3.5-py2.7.egg/kallithea/lib/auth.py", line 488, in __init__ self.anonymous_user = User.get_default_user(cache=True) File "/usr/lib/python2.7/site-packages/Kallithea-0.3.5-py2.7.egg/kallithea/model/db.py", line 649, in get_default_user user = User.get_by_username(User.DEFAULT_USER, cache=cache) File "/usr/lib/python2.7/site-packages/Kallithea-0.3.5-py2.7.egg/kallithea/model/db.py", line 559, in get_by_username "get_user_%s" % _hash_key(username) File "/usr/lib/python2.7/site-packages/Kallithea-0.3.5-py2.7.egg/kallithea/model/db.py", line 68, in <lambda> _hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest() ValueError: error:060800A3:digital envelope routines:EVP_DigestInit_ex:disabled for fips
The basic fix here is to switch to using a sha1 or better hash instead of an MD5 hash. A cursory grep of the source code results in the following files that would need to be swapped to a stronger hash:
./kallithea/lib/celerylib/__init__.py:from hashlib import md5 ./kallithea/lib/celerylib/__init__.py: md5(func_name + '-' + '-'.join(map(safe_str, params))).hexdigest() ./kallithea/tests/other/test_libs.py: _md5 = lambda s: hashlib.md5(s).hexdigest() ./kallithea/lib/dbmigrate/schema/db_2_0_0.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest() ./kallithea/lib/dbmigrate/schema/db_2_0_1.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest() ./kallithea/lib/dbmigrate/schema/db_1_5_2.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest() ./kallithea/lib/dbmigrate/schema/db_2_2_0.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest() ./kallithea/lib/dbmigrate/schema/db_1_6_0.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest() ./kallithea/lib/dbmigrate/schema/db_1_7_0.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest() ./kallithea/lib/dbmigrate/schema/db_2_0_2.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest() ./kallithea/lib/dbmigrate/schema/db_1_8_0.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest() ./kallithea/lib/dbmigrate/schema/db_1_4_0.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest() ./kallithea/lib/dbmigrate/schema/db_2_2_3.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest() ./kallithea/lib/dbmigrate/schema/db_1_3_0.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest() ./kallithea/lib/dbmigrate/schema/db_2_1_0.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest() ./kallithea/lib/dbmigrate/schema/db_1_5_0.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest() ./kallithea/model/db.py:_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest() ./kallithea/model/db.py: DEFAULT_GRAVATAR_URL = 'https://secure.gravatar.com/avatar/{md5email}?d=identicon&s={size}' ./kallithea/lib/helpers.py: return 'C-%s-%s' % (short_id(raw_id), hashlib.md5(safe_str(path)).hexdigest()[:12]) ./kallithea/lib/helpers.py: _md5 = lambda s: hashlib.md5(s).hexdigest() ./kallithea/lib/helpers.py: .replace('{md5email}', _md5(safe_str(email_address).lower())) \ ./kallithea/lib/markup_renderer.py: from hashlib import md5 ./kallithea/lib/markup_renderer.py: digest = md5(matchobj.group(0)).hexdigest()
Upgrading this hashing will allow companies who undergo NIST auditing and require NIST compliance to use Kallithea, however I could see that this might cause an issue for an upgrade to newer versions and hashes might need to be converted or regenerated somehow.
Attachments
Comments
Comment by Thomas De Schampheleire, on 2018-06-30 20:31
Hello, welcome to Kallithea and thanks for reporting this.
First of all, new development is happening on the default branch, in which many changes have already been done. For example, dbmigrate is no longer used, so these no longer need changes.
While I guess that some of the internal hashes could equally be some other hash algorithm, the gravatar one is an example that could not change. The Gravatar service (which is of course optional and could be skipped for FIPS compliance, requires emails to be hashed using MD5.
Change in hash algorithm for cache-related hashes would cause cache misses, but otherwise I would assume it's only a transitional problem at the time of the upgrade.
If you have an interest in this and have some programming skills, you are very welcome to propose patches. I think that would definitely be the fastest way forward.