Kallithea issues archive

Issue #191: 500 Internal Server Error while viewing some Changesets

Reported by: bitworld
State: new
Created on: 2016-02-14 03:09
Updated on: 2016-02-16 09:32


I'm using Python 2.7.11 and the newest Kallithea sources. My language is Chinese. the default_encoding in INI is gbk,utf8

Here is my log:

2016-02-14 11:07:21.676 INFO  [kallithea.lib.base] IP: User: <AuthUser
('id:2[zxp] auth:True')> accessed /MoistureTesting/changeset/29f93515da9ed21bf41
2016-02-14 11:07:21.721 INFO  [kallithea.lib.auth] user <AuthUser('id:2[zxp] aut
h:True')> authenticated with regular auth @ ChangesetController:index
Error - <type 'exceptions.UnicodeDecodeError'>: 'utf8' codec can't decode byte 0
xa8 in position 39: invalid start byte
File 'D:\\Kallithea\\Env\\lib\\site-packages\\weberror-0.11-py2.7.egg\\weberror\
\errormiddleware.py', line 171 in __call__
  app_iter = self.application(environ, sr_checker)
File 'D:\\Kallithea\\Sources\\kallithea\\lib\\middleware\\sessionmiddleware.py',
 line 62 in __call__
  return self.wrap_app(environ, session_start_response)
File 'D:\\Kallithea\\Env\\lib\\site-packages\\routes-1.13-py2.7.egg\\routes\\mid
dleware.py', line 131 in __call__
  response = self.app(environ, start_response)
File 'D:\\Kallithea\\Env\\lib\\site-packages\\pylons-1.0.2-py2.7.egg\\pylons\\ws
giapp.py', line 103 in __call__
  response = self.dispatch(controller, environ, start_response)
File 'D:\\Kallithea\\Env\\lib\\site-packages\\pylons-1.0.2-py2.7.egg\\pylons\\ws
giapp.py', line 313 in dispatch
  return controller(environ, start_response)
File 'D:\\Kallithea\\Sources\\kallithea\\lib\\base.py', line 449 in __call__
  return WSGIController.__call__(self, environ, start_response)
File 'D:\\Kallithea\\Env\\lib\\site-packages\\pylons-1.0.2-py2.7.egg\\pylons\\co
ntrollers\\core.py', line 214 in __call__
  response = self._dispatch_call()
File 'D:\\Kallithea\\Env\\lib\\site-packages\\pylons-1.0.2-py2.7.egg\\pylons\\co
ntrollers\\core.py', line 164 in _dispatch_call
  response = self._inspect_call(func)
File 'D:\\Kallithea\\Env\\lib\\site-packages\\pylons-1.0.2-py2.7.egg\\pylons\\co
ntrollers\\core.py', line 107 in _inspect_call
  result = self._perform_call(func, args)
File 'D:\\Kallithea\\Env\\lib\\site-packages\\pylons-1.0.2-py2.7.egg\\pylons\\co
ntrollers\\core.py', line 57 in _perform_call
  return func(**args)
File '<decorator-gen-23>', line 2 in index
File 'D:\\Kallithea\\Sources\\kallithea\\lib\\auth.py', line 793 in __wrapper
  return func(*fargs, **fkwargs)
File '<decorator-gen-22>', line 2 in index
File 'D:\\Kallithea\\Sources\\kallithea\\lib\\auth.py', line 838 in __wrapper
  return func(*fargs, **fkwargs)
File 'D:\\Kallithea\\Sources\\kallithea\\controllers\\changeset.py', line 324 in
  return self._index(revision, method=method)
File 'D:\\Kallithea\\Sources\\kallithea\\controllers\\changeset.py', line 312 in
  return render('changeset/changeset.html')
File 'D:\\Kallithea\\Env\\lib\\site-packages\\pylons-1.0.2-py2.7.egg\\pylons\\te
mplating.py', line 244 in render_mako
  cache_type=cache_type, cache_expire=cache_expire)
File 'D:\\Kallithea\\Env\\lib\\site-packages\\pylons-1.0.2-py2.7.egg\\pylons\\te
mplating.py', line 219 in cached_template
  return render_func()
File 'D:\\Kallithea\\Env\\lib\\site-packages\\pylons-1.0.2-py2.7.egg\\pylons\\te
mplating.py', line 241 in render_template
  return literal(template.render_unicode(**globs))
File 'D:\\Kallithea\\Env\\lib\\site-packages\\mako-1.0.0-py2.7.egg\\mako\\templa
te.py', line 452 in render_unicode
File 'D:\\Kallithea\\Env\\lib\\site-packages\\mako-1.0.0-py2.7.egg\\mako\\runtim
e.py', line 803 in _render
  **_kwargs_for_callable(callable_, data))
File 'D:\\Kallithea\\Env\\lib\\site-packages\\mako-1.0.0-py2.7.egg\\mako\\runtim
e.py', line 835 in _render_context
  _exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
File 'D:\\Kallithea\\Env\\lib\\site-packages\\mako-1.0.0-py2.7.egg\\mako\\runtim
e.py', line 855 in _exec_template
  _render_error(template, context, compat.exception_as())
File 'D:\\Kallithea\\Env\\lib\\site-packages\\mako-1.0.0-py2.7.egg\\mako\\runtim
e.py', line 864 in _render_error
  result = template.error_handler(context, error)
File 'D:\\Kallithea\\Env\\lib\\site-packages\\pylons-1.0.2-py2.7.egg\\pylons\\er
ror.py', line 22 in handle_mako_error
  raise (exc, None, sys.exc_info()[2])
UnicodeDecodeError: 'utf8' codec can't decode byte 0xa8 in position 39: invalid
start byte

CGI Variables
  CONTENT_TYPE: '; charset="utf-8"'
  HTTP_ACCEPT: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'

  HTTP_ACCEPT_ENCODING: 'gzip, deflate'
  HTTP_ACCEPT_LANGUAGE: 'zh-CN,zh;q=0.8,en-US;q=0.5,en;q=0.3'
  HTTP_CONNECTION: 'keep-alive'
  HTTP_COOKIE: 'kallithea=558df108640fbadb80322c0b409b0e907324bfae1f938d96ea8643
  HTTP_USER_AGENT: 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:44.0) Gecko/20100101
  PATH_INFO: '/MoistureTesting/changeset/29f93515da9ed21bf4179166eb3f8fe5af4e588
  SERVER_NAME: 'de.as.pptv.com'
  SERVER_PORT: '5000'
  SERVER_SOFTWARE: 'waitress'

WSGI Variables
  application: <kallithea.lib.middleware.sessionmiddleware.SecureSessionMiddlewa
re object at 0x0496E1F0>
  beaker.get_session: <bound method SecureSessionMiddleware._get_session of <kal
lithea.lib.middleware.sessionmiddleware.SecureSessionMiddleware object at 0x0496
  beaker.session: {'_authentication_token': '80169651534697644789578819454807252
033', 'authuser': {'is_external_auth': False, 'user_id': 2}, '_accessed_time': 1
455419241.676, '_creation_time': 1455418349.716}
  paste.registry: <paste.registry.Registry object at 0x05E77330>
  paste.throw_errors: True
  pylons.action_method: <bound method ChangesetController.index of <kallithea.co
ntrollers.changeset.ChangesetController object at 0x05E66B50>>
  pylons.controller: <kallithea.controllers.changeset.ChangesetController object
 at 0x05E66B50>
  pylons.environ_config: {'session': 'beaker.session', 'cache': 'beaker.cache'}
  pylons.log_debug: True
  pylons.pylons: <pylons.util.PylonsContext object at 0x05E66D30>
  pylons.routes_dict: {'action': u'index', 'controller': u'changeset', 'revision
': u'29f93515da9ed21bf4179166eb3f8fe5af4e5880', 'repo_name': u'MoistureTesting'}

  routes.cached_hostinfo: {'host': '', 'protocol': 'http'}
  routes.route: <routes.route.Route object at 0x047063D0>
  routes.url: <routes.util.URLGenerator object at 0x05E66A50>
  webob._parsed_query_vars: (GET([]), '')
  wsgi process: 'Multithreaded'
  wsgi.file_wrapper: <class 'waitress.buffers.ReadOnlyFileBasedBuffer'>
  wsgiorg.routing_args: (<routes.util.URLGenerator object at 0x05E66A50>, {'acti
on': u'index', 'controller': u'changeset', 'revision': u'29f93515da9ed21bf417916
6eb3f8fe5af4e5880', 'repo_name': u'MoistureTesting'})
2016-02-14 11:07:22.627 INFO  [kallithea.lib.base] IP: User: <AuthUser
('id:2[zxp] auth:True')> accessed /MoistureTesting/changeset/29f93515da9ed21bf41
2016-02-14 11:07:22.627 INFO  [kallithea.RequestWrapper] IP: Request t
o /MoistureTesting/changeset/29f93515da9ed21bf4179166eb3f8fe5af4e5880 time: 0.96



Comment by bitworld, on 2016-02-14 03:22

Comment by Mads Kiilerich, on 2016-02-14 03:59

What does "newest Kallithea sources" mean?

What does "gbk,utf8" mean? It is not just utf8?

Comment by bitworld, on 2016-02-14 05:42

The "newest Kallithea sources" means that I have built Kallithea from the latest sources which is obtained from https://kallithea-scm.org/repos/kallithea.

The default_encoding setting supports mixed encodings. I use gbk to keep all the Commit Message titles displayed correctly. My codes are submitted from Chinese Windows system. I have no idea how to submit the codes with the utf-8 encoded Commit Message. Here are two picture: one is ok with "default_encoding = gbk,utf8" settings, and another is error with "default_encoding = utf8" settings.

OK: ok.png

Bad: bad.png

Comment by Mads Kiilerich, on 2016-02-14 14:00

Are you using the default or the stable branch?

Are you using Mercurial or Git?

Do you see the same problem if you don't use localization of the Kallithea UI but see it in english?

Can you share a repository the problem can be reproducted with?

Comment by bitworld, on 2016-02-15 12:55

I'm using the default branch of Kallithea Repository.

My repository is using Mercurial.

I have test the English UI in Kallithea, and got the same problem.

I have tried to creat a demo repository to reproduce the problem, but failed.

I checked the changeset that causes my problem. It's OK in TortoiseHg. See this snap. snap2.png

Comment by Thomas De Schampheleire, on 2016-02-15 13:04

You say "I have tried to creat a demo repository to reproduce the problem, but failed." What exactly failed? Is it possible to do 'hg export 13 > somefile' and attach that file?

Comment by bitworld, on 2016-02-15 14:04

"but failed" means that I can't reproduce the problem.

I have no the rights to upload a file. I can mail it to you, if you will give me your email.

Here are tow snaps, just to show the main difference. One is for no HGENCODING settings: 13-mixed encoding.png

and another is for HGENCODING=UTF-8: 13-utf-8.png

In my demo repository, I have test the same commit message as my problematic changeset 13's. I didn't set HGENCODING=UTF-8. I can view the changeset in the demo repository correctly.

Thanks for your patiences and helps. Maybe, I should set HGENCODING=UTF-8, and keep all my files encoded with UTF-8.

Comment by Mads Kiilerich, on 2016-02-15 17:06

Yes, Mercurial works just fine when everybody use UTF-8. Unfortunately that is not enforced ... and quite hard to do consistently on Windows. Especially when it comes to non-ascii file names ... but that doesn't seem to be the problem here.

Comment by domruf, on 2016-02-16 09:32

I haven't looked into this (yet). But FWIW, I ran the tests on windows a few weeks ago and got an error related to spezial characters in file names.

I don't remember which test it was and if it is related to this problem. There were multiple problems with the tests on windows and I have only fixed a few of them yet https://bitbucket.org/domruf/kallithea/commits/5dc2bdbd01297e2a7acaf657fd762722d23314e5.

I'll let you know when I found out more about this.