Issue #308: Internal server error when showing full diff for file that contains '{' in its name when using Mercurial repository
Reported by: | Branko Majic |
State: | resolved |
Created on: | 2018-02-10 00:11 |
Updated on: | 2018-05-18 19:38 |
Description
When trying to view a full diff for a file that contains '{' in its name, an internal server error will be thrown. This happens with Mercurial repository. Reproduction steps within this issue are provided for the default
branch (changeset 228dd29e79da
as of this writing), but same issue happens with stable branch/version 0.3.3
as well (with different traceback).
Reproduction steps
-
Set-up Kallithea (minimal development environment described in
contributing.rst
should suffice. Below instructions assume the user will be calledadmin
for simplicity sake. -
Log-in into Kallithea and create a new Mercurial repository called
test-diff-filename-with-bracket
. -
Clone the empty repository:
cd /tmp/ hg clone http://admin@localhost:5000/test-diff-filename-with-bracket
-
Add a file to repository that includes curly bracket (
{
) in its name, and push the changes:cd /tmp/test-diff-filename-with-bracket echo "This is a funny README file." > 'README{' hg add 'README{' hg commit -m "Added a funny README file with bracket" hg push
-
Log-in into Kallithea and open the repository page.
-
Click on the top revision in the list of latest changes.
-
Click on the Show full diff for this file link right next to the
README{
filename (icon should resemble a paper sheet with<>
symbols in it).
Expected results
- In step (7), a full diff is shown for the file.
Actual results
-
In step (7), an internal server error is thrown with the following traceback:
2018-02-10 01:05:42.099 DEBUG [backlash] Traceback (most recent call last): File "/home/user/.virtualenvs/kallithea/lib/python2.7/site-packages/tg/appwrappers/session.py", line 71, in __call__ response = self.next_handler(controller, environ, context) File "/home/user/.virtualenvs/kallithea/lib/python2.7/site-packages/tg/appwrappers/i18n.py", line 71, in __call__ return self.next_handler(controller, environ, context) File "/home/user/.virtualenvs/kallithea/lib/python2.7/site-packages/tg/wsgiapp.py", line 285, in _dispatch return controller(environ, context) File "/home/user/projects/kallithea/kallithea/lib/base.py", line 553, in __call__ return super(BaseController, self).__call__(environ, context) File "/home/user/.virtualenvs/kallithea/lib/python2.7/site-packages/tg/controllers/dispatcher.py", line 119, in __call__ response = self._perform_call(context) File "/home/user/.virtualenvs/kallithea/lib/python2.7/site-packages/tg/controllers/dispatcher.py", line 108, in _perform_call r = self._call(action, params, remainder=remainder, context=context) File "/home/user/.virtualenvs/kallithea/lib/python2.7/site-packages/tg/controllers/decoratedcontroller.py", line 119, in _call output = controller_caller(context_config, bound_controller_callable, remainder, params) File "/home/user/.virtualenvs/kallithea/lib/python2.7/site-packages/tg/decorators.py", line 44, in _decorated_controller_caller return application_controller_caller(tg_config, controller, remainder, params) File "/home/user/.virtualenvs/kallithea/lib/python2.7/site-packages/tg/configuration/app_config.py", line 127, in call_controller return controller(*remainder, **params) File "<decorator-gen-88>", line 2, in diff File "/home/user/projects/kallithea/kallithea/lib/auth.py", line 810, in __wrapper return func(*fargs, **fkwargs) File "<decorator-gen-87>", line 2, in diff File "/home/user/projects/kallithea/kallithea/lib/auth.py", line 860, in __wrapper return func(*fargs, **fkwargs) File "/home/user/projects/kallithea/kallithea/controllers/files.py", line 682, in diff enable_comments=False) File "/home/user/projects/kallithea/kallithea/lib/diffs.py", line 202, in wrapped_diff context=line_context) File "/home/user/projects/kallithea/kallithea/lib/diffs.py", line 252, in get_gitdiff ignore_whitespace, context) File "/home/user/projects/kallithea/kallithea/lib/diffs.py", line 262, in get_diff ignore_whitespace=ignore_whitespace, context=context) File "/home/user/projects/kallithea/kallithea/lib/vcs/backends/hg/repository.py", line 260, in get_diff file_filter = match(self.path, '', [path]) File "/home/user/.virtualenvs/kallithea/lib/python2.7/site-packages/mercurial/match.py", line 158, in match listsubrepos=listsubrepos, badfn=badfn) File "/home/user/.virtualenvs/kallithea/lib/python2.7/site-packages/mercurial/match.py", line 385, in __init__ root) File "/home/user/.virtualenvs/kallithea/lib/python2.7/site-packages/mercurial/match.py", line 838, in _buildmatch regex, mf = _buildregexmatch(kindpats, globsuffix) File "/home/user/.virtualenvs/kallithea/lib/python2.7/site-packages/mercurial/match.py", line 874, in _buildregexmatch raise error.Abort(_("invalid pattern (%s): %s") % (k, p)) Abort: invalid pattern (glob): README{
Additional information
The error occurs in kallithea.lib.vcs.backends.hg.repository.MercurialRepository.get_diff
, in call file_filter = match(self.path, '', [path])
.
The match
function comes from mercurial.match
module, and treats the passed-in patterns ([path]
) as extended glob patterns by default.
In order to make this call work for paths that include globbing characters, the path
must be either escaped, or (more easily), the match
function should be told to treat all provided patterns as plain paths by passing-in additional default='path'
parameter to it.
Attachments
Comments
Comment by Mads Kiilerich, on 2018-02-10 14:54
Thanks.
I guess it would be easy-ish to reproduce in kallithea/tests/functional/test_compare.py ?
Considering the other case where we also might want handle filenames starting with glob:
, it would perhaps be better to prefix the filename with path:
before passing to match?
Can you spot/test similar problems in other places?
Comment by Branko Majic, on 2018-02-10 15:09
From what I can tell, the only occurrences are on controllers compare.py
, pullrequests.py
, and changeset.py
. So, theoretically, a fix could be done there instead (in those three places).
I can't see any other piece of code that might pass on a glob (or something else) towards this code path.
I already made a PR that fixes the issue by enforcing path
type in Mercurial's get_diff
itself. It should be noted that Git repository backend does not support any sort of pattern-matching, so any ability to specify the glob or regex paths would be Mercurial-specific - so not sure if this kind of dual behaviour should be introduced for something that is more of a standard interface method.
Comment by Branko Majic, on 2018-02-10 15:09
Wait a minute... I didn't send out PR for this one, nevermind :)
Comment by Branko Majic, on 2018-02-10 15:12
I just did a quick-hack testing with default='path'
. So, fix can be done on controller or Mercurial backend level. I'm inclined towards the backend just so any code would benefit from it, but if you have other ideas, let me know. I'll try to implement a test for this - if it gets implemented on the backend side, I guess regular unittests should be more appropriate.
Comment by Mads Kiilerich, on 2018-02-10 18:00
It seems like a fix at vcs level would be the best option. But it is quite messy anyway, so no big deal.
But I don't see how specifying default='path'
can help handle a file named glob:foo
- that will override the default and use globbing for foo
?
Comment by Branko Majic, on 2018-02-10 21:52
Ah... Right, good point. It completely slipped my mind :)
Looking into the match
function signature again, there is also an option exact
, which states patterns are actually filenames (include/exclude still apply)
. Doing some quick testing seems to result in correct behaviour (I tried some glob:
and re:
filenames with brackets etc.
This might be the better option then?
Comment by Mads Kiilerich, on 2018-02-11 14:11
right - exact=True
seems perfect
Comment by Thomas De Schampheleire, on 2018-05-18 19:38
Fixed with b4a5632733d9