Kallithea issues archive

Issue #295: Creating a repository with a clone causes the UI to hang until clone is complete

Reported by: shachar_gluska
State: new
Created on: 2017-08-22 19:28
Updated on: 2017-09-01 18:53

Description

When you create a new repository and choose to clone an existing one, the UI won't indicate that the server is busy cloning. Only after the clone is complete, the server will kindly respond with "please wait while creating the repository".

Existing Behavior: The webpage is simply waiting for a response from the server. Appears to be "stuck".

Expected Behavior: The user will be be shown the "Please wait while creating your repository" page while the server is busy cloning.

Steps to reproduce:

Tested with Kallithea 0.3.3.

Optional: Run Kallithea in a terminal / with access to its run logs. This will later enable to see that the webui appears frozen, but kallithea is actually busy cloning.

  1. Login to Kallithea.

  2. Click Add Repository

  3. Insert a URL into Clone remote repository:

  4. Choose a repository that takes a few seconds to clone. This will probably change a lot depending the server's hardware and connection. For me it was sufficient to choose https://kallithea-scm.org/repos/kallithea

  5. Click Add

  6. Watch as the webui appears to be stuck.

  7. Optional: look at Kallithea's run logs. You'll clearly see it busy cloning.

  8. Watch as Kallithea finally responds with "creating your repo, will redirect when done".

Attachments

Comments

Comment by shachar_gluska, on 2017-08-22 19:30

Comment by Mads Kiilerich, on 2017-08-22 19:36

That's pretty much a limitation of running as a pure WSGI web application. But the UI and messaging could perhaps be tweaked.

If using celery, such tasks will run in another process - see http://kallithea.readthedocs.io/en/stable/setup.html#celery-configuration .

Comment by shachar_gluska, on 2017-08-22 20:17

My intuition tells me that it's a simple problem with the order of operations. display the "please wait" and only then start the actual process.

Would the UI still look frozen? Yes. But the user now knows that the request is handled. It seems reasonable for the ui to be non responsive if the correct message is displayed.

Of course, I could be wrong.

Comment by Thomas De Schampheleire, on 2017-08-29 20:04

I did some analysis of this issue. The current behavior is as follows: clicking the "Add Repo" button after filling in the new repo form, triggers method 'create' of kallithea/controllers/admin/repos.py. This triggers the creation of the repository (via Celery or directly in this thread), then (after the repo has been created, indeed), redirects to another page that shows a 'the repository is being created' message, via method 'repo_creating' in the same controller.

This repo_creating method renders a page with embedded javascript, that uses AJAX to check the status of the repository creation, via a URL that ends up in method 'repo_check' still in the same controller.

Once done, there is a javascript-based redirect to the repo summary page, if not we continue polling.

I agree that this situation is not ideal in the case without Celery. In order to solve it, the process would need to be:

  • from create_repo, redirect to 'repo_creating' without having started anything yet
  • in repo_creating, render the html page, still without having started anything yet
  • from repo_check, initiate the repo creation/cloning process on the first call, then check the status on subsequent checks.

Some problems with this approach are:

  • need to pass the necessary form elements all the way through the three involved methods and associated html, each time from python to get elements and back. I made a first attempt at solving the issue but bumped into this -- it requires more changes than I thought it would.

  • the initiation of the creation/cloning process would be triggered by Javascript from the client side. Previously, after submitting the form, the repo clone would be initiated directly, Javascript is only used for polling the status. If Javascript did not work for any reason, waiting a while and checking the result manually would work fine. WIth the proposed approach it does not.

Perhaps there is a way to avoid these problems: when submitting the initial form, use Javascript to show a 'Please wait' banner on the form submission page already, possibly only done when celery is disabled, possibly greying out the form upfront, ...

In the Celery case, do as before, redirect and show a real 'ongoing' page. In the no-Celery case, leave out the intermediate redirect to the ongoing page and directly go to either an error page, or the repo summary.

Thoughts?

Comment by Mads Kiilerich, on 2017-08-30 01:31

Nice analysis!

Another option could be to do the "repo create" as an async call from the main page, only with one slow call if not using celery, but a fast call followed by polls and liveness updates if using celery?

Comment by Marcus Marcus, on 2017-09-01 08:01

Maybe its also a way to start more instances and use Apache with a Proxy/VConf setup Simply set the Port and instance_id inside the .ini

Comment by Mads Kiilerich, on 2017-09-01 09:00

Yes, the documentation recommends running multiple worker processes. That prevent slow responses from blocking other requests. The slow request will still take the time it takes and can't respond before it is done.

Comment by Thomas De Schampheleire, on 2017-09-01 18:53

@kiilerix: could it be that your "Another option" boils down to the same as my "Perhaps there is a way out..." thing?

If not I don't think I understand your suggestion :-)