Kallithea issues archive

Issue #165: [feature] Interval sync/update for "remote repositories"

Reported by: André Klitzing
State: new
Created on: 2015-10-09 07:04
Updated on: 2018-05-18 20:13



I like to request a feature to set an interval to automatically sync a remote repository. We have some repositories that were cloned with the "remote repository"-feature. As we use this as an internal read-only mirror we like to see that the mirror will be updated after X minutes.




Comment by opensourcer, on 2016-03-18 20:44

I would love this feature as well! We need a way to disable/enable/schedule this function per repo so that each individual repo can be scheduled differently.

Comment by Mario Juric, on 2016-10-25 10:06

This would be beneficial for my company as well .. either to have this built in kallithea or some documentation on recommendations how to achieve this reliably

Comment by domruf, on 2016-10-25 13:13

I have a working implementation for this. But it requires celerybeat. So it makes the setup a bit more complicated because it needs 2 celery processes.

Comment by Søren Løvborg, on 2016-10-25 19:49

FWIW, this is quite simple to do by using an external scheduler (like cron on *nix or the equivalent Windows thing I don't remember the name of) to send a requests to the Kallithea JSON-RPC API. There's even an example on the API docs page:

curl https://kallithea.example.com/_admin/api -X POST -H 'content-type:text/plain' \
--data-binary '{"id":1,"api_key":"xe7cdb2v278e4evbdf5vs04v832v0efvcbcve4a3","method":"pull","args":{"repo":"CPython"}}'

Comment by Mario Juric, on 2016-10-25 21:19

@kwi thanks for the example, this would most certainly be doable. The thing is cron is a bit unreliable for what we're trying to achieve, and that is near instant sync after the change is done to the remote repository, not to mention that we have 100+ repositories, and some of them are not that active, so we'd be requesting for update when there are no changes.

My thought was setting up a hook on the remote repository that would notify Kallithea to pull on every successful change to the repository source, and your example with using the JSON-RPC API might just do the trick. That way Kallithea would pull the changes from the remote only when there are actual changes on the source.

To be fair this issue is opened with scheduled updates in mind so your suggestion is very much valid :)

Comment by Søren Løvborg, on 2016-10-27 14:27

Yeah, for "live" syncing, you'll need a hook on the source... and in that case, it might be preferable to have that hook push to Kallithea, ensuring that the original "hg push" does not return before the changes have been synced to the mirror.

Comment by Thomas De Schampheleire, on 2018-05-18 20:13