Kallithea issues archive

I have no doubt you are right but I have no idea what PEX is (in this context), how widely used it is, and why we should care.

Please try to come up with a change to the documentation that describes to uninformed people like me what problems it solves and how it does it. See http://docs.kallithea-scm.org/en/latest/contributing.html .

@lucassmagal This topic of deployment of Kallithea in environments like PEX, Docker, Vagrant is currently indeed not documented. I think it is an area of the documentation that can and should be improved. The 'deployment' of Kallithea has at least two different aspects: how to handle WSGI: using the built-in waitress/gunicorn servers, apache+modwsgi, uwsgi, etc. The second aspect is what you are referring to here: easy launching/shipping of an instance.

Note that the instructions you listed are fine for demo purposes, but are not necessarily fit for all production environments: they do not include things like celery, for example.

The same problem is present for the Dockerfile available on the docker registry: https://registry.hub.docker.com/u/petergrace/kallithea/dockerfile/

I think that these limits should be clearly clarified in the associated documentation, but ideally a full solution would be made available that does cater for production-ready environments.

I see @patrickdepinguin and @kiilerix , I'll look into it. My original intention was just informing you there was an alternative to virtualenv (and one I personally like).

About what PEX is, it is for Python what a jar is for Java applications, it was made by Twitter to help with deployments. It's pretty new and still has problemas with, say, uWSGI as an entrypoint, but I've been testing in staging for some days and it works fine (the goal for me is having something to freeze my dependencies in a portable way without relying on docker).

I believe there's no problem in running Celery with a PEX, in the end it's just a bundled Python script, but I'll test it to know for sure. Actually, I'll test in a real production environment (with postgres, nginx and stuff) today, and I'll keep you guys posted.

Ok, I deployed Kallithea using a PEX file with sqlite and celery (even though I don't need it) and it works well, it's basically just the same as calling paster with the exact same commands (to setup, to serve the website, etc.), but with all dependencies bundled in a single, portable file.

I suspected Celery would "just work" because in the end it's not a container like Docker (with I agree can make things more complex), it's just a zipped python file. I deactivated Celery to use just gunicorn in production, here's how I created it:

$ pex kallithea gunicorn futures meinheld -c paster -o kallithea.pex
# After that `kallithea.pex` will be created and can be used in place of the `paster` command
# whenever it appears on documentation

The first four arguments(kallithea gunicorn futures meinheld) are the dependencies I want to be bundled in. The second one (-c paster) is the entrypoint to the bundle (if I don't specify one, it'll just open the shell). I updated my my.ini accordingly to use meinheld's gunicorn worker. Finally, my systemd service file calls /usr/bin/python /path/to/kallithea.pex serve /path/to/my.ini.

If you guys like this approach, I can update the docs adding it.

Celery needs a broker like rabbitmq, how did you handle that? I guess it must already be installed? Or can PEX also take care of non-python packages?

I haven't used Docker myself yet, but it has ready-made images for celery, rabbitmq, postgres and python. Putting together a solution for Kallithea with these components looked quite easy, I still plan to look at this one day. Disadvantage of Docker is that it requires a recent enough kernel (3.8+ I think).

Docker brings other concerns to the table like proper persistence through volumes, coordination between containers if you separate them somehow, and etc.. Recently runC and the "Open Container something" was announced, I don't know much about how Docker deployments are going.

About rabbitmq: sure, it has to be installed on host, like postgresql and nginx. PEX is nothing like Docker and more like virtualenv, it just bundles python packages in a zipped file you can then transfer for the server and use as you see fit just like a regular python file (python mybundle.pex do-something).

As said before, I think this topic deserves some attention in the manual. Final goal would be to cover all relevant solutions, but PEX alone is already a good start.

Would you be able to add some info to the documentation and send a patch/pullrequest? Thanks!

Of course. I'll try to help with the other solutions as well.

Hi @lucassmagal, kindly checking on possible contribution to the manual...