Monday, May 31, 2010

GitFarm source code?

It is a popular request to GitFarm for its source code, but these requests are mostly ignored by its provider, Bo (yboycn). GitFarm allows you host a Git repository on Google AppEngine. It turns out that if you know a bit of Java, you can put together something like GitFarm over the weekend. Most of the pieces you need are already available as open source.
  • JGit, Git implemented in Java (EDL licensed).
  • GaeVFS, a filesystem for Google AppEngine using datastore and memcache API (Apache licensed).
And if you build your Git hosting on AppEngine that way, you are not compelled to publish your source code. Both EDL and Apache License are only triggered by distributing source or object code, not software as a service. You are not even compelled to reproduce a copyright notice for providing software as a service.

2 comments:

Big Rich said...

I'd seen JGit but not having an in-depth knowledge of Git implementations I'd assumed it was solely a client-side library, so thanks for putting me on to it again. (I'm a Java dev, so I'll be able to navigate around the code).

Just to throw a bone to those guys getting snake eyes from GitFarm, there is also a Python-based Git implementation, which, from the docs, looks as though it also implements the Git server logic. It can be found at:

http://samba.org/~jelmer/dulwich/


And if they look/search real closely, it is possible to store files up to 2gb each on GAE using a GAE/Python API (I could tell, but, hey, they'll feel better finding it for themselves...really...)

Jonas Finnemann Jensen said...

I think gitfarm uses WebDAV for access. Their push and pull addresses uses HTTP://... so that's the only logical explanation.

An implementation of WebDAV for AppEngine can be found here:
http://code.google.com/p/gae-webdav/

However, it's only possible to upload files smaller than 10 MiB, according to [1] and [2] (Request size max 10 MiB, request duration max 30s).

You might think that most git projects doesn't have files larger than 10 MiB, however, git can compress stuff into pack files. For instance a checkout of qt has only only pack file size 320 MiB.

And no, you can't upload more than 10 MiB unless, you're using blobstore, see [3].
Maybe it's possible to do a 307 redirect on a WebDAV PUT request, but I somehow doubt it.
- Perhaps I should try it out someday :)

Anyway, if blobstore can't be used, you can be sure than it will work for repositories larger than 10 MiB. And if users suddenly decided to run "git gc; git push" everything may suddenly break (Correct me if I'm wrong).

[1] http://code.google.com/appengine/docs/java/runtime.html#Quotas_and_Limits
[2] http://code.google.com/appengine/docs/python/runtime.html#Quotas_and_Limits
[3] http://code.google.com/appengine/docs/python/blobstore/overview.html