I started patch-tag coming on two years ago, and a blog post has been brewing about what went right, what went wrong, what I didn’t expect (or what was harder than I thought) and where I see things going. Here goes.
UPDATE: After I posted the initial version of this communique, a couple of people have asked if patch-tag is here to stay. Short answer. Yes Long answer, see my reply to Eric Kow below.
What went right.
Haskell is a joy to work with, the haskell community continues to be friendly for the newbie and fun for the expert, better now than when I started. The explosion of creativity on hackage has kept making simple things simpler while pushing the limits of what you can experiment with, with a simple library include.
Darcs worked out too. It is still my favorite versioning system and I use patch-tag for offsite versioning almost all my new work, and keep trying to convert my friends and cubicle mates to it. Darcs users are nice and a lot of them see to be working on interesting things.
Happstack as a server — overall I’m sold, though there are a lot of rough edges, and I watch snap and yesod with interest. With happstack too many monads and the documentation is sparse… but it got the job done for me, so ok. I really want to do a yesod test drive too — Michael Snoyman has put a lot of himself into this framework, and from browsing the docs and watching the intro videos I have to say he is doing an awesome job.
View Code: HSP/HStringTemplate.
HSP is a bit of an odd one becuase I don’t use HSP in patch-tag, but I use it for my other happstack work, and I like it. It isn’t all win. If you are using HSP you have to recompile every time you change the view code, and the time lag can get noticeable, particularly if you have monolithic View files with a lot of code, and rely on cabal install. HSP also relies on the trhsx preprocessor which means that projects that require it won’t cabal install ootb without ~/.cabal/bin
(or similar) is in your PATH. And there are some mystifying error mesages. I think it also uses template haskell, so slightly slower compiles. But even so, I like it. I like having the compiler check that my tags match up, I like being able to copy html from the wild directly into my view code, and I like that designers find it easy to work with.
HStringtemplate is what I use in patch-tag and I like that too. The compiler won’t check my html tags match up, but it is fast, no recompile is required, and it is pretty smart. I will probably depend on hsp in the future, and might even switch patch-tag to hsp at some point, but overall I have to say hstringtemplate worked out pretty well.
Linode/amazon cloud. I use linode to host patch-tag, and the aws cloud to occasionally spin up a dev server and experiment. Odd combination perhaps but so far so good.
Help from unlikely places. At different times and in different ways Matt Elder, Dan Patterson, and Ram Durbha had a major impact in moving patch-tag forward, for no tangible material reward. I hope to get more detailed in another post, but for now I would just like to say thanks. These guys taught me, maybe the most important lesson I learned from patch-tag: that the universe is basicaly a friendly place.
What went wrong.
The hyphenated name. Mama mia. Why didn’t I just call it patchtag? The actual reason is that I showed logo mockups with and without hyphens to a couple of friends and everybody seemed to like the hyphen version better. If I could go back in time and do-over, I’d probably ditch the hyphen.
The macid data store. I will probably be switching patch-tag back to some more traditional data store, probably a vanilla db. No one big reason, just a lot of small ones.
Well there is one big one. Patch-Tag app state sometimes just loses data. So, a new user account will get created and then vanish. I think this happens when the out-of-memory killer kills patch-tag before an event has been written to the macid serialization log. Honestly though, I’m not sure. Patch-tag can soldier with a bit of human intervention because the most valuable artifacts (repositories and namespaces) are on the file system and not in macid. Still not good.
Then there are the minor issues with macid. Small user / dev community. The documentation just isn’t all that great, and not that many people know how to use it. It was started by the happstack originators (alex jacobsen and co I suppose), and found a home in the current happstack cabal — kudos to jeremy shaw and the other regulars on the happstack list. But it just somehow never seems to have “clicked” as a technology from my perspective. It seems to me, somewhat impressionistically I admit, to use an inordinate amount of memory, and slow down compile time — all those template haskell directives. And I don’t really understand it all that well even after using it for a long time. Finally, macid isn’t as easy to roll back on corruption as I had initially thought — you have to edit a binary file and… ugh. Macid *wants* to be a silver bullet kind of nosql solution that will scale to multiple servers transparently, use native data structures, and make life easy for the developer. It is almost that, but it isn’t there yet and other
nosql solutions have gotten a lot of traction in the meantime.
Finally… it is somewhat painful to admit, but patch-tag really doesn’t need transactions. I probably could have just used text files and read/show serialization for state. What the heck was I thinking? I can’t even remember anymore.
Cabal/dependency management. I was a bit conflicted on whether to put cabal under the things that went wrong category because I have no intention of giving up cabal and it is clearly a core technology for this kind of project. That said, dependency wrangling proved to be a continuing time and energy drain throughout the project. On a nearly daily basis it seems, things would install one day and wouldn’t the next because somebody had updated a dependency on hackage and this broke something upstream. It doesn’t seem to happen that often in small to medium size haskell codebases, but projects that have a lot of moving parts (eg use happstack) appear more vulnerable to this problem. I suspect yi, gitit, gtk2hs have a similar installer experience.
There is a second thing about cabal that actually has an easy fix, but I didn’t discover for a long time. (Thanks Matt Brown for pointing this out!) Cabal install, which was how I compiled for the first year or so, started taking a long time. Like, over a minute. The fix is to just compile with ghc –make, after using cabal install just the first time to pull in all dependencies, and periodically to get updates. This seems to usually go about 4 times as fast, and when trying to stay in flow mode 15 seconds versus over a minute can start making a big difference.
Gitit fork. Gitit on the whole, I am happy with. I don’t think that many patch-tag users have taken advantage of this feature, including myself, but I really like the portability of keeping documentation alongside repo. Cross polination with another major happstack consumer is another win. Where I have regrets is having forked the gitit code somewhere along the line to get some look and feel features I wanted because I was in a hurry. Now gitit mainline has progressed several versions and I have to undo the fork I did and clear up all these niggling details to sync up, or maintain the fork forever. Whoops.
Unix security model alongside app security model to get ssh working. This is a mess for portability. If I want to switch servers I need to recreate system accounts, ssh keys, it’s a real drag. Alex Suraci’s Darcsden has an ssh server built in, written in native haskell, and apparently this makes it a lot easier to maintain. Transitioning patch-tag to this library and ditching the unix security details is a high priority for me in terms of future maintainability.
(Eventual) Moneti$ation. I thought I would go after darcs first, add git/svn other systems once darcs was solid, which would be a big enough market to make a sustainable business. Turned out not to be so easy!
On several occasions I was about to do payments, but then I got distracted fixing something without which it wouldn’t have felt right to start charging. Looking back on it, I kind of wish I had already gone after paying customers by this point.
The good news for me on a personal level is that having done patch-tag has really helped me get the kind of work I want to have, and advocate for using the technology I like, namely functional programming.
End ramble down memory lane, and announcing open source.
Patch-Tag source is open source at
The install is non trivial, so for anybody that wants to check it
out without jumping through a whole lot of hoops (highly recommended) there is an amazon ami. Find the ami code by getting your amazon key, setting up ec2 command line tools and executing
thartman@ubuntu:~>ec2-describe-images -a | grep -i patchtag
IMAGE ami-febe5597 072945664613/patchtag dev
ami 072945664613 available public i386 machine aki-5f15f636 ari-d5709dbc ebs
You can run this ami from the comand line with ec2-run-instances, or use the aws gui or any of the numerous third party guis. Once you’re in, cd to the patch-tag directory, darcs pull the latest code, cabal install, and you should be good to go.
The main reason I am open sourcing patch-tag is to stimulate the haskell web-devel ecosystem by putting a “real world ready” app out into the open.
If anybody playing with the opened patch-tag wants to help me out with the project, here are my highest immediate priorities.
* Help me choose an open source license for patch-tag. I am considering gnu, bsd3, and CPAL (same license as reddit). What does the community think?
* Get patch-tag easier to install and on hackage. Patch-tag is not on hackage, because my feeling is that a program on hackage should Just Work or it violates legitimate expectations, and patch-tag cabal install is far from just working. Anybody is free to throw patch-tag on hackage under their own user account but I would say better to wait because the install issues *are* fixable with a little love and care.
Key subtodos for this are
* Unfork gitit and peg patch-tag to mainline gitit
* Use same ssh lib as darcsden so I can ditch most or all of the linux sysadmin stuff
* Make patch-tag machine instance for platforms other than ec2 (eg virtualbox, vmware, make it runnable on windows, etc).
* Diagnose and fix a suspected memory leak that results in regular visits from the dread oom killer.
* Transition patch-tag to a better understood / supported persistence solution than macid. My current thinking is sqlite, or possibly just text files with read/show serialization.
* Work on macid, and help realize the macid dream of straightforward nosql style persistence that uses native data structures for an easy start and scales to zillions of users with no unpleasant surprises.
Finally, and most importantly: Use haskell web-devel. Write documentation, ask questions, get on irc, and make the world a friendly place for people that like haskell and want to write the next facebook/ita/viaweb killer.
Thanks for tuning in, folks, and happy tagging!