Saturday, March 31, 2007

EclipseDarcs 0.4 available

A new version of EclipseDarcs, the Darcs integration in the Eclipse IDE, is now available. This is version 0.4 (still a preview, or milestone, version).

Thanks to Radek, we have now support for ssh repositories (note: this works only if you have passwordless login enabled, and on Windows you need the latest Darcs 1.0.9 release candidate build), and for pulling and pushing all patches from withing the workspace.

I think this is a great step forward. We're still not yet quite feature complete, and there are some bugs, so please be patient for a little while longer :-)

There are various options to get the latest version, please look at the installation instructions on the EclipseDarcs homepage. All feedback is very welcome. Enjoy.

Sunday, March 18, 2007

Command-line options in Haskell

I'm currently writing a small command-line application, and one of the tasks that needs to be done in that context is handling of command line options. So I've learned the basics of handling command line options in Haskell, and have logged my results. Nothing in this post is my own idea, I have just collected what I found when looking around :-)

My original intention was to post this as a regular blog entry here, but I did not manage to get the Haskell code listings properly formatted. I've tried systematically lots of combinations of the HTML 'code' and 'pre' tags, nbsp escapes and so on - nothing worked. Finally I gave up and put it all into a separate page.

Friday, March 16, 2007

Cohatoe 0.3 preview

The next preview of Cohatoe is now available from the download page, from the update site, and in the Darcs repository (all to be found here). The usual disclaimer (experimental state etc.) applies as before, but my feeling is that it is getting nearer some state where one can actually start some serious playing around with it. (I'm especially hopeful that there will be fewer API changes from now on.)

I have left out all the progress monitoring stuff I wrote about, but I have re-worked the Cohatoe API to execute Haskell functions within the IO monad. This means that the main entry point for Haskell code that is contributed in an Eclipse plugin (pluginMain, as declared in the Cohatoe API) is now of type [String] -> IO [String]. Please note that you have to re-compile any object code against that new API in order to run with the new version of the plugins.

Another important piece that is now in place is exception handling. Several sorts of exceptions that are raised during the execution of contributed Haskell code are caught and re-raised on the Eclipse side - instead of just making the server executable break and exit. This makes Cohatoe much more robust.

Any feedback is most welcome - have fun :-)

Wednesday, March 14, 2007

EclipseDarcs 0.3 available

A new version of EclipseDarcs, the Eclipse IDE support for the Darcs RCS, is now available. This is version 0.3. It is still a preview (milestone) version, and it contains mostly several fixes and minor improvements that have been done over the past months since the 0.2 release. (But watch out for more soon, there are things happening at the moment :-).

Tuesday, March 13, 2007

Progress on Progress?

In the past few days, I have been exploring a side alley of the main topics in Cohatoe: progress monitoring. I think it would be actually nice to have something like a progress monitor API on the Haskell side so that you can send back progress feedback to the omnipresent progress dialogs in Eclipse (and by the way, Eclipse users are very much used to a responsive UI that allows them to send everything into the background that takes more than a few hundred milliseconds time). So I poked around in that area for a while. But it looks as if I have hit a dead end.

On the Java side, taking progress messages and pass them on to an IProgressMonitor instance isn't a real problem (although it took a bit to implement, mostly for the message encoding and decoding). But I'm somewhat stuck on the Haskell side.

Once more, this is more a question of API design than anything else. I don't think there would be a problem at all if we would in general pass a progress monitor 'handle' to the main entry point function. An example would then look like so:

import Cohatoe.API

resource = plugin {
pluginMain = sayHello

sayHello :: ProgressMonitor -> [String] -> [String]
sayHello _ _ = ["Hello from Haskell"]
that is, adding another parameter to the signature of the main entry point function. The progress monitor type would be declared by the Cohatoe API, and the framework would actually create a channel behind that value that sends all progress feedback to the Eclipse side.

However, I'm uneasy with that. True, there will be a lot of occasions where it makes sense to send progress feedback, but there will be as may where it seems not necessary. Of course, in these cases one can always ignore the progress monitor argument, but it still seems not nice to me to make it a part of the function signature everytime. I would very much more like something similar to this
myFunction :: [String] -> IO [String]
myFunction args = do
pm <- getProgressMonitor
pmBegin pm "Starting" 100
-- actually do something
pmDone pm
But then where to get the progress monitor from? I have tried some approaches that seemed to lead nowhere. I suspect I could do it by designing my own monad, but I'm not deep enough into Haskell yet to do that :-) And I also can't see how that would keep the API neat. So I think for now I'll leave the progress monitoring out in the interest of simplicity and revert the code base.

(Btw, this was more of an experiment anyway; I don't think that it is a good idea in general to re-model the Eclipse APIs, such as the progress monitors, in Haskell. That's far too much work, and there is a limit anyway, since the Eclipse APIs really assume to be dealing with stateful objects all the time. I've written a bit more about that in my earlier contribution to the Eclipse Languages Symposium.)

Friday, March 09, 2007

Using Darcs (and EclipseDarcs)

Radek Grzanka has asked, on the EclipseDarcs mailing list, whether I could share some experiences using Darcs in my programming team (and I'm happy to do so :-). I have introduced Darcs to my team about one and a half year ago when we started to do some projects that involved developers sitting in locations other than our office in Karlsruhe, Germany (including external developers who were located in the US), and which also involved people who are, like I was myself at the time, frequently travelling.

For me this is one important plus for Darcs: that it allows fine control over what goes into the repository while being offline (on the train, on the plane, or in a Hotel room that hasn't internet access). With source code control systems that require the presence of a server you can check in less frequently, and therefore you tend to either check in too coarse-grained, or you spend much more time figuring out which source changes belonged to which activity when you are checking in. With Darcs, on the other hand, you can code, then record, then code, record and so on, and when you go online the next time, you can still push selectively.

The teams we are working in are relatively small (up to six people, with some fluctuation), so I couldn't say much on how it works with really big teams. But for us it worked out really well. Darcs is pretty easy to learn and also definitely fun to work with, and that was consistently the feedback I got from all developers. (It also has the occasional bug or sluggishness, especially on Windows, but that's far short of the big-time sucking that CVS can be, in my experience.)

Darcs is also helpful with branching, which we do a reasonable amount of. However, in this area one still needs some tools or techniques around it, mostly for figuring out diffs between branches. This is one of the areas where EclipseDarcs comes in very handy, namely the repository browsing perspective that allows to conveniently inspect patches. (I have also started work on a repo diff view for the Darcs repository browsing perspective, exactly from this background.) The most needed tool there is a viewer that supports viewing patch dependencies. For instance, imagine you have one release repository and one development repository. The latter holds a bunch of patches which are not yet in the release repository. Someone has the job to 'release' some patches from the development repo to the release repo. So it is important for that guy to see which patches depend on which others, in order to figure out what minimally must be pushed.

We typically have one central repository on an intranet server that we call the integration repo, or development repo. Developers work on local repos on their machines, and record changes there. All our work items, bug fixes and tasks that we do are structured in an issue tracker. So typically, when somebody has done some implementation task, that's associated with an issue tracker ID and recorded so (stating the issues tracker ID first thing in the patch comment). After that, it is usually either pushed to the integration repo, or sent to another developer who reviews it and then pushes it. On the integration repo, we have automated processes that build, collect compiler warnings, run CheckStyle, run JUnit, run PDE JUnit and so on. That way, we get early feedback about integration problems (that is, problems that go further than simple conflicts in the source text). This sort of setup is relatively common today (much more so than when we introduced it, some five years ago), and often called 'continuous integration'. It is one of the most valuable practices in software development in my view; however, it requires a certain insight and co-operation from the team. (As with many of these so-called 'agile' practices, the people in your team need to understand and value them - else they won't work much better than any other methodologies.) In our practice, we try to push and pull as often as possible (several times a day). This helps to have the entire team up-to-date with recent changes to the code base, and reduces the chance of integration problems.

In addition to the integration repo, we have branches that hold the released versions. In some cases, they are just subsets of the corresponding integration repo (the latter one holds more patches because some functionality is not yet mature enough to be released to the public, but ok to be integrated with the rest of the code base). In some cases, they are real functionality branches (branches of the main stem that contain some different functionality, e.g. for deployment of a customized version). Naturally, we try to keep the second variant rare - but in my view it is definitely more pleasant to do this with Darcs than with other systems I've worked with.

And of course we have also learned to love such techniques as those which are described on the Darcs wiki as spontaneous branches and preparation branches.

We use EclipseDarcs together with the Darcs command line (the latter one for more complicated recording that spans over multiple files, and for pulling and pushing). EclipseDarcs comes in helpful for inspecting repo contents (as I remarked above), and for adding files, quickly recording single-file changes, and it helps of course by showing which files in the repo have changed since the last recording. Well - and we all know that there is plenty of room for improvement :-) It would be nice to be able some day to have all Darcs functionality integrated in Eclipse, of course. We're only half-way there. But while that is a little inconvenient, I don't think it really should be a reason not to use Darcs. For one thing, the command line interface of Darcs is very convenient, and apart from that I'm convinced that a professional programmer should be able to use a command line tool.

The source trees we work on are almost always Eclipse Plug-In projects. This is a bit nasty when the contents of a repository must be imported into the workspace for the first time. (Especially because the contents of the pristine tree, under _darcs/pristine/, are detected by Eclipse and offered as projects to import; one must be very careful not to select these - the best technique for this is to temporarily move the entire _darcs folder out of the repository, import the projects all at once, and move the _darcs folder back.) On the other hand, this tends naturally to a well-structured and fine-grained organization of the source tree, which helps when recording, and probably also helps with the Darcs performance. For our purposes, the performance is ok (but again, that pretty much depends on the layout and size of your source trees).

Perhaps one or two more misc notes: one should checkpoint repos from time to time, it really helps to reduce the time to initially get repo contents. (It is also definitely faster to use darcs get instead of darcs init && darcs pull --all, but that's probably old news :-)

Sunday, March 04, 2007

A focused braindump

Over the last months, I have written an informal paper about some of the more arcane details of Eclipse technology, related to its update/install story. Currently, the approach in this area is based on so-called 'features' (roughly groups of plugins, which in turn are the functional units in Eclipse). The paper gives an overview over the different relationships that can obtain between features, and exposes some problems in the logical system that emerges from these relationships. It also highlights some issues that have arisen in the real-world use of the features approach.

My goal was to collect some thoughts that I have posted to mailing lists and points I made in technical discussions, mainly to have them all in one place. There is an ongoing discussion at the Eclipse project about this topic, and about replacing the features-approach in future versions of Eclipse, so hopefully this is somewhat useful for summarizing a few lessons learned. I’m confident that the problems described here will soon be a thing of the past.

Btw, if you are interested in the latest developments, here are a few pointers: the requirements pool for a new Update Manager, the Equinox provisioning project proposal, the Maya project proposal (somehwat related), and the Eclipse Packaging Project homepage (dito).