Thursday, December 28, 2006

Simple example walkthrough II

In the last post, we have prepared an Eclipse plugin project for a Hello-World Cohatoe plugin. Lets have a look at the Haskell side now.

The Haskell code that we are going to contribute will be loaded at runtime by the Cohatoe server core plugin, which in turn delegates the work to hs-plugins. hs-plugins is a library by Don Stewart written for exactly this purpose: dynamically locating compiled Haskell code at runtime, loading it and executing it, and all this in a typesafe manner. (hs-plugins can also load and on-the-fly-compile Haskell source code, but that's not a functionality I'm using.)

Haskell code that can be loaded this way has to fulfill a few conditions: it must provide, by convention, a symbol called 'resource', which is a record and has a field, pluginMain :: [String] -> [String], which is the entry point function that Cohatoe will call. (Think of it as a fancy sort of main function.) Therefore, all Haskell code that is contributed via Cohatoe will include something like this after the module declaration:

-- We must import Cohatoe.API and implement resource 
so that this code can be dynamically loaded
import Cohatoe.API

resource = plugin {
pluginMain = sayHello

-- our code starts here
-- ...
sayHello = ...
As you can see, this needs to import the Cohatoe API. Let's make sure that we have that API available.

Remember that you downloaded a second file, Unzip that archive now and open a command prompt in the cohatoe-api/ folder:
> cd C:\cohatoetest\cohatoe-api
Now run the following three commands:
> runhaskell Setup.hs configure
> runhaskell Setup.hs build
> runhaskell Setup.hs install
This compiles the cohatoe-api package (which is very small and simple), installs it and registers it as a package for GHC.

Now that we have this in place, we can compile the Haskell code that we want to call from our Eclipse plugin. Create a file, e.g. Example.hs, with this content:
module Example where

import Cohatoe.API

resource = plugin {
pluginMain = sayHello

sayHello :: [String] -> [String]
sayHello _ = ["Hello from Haskell"]
and compile it like so:
> ghc -c -package cohatoe-api Example.hs
Put the resulting Example.o and Example.hi files into a folder called os/win32/x86/obj/ in your plugin:

(You might have guessed that there is a naming convention for OS platforms at work here in this particular name for the folder. You're right. I'll elaborate on this in a later post.)

The next thing to do now that we have our Haskell object code, is to register it with Cohatoe as an executable function. This is done via Eclipse's extension mechanism. Cohatoe provides an extension point, and we'll provide an Eclipse extension to that extension point now.

Open the Plugin Manifest Editor by double-clicking the file plugin.xml in the plugin project. First switch to the Dependencies tab. In the Required section, press Add and select de.leiffrenzel.cohatoe.server.core. By doing this, we make our new plugin dependent on the Cohatoe server core plugin, which we must do because we want to use its extension point. Now switch to the Extensions tab and press Add. Choose the haskellFunctions extension point (actually, it appears under its fully qualified name, which is de.leiffrenzel.cohatoe.server.core.haskellFunctions), then right-click it and select New > haskellFunction. Here you get a form where you can fill in the details for the Haskell Function extension that we are just creating:

As you can see, we have basically to specify three things: a Java interface, a Java class that implements that interface, and the location of the object code of our Haskell function. For the latter, just specify the .o-file, i.e. $os$/obj/Example.o. We only need to specify the location of the .o-file, the location of the corresponding .hi-file is automatically determined by Cohatoe from that location. (But the .hi-file must be there in the same folder as the .o-file, otherwise hs-plugins will not be able to load the code.)

And what about the interface and implementation class? These are registered together with the object code so that Java code from Eclipse plugins can call our Haskell function. That Java code will always only see the interface that you specify here. (We will see in a moment what exactly will be visible for any Java code that wants to call our Haskell function.) As a provider of a Haskell function, however, you also have to provide an implementation class for that interface. This is necessary for Cohatoe itself. It will instantiate that implementation class and pass it on to the clients that want to call the Haskell function (they will only see the specified interface as the type of the object that they get). The primary role of this implementation class is that it gives the provider of a Haskell function a hook to implement data marshalling. In our case, there is no data to be transported from the Java to the Haskell side, and only a minimum from the Haskell side to the Java side - thus the implementation class will not have to do much.

In the screenshot above you see that I have specified cohatoetest.IExampleFunction as the name of the interface and cohatoetest.ExampleFunction as the name of the implementation class. These two types do not yet exist. You can easily create them by clicking the link before the type name in the Manifest editor.

After you have created the source files, their content should look like this:
// cohatoetest.IExampleFunction:
package cohatoetest;

public interface IExampleFunction {
String sayHello();

// cohatoetest.ExampleFunction:
package cohatoetest;

import de.leiffrenzel.cohatoe.server.core.CohatoeServer;

public class ExampleFunction implements IExampleFunction {

public String sayHello() {
CohatoeServer server = CohatoeServer.getInstance();
String[] retVal = server.evaluate( IExampleFunction.class, new String[ 0 ] );

String result = "[Got no answer from Haskell.]";
if( retVal != null && retVal.length == 1 ) {
result = retVal[ 0 ];
return result;
The only interesting thing here is the implementation of sayHello() in our implementation class. You see that there is a singleton object of type CohatoeServer, to which the call to our function is delegated. The Cohatoe server singleton knows about all registered Haskell functions, and it knows how to forward calls to them. Since it is a singleton and shared between all plugins in the running Eclipse instance, we must pass a key to it so that it knows which Haskell function we actually want to evaluate. The Cohatoe server uses the class objects of the interfaces under which functions are registered as keys; you can see that we pass IExampleFunction.class as a key here. We also pass an empty string array as parameter list. The return value (if everything goes fine) is a one-element String array that holds the single string returned by our Haskell function.

(Note that this is a really, really trivial example. What you usually want to do in this implementation is to accept objects of all types as parameters and convert them into a string representation for marshalling, then convert back from strings what you receive as result from the call to the Haskell function. This is deliberately designed so that the interface for the Java code that calls the Haskell function is typesafe. So for example, instead of accepting a String for a file name, you would use a object in the signature of your interface function, and only convert it into a string inside the implementation. Similarly, you should not return the bare strings that you get from the Haskell side, but instead convert them into objects of a type that really represent your data.)

This is all - we have now made our Haskell function available to any Java code in any Eclipse plugin that wants to call it. (Strictly speaking, we have made it available to any Eclipse plugin that depends on our own plugin.) To demonstrate how one would call it, we'll put such a call into the run() method of our 'Hello-World' action. Remember, this was in the class cohatoetest.actions.SampleAction. Modify the run() method so that it looks like this:
  public void run( final IAction action ) {
CohatoeServer server = CohatoeServer.getInstance();
Object fun = server.createFunction( IExampleFunction.class );
if( fun instanceof IExampleFunction ) {
IExampleFunction helloFunction = ( IExampleFunction )fun;
"Cohatoetest Plug-in",
helloFunction.sayHello() );
You see again that we use the type of our declared interface as key, this time to obtain an object that (sort of) represents our Haskell function, so that we can call it via that object. The rule is that you can get an object complying to the interface that you pass in as key, or null (if no function is registered for that interface). Then you can cast and pass your function arguments in a typesafe manner.

Voila, if you run this, you'll see a friendly message from the Haskell side :-)

In my next post, I'll explain some troubleshooting aids that come with Cohatoe - just in case you encountered any problems on the way.


Thiago Arrais said...

Have you considered using YHC with a Java backend? Maybe that way the Haskell code won't need to be compiled to multiple platforms. I don't know that much about YHC and may be terribly wrong here, but my idea is that the Haskell code would run inside a YHC VM hosted inside the same Java VM that hosts the Eclipse code. The YHC VM could be distributed as Java bytecode with the plugin or with Cohatoe itself (if licenses permit, of course).

The only downside seems to be that the Java runtime for YHC code isn't written yet. But a dotnet one has already been written and the folks at YHC say it shouldn't be too hard to have a Java one. Maybe someone even has already started it.

Leif Frenzel said...

Yes, that's an old discussion. The problem here is, as you quite correctly say, that there is no actually existing solution that compiles Haskell to JVM bytecode.

I've been following these discussions for more than two years now, and the opinions differ as to how difficult it would be. Sometimes I've heard that this is much easier to for .NET bytecode (i.e. MSIL) than for Java bytecode, sometimes I've heard it should be simple for both. Fact is that there is no working solution, and has not been for years.

Also note that it would not be an ideal solution either. First of all, just a prototype will not do. We're talking about linking to the GHC API, or HaRe, and many other tools from the Eclipse side, so the compiler would really have to be as production quality as GHC is now, it wouldn't suffice to have one that can do only 'most Haskell' code. Second, we still would have to compile GHC and HaRe into Java bytecode that works on all platforms, and building these is not quite trivial even from their original sources ;-). In order to get them compiled and running in such a platform-independent manner, we might have to change the code, and thus maintain a source branch. Third, we would have to ship a JVM bytecode version of all these tools, and for all their releases again, which would be a significant maintainance task.

Compare this to a solution where the Haskell part of things is compiled natively. Haskell programmers will use the tools of their choice (GHC, Cabal, or whatever their favourites are). They will compile against installed libraries, e.g. GHC-API, which are on their system anyway. So they really can concentrate on contributing their Haskell code and then use it from within Eclipse.

I have already started to prepare a somewhat long post about the different options that have been discussed and tried so far (not least by yourself :-), where I want to compare the pros and cons of the different approaches. But the significant advantage of the approach taken here in my view is that it makes it primarily simple for the programmer who wants to add some functionality. He would, on the Haskell side, really just have to do some marshalling code, and he could just simply use all the existing Haskell code that is there. And he can do it now, already.

Thiago Arrais said...

The YHC approach is quite different from that. YHC programs are compiled to some form of intermediate language that is supposed to be interpreted by an YHC machine (just like Java programs). That virtual machine can theoretically sit on top of another virtual machine, like the JVM. Maybe that is too much virtual machine stacking. Maybe not. Only a real world experiment would tell us.

The fact is YHC is quite experimental right now and can't compile much more than Haskell 98 just yet. The native approach really doesn't have this (quite large) disadvantage. After all, native tools have been around for quite some time and the interpreted ones are just a promise. There is a maintenance burden with distributing multiple binaries whose size we don't know yet, though. Maybe we should start trying something as soon as possible. Have you tried working with Cohatoe under Linux? Do you need any help?

The native approach seemed to present some annoying bugs in the past (just think EclipseFP using the GHC parser), but I think by this time you have already dealt with most of them.

Keep up with the good work! I will certainly make sure to follow your progress.

Leif Frenzel said...

Well I'm definitely interested in the developments at YHC :-) And the nice thing is of course that there would be not problem at all to convert any contributions done via Cohatoe to code contributed via another approach, such as the YHC-based one. Seeing it like this, everybody is on the safe side, as long as things are written in Haskell :-))

Leif Frenzel said...

>Have you tried working with
>Cohatoe under Linux?
Not yet. I am myself working on Windows right now, but I think I'll be able to provide a Mac-binary too, and do some testing with it.

>Do you need any help?
Yes, I would appreciate very much any help. In particular, if you could give it a test-run under Linux and perhaps provide a server binary for Linux, that would be cool. (I'm just preparing another post that describes what is involved in building a server binary for a different platform, but it's not difficult.) Any Darcs patches or suggestions are very welcome.

>The native approach seemed
>to present some annoying bugs
>in the past (just think
>using the GHC parser), but I
>think by this time you have
>already dealt with most of them.
Yes, the approach that I'm currently following avoids two heavy problems that we had with the native lib before: it does not depend on dlls, and therefore can be used on other platforms than Windows; and it runs in a separate process from the Java VM process, so that a crash in the native code doesn't kill the JVM.

The bridging code that one has to write is also much simpler, often even trivial, compared with the lots of stuff one had to do in those earlier experiments (there is no c and JNI necessary anymore).