A few years ago, when Eclipse's popularity was quickly growing, an interesting phenomenon was going on: a number of small projects were popping up (for a time, at the rate of a dozen or more per week) that implemented some extension to Eclipse, often for supporting some programming language. Some of them died quickly away when the interest of the founders moved on to something else, some of them became obsolete when their main idea was implemented by the Eclipse project itself. But there were some language support plugins that remained active and continued development until today. (EclipseFP is one of them, others include PHPEclipse, RDT (Ruby), Erlide (Erlang), PyDev (Python), EPIC (Perl) and Eclipse.org's CDT project.)
The obvious paradigm for all Eclipse-based IDE's are are the Java Development Tools (JDT) from Eclipse.org. They were a most important factor in making Eclipse popular, and they pioneered many of the cool IDE features that we have come to expect from Java IDEs nowadays. However, keeping up with JDT proved hard for each of the language support projects. One obvious reason for this is of course that the effort that went into the JDT project exceeded by far the means available to the other projects, all of them (with the exception of CDT) being volunteer projects. But there are other reasons, too; I've got a few observations about one of them.
The small step and the big step
For building language support on the basis of the Eclipse Platform, there is a small step and a big step: the small step is to get basic support, the big step is deep support. In order to make the small step, you just have to do a bit Java coding, learn a few Eclipse APIs, and you need some modest understanding of the target language. To make the big step, you have to implement, or integrate someone else's implementation of, deeply language-aware functionality (i.e. functionality aware of the syntax and semantics of the target language).
The equivalent of the small step can be done even with many text editors: they often have a feature where you can enter some syntax rules and get syntax coloring for content formats such as HTML or Java source code. In Eclipse, the small step includes not only syntax coloring, but project types (called 'Project Natures' in Eclipse terminology), integration of external build tools (normally compilers), output parsers to populate the Problems View, import and export wizards, etc.
The big step, on the other hand, includes all the attractive features such as automated refactorings, search for references to language elements ('Find all calls to this function'), navigation to declarations, debugger integration and so on. (A while ago, I've co-authored an article in a German journal that discussed which features belong into the language-sensitive category; my co-author exemplified the main points from the RDT project.)
What the big step involves
What everybody wants, of course, is language support that has taken the big step. But this means that a backend must be built that can analyze the syntactical and semantical structure of source code written in the target language. (And not only source code in the language itself, but also supporting formats, for example - in the Haskell world - Cabal configuration files, GHC package configuration files, etc.) It must understand the concepts behind the structure of the language (e.g. it must know what a module is, and what it means to find the name of a module in the name of a source file, or in a declaration in a Cabal configuration, and so on).
JDT has achieved this. The Java IDE in Eclipse includes a fully standards- compliant Java compiler, and much of the interesting functionality relies on the source code parsing and incremental compilation facilities provided by it. In addition, JDT maintains a model of all language elements in the workspace; it has representations for all methods, types, (Java) packages and so on, both from source files and attached libraries. All this is naturally implemented in Java. (It is also certainly helpful that the JDT project can practice 'Eat your own dogfood', i.e. that Eclipse can be built using Eclipse itself.)
But what are you going to do if your target language isn't the same language as the one Eclipse is implemented in? First of all, there is not really a way around implementing Eclipse plugins at least partially in Java. That's unfavorable: even if you actually want to build all that functionality yourself, chances are that you want to do it in the target language, not in Java. If you don't want to re-implement everything (and that should be the common case), then it's squarely improbable that the existing stuff is written in Java. Haskell is a very good example: there is almost everything you'd like, in various degrees of maturity: source parsers (both for Haskell 98 code and many language extensions), type inference and typchecking engine (GHC API), refactoring (HaRe), API search (Hoogle), a debugger (in GHCi), and lots and lots more. But guess in what language all that is written ... ;-)
There are the usual options for language interoperability: you can run executables written in other languages and communicate via standard input and output, or use language interoperability interfaces (such as Haskell's FFI) in conjunction with Java's native interface, JNI. Some languages can be run in an interpreter; others can be compiled to Java Virtual Machine bytecode. Although this is feasible, it's sometimes ugly and complicated, and it often limits deeper integration (there's only so much information that you can send via console i/o, and in any case if your data is highly structured, you have to put additional effort in serializing and de-serializing it).
Apart from that, Eclipse has its own dynamic module system, which relies on loading code at runtime on demand - most of the plugins that are actually in an installation may not be loaded at all in a particular session. Furthermore, the entire extensibility model in Eclipse relies on the idea that plugins cannot just plug into pre-existing extension points, but may very well (and very easily) provide new extension points themselves, which are available for anyone else to extend. In order to honor these features of Eclipse, there is much more to interoperate with than just the programming language (Java) and its runtime (the JVM). It's the Eclipse platform concept that must be taken into account. And there is no obvious way to do that in any other language than Java. (There has been some experimentation recently with executing scripts in the place of Eclipse extensions, usually scripts written in languages that can be compiled to Java bytecode. If you are interested, look at this Google Summer of Code project and this post about Eclipse plugins in Scala).
Thus taking the big step is not just more difficult for others than JDT for lack of resources. It is inherently more complicated because it adds the burden of interoperating with the platform from a different language and/or runtime. It also doesn't help of course that naturally the group of potential contributors is smaller if they are required to be fluent in two different languages and sets of APIs. (I know from years of experience as a trainer for Eclipse plugin development that it takes some time to learn to find your way around in the set of Eclipse APIs, even for experienced Java programmers - simply because of their substantial size and the number of associated concepts.)
The way out
The lesson to be learned from this is, I think, clearly that a crucial piece in building an Eclipse-based IDE for a language must be to make code written in that language interoperate with the Eclipse Platform - that is, it is necessary to make it possible to write Eclipse plugins in that language. Once this is in place it becomes possible to
- re-use existing tools written in the target language,
- find and motivate contributors (i.e. who have an interest and a good knowledge about that language), and finally
- make the big step :-)