Archive for January, 2008

The future of programming language runtimes

January 26, 2008

There is a saying in linguistics, “A language is a dialect with an army and a navy. I think this applies in a way to computer programming languages too. The most widespread programming languages aren’t necessarily the ones that are “perfect” in an Esperanto sort of way, but the ones which have an army and a navy – i.e. large corporations backing them.

Fragmentation

The current fragmentation in the language runtime space is quite disheartening. It’s still best described by this 7-year old hub paper by Havoc and Owen. I hope they won’t be mind my linking to it. Examples of this fragmentation problem are still prevalent today. The difficulties in using gtkmozemebed from Python due to two garbage collectors and type systems in one process. To use the Linux inotify system from Ruby, you need a random third party binding that may or may not be up to date. If you want to use the Eclipse rich text editor in your next OCaml program, well you’re just screwed. The examples are almost endless…most pairs of (API X, runtime Y). Not all of those are sensible combinations, but a lot of them are.

Now, the problem with Hub I think was that it assumed you could still sensibly mesh unmanaged languages like C/C++ as they are today with managed VM-based runtimes like Python and Java. If you’ve been following the Mozilla project, you know that they are embarking on a huge effort to automatically rewrite their 2+ million lines of code to go from C++ with reference counting to C++ with garbage collection, because the problems with leaks between the two systems are too hard. (By the way, this stuff is really cool and worth watching as I’ve mentioned before).

In the GNOME community, we largely get by with systems such as PyGTK and Gtk# that do mix runtimes with reference-counted C because we’re not running untrusted code from the Internet, like Mozilla has to. If you forget to destroy that widget you removed but that still had signal handlers calling managed code connected to it, it’s a bug in your application that is your fault – not the fault of some college student writing his first JavaScript application that was an intern at a website you visited.

The technical future

This prediction has been made plenty of times for years, but I think it’s worth repeating. The future of language runtimes is pretty clearly going towards VM-based runtimes. C/C++ are dead ends for pretty much anything except a kernel. More modern AOT-compiled languages such as OCaml and Haskell have a lot to gain by switching to a VM. Concurrent, generational garbage collection isn’t trivial, and a modern VM gives you that. Being able to load code dynamically, or programmatically construct new optimized functions at runtime for specific data is very appealing. Debugging is significantly easier.

The political future

So, here we are in 2008. Is the situation much better? In some ways, yes. The Free Software community now includes Java. Popularity of truly open, community-driven languages and runtimes such as Python and Ruby seems to be rising. And finally, there is a project which is attempting to create a .NET base language runtime, Mono.

Mono

The Mono project is the marriage of two major ideas. A runtime designed for multiple languages is a very unifying concept. It would solve almost all of the issues Hub was attempting to address for managed, VM-style languages. It leaves C/C++ out unless they’re manually bound, but there’s no helping that.

The second idea though of Mono is that it is defined as a .NET clone. This idea has proven very divisive. One company, Microsoft, is the upstream. This is not a good idea for the Free Software community; we don’t have control of our technical destiny anymore. If we want to add a bytecode that makes Python run better, we have to wait for decisions to be made in some Microsoft boardroom.

If I was in charge of Mono, what I would do immediately is reach out to 3 communities that are much more clearly part of the Free Software world: Python, Ruby and finally – Mozilla Tamarin. Mono could say to these communities, “Hey, we’ll be your JIT, garbage collector, and common library runtime” – but with the freedom to adapt and change to the needs of these communities. Especially having Mozilla on board would be a coup, because it’s shipped to so many people. I certainly wouldn’t have a problem making my application depend on Firefox being installed on Windows.

The problem with separate projects like Jython and IronPython, JRuby etc., is that they’re essentially forks. They have connections with the upstream, but that’s not the same as being the upstream. If Mono can unify the truly open Free Software languages around one common runtime, it would greatly circle the wagons around

Java

Sun has finally realized that the idea that every single application would be written in Java and only needs a least-common-denominator API across every platform doesn’t work. Even more importantly, the JDK is now Free Software, and though the JCP isn’t ideal, it at least exists.

It certainly seems plausible to me to imagine that Sun would be willing to buy into a shared runtime that wasn’t .NET, but a true cross-corporation, community effort. They’re already trying to add new bytecodes to better support other languages, but instead we could translate IKVM-style from Java 5.0 bytecodes into a new shared runtime.

5 years from now…

I really hope I’ll be writing my Python app that

  • Gets efficiently JIT-compiled
  • Can access various OS components like inotify automatically as they’re added to a common shared library
  • Can seamlessly blend in a Web-like widget that executes other languages such as JavaScript without pain
  • Is able to call into frameworks like Hibernate

Oh, and I want my jetpack.

Hotwire hypershell 0.700 alpha released

January 24, 2008

Hotwire 0.700 (mirror) is now available.

So, about 8 months ago you may remember I got really frustrated with bash for carelessly losing my history, and in my spare time, I set out to create a new developer shell. Not just a plain old Unix shell; I wanted a shell that took advantage of my computer (using a modern language runtime, and integrating with the desktop environment), rather than being limited by the “terminal emulator” concept. I’d recently tried Windows Powershell around that time as well, and thought they had some good ideas but also made some bad design decisions. I thought it was possible to do better.


Looking for a firefox process

In short, I wanted to create a “hypershell” that was better than the Unix concept of a shell, and that was also improved on the Windows PowerShell model. Hotwire 0.700 is the first release where I think that goal is, if not achieved in many ways, much closer at hand. Previous versions were more like demo toys, but this is the first one I’d call alpha. In other words, usable in many ways, although far from complete. Many developers and power users will hopefully find it a suitable replacement for a terminal+Unix shell running as the same uid on a desktop. For system administrators, Hotwire doesn’t yet have a really strong story for improving the remoting experience (well, you will probably like the included ssh client), but I’m going to be taking a close look at things like func.

The new webpage (mirror) has a longer explanation of things, screenshots, etc. The ChangeLog is also a useful read, especially if you’re upgrading from a previous version. And don’t forget to try the updated tutorial to get a hands-on introduction to the system.