Archive for May, 2008

Some small tweaks

May 26, 2008

One of the things I’ve been looking at again recently is the initial experience for the desktop. Previously we investigated reducing the number of steps by starting the browser by default; incidentally I have plan for a more refined version of that kicking around that isn’t quite ready but should hopefully be soon.

In the context of the Online Desktop work we want to get the user logged in online. However, this requires a network connection. Pretend for a moment you’re new to the system. Look at a default desktop, say one from the Phoronix Fedora 9 review, without the obligatory window to 1970 in the middle.

How do you know what to do to get online? If you’re using a wired connection then you don’t have to know anything; NetworkManager does it automatically. However a much more common case now is wireless networks, and you would have to know to click on the “two computer and broken X” icon in the top right. A simple solution is to have a notification pointing to it:

How to connect to a wireless network

Cooking up the patch for that, I quickly ran into the problem though of notifications pointing to the wrong place. If you’ve used GNOME for long enough you probably know what I mean; on login sometimes getting a notification in the top left instead of the top right, or it could just be off by 10 pixels. This usually wasn’t too bad because the notification was more about the content, but in this case a notification pointing to the wrong thing was actually worse than nothing.

Fixing this right ended up taking me though a number of layers in the stack, from the application displaying the notification, to libnotify, to GtkStatusIcon, to the notification protocol, and finally to the notification daemon, which needed a fairly big patch. However after spending a bit of time this weekend I finally got it to work:

Link to Theora video

Not earth shattering, but it has been a pretty longstanding bug in the desktop. It was actually sort of fun to get back to grubbing around with the low level of Xlib.

Polyglot programming

May 10, 2008

As you may have gathered from previous entries, I’ve recently become interested in programming languages again.
I’m almost done (though skimming some parts) with my copy of the aforementioned Ruby book. and though I haven’t done any substantial Ruby programming (just playing around in jirb while reading), I think I have a good idea now why so many people love the language.


Most of my real code over the last number of years has been in C, Java, and Python, and I know those languages and their runtime libraries pretty well, but reading the Ruby book I was struck by how really useful closures can be in an Algol-family language (i.e. not Lisp). Well, Ruby calls them “blocks” and has infrastructure on top in the form of yield, etc., but that’s fundamentally what they are. C/Java/Python all lack them (no, Python’s single-line lambdas are too restrictive to count).
Closures…environment, get it?

Closures are incredibly powerful, in fact you might say they’re the ultimate language construct. Neal Gafter has a good description of the kinds of things you can do with them, going from a language that doesn’t currently have them.

Polyglot programming

While I was thinking about this blog entry, I reread one of Steve Yegge’s great blog posts, and decided to look up more about his reference to the author of a design pattern book “leaving Java to go to Ruby”. After going to Martin’s home page, I found on his wiki he has a good entry which is pretty close to what I wanted to talk about. That is basically: it makes sense for large software systems to have multiple layers in different languages.

Now, if you’re thinking “Wow, that’s obvious”, that’s good; but there is more to the story here. So let’s look at some rationales. If your app is large enough, you probably have parts which need to be fast. And you probably have other parts which cry out for a domain-specific language.

So for speed, you’ll want a lower layer which is usually characterized by manifest typing and direct vtable function dispatch. Read: C++/Java/C#.

But it makes sense often to have a higher layer which is agile. It is better for the parts of your program which change rapidly – this could be user interface bits you’re prototyping, or rapidly creating test cases. This layer is usually characterized by implicit typing (possibly with type inference), metaprogramming capabilities, and (ideally) good integration with the lower level language. Read: Groovy,JavaScript,Python,Ruby

There is a lot of software out there split in exactly this way; in fact, you’re almost certainly reading this blog entry in one of them, Firefox, where the answer is C++ and JavaScript. A lot of computer games are built in this way too – for Civilization 4, the answer is C++ and Python, and for World of Warcraft it’s C++ and Lua. If you’re familiar with Java, just think about JSP and Ant – they’re really DSLs. If you mostly know Python or Ruby, think about how much of the underlying platform is actually written in C/Java/.NET.

So it’s fairly easy to dismiss anyone who says something like “everything must be written in language X”, for values of X like C,Ruby,Python,Java. Which reminds me to say: Eclipse really needs to embrace Eclipse Monkey.

The impedance mismatch

So we accept that it makes sense to have multiple languages with different characteristics. One important issue then becomes – how similar are our two different layers? Taking the example of C++ and Python as in Civilization 4. The gap is enormous. C++ containers are not the same as Python containers. C++ strings are not the same as Python strings. C++ objects and Python objects are wildly different. The answer to this problem is to create a special glue layer; in Mozilla, it’s called XPCOM. In GNOME, it’s called pygobject. These layers are very painful to create and maintain.

An interesting question is – what if our two languages shared more? Do we really need to have separate container types just to get agility and dynamism? The answer turns out to be – no, which we’ll get to in a minute. As we know, the fact that there are a lot of things that every modern language shares lower level components (like garbage collection, JIT compilation) led Microsoft to brand .NET as a multi-language runtime (as an aside, plenty of languages ran on the JVM long before .NET was created; for example Kawa, which dates to 1996). Now here’s the thing, though. Running on .NET does not make Python objects same as .NET objects, nor does it make their containers the same.

What is an object?

Let’s briefly take a look at what a Python object is. It’s a fairly illustrative example of just how different languages can be. In Python, every object instance is by default a dictionary (hash table), with data stored in the __dict__ member. Every property lookup or method call has to in general traverse a chain of hash table lookups. At any point, some other code can come along and add a new entry in an object’s dictionary:


class Test(object):
def __init__(self, a):
self.a = a

t = Test("hello")
t.b = 42
print t.b

Supporting this level of dynamism is expensive, both in time and space, again because every object instance carries along a mutable hash table under the covers. It means you can’t share very much between processes. It makes multi-threading much slower because everything has to be synchronized on that dictionary. Besides being expensive, it’s almost never what you actually want, at least by default. You usually want t.b to be an error. This is by far my biggest issue with Python. In fairness to Python, it predates almost every other language discussed here.

Stealing and language evolution

Languages are clearly stealing things from each other, and evolving together. In the Ruby book they often mention how certain parts were taken from other languages. Java and C# are stealing ideas from each other. ECMAScript 4 is clearly rebuilding itself on a more JVM/.NET like class model.

What I’ve been looking at lately is a new dynamic language that has clearly stolen a lot of the good ideas from Ruby and Python, but is a lot more “native” to a modern runtime (in this case, the JVM): Groovy.

#!/usr/bin/env groovy

class Test {
String a

def t = new Test(a: "hello")
t.b = 42
println b

This results in an exception about a missing b, because its idea of a class is exactly the same as the underlying JVM platform, where objects are much more static by default (this is also true of .NET). Note we can even declare types if we like (or we can just use def). I really like how default constructors work – it’s even less typing than both Python and Ruby! It has useful closures, regular expression and hash table literals. Pretty cool. For a more complex example, here’s an example of a fairly typical scripting task of log file processing I wrote a few days ago. I’m fairly sold so far, but there is still more to learn. I spent a bit of spare time poking at getting it packaged, but ran into some Maven bootstrapping issues.

More on languages

One random link: An awesome feature of Python is Generators, and if you aren’t familiar with them and think of yourself as a “systems programmer”, check out this very good slide set.

Second to last: some recent additions to my Google Reader feed: Charles Oliver Nutter, John Rose, Lambda the Ultimate.

As an aside that’s not directly language related, but also new to my feed list is Why, who is like a great artist-programmer churning out amazing works like Shoes. Does anyone else have the feeling that for Why all of these code projects are just what he does in his idle time, and in the next few years he’ll emerge from his underground hideout with an army of giant robots and take over the earth?

An even better OpenSSH interface

May 9, 2008

As I’ve mentioned before, in my spare time I’ve been working on a fun project to replace the Unix terminal and shell with native graphics and a high-level language runtime. One thing that I quickly found out though was that certain applications require a real VT100 terminal emulator; for example, less is a highly optimized program, and while replacing it with a proper X11 app would be possible, it would not be trivial and would largely be a distraction from my goals anyways. Another situation where you need a terminal emulator is connecting to remote Unix hosts; while I have complete control over the software on my laptop, it’s hard to change all the existing servers out there.

SSH in particular is something that really cries out for a nice application around it, rather than launching it primitively from an existing terminal emulator. Earlier I mentioned the result of a few hours of PyGTK hacking. Over the last week or two I’ve been working on improving it. Today David Ascher of Thunderbird fame asked for it, so I should probably kick it out the door (though HotSSH is primarily for X11/Unix and not the Mac).

New connection with history

As you can see, the new version of HotSSH now remembers what you do, automatically. For example:

  • If you connect to a host and use a custom username, the next time you open HotSSH and type the host again, it defaults to that custom username, without you having to do anything
  • It has pretty good completion on the Host entry, based both on ~/.ssh/known_hosts as well as its own history
  • Open windows and tabs are saved automatically, Firefox style

The only dependencies are CPython, GTK+, DBus, and OpenSSH. I’ve broken it out of where it previously lived in the Hotwire tarball into a separate source tarball. You can currently get it from the Hotwire downloads page, though my long term plan is to import it into GNOME SVN and ideally have it come with GNOME out of the box. The code is currently here, and bug reports here. It works well enough for me now; about all it might make sense to add is some sort of color scheme (we currently just pull from the gnome-terminal defaults). To run, you can just unpack the tar and python ui/hotwire-ssh. If you install it, it should create a default bash alias for ssh, but this is something that package distributors will need to make sure integrates well.