The Old Way:

1. Developer in libfoo (a C library) works on solving a problem, decides to create foo_bar() function
2. Function foo_bar works, and is committed
3. A period of time later, a new shared library appears in OS update package
4. You install the package
5. You try to use function bar from your favorite $LANGUAGE bindings, but it isn’t bound yet
6. You curse and go searching bugzilla to see what the status is
7. If upstream is active, some manual review is done to make sure the function is sanely callable from $LANGUAGE
8. If the function has some special requirements, sometimes metadata is manually added to hackish .defs files
9. A patch is eventually made and committed
10. Another period of time later, new bindings appear
11. You can call foo.bar()

Note that steps 7,8, and 9 have to be repeated for each binding, be that PyGTK, Gtk#, java-gnome, etc. Even worse they all have divergent .defs files.

The New Shiny Future Way:

1. Developer in libfoo (a C library) works on solving a problem, decides to create foo_bar() function
2. If it is not sanely bindable, author gets warning from g-ir-scanner, reworks it or adds metadata, commits
3. A period of time later, a new shared library appears in OS update package
4. You install the package
5. You can call g.foo, because your $LANGUAGE bindings use GObject-Introspection

The new way, obviously, is going to be significantly better. By actually including all the metadata necessary to interface with the system in the shared library (.so) itself, we eliminate a large lag time before improvements in the lower levels of the stack become usable to bindings. Even more important I think is that by integrating into the build process of the GObject-based C libraries, we can help ensure that authors don’t add new interfaces that are difficult to bind.

Why am I talking about this? Well, I just reached a major milestone in a side project I’ve been working on. Here’s a screenshot:

Instant GObject/WebKit bindings for the JVM

And not just WebKit of course – this is a large step forward for Java bindings for Gtk, Clutter, libnotify, libgypsy, various canvases like hippo canvas and in general the entire “Free desktop stack”.

Of course because it’s Java, it’s also instant bindings for Jython, Rhino, JRuby, Groovy, etc. Just ran this program with Jython trunk:

from org.gnome.gir.dynamic.WebKit import *
from org.gnome.gir.dynamic.Gtk import *
from org.gnome.gir.gobject import *

GObjectGlobals.init()
GtkGlobals.initCheck(None, None)
win = Window(WindowType.TOPLEVEL);
sw = ScrolledWindow(None, None)
win.add(sw)
wv = WebView()
wv.open("http://www.gnome.org")
sw.add(wv)
win.setSizeRequest(640, 480)
win.showAll();
GtkGlobals.main();

I just tossed up a web page for the project here. Right now it’s basically blocked on finishing gobject-introspection itself; finishing the .gir to .repo compiler. So I’ll be looking at that. My personal goal was to get bindings for some canvas (probably hippo), and clutter, along with all of Gtk+.

Let’s take a step back though. What I think is interesting about this is that it enables much more seamless multiple layer applications. I blogged about this before. At the bottom, you have unmanaged C (GObject) code. In the middle, you have a managed, statically typed core (Java/.NET). At the top, you have a managed, dynamic language (Groovy,Boo,Python,JavaScript). GObject-introspection dramatically lowers the pain for going between the bottom and middle layers. Using the JVM (or .NET) makes the middle and top layer transition much nicer, especially for a runtime-native language.

Realistically landing and finishing all of this is probably a year out at the earliest still. But it’s fun to hack on!

Edit: By the way, HotSSH version 0.2.5 is out, and this one actually has a README that tells you how to install it! It fixes various bugs, and most interesting it has the favicon work.

Awesome. In the near future, I can record a .OGG file, toss it up on the web somewhere (say my personal S3 bucket), and link to it in my blog with <video>. Now I have another motivation to make sure my family is using Firefox. Of course really what we also need is a general video hosting site that uses <video> – does one exist yet?

Today I spent a good chunk of time going through DBus bugs and pending patches. There are still some major things outstanding – large patches for Mac OS X and Windows integration, Scott’s timeout work, but I have a feeling after those we’ll be at the point where it really is basically done. Now we’re moving on to replacing the init system. Woo.

But, that’s not what is exciting about today. Is there anything exciting about today? Well, today is the day that Unix gained /etc/favicon.png. No, I’m not talking about HTTP, I’m talking about a HotSSH patch that I finished on the shuttle ride back:

Note the tabs have icons

The GNOME sysadmins haven’t yet put /etc/favicon.png on window.gnome.org, and I stole the freedesktop.org favicon for my local machine to test

Basically when you open a connection, we asynchronously run some Python code on the remote machine to test for the existence of /etc/favicon.png and if it exists, its mtime. If the cached favicon we have is old or nonexistent, we (again asynchronously) retrieve it via scp. All of this reuses OpenSSH’s connection sharing so you don’t have to reauthenticate for any of this.

A quick Q&A:

• OMGWTFBBQ! – That’s not a question.
• I’m a system administrator and I have a favicon.ico for our website, how do I set this up? – Run convert /path/to/favicon.ico /etc/favicon.png, copy around to all machines as appropriate, put it in Puppet or whatever.
• Are favicons associated with hosts or keys? – Good question, right now I attempt to associate them with the host key, but OpenSSH’s host key hashing foils that sadly. So probably with the host.
• Why? – Because it’s faster to recognize images than text.

Thomas blogged about my nemesis, bug 482354. I’ve been trying to upstream the fix for unbreaking clicking on links for quite a while now.

In it, Christophe brought up Rhythmbox, which is quite similar to Pidgin in how it acts with the tray icon.

Several years ago I was chatting with Seth about Rhythmbox and he mentioned that he thought it was a fairly special kind of application because it is generally used in a very “transient/background” way. For example, a normal way to use it would be minimized to the tray (not even in tasklist at the bottom); then when you want to switch songs or albums, you click the tray, do a quick search, and then minimize again. This contrasts with “regular” applications like Eclipse, Firefox, Evolution where it’s expected that you will often spend a substantial amount of time in them in one go.

The tray icon approach sort of works, but I think we could do better. Following is a potential approach that I’m recording in my blog so I don’t forget, and of course to gather comments.

Transient Application

First, for reference here is how Rhythmbox currently looks:

Unmodified Rhythmbox

And here’s a mockup of how it could work:

How Rhythmbox could be a “transient” application

The general idea is to treat the application window like a really fancy GtkMenu. Here’s a concrete list of user-visible behavioral changes:

• Clicking anywhere outside of the window causes it to minimize back to the tray icon.
• It does not appear in the task list
• There are no minimize/maximize buttons
• The close button does not actually exit the app (i.e. stop your music), but just minimizes
• There is a visible arrow showing you the association with the tray

Overall I think this approach will make the “show Rhythmbox window, choose song/album, start playing, make it go away” task nicer since you’ll only have to move your mouse to the tray icon and click once (to open) instead of twice (once to open, once to close). It will make it a lot clearer to the user what’s going on (in the current Rhythmbox we have the minimize/maximize animation, but no arrow).

Implementing this would require window manager changes, new GTK+ API for GtkStatusIcon, and updating several applications that fall in this category (Rhythmbox, Pidgin, Banshee, etc.) to use it. Also someone with actual graphics skills would have to draw the arrow. Does that sound like a lot of work just to save you one mouse click? Not when you’re applying the forehead mashing method of user experience improvement!

Now to find some time to implement it…

Edit: – I forgot to mention this would also be perfect for the new NetworkManager connection dialog.

An excellent update from one of the Jython hackers (and followup). I particularly liked this:

>>> from __future__ import GIL
Traceback (most recent call last):
(no code object) at line 0
File "", line 0
SyntaxError: Never going to happen!

For me the most useful thing is that Jython SVN now supports decorators. I use decorators fairly extensively in my Python code, and they were the main thing blocking me from using Jython more widely.

Some people may wonder – why Jython? One reason is that the tooling around the JVM is awesome. JDWP (and clients like Eclipse‘s debugger) blow away gdb. For a while now I’ve been trying to debug a periodic lockup in a multithreaded Python process, and it’s been immensely painful with gdb. There are a lot of other extremely useful FOSS tools for heap analysis, profiling, etc. Even if your entire world currently runs on Python, and you aren’t interested in easily taking advantage of some of the other awesome JVM software out there like Hadoop, running on Jython makes sense alone just for OpenJDK being among the best Free VMs out there.

I liked this blog entry from Planet KDE. That entry reminded me to talk about one way I like to think about how to improve the user experience with software – and that is by what I call the “forehead mashing” method.

This is where as you’re developing software, write out a few problems that you’re trying to solve. Then, for every step the user is required to make – for every dialog that pops up, for every few seconds of keystrokes, every dialog to click, every checkbox to check…lower your head onto the desk and hit your forehead.

You don’t have to do this with a lot of force. Just enough so that you feel it.

This calls for several head-desk hits

One of the top things that will quickly make your forehead sore are Wizards/Assistants. The way I think of these is that the programmer is so proud of their work and they think it’s so awesome and complicated that they want you to acknowledge it.

What if programmers inflicted these kinds of things on ourselves? Say in gcc.

If you all keep making wizards, I’ll create this patch for GCC

It’s not just wizards of course that are the source of pointless user time spent reading text and clicking buttons; random dialogs are another source. Often with some thought, it’s really not hard to reduce and simplify.

This is davidz’s theme, don’t blame me

So I promised in the comments in my previous post that I would do something to reduce the pain from the notification bubble I added; to keep the universe in balance as it were. So I’m happy to say that in Fedora rawhide now, the above dialog is no more. It was just a bad idea, and you will never see it again.

One of the things I’ve been looking at again recently is the initial experience for the desktop. Previously we investigated reducing the number of steps by starting the browser by default; incidentally I have plan for a more refined version of that kicking around that isn’t quite ready but should hopefully be soon.

In the context of the Online Desktop work we want to get the user logged in online. However, this requires a network connection. Pretend for a moment you’re new to the system. Look at a default desktop, say one from the Phoronix Fedora 9 review, without the obligatory window to 1970 in the middle.

How do you know what to do to get online? If you’re using a wired connection then you don’t have to know anything; NetworkManager does it automatically. However a much more common case now is wireless networks, and you would have to know to click on the “two computer and broken X” icon in the top right. A simple solution is to have a notification pointing to it:

How to connect to a wireless network

Cooking up the patch for that, I quickly ran into the problem though of notifications pointing to the wrong place. If you’ve used GNOME for long enough you probably know what I mean; on login sometimes getting a notification in the top left instead of the top right, or it could just be off by 10 pixels. This usually wasn’t too bad because the notification was more about the content, but in this case a notification pointing to the wrong thing was actually worse than nothing.

Fixing this right ended up taking me though a number of layers in the stack, from the application displaying the notification, to libnotify, to GtkStatusIcon, to the notification protocol, and finally to the notification daemon, which needed a fairly big patch. However after spending a bit of time this weekend I finally got it to work:

Link to Theora video

Not earth shattering, but it has been a pretty longstanding bug in the desktop. It was actually sort of fun to get back to grubbing around with the low level of Xlib.

As you may have gathered from previous entries, I’ve recently become interested in programming languages again.
I’m almost done (though skimming some parts) with my copy of the aforementioned Ruby book. and though I haven’t done any substantial Ruby programming (just playing around in jirb while reading), I think I have a good idea now why so many people love the language.

Closures

Most of my real code over the last number of years has been in C, Java, and Python, and I know those languages and their runtime libraries pretty well, but reading the Ruby book I was struck by how really useful closures can be in an Algol-family language (i.e. not Lisp). Well, Ruby calls them “blocks” and has infrastructure on top in the form of yield, etc., but that’s fundamentally what they are. C/Java/Python all lack them (no, Python’s single-line lambdas are too restrictive to count).
Closures…environment, get it?

Closures are incredibly powerful, in fact you might say they’re the ultimate language construct. Neal Gafter has a good description of the kinds of things you can do with them, going from a language that doesn’t currently have them.

Polyglot programming

While I was thinking about this blog entry, I reread one of Steve Yegge’s great blog posts, and decided to look up more about his reference to the author of a design pattern book “leaving Java to go to Ruby”. After going to Martin’s home page, I found on his wiki he has a good entry which is pretty close to what I wanted to talk about. That is basically: it makes sense for large software systems to have multiple layers in different languages.

Now, if you’re thinking “Wow, that’s obvious”, that’s good; but there is more to the story here. So let’s look at some rationales. If your app is large enough, you probably have parts which need to be fast. And you probably have other parts which cry out for a domain-specific language.

So for speed, you’ll want a lower layer which is usually characterized by manifest typing and direct vtable function dispatch. Read: C++/Java/C#.

But it makes sense often to have a higher layer which is agile. It is better for the parts of your program which change rapidly – this could be user interface bits you’re prototyping, or rapidly creating test cases. This layer is usually characterized by implicit typing (possibly with type inference), metaprogramming capabilities, and (ideally) good integration with the lower level language. Read: Groovy,JavaScript,Python,Ruby

There is a lot of software out there split in exactly this way; in fact, you’re almost certainly reading this blog entry in one of them, Firefox, where the answer is C++ and JavaScript. A lot of computer games are built in this way too – for Civilization 4, the answer is C++ and Python, and for World of Warcraft it’s C++ and Lua. If you’re familiar with Java, just think about JSP and Ant – they’re really DSLs. If you mostly know Python or Ruby, think about how much of the underlying platform is actually written in C/Java/.NET.

So it’s fairly easy to dismiss anyone who says something like “everything must be written in language X”, for values of X like C,Ruby,Python,Java. Which reminds me to say: Eclipse really needs to embrace Eclipse Monkey.

The impedance mismatch

So we accept that it makes sense to have multiple languages with different characteristics. One important issue then becomes – how similar are our two different layers? Taking the example of C++ and Python as in Civilization 4. The gap is enormous. C++ containers are not the same as Python containers. C++ strings are not the same as Python strings. C++ objects and Python objects are wildly different. The answer to this problem is to create a special glue layer; in Mozilla, it’s called XPCOM. In GNOME, it’s called pygobject. These layers are very painful to create and maintain.

An interesting question is – what if our two languages shared more? Do we really need to have separate container types just to get agility and dynamism? The answer turns out to be – no, which we’ll get to in a minute. As we know, the fact that there are a lot of things that every modern language shares lower level components (like garbage collection, JIT compilation) led Microsoft to brand .NET as a multi-language runtime (as an aside, plenty of languages ran on the JVM long before .NET was created; for example Kawa, which dates to 1996). Now here’s the thing, though. Running on .NET does not make Python objects same as .NET objects, nor does it make their containers the same.

What is an object?

Let’s briefly take a look at what a Python object is. It’s a fairly illustrative example of just how different languages can be. In Python, every object instance is by default a dictionary (hash table), with data stored in the __dict__ member. Every property lookup or method call has to in general traverse a chain of hash table lookups. At any point, some other code can come along and add a new entry in an object’s dictionary:

#!/usr/bin/python

class Test(object):
def __init__(self, a):
self.a = a

t = Test("hello")
t.b = 42
print t.b

Supporting this level of dynamism is expensive, both in time and space, again because every object instance carries along a mutable hash table under the covers. It means you can’t share very much between processes. It makes multi-threading much slower because everything has to be synchronized on that dictionary. Besides being expensive, it’s almost never what you actually want, at least by default. You usually want t.b to be an error. This is by far my biggest issue with Python. In fairness to Python, it predates almost every other language discussed here.

Stealing and language evolution

Languages are clearly stealing things from each other, and evolving together. In the Ruby book they often mention how certain parts were taken from other languages. Java and C# are stealing ideas from each other. ECMAScript 4 is clearly rebuilding itself on a more JVM/.NET like class model.

What I’ve been looking at lately is a new dynamic language that has clearly stolen a lot of the good ideas from Ruby and Python, but is a lot more “native” to a modern runtime (in this case, the JVM): Groovy.

#!/usr/bin/env groovy

class Test {
String a
}

def t = new Test(a: "hello")
t.b = 42
println b

This results in an exception about a missing b, because its idea of a class is exactly the same as the underlying JVM platform, where objects are much more static by default (this is also true of .NET). Note we can even declare types if we like (or we can just use def). I really like how default constructors work – it’s even less typing than both Python and Ruby! It has useful closures, regular expression and hash table literals. Pretty cool. For a more complex example, here’s an example of a fairly typical scripting task of log file processing I wrote a few days ago. I’m fairly sold so far, but there is still more to learn. I spent a bit of spare time poking at getting it packaged, but ran into some Maven bootstrapping issues.

More on languages

One random link: An awesome feature of Python is Generators, and if you aren’t familiar with them and think of yourself as a “systems programmer”, check out this very good slide set.

Second to last: some recent additions to my Google Reader feed: Charles Oliver Nutter, John Rose, Lambda the Ultimate.

As an aside that’s not directly language related, but also new to my feed list is Why, who is like a great artist-programmer churning out amazing works like Shoes. Does anyone else have the feeling that for Why all of these code projects are just what he does in his idle time, and in the next few years he’ll emerge from his underground hideout with an army of giant robots and take over the earth?