How to share state with applications for Free Unix-based desktops

So you’re writing an application. At some point, you have a problem where you want to be able to talk to an instance of your program while it’s running. It turns out, this is fiendishly difficult for Free Unix-based desktops, and many programs get it wrong. On my Fedora 9 system, looking in /tmp, I see listings like gconfd-walters (GConf), hsperfdata_walters (OpenJDK), pulse-walters (Pulseaudio), emacs500 (GNU Emacs). All of these programs are broken. The root cause of this is the failure of Unix to provide a standard per-user, per-machine temporary directory. If we had that, things would be dramatically simpler, but we don’t.

You can’t use /tmp because it’s not per-user. Any time you rely on a well-known name in /tmp (like the above list), your application is subject to denial-of-service attacks if another user creates that file before you log in. Security aside, it’s also just harder to deal with because you have to check for whether or not the data in your temporary directory is stale or not (is something listening on that socket, or is it left over from a session crash?), and that’s yet more painful code.

You can’t use $HOME because it’s not per-machine (think of all the sites that subject users to the pain of NFS), and won’t always support Unix domain socket files.

Now, there are other solutions; you can use the X server, which is what Firefox does I believe. If your application is single-instance this isn’t too bad, but there is an easier way:

Enter solution: DBus

The best solution is to use DBus. The key feature here is that the DBus session bus provides exactly what we want: a per-user, per-machine namespace, as well as a fairly sane IPC system we can use to pass messages. Here’s how you use it.

  • Decide on a service name for your application. For the purposes of this discussion, we’ll call it org.openssh.Ssh, for no particular reason.
  • In your application startup, try to acquire this name on the session bus, using DBus. If you’re successful, your application can now provide a service. To pick a completely random example, this service might be multiplexing SSH connections. If something else already owns the service, then you can act as a client; you might send the message org.openssh.GetControlMasterSocketPath to return the path to another Unix-domain socket, because your application has a non-DBus protocol for communication.

In this example, we just used DBus to retrieve the path to another local, randomly-named Unix domain socket. That’s perfectly fine – DBus is not the one true IPC system. Programs like Pulseaudio and OpenSSH already have defined protocols, and there is nothing wrong with that, we don’t need to replace them with DBus. However, DBus is the best candidate for a one true atomic session namespace.

Again, all examples used in this blog post are completely hypothetical, and have no relation to the silliness of requiring users to try to hack around this issue on their own computers with variables like ControlMaster and ControlPath to enable a useful feature of your application.

Advertisements

Cute..

I see from this history meme that a lot of people are still using prehistoric shells! Low numbers like 303 or even 24 total times for top execution? Now, my shell knows how to store data properly, so for example it doesn’t toss away your entire session if you don’t exit it cleanly, and has scaled so far with near-instant searches for a 20000+ history size.

My code:

import os,sys,sqlite3
c = sqlite3.connect(os.path.expanduser('~/.hotwire/state/history.sqlite'))
cmds = map(lambda x: x[0].split(' ', 1)[0], c.execute('SELECT cmd FROM Commands'))
stats = {}
for cmd in cmds:
count = stats.get(cmd, 0)
stats[cmd] = count+1
cmds = stats.keys()
cmds.sort(lambda a,b: cmp(stats[b],stats[a]))
for cmd in cmds:
print "%s: %s" % (cmd, stats[cmd])

Output:

cd: 2280
ls: 2003
ed: 1823
git: 1362
python: 1187
make: 817
svn: 751
sudo: 669
py: 543
cat: 442
rpm: 380
rm: 370
less: 366
cvs: 306

Various

Mailing lists

Benjamin discovers that yes, pretty much no one except hardcore hackers use mailing lists (my thoughts on them here). I don’t see the disconnect between normal users and developers as “scaling” – it’s not a good thing. I’ll try to look at Fedora Forum more…must train self to ignore the flashing “NEW” icon.

New journal

I spent a little bit of free time a few days ago to make the Firefox Journal extension work again. As mentioned before, you’ll need to create an AMO account to install because it’s still in the “sandbox”.

Now it needs some CSS love for sure. But, it works and I find it pretty useful for a few tasks, such as “go back to that new bug I was looking at 10 minutes ago”.

Speaking of extensions…Password Maker is awesome. I’ve now redone most of my web passwords using it.

Account system

I’ve been thinking lately about how to improve the online.gnome.org account system. If you’re curious, my proposal is linked from here

Languages

  • Clojure definitely shows that there is a lot of room left for innovation in programming languages. I’d only briefly heard of transactional memory before; this is something to learn.
  • ECMAScript 4 looks like a massive improvement over the old JavaScript. I, for one, will not miss the old bizarre prototype system.

A good post by one of the PHP authors on dynamic vs static languages (though he phrases it as Ruby/Python/PHP vs Java). I think my feelings on this lately are that if I were going to work on a website frontend process, it makes total sense to use Ruby or Python. But there’s a lot more to the world than website frontends. For example, I don’t think I would choose a dynamic language to write Hadoop or HDFS. The dynamic vs static debate is of course a very old one, but what is new is the dramatic progress of dynamic languages on VMs designed for static languages, which I was playing with before.

What I think people in these debates greatly underestimate though is the importance and power of the Free Software ecosystem, which could also be called the “default factor”. /usr/bin/ruby has always been a simple yum/apt-get install away on Free operating systems if it wasn’t already there; no license clickthrough web pages. That matters.

It’s the community, stupid

Speaking of technology though, a lot of us sysadmins, programmers and free software hackers can get easily get caught up in the implementation details of our systems. Now for sure, some things are just technically better than others; for example, the JVM is years ahead of CRuby’s green threads and basic bytecode interpreter. But what ultimately matters is the people behind the technology. It’s easy to get confused because the technology names are staring us in the face; when you want to run your program, you type /usr/bin/ruby, and not /usr/bin/a-language-from-matz-and-and-now-a-global-community.

A key example where I see this happen often is in “packaging systems”; many people confuse the program apt-get, which is just an overgrown wget, with all the hard work that goes into packaging and maintaining software. In the beginning, people had to go through and make sure it worked. For certain types of software, it is actually hard and there’s no getting around it; an example is JNA which deals with some lower-level JVM access. In that case, a lot of the work was done by two Fedora community members. And we even sent the patch for our work upstream, so that all Free operating systems can benefit.

Anyways, that’s enough for a Friday pre-lunch update…

ToolKit

“What it lacks is an extensive screen reader like GNOME’s Orca”: Do you really think our time is best spent spending time reimplementing Orca just so we have something in Qt? Maybe someday someone will, but right now we have better things to do. This kind of mentality of “my toolkit or DIE!” is a disease. It makes us waste more effort and time when it isn’t specifically needed. In this case, just use Orca with KDE4 apps.
Aaron Segio.

Agreed! — Colin, who is currently using the official Last.fm client written in Qt because, hey it worked, and it’s Free Software.