Archive for the 'Uncategorized' Category

Platforms as a side effect

January 27, 2012

What I want to talk about here is a simple statement that I believe to be true:

The best platforms are written by the people who are forced to invent them as they make a product.

Years ago I learned a bit about J2EE; never actually wrote an app using it, but enough to get a sense. I came away with the very strong impression that the people working on it were driven by committee, with managers in their respective contributing corporations telling them what to do. They weren’t the same people out in the field writing apps using it, day in and day out, under time pressure to produce as much as possible. On the other hand, from Ruby On Rails Wikipedia:

David Heinemeier Hansson extracted Ruby on Rails from his work on Basecamp, a project management tool by 37signals (now a web application company).[10]

Now, I’ve never written a Rails app either, but it’s pretty clear from the Internet which one of these wins. Another excellent example is the Amazon Web Services. Amazon had a lot of this internally because they were forced to in order to make a web shopping site before CEO Jeff Bezos made the key decision to spin it off as a platform.

And the most topical example here – GTK+ was originally spun out of the GIMP project because Motif sucked. Anyways, some food for thought. Basically if you’re one of those people in the trenches writing a platform – you should consider asking your manager to switch to writing apps for a bit. At least hopefully this blog post reminds me later that I have a few GTK+ apps that I really should get back to hacking on…

The GPL and distributing binaries

December 6, 2011

Of late I’ve become the “build guy” in GNOME it seems. One thing I want to clear up is I do not actually care about building just because I think it’s fun or interesting in and of itself. No, the reason I care about building is because if software doesn’t build, then clearly it’s not being run. And if it’s not being run, then it’s not being tested. And if it’s not tested, then it will be crap. In other words, a competent build system is necessary for not producing crap (but not sufficient, obviously).

That motivation established, what I want to talk about is the GPL (and the LGPL). Specifically, this section:

The source code for a work means the preferred form of the work for
making modifications to it. For an executable work, complete source
code means all the source code for all modules it contains, plus any
associated interface definition files, plus the scripts used to
control compilation and installation of the executable. However, as a
special exception, the source code distributed need not include
anything that is normally distributed (in either source or binary
form) with the major components (compiler, kernel, and so on) of the
operating system on which the executable runs, unless that component
itself accompanies the executable.

The goal (spirit) of the GPL here is that if you receive binaries, you should be able to rebuild those binaries from source. Probably a number of people who work on GPL’d code may simply think that saying “The source is this git SHA1 from this repository” or “The source is this tarball” is enough – but things get interesting with the sentence plus the scripts used to control compilation and installation of the executable. It’s my understanding that this provision hasn’t been heavily explored, but a rough consensus is embodied here.

For example, an interesting question you might ask is – does that GPL provision apply to network build servers, like Open Build Service? I actually don’t know the answer, and OBS is under the GPL itself, so clearly they don’t have a problem distributing it. Now, the Xen stuff aside, OBS is basically just a wrapper around running RPM or dpkg, which in turn are just wrappers around Makefiles, which are in turn wrappers around the stuff that actually matters (the code). But still – were I to run a network build server like that, I’d probably be extra careful and embed in each build log the version of the build server (and link to source for that version), which they don’t appear to do right now.

How about the WebOS sources? There’s no links there to any scripts or details about their build system, or in general any clues for how you’d take those sources and rebuild them, and update your WebOS device with the updated binaries. Maybe that’s covered elsewhere – I didn’t look extensively.

I also recently looked at Zeroinstall, and I think they’re flirting with non-compliance, at least in some cases. For example, there’s little information on the ClanBomber feed about how it was built (with what version of 0install, with what version of what underlying distribution, etc.)

To be clear – I strongly believe in the GPL (and LGPL) – they’re a key foundation of our community and what we’re building. But complying with the provisions are not as easy as one might think, and I’d argue that this requirement is a key driver of “packages” as you see the in Debian and RPM worlds. They’re about having a full story for how you can reliably rebuild software and reinstall it on your computer. All the rest of the stuff they also do (configuration management, being able to dynamically turn a minimal install image into a desktop, etc.) is secondary.

TL;DR – If distributing binaries of GPL’d (or LGPL) software, “packages” (Debian/RPM style) for self-hosting builds are basically the state of the art. For cross builds, the Yocto project exudes competence. If you’re not using one of those systems and you skipped to this TL;DR section, you should go back to the top and read the whole post.

OS APIs: Windows 8/WinRT and GNOME/GTK+

November 30, 2011

If one is making an operating system, clearly the API that application authors use is extremely important. The whole point of an operating system is to run them. What I want to talk about is fundamental APIs, or the lowest stable level.

If you have even a passing awareness of the evolution of Microsoft Windows over the last 25 years, you know there have been a lot of APIs that have appeared, been promoted and marketed to Windows developers, and then either deprecated or relegated to just an “option”. For example, MFC. However, these APIs are all wrappers. The only time the fundamental Windows API broke incompatibly (I believe) that 16 bit applications don’t run on 64 bit; see X86-64. But the point here is that since the introduction of 32 bit Windows NT in 1993, if you coded to that API, your application will still run. If your application still runs, that means it’s the same operating system.

More recently, Microsoft for a while was promoting .NET heavily, arguably more than any of their frameworks before. It’s important to understand that at its introduction, .NET was still fundamentally a wrapper. For a while, there were rumors that it might become a fundamental API, but with the introduction of Windows 8, that won’t happen. For a really great read on this, see this ArsTechnica article. Especially fun to read are the bits about the politics, which the article only mentions in passing – you can find more scattered about the internet, like here.

Now Microsoft is saying something huge: all new APIs will be based on this new WinRT thing. Your old Win32 apps won’t break, but this time we have reason to believe they’re pretty serious – this really is the new fundamental API, and if you want your app to use new features, you will have to use WinRT. You can access WinRT from plain bog standard C, even if it’s not beautiful.

How does all of this relate to GNOME and GTK+? I think WinRT validates where we’ve been doing in GNOME with GObject Introspection. GObject may not be the most concise thing in the world (though it definitely beats C/COM), but the combination of a pure C base with added metadata and runtime support mean that all of our fundamental APIs (basically the GTK+ stack, and notably GIO for non-GUI programs) remain accessible to C and also available in other runtimes and languages.

There’s a lot more to do on introspection – we desperately need a complete documentation generator for example. It’s also pretty clear to me in that in order to truly succeed, we need to “downgrade” C to be a consumer of the API rather than a source, i.e. we need to do what Microsoft has done and define interfaces in an IDL. That will be interesting to do while still keeping around the old C APIs that don’t match the projected C binding.

TL;DR – I believe GNOME’s approach of using C with metadata and minimal runtime hooks as a fundamental operating system API is the right course, and we should keep doing what we’re doing.

Thoughts on being an upstream

October 9, 2011

I’ve been reading things people report in Bugzilla for years. How I feel about this now is that there are really several, entirely different things that we presently lump under “bug”. For example, I think it’s pretty clear that someone’s random ideas for a change to the design are totally different from say identified code regressions, which are in turn different from proposed patches.

Given that I am only one human and can’t respond to every bug, much less actually fix them, I’ve had to invent a prioritization mechanism. Note this is in my upstream GNOME role – I do work on Red Hat Enterprise Linux too, and there my priorities are obviously influenced by customers, management etc.

My goal in upstream is simply to make good Free Software. Thus, prioritization I’ve settled on looks something like this:

  • Is the bug reporter an important Free Software contributor to anywhere in the stack? Then the bug is important. If they’re also an engineer or knowledgable tester, then things are ideal because I can expect them to try things out and help me debug.
  • Is the bug a valid regression with enough data to debug it? For example, “I upgraded from GNOME 3.0 to 3.2, and application foo crashes with this stack trace…”. Then it’s important. Ideally it’s bisected – something I’d like to make easier for both developers and testers.
  • Is the bug something embarrassing? For example, badly leaking memory. Then it’s important.

Obviously, a bug could be all three of these things – e.g. a Mozilla hacker could report a regression in GNOME that causes us to leak memory. Then it’d demand a reply =) Things beyond this though get fuzzy. One thing that’s important to keep in mind is that GNOME does not have a business model that scales directly per user. Someone new using GNOME doesn’t necessarily mean there are more people working on it able to respond to, diagnose, and fix bugs. This is why as upstream, I focus so much on the things above – Free Software contributors (in some form) and unintentional regressions.

Why regressions? Well, I obviously have no obligation to someone who just happens to use and deploy my software for free – it’s not like we can e.g. work perfectly on all hardware in all situations. But it important for anyone contributing a change to the FOSS pool to make sure that while their change may take a step forward, it’s not taking two steps backward somewhere else.

I apply that rule strongly even to myself – while I don’t personally care about OpenBSD, I took the time to diagnose a regression I introduced, following the principles above. Hopefully others feel the same!

Montreal 2011

October 8, 2011

So I’m at the GNOME Summit in Montreal, and so far it’s looking good. We have a lot of sessions lined up, and a good collection of core GNOME hackers here, along with some interested outsiders.

One thing that’s been on my mind a bit is that I’m now approaching about 10 years of contributing to GNOME. Now, GNOME is two things – the people, and the code. We have a lot of good code, and some bad code. But when I originally got involved, it was more the people, probably most by Havoc Pennington. He’s one of those rare people that had a grasp of a lot of issues, and can both write well-reasoned English, and also write a lot of good code.

What I hope to do by both action and word is pass down to the next generation of GNOME hackers that are coming in now some of those principles and ideas that have guided the project, and heavily influenced me. Havoc’s blog has a lot of good stuff – I’d start with his Free Software UI post.

If you know me, then you also know that I really care about quality in Free Software. One thing Havoc doesn’t explicitly mention is that many preferences and settings (and system state) interact with other preferences and settings. In the GNOME 2 days for example, we just never figured out how upgrades would work if you customized the panel at all. For example, if you moved the clock, what would happen if a UI redesign integrated the clock closely with another applet? There’s a huge intersection of concerns here, and in practice it was entirely possible after upgrades you’d get a half-state between the two. It was gross.

In particular, what I as a software engineer live in mortal fear of is combinatorial explosion. The number of possible states exponentially grows as options and system states change, and that kind of thing makes QA and testing near-impossible. Besides just being bad UI, it makes the entire system buggy. And to me, that’s not what GNOME is about, because I’m following in the footsteps of those who came before.

Paying for email

September 19, 2011

I’ve recently been growing increasingly uncomfortable with my dependence on some of Google’s vast array of free (as in beer) products like Gmail. Now, I’m not really very concerned with privacy per se; what bothers me a lot more is the very unclear business model behind it. And particularly for email, which is (still for most people) the root of identity on the Internet.

From Google’s Q2 2011 earnings, we see that advertising on Google sites are 96% of its revenue. We don’t know how much of the advertising revenue comes from clicks on the search engine versus say GMail, but my guess is the GMail portion is very very small. Now I know GMail gets packaged up and sold to business as part of Apps For Your Domain, but how much income they get from that is unknown.

Under this premise that GMail is being subsidized by the core search business (which it may or may not be, I don’t know), what makes me particularly uncomfortable is that I’ve almost never clicked on a sponsored result in Google Search, and particularly not done so and paid for something. This means my email is effectively being paid for by other people. Who are these people and what are they doing? Seniors searching for online pharmacies I guess? I have no idea.

While at first I admit it was hard to resist “free”, I decided to just start paying actual money – directly – for email. By the recommendation of a few other people I chose Fastmail.fm, which is working out fairly well so far and I’m about ready to stop using GMail.

This means I’m also back to using Evolution. The only hiccup so far is that I find myself really, really missing the GMail conversation view. It’s simply a far faster and more effective way to read lots of email. Oh well, at least I have the source code to Evolution! I wonder what happened to that Summer of Code project for the conversation view…

Why I joined GNOME – fixing the clocks

August 22, 2011

Desktop Summit

Briefly – let me say it was great to talk to everyone I did at the Desktop Summit! I had a lot of fun. And thanks to Openismus for loaning us their offices for a successful introspection hackfest.

Clocks – maybe not so easy

So a little over 10 years ago, I wasn’t really a GNOME contributor. Actually at that time, I was becoming a Debian developer. I was making Debian run better on my (then PowerPC) MacBook, helping Debian/GNOME integration, and then after that moved into upstream GNOME where basically more interesting things happen than packaging (except for installers which sadly we haven’t pulled into GNOME and fixed yet, but that’s another story).

One thing I remember that really resonated with me a lot in 2001 was when Sun Microsystems contributed a Usability study to GNOME (the link there is obsolete, the study now lives here for the curious). Specifically, the section on clocks.

The usability study in general created a big collective realization for the GNOME project that we were basically just doing a lot of stupid and pointless things like having 4+ different clocks. That realization led to a big push for usability and simplicity, and you can find the results in things like Havoc’s preferences page and of course in the GNOME 2.0 release which followed.

A lot of Internet dwellers seem to interpret the cultural aversion in GNOME to excessive settings/preferences as us just being sadists, but what it really means to me is that there are real bugs to fix. And GNOME to me is really about that – people who believe in Free Software, but also making something usable and not buggy. Having a willingness to solve hard problems.

What does this have to do with clocks? Well, you see, it turns out clocks are a bit hard. During the Summit, Dave Airlie was complaining the GNOME 3 clock didn’t show the right time immediately after unsuspending, which honestly is pretty embarassing. Another example is leap seconds. If we can’t get the clock right, we should probably just give up =)

So I fixed it with a quick hack, but in reality there are other situations which the clock display needs to be aware of – for example, if the user changes the time via the Date & Time panel. We could bodge around this in userspace, but it turned out that relatively recently the Linux kernel had gained exactly the API we need. I spent some downtime after the summit working on it, the resulting bug for the curious lives here. One of the nice things with this approach is that when we’re showing an HOUR:MINUTE clock, we no longer need to wake up the display process once a second just to see if the system clock got set from underneath us, which is useful for power saving.

It’s just exactly the kind of thing that GNOME means to me – the tradeoff for us not working on supporting a combinatiorial explosion of options is that we spend our time making the core good. Are there more important issues to fix than the clock? Yes, but it’s a visible one and I was offline while traveling so it was something I could easily do then. Oh and also to continue to prove Linus wrong that no one uses Linux-specific interfaces by using them in GNOME =)

The transition from “them” to “we”

August 6, 2011

While a lot of large Free Software projects do have some sort of formal “membership” structure (e.g. for GNOME there is the foundation), in reality being part of a project is more about your mindset. It’s easy for anyone to complain from the outside about something – and then the project is “they”.

I think many people, even ones who have been in the FOSS community for a long time, forget that it’s often not that hard to transition to “we”. As long as you can point to some sort of contribution (that could be any of code, marketing, documentation, art), even just periodically. By contributing even in a small way, you make a fundamental shift from “they” to “we”. From consuming to producing.

One common problem though is that of those who do propose changes, is to start out with a controversial change. This is not the right way to do it – while you may succeed, it’s going to be an uphill battle. Start by contributing non-controversial things – in the mindset of many people working on a FOSS project, this builds up karma or goodwill. And it makes it far more likely the people involved in the project will listen to your concerns.

It’s pretty basic stuff really – but easy to forget apparently.

You want to be up for this

August 2, 2011

Since every else is doing it, let me pimp my talk! Yeah, it’s at 9am. But it’ll be worth it!

Desktop Summit 2011

Tweaking the Linux scheduler for “background” jobs

July 18, 2011

TL;DR version: If you’re building software (or some other “background” job) on your laptop, on Linux, prefix the build command with chrt --idle 0 instead of nice, e.g.: $ chrt --idle 0 make -j 12

So a while ago, a “Linux wonder patch” was making the rounds in the news. This patch happens to basically only be useful if you’re running things like make -j 12 on your laptop, which is hardly newsworthy really, and on top of that, many people (me included) think it’s a gross hack. This was discussed several times, but probably one of the best articles on it is a LWN article.

Now, in that thread, I argued that the Linux scheduler should do something better with the traditional Unix nice scheduler value, and Linus basically said that while nice wasn’t very useful, it did have some semantics that we couldn’t break.

Now as it happens, this is is one of those half-baked Unix things for which Linux has a better replacement. Just like how fork is obsoleted by clone and mmap, it turns out that nice is obsoleted by sched_setscheduler.

In particular, SCHED_IDLE is quite nice, pun intended. If you’re building software (or running other “background” jobs) on your laptop (or a server even), you don’t need a gross patch in the kernel or to set up cgroups manually; just patch your build process to use chrt --idle 0 (or type it manually), and you will notice the difference. If you’re a GNOME developer and use jhbuild, this is actually done by default now.

Follow

Get every new post delivered to your Inbox.