FUDCon Toronto, on the bus

At FUDCon Toronto 2009, finally got a reliable connection at the hack room to give an update. On the bus from Westford to Toronto, I spent some time optimizing the GNOME Shell JavaScript engine Gjs, specifically how we do invocations from JavaScript into our C core.

As always, sysprof is an a great tool, and revealed we had some relatively low-hanging fruit. (Side note – in Fedora 12, sysprof no longer requires an external kernel module, which is awesome). Said fruit was partly in Gjs, partly in the GObject Introspection layer. We could clearly be doing more caching in the latter, but Gjs was also using the basically toy/demo invocation function g_function_info_invoke.

I’ve now got it so the invoker can take a bunch of JavaScript jsval pointers, and schlep them more directly into libffi argument types. The end result is currently a 25% speedup on my benchmark, which is fairly good as far as 5 hours in performance work goes. I’m now looking at some other bits, and I think I’ll be able to squeeze out another 7-10% at least with just another hour or two of work.

Now, in the the farther future, I could imagine teaching SpiderMonkey‘s JIT compiler about how to invoke directly from JS to C (and the reverse); the amount of generic work we still do to take say a single double across is fairly high, but the JIT could know about the platform ABI, and be able to trace right up until a native method, which would be a large speedup.

And for now, I’ll leave you with a picture of a tasty waffle from this morning:

From FUDCon Toronto 2009

Netbook in the coal mine

For a long time my primary workstation was a (now 3 year old) Dell Inspiron E1705, or “bricktop” as ajax liked to call it. However the hinge broke a while ago when I closed my car door while the bag was too close, and when the laptop started to fall apart I decided to get a new machine. Actually, I got two. The first thing I did was get a serious desktop computer, a Dell XPS 630. It’s quite a beast, especially since I got 8GB of RAM with it. I pretty typically have over 2GB of memory used purely for page cache, which is probably my entire working set of files. It’s hooked up to a nice 24″ Dell 1920×1200 monitor. This computer is named megatron.

Needless to say, the machine is fast and nice to use. However, it’s rather unrepresentative of the general personal computing space. So I wanted another machine that was mobile, and also closer to what “most” people use. So I picked up a MSI Wind U123, known as pocket. It has just 1GB of RAM, which is probably the realistic minimum we want to support going forward (ideally we’d be workable with 512MB, and we probably are if you’re just running one application or one or two websites, but…). The machine also has an Intel 945 video, which is also near the lowest end graphics card we’ll be supporting for GNOME Shell.

The difference between the two computers is rather extreme, but I use the netbook very often, and working on it has forced me to optimize some things in the shell. It serves a similar role for performance issues that canaries used to do in coal mines (hence the title of this post). Specifically, I’ve been working on search performance, with fairly good results. We had a few sillies in the old search system like creating lots and lots of ClutterActors we’d never display, not caching lowercased strings, etc. (the other goal by the way of my search work is to move us closer to the current search mockup, which should be cool).

But the biggest performance problem I’ve been running into so far is synchronous I/O. A lot of GNOME libraries like gnome-menus were designed for gnome-panel, which typically you don’t interact with often, and so it’s not a big deal if the process is blocked. However if the shell is blocked, because we’re acting as a compositor, we won’t repaint the desktop or process input, which is a serious user experience problem. An example of this was that we would synchronously stat (check for existence) of recent documents; this could easily take several hundred milliseconds if the data isn’t currently in the kernel cache.

I’ve been reworking our docs handling specifically to be smarter; we use async I/O now, and only stat ones we’re actually going to show in the UI. We could be smarter still, but this has helped a lot. GIO has really nice APIs for this kind of thing.

I mention this for two reasons; first, just because you might be interested in a status update on Shell work. The second is because I’m hoping to Real Soon Now land my extension system patch, so if you create an extension using it, for a good user experience, you’ll need to be very careful about I/O too =)

It’s coming

GNOME Summit went really well, and I had a great time. One thing that I was really heartened to see is that we’ve agreed on a GLib/dbus plan, and this will enable a number of cool things. First, dbus-glib is just not very good, and we’ve needed a better story. I hadn’t realized before just how much nicer GVariant can make doing DBus calls. But beyond that, having DBus underneath the GTK+ stack will better unlock things like GApplication which will in turn let us do nifty things like exporting GtkAction over the bus. If that’s still a bit abstract, look at say Stuart Langridge’s post on Linglish.

Pay no attention to the processes and X Windows behind the curtain…

A major change we’re trying in the GNOME 3 Shell is to be application-based instead of window-based. In this we’ll be in good company with newer releases of other operating systems, but it’s still a major change. What I want to explain in this blog entry is what that means from a user perspective. For the developer perspective, see this wiki page.

Let’s first look at one of the most venerable (and yet apparently still obligatory) applications, the Calculator. In GNOME 2, launching the calculator looks like this:

From GNOME Shell

Launching Calculator

When we click on that menu entry, the application is started (for more details about under the hood, see the linked wiki page above). The visual result is this:

From GNOME Shell

Calculator and the window list entry

There’s a window for the application, and a task list entry. Now in GNOME 2, if we go to the menu and choose Calculator again, we get this:

From GNOME Shell

Two calculators

Technical people will know that under the covers, there are two gcalctool processes, each of which is creating one window. What this example emphasizes is that in GNOME 2, the bottom panel has a list of windows (or tasks), not applications.

Moving on to GNOME Shell, the “window list” and “menus” part are merged into the overview. Let’s take a look at the application well when Calculator is not launched:

From GNOME Shell

GNOME Shell overview application well

Here I’ve added Calculator to my favorites, so it always shows up. It’s not running yet. When I click on it, the active application at the top immediately changes to show that (GNOME techies: this replaces startup notification), and then the widow appears:

From GNOME Shell

Running Calculator in GNOME Shell

So now that Calculator is running, let’s go back to the overview and see what changed there.

From GNOME Shell

Application well, with Calculator running

You can see that the Calculator gained a glowing status indication, like the other applications I had running already. When I click on that icon again, I am switched to the running calculator window:

From GNOME Shell

Looks exactly the same!

In other words, it looks exactly the same, it just shows you the window again. (Under the hood, the program is not re-executed, there won’t be multiple gcalctool processes, etc.)

Ok you say, but Calculator is a pretty boring application and you don’t use it anyways. How am I making your life better? Well, there are two major things.

One of them is that many programs fail in some obvious and other times less than obvious ways if you click the menu entry twice in GNOME 2 (technically, by default this will start two processes). They’ll overwrite each other’s data, etc. For very few instances does it make sense to have multiple if the app is not explicitly designed for it, and this will avoid you accidentally launching two. Personally I get annoyed when I accidentally launch two xchat-gnome instances and appear on IRC twice.

The second improvement here will come when we get a bit better application integration; the mockup we’d like to implement for say Firefox and multiple windows looks like this:

From GNOME Shell

This needs application-specific work though.

As a brief unrelated aside, I think recent Chromium builds with the tabs-in-window borders (technically, client side decorations) looks cool fullscreen in GNOME Shell with the application menu:

From GNOME Shell

Chromium in GNOME Shell

Next post I’ll talk about how GNOME Shell will save you time and get you back into your applications faster.

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 United States License.

Just when you think you have a good idea…

…it turns out someone on the Interwebs already did it. Curse you interwebs for how easily you crush one’s brief pretensions to originality. Anyways, in GNOME Shell we finally got bug 589652 in which means we’re ready for translations! We don’t have a lot of strings, but it’d be good to get the ones we have translated for 2.28. Ready, set, go! I’ve given French a leg up with a translation for Activites as a test case.

Along the way it took a bit of gjs hacking to make it easier to create native modules; if you found yourself embedding Spidermonkey but yearning for PyArg_ParseTuple, gjs_parse_args is your friend.

Speaking of programmer stuff, the built-in console mentioned in my previous post gained a parent-list inspector and property inspector which bring it very slightly closer to Firebug-like real, ultimate power. True story, I originally called it lookingGlass.js completely forgetting about Sun’s project ironically also about 3D-based desktops. Anyways, since that doesn’t seem to be actively developed anymore, and it’s a really great name, I think I’ll just…if you don’t mind….(yoink!) claim that name.

There have been a lot of other user-visible changes recently, in visual terms I really like the result of both Sander’s excellent work in bug 584609 in combination with the progress recently on moving visually to the mockup Jon referenced here. Another promising change is our start on the application menu area.

Which gets me to the subject of the next blog post; one the major changes in GNOME 3 from both a user experience point of view and application developer view is that we’re application based, not task based. Work continues in this area – in the next post I’ll talk about how this affects the experience.

It’s hot

It’s hot and humid in Boston, and I’ve been finding it a bit hard to sleep sometimes. I keep telling myself I’m going to get an air conditioner, but, well, laziness and money. Anyways tonight around 3am I just gave up, and from then till now-ish I was working on this cool little hack for Gnome Shell.

So here’s the problem. I have almost no visual skills personally, and in a freeform environment like Clutter or HTML I have a really tough time making things I code look good without basically being handed the pixels directly by a visual designer. But one thing I have noticed though is we have some people in the GNOME community with awesome visual skills. So I wanted to lower the barrier to entry a bit for them.

My inspiration was Firebug (which again if you have to do HTML and haven’t tried, do so immediately); I wanted to make something where you could interactively edit and debug the visual layout. So the result of the hacking landed in bug 590515. But since you’re unlikely to do a git pull and git bz apply, I used Owen’s cool builtin facility for making .ogg recordings of the shell.

Note however that at least parts of the shell aren’t redrawing correctly in the beginning, and I gave up debugging that for now. The top chrome will appear in the middle of the video.

Ok, you don’t have the video tag, try the direct link.

Being able to hack something like this up in a short span of time is really neat I think, and definitely is one of the high points of having one’s app scripted with JavaScript. You can go in and play around, even monkey patch things.

To actually make this useful for artists there’s a lot more to do; a Firebug style properties display/layout debugger probably wouldn’t be too hard, and would help you get past the problem of even knowing what properties there are. But besides this tool, in the big picture we really need to switch to CSS, but that’s coming.

One question you may have is why when I moused over the window area, we didn’t see different boxes for the windows? Well, that comes down to the fact that there’s a transparent input-only box over the windows. A nice future enhancement for this tool would be to be able to skip layers; then you could type in a bit of code and animate/color etc. your windows.

Another thing that would be neat from a code perspective to make this more useful for hackers is to allow JavaScript subclassing GObject; the problem is most of the shell there’s no good way to get the JavaScript framework from a given actor. If only I had a time machine to go back to when Brendan was typing in code for prototypes, and hand him an extends keyword. But, speaking of dreaming…it’s 8am now.

There’s more going on in the shell land, among other things we’re taskbar-less now. Yay for less visual chrome, though we have to work through the details. More on this later.

git-un-diff-whitespace

So among the major burning problems we have in Free Software, one of the biggest is trailing whitespace in source files. If you too have a project where people really care about this, you may be interested in my quickly hacked up git-un-diff-whitespace script. It removes trailing whitespace *only* in areas modified by “git diff” (i.e. it won’t pollute your diffs with lots of spurious whitespace changes). I briefly tried to find something like this elsewhere on the interwebs but failed. Oh, and if someone fixes it to handle new files I’d be happy.

Full experience

One of the things I wanted to talk about more was what I think does matter about the desktop, which is the total experience and how it relates to branding. People with extensive computer experience have a significantly better understanding of the individual components in a system (though they can and often are wrong of course about details), but the problem is that this leads to a different experience than someone without that understanding.

A specific example I’m thinking of here is the Java updater on Windows. I overheard one Java developer expressing surprise that someone they knew without much computer experience thought Java was a pop up. Now you may laugh, but really, they’re not wrong. The pop up is their total experience. People with the computer knowledge know they need it to play some online game, or they don’t. They know it’s from a different company, and added on by the hardware vendor, or that the pop up is a tiny piece of a much larger system. It ends up being a very different brand.

Speaking of hardware vendors and brands, this brings me to Windows itself. I’ve kept around a Windows install on my PC for various reasons, among them that I like to keep up to date on what’s happening in Windows land and be able to carry on an intelligent discussion about operating systems in general (not that you can tell from some coffee-fueled rambling blog posts), and also because I’ll probably have to work on porting GObject Introspection to Windows (pure pain) at some point. So here’s the problem with the Windows brand, and I think Microsoft is catching on to this – you’re carpet bombed with it in your interaction with the system, and it’s associated with very good things, very bad things, and everything between. Some things that are Microsoft’s fault, and some things that aren’t.

I was browsing the menu one day and came upon a Windows Dancer menu item. Microsoft made this, associated it with the Windows brand, and shipped it. It boggles my mind. Did you know that 29% of Windows kernel crashes are caused by NVidia? Except here’s the problem – it just says “Windows crashed”. So the user is going to associate that crash with the Windows brand, even though it it’s really not Microsoft’s fault. By far my favorite though is the Dell Dock. Definitely a lesson there in letting hardware vendors do arbitrary things to one’s OS.

The point here is that the design and experience of Microsoft’s core system (or any operating system) in a vacuum are far less relevant than the total experience of someone who just gets a computer. And this I think is actually a strength of the free software operating system, for a few reasons. Following the “rough consensus and working code” approach inherently avoids the worst things. If someone proposed a Message from your mouse vendor pop up in GNOME, we’d just laugh. Second, because we’re fundamentally cooperative, there’s less tension about replacing higher level things that people try to add on top. If memory serves correctly, for years Windows didn’t ship with an unzip utility in the core OS, presumably because of concerns about undercutting Winzip. We’ll never have that problem.

A bit more on this later, and how it relates to a feature I’d like to work on for the shell.

Uncanny valley? I don’t think so.

So Henri Bergius discussed John Gruber‘s claim that the free desktops are in an uncanny valley. Let’s leave aside the obvious bits that 1) desktops are absolutely nothing like robots trying to be people, and 2) clearly the fundamental basis of the uncanny valley is the “Pathogen avoidance/mate selection”, since in the tenth of a second it takes for your reaction to form, you obviously don’t have any time to ponder “Western constructions of human identity”. Given that people don’t get pathogens from their computer (unless you’re using a public internet terminal) nor attempt to mate with them (no comment), the uncanny valley isn’t very useful as an analogy.

That said, it’s amazing to me that John managed to go on for so long about operating systems in general (not just desktops) without mentioning the most important point about them. Actually, really the point, which dwarfs all others. That is:

To run the applications you want

Now, don’t get me wrong. There’s a lot of details about desktop operating systems that are important; more on this in a minute. Obviously, a web browser is essential, but that’s a commodity. Now, I don’t want to get into a debate about exactly what percentage of people can be fully served by solely a web browser, but let me just note that for a lot of people, especially ones with overseas families, Skype is still a killer app. And you can’t write Skype in the default web security model (which is the relevant bit here, not a missing-the-point debate about AJAX versus platform widgets).

Continuing on the subject of the uncanny valley, clarkbw told me a while ago there’s been HCI research to the effect that people are not generally too troubled by visual differences. Consider for example the evolution of the Start menu; it looks really different between each of Windows 2000, XP, and Vista, but generally people don’t have a problem with that. However what they do have trouble with is something that looks similar, but does something completely different. A good example here is how in the default GNOME panel configuration, we ship the “make all your windows go away” button in the bottom left. Kind of a bad idea, one that will go away with the shell design.

But in general, I don’t think the problem has anything to do with an “uncanny valley”. If we’re talking about the consumer space, basically people just use the operating system that came with their computer, until it’s time to replace the computer. Overcoming this is I think largely a business problem, not a desktop interface problem.

So on the subject of looking like Windows in the context of GNOME, my take is that it is important to look good and be functional, and having a distinct visual identity is important – but – keep in mind:

Given that data, considering that our target market is, like Windows, general purpose computing, it makes a significant amount of sense to consider issues that occur during transition and what the expected target audience needs.

What did I mean by “general purpose computing”? Well, it’s a big world. Think of scientists and engineers using Scilab, or all of those fixed-function registers that currently run Windows. OLPC-type projects. Content creators who need to edit video. System administrators and developers (count me as a skeptic of editor as a service).

What about targeted “netbook” operating systems? There is definitely a large number of people who only need a lightweight computer. I think the core desktop should adapt automatically to those kinds of conditions; for example, you should be able to drop bits of GNOME you’re not using (say Evolution), and the panel should adjust Ubuntu Netbook Remix style. There’s definitely a place for Moblin type specialized interfaces, and as long as they’re generally compatible with the general purpose OS. Concretely, if say Skype has to rewrite or do significant work to adapt their app, there’s a problem. But regardless, if Moblin achieves its goal of being shipped by hardware vendors on netbook hardware regardless, it’ll be a huge step forward.

Now, different people have different visions of where the GNOME Shell should go, and that’s healthy and fine. To an extent these different visions can be accomodated by different means. Ideally of course, we fix the problem in the default design. But failing that, it might work as a Firefox-style extension mechanism that we want to add, a separate project that builds upon shared infrastructure, or automatic interface adjustments like the netbook screen size one I mentioned above.

But you can count me as believing that it makes sense to be sure that the apps you use today continue to work, while achieving the goals of making GNOME look nicer and more interactive than two grey bars, and fixing the obvious problems like poor access to your recent files, no application search, etc. Some of this we’ve done already, but there’s a lot more to do. If you’re interested and able, it’s pretty easy to dive in to the project!