HCoder.org

Some time ago, Opera announced the Opera Web Standards Curriculum project. It’s a very interesting collection of articles that can be used as “curriculum” to learn about web development. It gets extra geeks points for using a Creative Commons license for the articles themselves. Even the W3C mentioned it :-) I just found some time to have a look at it, that’s why I’m posting now :-)

The other news is that finally the Opera QA blog is online, and has the first non-hello-world-article (written by yours truly), “Continuous Integration: Team Testing”. I’m very excited about this, because it’s the first time I’ll participate directly in a company blog, and because the IT world needs more (and better) QA, so hopefully we’ll be able to spread the word and make the world a better place :-D

I don’t really do any Python development myself, but at work I do support some automated testing infrastructure (in this particular case, I’m talking about a CruiseControl.rb installation), and some of the projects that use that infrastructure use Python. The setup is so that the tests are actually executed in the CC.rb server, so I have to have Python installed there, and it has to have some basic dependencies to be able to run the tests.

A couple of times something strange happened: suddenly, those tests would start failing with no apparent reason, and looking at the logs, it looked like some dependencies were not installed (error messages such as ImportError: No module named sqlalchemy). Of course that didn’t make any sense, because SQLAlchemy is needed for the tests and they were working like a charm for weeks. I was totally and completely confused by the error message, and I tried to install SQLAlchemy again. That solved the problem, luckily, so I decided to forget about it because it wasn’t my thing anyway.

But the problems appeared again. And again. And I got another error message that was really confusing, because it looked like Python was using some old version of some module (a version that wasn’t there anymore, because the code had been updated from SVN). So I just got tired of not knowing what was going on, and decided to investigate enough to find out the root of the problem. And I found something surprising.

What I found is that the famous python setup.py develop (that everyone told me to use) actually adds the “current directory” to the list of paths where Python searches for modules, so you can develop your module and use it from anywhere. I had heard some comment on that, but I didn’t quite get what it meant, and I don’t think the person that said it realised either.

The fun thing with setup.py develop is that when you have several branches of the same project in the same machine, and you use that to make the modules available… well, I guess that knowing which versions of which modules Python will use becomes an interesting question to say the least. I’m not saying that the way it works is necessarily wrong, but I do think it is dangerous, and people shouldn’t think of it as the “normal” way of developing modules in Python. It should be used with care.

After having realised that and thought about it a bit, I still don’t understand why those modules simply “dissappeared”, but it seems that there was some corruption of /usr/lib/python2.5/site-packages/easy_install.pth or similar (that file seems to be modified when you install packages with easy_install, and it had references to the directories I ran setup.py develop from, so that’s my main suspect for now). At least I know now that I could backup a working easy_install.pth file, and restore when we have problems again, but I’m far from happy with that “solution” ;-)

Also, I’m wondering what the hell should I do in the future to prevent more problems, because using setup.py develop sounds like a terrible idea to me. I tried to set PYTHONPATH instead, but apparently I failed. Any suggestions?

EDIT: I’m finally using PYTHONPATH. I have no idea what I tried last time, but using it was easy, quick and clean. I still have no idea why the hell Python sometimes forgets where some modules are, though.

So, today I was working normally, and suddenly I mispress something… and I can’t switch to other desktops anymore.

First thing I think: maybe some KDE global shortcut manager or whatever went nuts, and redefined my “Switch to Desktop” keys. So I go and check the preferences, and I find that everything is alright.

So I try to redefine the shortcuts again, and I notice that according to KDE, F1 produces XF86Launch0, and the rest of my F-keys just don’t do anything. I panic, think for a moment about changing the shortcuts to Ctrl-1, Ctrl-2, etc., discard the idea because sooner or later I’ll need the F-keys anyway… and decide to reboot. But still I can’t use my F-keys.

Totally desperate, I ask on IRC and someone says “F Lock”. And I go “WTF is that?” but look at my keyboard, and see some key that is indeed labelled “F Lock”. I press it and everything goes back to normal.

Then, the person goes on to explain that Microsoft has very retarded keyboards (in particular, I was using a Microsoft Natural Keyboard) that “feature” a key called “F Lock”, that redefines the “F keys” (F1, F2, ...) to be some “useful” idiotic retarded shortcuts for Office applications or who knows what. I was also told that apparently some of those keyboards, when they boot, they are by default in “retarded mode” (mine seemed to somehow remember the setting in my last reboot, because it has never done that).

I just had to blog about this. Amazing.

As part of my QA work on several projects, months ago I was looking for a continuous integration server. I looked at several, but most of them seemed really scary judging from the documentation. I finally went for CruiseControl.rb, and I have been really happy with it all this time. It’s a really nice, very simple continuous integration server written in Rails. I had it up and running before I even understood how to install the others I looked.

Even though is a really cool piece of software, I was missing some better test result reporting. It was actually there, but only for Rails projects, and unfortunately we don’t have any Rails (or Ruby, for that matter) projects at work. So, I just had a look at the sources to see if I could hook my own reporting there, and the code turned out to be impressively easy to understand (especially taking into account that it’s a rather non-standard Rails application, as it has builders running as daemons, it doesn’t really use a database, etc).

The result is a patch for CC.rb, already submitted to their BTS, that adds a plugin-based result reporting, that can be extended to understand any kind of testsuite. It’s basically a parser that collects all the test passes and test failures from the testsuite output log.

Also, the other day I had another need, which was even easier to make because it could be implemented as a simple CC.rb notification plugin. It depends on the above patch, and it collects all the bugs in the current build, searches in the history of the project, finds out who made the commits that produced the regressions, and bugs all those people by e-mail, pointing out which failures were supposedly made by them, and which build they started failing (so it’s easier to locate the offending code).

It’s not perfect, and it cannot be, but it’s a nice addition to continuous integration. This notification plugin is not public yet, but it might be in the future (especially if they accept my patch as part of upstream), so stay tuned if you’re interested.

Today I have been one year working in Oslo! Yay! So far the experience has been quite good, so I’m staying here for some more time still.

I’ve also slowly becoming kind of active again in Debian (especially helping dhelp), although I admit not being very active in any other software project (Haberdasher feels kind of abandoned, because I don’t have any urge for new features). Hopefully that will change…

Lately I have been thinking a lot, not to say “obsessed”, with the big picture. I can’t but wonder if that is a general IT industry problem, that big picture. I mean missing it.

My current theory is that computer work is just too hard (or tools not advanced enough?), and there is too much pressure and too hard time constraints to allow people to step back and think about the big picture from time to time, to make sure everything makes sense.

And perhaps that’s why you hire and have QA people, perhaps that is the real purpose of QA. At least I feel that now. I mean, what’s the use of something that has a high “technical quality”, if it just doesn’t make sense? That is actually a big part of the quality, “making sense”. Because of that I’m starting to feel like my job is being a developer that does important things that “nobody has time to do”, because they’re too busy fighting with details. Not in the sense of a “manager”, but in the sense of some “responsible” developer. It’s a really strange job position I think :-)

It’s really hard to measure the impact of good QA in a software project, but I’m sure that is high, probably higher than people use to think. For me, thinking about software projects without QA is a bit like thinking about programming without a Version Control System: I wonder how I had done it in the past, and feel really unconfident without it. How many projects have failed (both from the resources and goals point of view, and from the business point of view) for not having good QA? How many projects have been delayed, or even cancelled, because they lacked someone caring about the Big Picture?

EDIT (2008-5-18): I have disabled comments in this post due to insane amount of spam. If you want to comment, please comment in some other entry :-/