Drupalcon - part 2... testing


There were a lot of very interesting sessions at Drupalcon, 4 full days packed with them in fact. Some of my personal favorites were openid, form api 3, panels 2, and the Drupal association panel. One of the things that really stood out (and I also really enjoyed) was the configuration management discussions. A decent percentage of the talks covered or had some overlap with configuration management and testing. Automated testing is a huge issue for "Enterprise" Drupal right now. There's not really a "safe" way to upgrade a Drupal site at the moment. A major upgrade pretty much goes as follows:

  1. backup
  2. upgrade the code
  3. update the database
  4. cross your fingers
  5. tap your toes
  6. hope it works.
  7. repeat if necessary (*always* repeat if necessary ;-))

Once you've upgraded you can manually go through the pages and try to find any glitches in the site, and if you're lucky you'll get them. Maybe the users of the site will find and report the errors. If you're lucky you won't lose any data. There's currently no unit testing, regession testing or smoke testing (sure, there's some test code... but as far as I'm concerned the amount that exists is really only proof of concept that it can be done). There's roughly half a dozen companies that put the majority of paid development into Drupal and are also hiring up most of the good Drupal developers. These companies have a vested interest in getting automated testing working in Drupal.

At one of the more ad hoc sessions (which included people from most of the big Drupal comapnies and prominent community members) NowPublic promised a large chunk of one of their developers' time to work exclusively on writing unit tests. There was obviously a lot of support for this. This session was more or less a discussion of how to approach the problem and solve it and I think some good ideas and points came out of it.

Writing tests is a bitch. I can say this as I use to write *very* extensive tests for code I wrote (python has some very nice tools for unit testing that I miss in almost every other language I use). The test code can often be significantly larger than the code it's testing. It was kind of fun at first, but all the typing and copy-and-pasting quickly started aggrevating the carpel tunnel ;). Writing the tests is the easy part. Keeping the tests up to date is the tough part. Getting developers to keep improving the code at the same rate while forcing them to write and update the test code on a mostly volunteer project... I don't even know if that's possible.

There's a popular idea in the Drupal business community that if you develop something, give it back to the community and let the community maintain it. I think this works great for both the companies developing Drupal and the volunteers working on the project. Keeping their code a secret hurts both parties. Let the company pay for the initial development of the code, and if there's interest someone will pick it up. Unfortunately, I don't see this working for testing. Even if the initial set of tests is paid for, I really can't see the "community" picking up the maintenance of the tests. It's a bitch, and doesn't scratch enough peoples' itch.

Regardless, automated testing in Drupal will be a Good Thing. And what do I know, maybe volunteer developers will be totally into keeping the test code up to date. "All tests passed" messages make people feel warm and fuzzy inside :).

One last point... since I'm in a ranty mood. This is to the person who likes to preach "...every time you touch a contrib module ... run coder module ... write and update simpletest tests ... yada yada". Perhaps it's about time you put your money where your mouth is and start getting your own developers to do it. :-p

Drupalcon Barcelona trip wrap-up

1 comment

...because your tiny amount of liquids won't explode if they're in little plastic bags...

I'm just on my way back to Vancouver from Barcelona, sitting in the Barcelona Airport.
It's been a pretty intense week, to say the least. Let me just say, Drupalcon Barcelona, effing awesome. Met lots of awesome people, who I'll hopefully be able to keep in contact with a bit still.

We (Boris, Steven, Djun, and I) arrived Sunday night after about 20h of travelling the trip there was pretty uneventful... ran into walkah in the bathroom of the Amsterdam airport, which was nice, because I was starting to get tired of Steven ;).

Monday we did a brief walking tour of Barcelona, stopped in at the Market and grabbed a quick bite to eat. We met up with a dude from the Zope project for dinner later that day and had some great techie conversations. Ended up cleaning out the beer selection at the place we were eating too.

On Tuesday, Boris and I decided we needed to cook a BIG "spanish" dinner. We hooked up with the Now Public crew at the Market and made a quick dash to get all the food we needed for the night, with only minutes to spare before everything closed for siesta. Boris is an awesome chef. Let's see if I can remember even half of what we cooked... muscles, prawns, baby octupus, squid, guacamole, lamb, fish, sausage (numerous varieties), and I'm sure a ton of stuff that I'm missing. We went through about a litre of olive oil, and basically ended up drinking and eating for 12 straight hours and the Now Public apartment, which was a pretty sweet place... almost had a bit of a greek feel to it, actually. Afterwards we hit up La Ramblas for some more beers, where I /may/ have stolen a 1.5L glass... attempting to hide it by stuffing it under my shirt... because I'm classy like that.

The conference ran from Wednesday through Saturday. I gave a pretty brief presentation of my loadtest module Wednesday afternoon and it went better than I thought it would. Ican't remember what happened Wednesday night, so it must have been pretty low key.

Thursday we hit an Irish pub. Yeah, nuff said.

Friday, Djun and I decided to pick up some food to cook and just chill and have a romantic Dinner with the two of us. Then we went out and took photos from 11:30-2am around the town.

Last night can't be discussed.

Went to bed at 5am, got up at 7am to head for my flight. I decided it would be a bright idea to catch the metro as we never really cracked the code for cabs in this city :). Unfortunately I went the wrong way down the line. Though I managed to realize early enough and hopped on a train back downtown, at which point I decided to grab a cab :). I'm now in Amsterdam, running on 2h of sleep for the past 30h and still have 12h left of travelling. Nnnst! I'll post some pics when I get back. I promise :).