DrupalCon Boston 2008 - Some Sessions

no comments

Just for fun... I thought I'd put up a list of a few of the DrupalCon sessions I'm excited about. I'm going to try to blog daily and maybe post photos daily too. I was also thinking... since it seems to be getting popular to take pictures of people taking pictures, I'm gonna try and take as many pictures of people taking pictures of people taking pictures as possible.

Monday 9:30 - Drupal Multimedia (Presenters: Aaron Winborn, James Walker, Darrel O'Pry, Nathan Haug) - "This session will guide you through the steps necessary to build image, video, and audio into your sites."

Monday 1:30 - Mapping business requirements to Drupal modules: a gap-fit process (Presenter: Boris Mann) - "Join me to talk through a process of building repeatable processes in selecting modules and implementing a gap-fit analysis of which modules your new Drupal project is going to need."

Monday 3:30 - OpenID and Identity in Drupal: the future of user.module (Presenter: James Walker) - "This session will not be why OpenID, but rather what’s next with OpenID."

Monday afternoon NowPublic will be at the job fair, so drop by and say hi! :)

Tuesday 9:30 - The Future of Fields (Presenters: Barry Jaspan (bjaspan), Nedjo Rogers (nedjo), Karen Stevenson (karenS), Larry Garfield)

Wednesday 9:00 - SimpleTest: Because clicking on forms is for suckers (Presenters: Angela Byron (webchick), Rok Zlender, Karoly Negyesi (chx), Jimmy Berry (boombatower))

There's plenty more... but I think this is a good start.

And now I'm off to the airport.


Missed one!

Wednesday 9:45 - Practical jQuery - how to act like you're JavaScript-smart (Presenter: Dimitri Gaskin) - Dimitri is Drupal's 12 year old developer. I'm really looking forward to watching this presentation.

Drupalcon - part 2... testing


There were a lot of very interesting sessions at Drupalcon, 4 full days packed with them in fact. Some of my personal favorites were openid, form api 3, panels 2, and the Drupal association panel. One of the things that really stood out (and I also really enjoyed) was the configuration management discussions. A decent percentage of the talks covered or had some overlap with configuration management and testing. Automated testing is a huge issue for "Enterprise" Drupal right now. There's not really a "safe" way to upgrade a Drupal site at the moment. A major upgrade pretty much goes as follows:

  1. backup
  2. upgrade the code
  3. update the database
  4. cross your fingers
  5. tap your toes
  6. hope it works.
  7. repeat if necessary (*always* repeat if necessary ;-))

Once you've upgraded you can manually go through the pages and try to find any glitches in the site, and if you're lucky you'll get them. Maybe the users of the site will find and report the errors. If you're lucky you won't lose any data. There's currently no unit testing, regession testing or smoke testing (sure, there's some test code... but as far as I'm concerned the amount that exists is really only proof of concept that it can be done). There's roughly half a dozen companies that put the majority of paid development into Drupal and are also hiring up most of the good Drupal developers. These companies have a vested interest in getting automated testing working in Drupal.

At one of the more ad hoc sessions (which included people from most of the big Drupal comapnies and prominent community members) NowPublic promised a large chunk of one of their developers' time to work exclusively on writing unit tests. There was obviously a lot of support for this. This session was more or less a discussion of how to approach the problem and solve it and I think some good ideas and points came out of it.

Writing tests is a bitch. I can say this as I use to write *very* extensive tests for code I wrote (python has some very nice tools for unit testing that I miss in almost every other language I use). The test code can often be significantly larger than the code it's testing. It was kind of fun at first, but all the typing and copy-and-pasting quickly started aggrevating the carpel tunnel ;). Writing the tests is the easy part. Keeping the tests up to date is the tough part. Getting developers to keep improving the code at the same rate while forcing them to write and update the test code on a mostly volunteer project... I don't even know if that's possible.

There's a popular idea in the Drupal business community that if you develop something, give it back to the community and let the community maintain it. I think this works great for both the companies developing Drupal and the volunteers working on the project. Keeping their code a secret hurts both parties. Let the company pay for the initial development of the code, and if there's interest someone will pick it up. Unfortunately, I don't see this working for testing. Even if the initial set of tests is paid for, I really can't see the "community" picking up the maintenance of the tests. It's a bitch, and doesn't scratch enough peoples' itch.

Regardless, automated testing in Drupal will be a Good Thing. And what do I know, maybe volunteer developers will be totally into keeping the test code up to date. "All tests passed" messages make people feel warm and fuzzy inside :).

One last point... since I'm in a ranty mood. This is to the person who likes to preach "...every time you touch a contrib module ... run coder module ... write and update simpletest tests ... yada yada". Perhaps it's about time you put your money where your mouth is and start getting your own developers to do it. :-p

Drupalcon Barcelona trip wrap-up

1 comment

...because your tiny amount of liquids won't explode if they're in little plastic bags...

I'm just on my way back to Vancouver from Barcelona, sitting in the Barcelona Airport.
It's been a pretty intense week, to say the least. Let me just say, Drupalcon Barcelona, effing awesome. Met lots of awesome people, who I'll hopefully be able to keep in contact with a bit still.

We (Boris, Steven, Djun, and I) arrived Sunday night after about 20h of travelling the trip there was pretty uneventful... ran into walkah in the bathroom of the Amsterdam airport, which was nice, because I was starting to get tired of Steven ;).

Monday we did a brief walking tour of Barcelona, stopped in at the Market and grabbed a quick bite to eat. We met up with a dude from the Zope project for dinner later that day and had some great techie conversations. Ended up cleaning out the beer selection at the place we were eating too.

On Tuesday, Boris and I decided we needed to cook a BIG "spanish" dinner. We hooked up with the Now Public crew at the Market and made a quick dash to get all the food we needed for the night, with only minutes to spare before everything closed for siesta. Boris is an awesome chef. Let's see if I can remember even half of what we cooked... muscles, prawns, baby octupus, squid, guacamole, lamb, fish, sausage (numerous varieties), and I'm sure a ton of stuff that I'm missing. We went through about a litre of olive oil, and basically ended up drinking and eating for 12 straight hours and the Now Public apartment, which was a pretty sweet place... almost had a bit of a greek feel to it, actually. Afterwards we hit up La Ramblas for some more beers, where I /may/ have stolen a 1.5L glass... attempting to hide it by stuffing it under my shirt... because I'm classy like that.

The conference ran from Wednesday through Saturday. I gave a pretty brief presentation of my loadtest module Wednesday afternoon and it went better than I thought it would. Ican't remember what happened Wednesday night, so it must have been pretty low key.

Thursday we hit an Irish pub. Yeah, nuff said.

Friday, Djun and I decided to pick up some food to cook and just chill and have a romantic Dinner with the two of us. Then we went out and took photos from 11:30-2am around the town.

Last night can't be discussed.

Went to bed at 5am, got up at 7am to head for my flight. I decided it would be a bright idea to catch the metro as we never really cracked the code for cabs in this city :). Unfortunately I went the wrong way down the line. Though I managed to realize early enough and hopped on a train back downtown, at which point I decided to grab a cab :). I'm now in Amsterdam, running on 2h of sleep for the past 30h and still have 12h left of travelling. Nnnst! I'll post some pics when I get back. I promise :).

GSoC - Loadtest module


As some of you know, I did Google's Summer of Code this summer. The result of that ended up being the loadtest module. This entry is basically just a description of how to use the module... so if you're not interested, please move along :-P.

The first part of this module lets you define and configure "states" for your site that you can run tests against.

You can use these states in two different ways:

  1. When running a loadtest, using the loadtest module, you can choose the state you wish to test.
  2. If you want to run more comprehensive tests using an external application such as ab, you can switch between states remotely allowing you to more easily script your benchmark tests.

The following screenshot is of the main testing interface. Each part of this form is described below.

  • The label is used for you to identify your test run at a later date. Each label will automatically have the timestamp for the test run appended to it.
  • A test suite is the "type" of test to run. By default there are two: "Single run", which just runs the selected tests once and "Test individual modules" which autotmatically disables all of the modules and then enables each module one at a time running tests at each step. Test suites can be added by module programmers as well so they can build tests that are more suited to their specific modules.
  • You can select a state from any of your pre-configured states. Leaving this empty will simply use the sites current settings.
  • There are currently two default tests that are included with the module:
    1. RequestRandomNodes - which does just that, page requests on random nodes. This is mostly just for testing, not overly useful for running load tests as you would probably want to compare test runs against identical page requests.
    2. RequestPages - This test requests a set of select pages a certain number of times. By default the pages "frontpage" and "tracker" are requested 10 times.

Unlike simpletest tests, loadtest tests can have individual configuration options that can be set through Drupal. The RequestPages test lets a user define which pages they want to request and how many times each page should be requested.

Depending on the options you choose running a test could take several minutes. Since site settings may be modified on the fly during a test run, it is important that you don't use this module on a production site, and don't interrupt it while it's running.

Finally, once the test has been run, you can view your statistics and compare multiple tests.

The code can be downloaded from the tarball on the module page, but I'd recommend taking it directly from CVS:

cvs -z6 -d:pserver:anonymous:anonymous@cvs.drupal.org:/cvs/drupal-contrib checkout -d loadtest contributions/modules/loadtest

Google Summer of Code

no comments

As some of you already know, I'll be doing the Google Summer of Code this summer. For those of you who just asked "WTF is Google Summer of Code?", it's basically (to sum up in one sentence) Google sponsored and organized open source development. Basically, Google is paying me (and ~900 other students) to work on an open source project. If you're still curious, you can visit here or here, for more info. The basic premise is to raise awareness for open source and get more people (specifically students) involved in open source development.

GSoC officially begins on Monday and I'll be working on the Drupal project, my project proposal consists of developing a framework for allowing Drupal admins to load test their sites and more easily determine bottlenecks and such. I have a more detailed description on the Drupal SoC group, if you want to learn more. I figured it made good sense to do a Drupal project... considering about 90% of my contract work at the moment is Drupal related. What's Drupal? I think you already asked that question, Carla. It's a content management system (CMS). What's a CMS? Yeah, I don't know, really. But basically Drupal helps you build ultra sweet community based websites in like no time at all, and that's all that really matters.

Anyway, I'll probably be posting other random tech stuff here about my project... so fell free to ignore my posts tagged with Drupal and GSoC if you don't wanna hear about benchmarking, bottlenecks, scalability, load balancing, unit testing, or other such terms, my feelings will only be mildly hurt.