Kevin's Weblog

I’m a huge fan of Norman’s previous book The Design of Everyday Things and I highly recommend it to anyone who builds things, physical or virtual. This book takes on some of the aspects of design beyond the functional or behavior, and discusses the other aspects of why people are attracted to certain objects. It’s quite interesting for the first 2/3s of the book or so, but then takes a strange hard left into a discussion of robots and why robots will need to have emotions too. Very strange. But, I would still recommend reading the earlier parts of the book.

It’s been a year and a half since my last political post. I’m a little surprised that it’s been quite that long, but not entirely shocked. I spent nearly all of this election cycle somewhat withdrawn from the whole thing; feeling disenfranchised from years of being told that my dissent was unpatriotic, and being disgusted by the amount of ugly politics-as-usual that was going on. I wasn’t spellbound by Obama like so many of my friends and neighbors, but slowly gained respect for his thoughfulness, general positivity, sense of context, and willingness to confront real, serious issues that most politicians shirk from.

So, it wasn’t until yesterday that I really started to feel a nervous excitement that maybe we really would see a change this year. By 5PM, I was compulsively reloading web pages, and when the counts hit 200 electoral votes with only the east coast reporting I got that “OMG it’s really happening” feeling. Around 7, I headed down to the Green Dragon with a friend to watch on the TV there. I think I started vibrating at some point shortly before they called it.

The whole room was silent for Obama’s speech. I’m pretty sure most of us cried. I think over the last couple years I’d resigned myself to only being able to have an effect on my local community. His speech inspired me to again turn my efforts to a larger sphere. My community will still be my primary, day-to-day focus; but I’ll definitely be more engaged at the national level as well. What Obama said is true: this election doesn’t change anything by itself, it just gives us the opportunity to change. This is a difficult time for our country, and it’s going to take hard work by many people to solve these problems. But there’s a huge population of people like me that had all but given up and are now re-energized, and I think we can really make some improvements.

Closing thoughts... To Obama: live your rhetoric and remember that you now represent all Americans, not just your supporters. To McCain: you won back some of my respect with your concession speech; please let this be a return to your old, pre-election self. To supporters of both: be as gracious in victory and defeat as your candidates were last night. Talk and listen to people you don’t agree with, and realize that your goals are largely the same, even when your methods differ. Find the places where you can work together.

Stay hopeful.

It shouldn’t need to be pointed out, but the root of ‘useful’ is ‘use’. If you don’t actually use an object, regardless of whether you abstractly feel it to be useful, perhaps it isn’t. There is a particular risk with objects that are both beautiful and useful that we fail to use them, whether because we feel they are too expensive or unique and irreplaceable to risk damage, or because we put them on display as works of art but then fail to reach for them when the need arises, or whatever excuse there may be.

Recently there are a handful of objects I’ve been treating this way: my pocket knives and my fountain pen. Yesterday, I actually did take the intentional action of putting a knife in my pocket, along with my wallet and keys, before heading out the door. Today I did some paperwork with my fountain pen. Hopefully this marks a return to these habits.

It’s now official that OSCON is leaving Portland and moving to San Jose next year. There’s a lot of unhappiness around here about it and a definite feeling that the stated reason (rising travel costs) isn’t the whole story.

Personally, I don’t think it’s clear that making the conference local for more people is a good thing. I’ve noted two years running that I felt like it was a very different experience to be living in Portland and attending the conference — you’re much less immersed in the conference. Increasing the number of people who are there for the sessions but don’t stick around for the evening social activities won’t necessarily improve the conference.

ZDNet thinks “Portland just doesn’t have the facilities to house a fast-growing conference covering a global audience”, but I’d like to hear more justification for that claim. OSCON doesn’t come close to filling the whole Oregon Convention Center.

Personally, this feels like the usual Silicon Valley bubble mentality, combined with a touch of corporate greed. I also have a fear, possibly unjustified, that this may also trigger a slip backwards on the “women in tech” front. My impression is that Portland has one of the best environments for trying to increase the participation of women in technology and open source, and I also have the impression that the Valley has one of the worst.

I’m trying really hard to look at this beyond the personal impact on me (because it certainly did just get less likely that I’ll be able to attend next year), and I’m hearing a lot of the same unhappiness from people outside Portland as well. I guess what I’d like is a bit more openness from O’Reilly about this. Because, really, openness is what OSCON is all about, right?

Have nothing in your house that you do not know to be useful, or believe to be beautiful.
William Morris

I’ve been reading Emotional Design, and the inclusion of this quote inspired me to try to take an inventory of my house using this criterion. Of course, ideally I’d have only objects that are both beautiful and useful, but initially I’ll just make it my goal to be rid of things that fail both and make some notes for future improvements.

This morning I tackled the easy rooms: the living room and dining room. No surprises there. Most things are both beautiful and useful, except for my futon and folding chairs which are only useful, and the plants and artwork which are only beautiful.

Now I wonder if I should make the kitchen or the office the next room. Either way, it’s going to be a much bigger task.

I’ve been tinkering a little more with the Beust Challenge, following up on my previous post. There are a couple significant new developments to report.

First, Hynek (Pichi) Vychodil wrote a faster version using a permutation generator.

Second, I also ported the first “CrazyBob” solution using a bit mask to Erlang.

Third, I discovered that my overly literal ports were actually slowing things down. The CrazyBob code uses an unspecified Listener class that receives the numbers in the series, and presumably computes the actual results from there. (Aside, I cannot actually benchmark against the Java code because I haven’t found what this implementation is.) I fudged this in my original ports by simply spawning a process, which then discarded all the messages it received. After I noticed that the code was pegging both of my CPUs, though, I realized that message passing might actually be the bottleneck in my code. Turns out this was the case, and removing the listener process and just computing the results in the main process actually sped things up substantially.

Finally, I got access to a machine with HiPE enabled.

So... here’s the results. First on my MacBook Pro, without HiPE:

The bitmask solution starts out the fastest, but loses out in the end to the more clever solutions. pichi edges out the crazybob solution by about a third.

Now on Linux 2.6 with HiPE:

And our new winner is... bitmask! HiPE barely helps the original brute-force solutions at all, while crazybob and pichi gain about a factor of two. bitmask, on the other hand, picks up an order of magnitude, and is now only a factor of 3 slower than the posted results for the Java crazybob solution (with unknown differences in hardware).

Conclusion: Erlang doesn’t suck!

During my time off between jobs, I decided to spend some time doing fun hacking. For the last couple days, that’s meant doing a little benchmarking with Erlang. Recently, there was a challenge posted by Cedric Beust. In the wrap-up he lamented that the one Erlang solution posted was “a bit frightening” and neither concise nor fast. Part of this conclusion, I think, was due to the poor formatting of blog comments, and part because the comment submitter included a whole lot of test code and also reimplemented some library functions for no obvious reason. I modified it to use the standard libraries where appropriate, and made a couple other minor optimizations and got about a 3x speed-up. It’s still much, much slower than the top submission, which is because the top submission is clever, and this is still a brute-force implementation.

I then set out to port the “crazybob” solution to Erlang to see how much better we can do. The Java version uses a hand-rolled linked list to keep track of the available digits. I decided to use a gb_set for my version. For small N it’s a bit slower than the brute-force version, but the scaling is much better. For N=10^10, it runs in about 26s on my laptop. This is still something like 50-100 times slower than the Java version, but I’m also not using HiPE, which might close the gap a bit.

I suspect this is not the fastest possible Erlang implementation, because the heavy use of gb_sets is undoubtedly causing a lot of memory churn. It’s been pointed out that this is basically a problem of generating permutations, which ought to be possible to do with near constant memory usage.

Here are the results of my benchmarking, and the code. For each data point, I ran the code 3 times and took the middle value. For the brute force versions the timings were very consistent, while the “fast” version had 10-20% variation between the run, presumably due to vagaries of memory allocation. I did not actually run the brute force algorithms for 10^10; the values are generated by extrapolating linearly.

Update: Hynek (Pichi) Vychodil wrote a faster version, bringing us within a factor of 10 or so of Java (and still not using HiPE). Oddly, there’s a bigger speedup for this version on my machine (26s vs 5s) than on Hynek’s (18s vs 8s).

See important updates in the following post

After nearly 6 1/2 years, on Monday I gave notice at Yahoo. After next week, I’ll be moving on to new things (to be described in a later post). The short version is that I was not finding happiness or the career opportunities I wanted at Yahoo any more. The long version... well, you’ll have to come buy me a beer or two if you want that.

I couldn’t take detailed notes this year on account of a sprained wrist, but here’s the highlights of my week.

I spent most of Monday and Tuesday working on my talk. Tuesday afternoon, I bounced back and forth between People for Geeks and Francesco’s Erlang Tutorial. Nothing new for me in the Erlang talk, but I wanted to get an idea of what he was covering. There was a lot packed into 3 hours, but people I talked to said positive things — that they learned a lot about the language. Over in “People for Geeks”, Andy Lester had quite a bit to say about working with your manager which I considered questionable. For example, “don’t go to your manager with problems” (I thought a manager’s job was to clear obstacles) and “your job is to make your manager look good” (I wonder what our shareholders and upper management have to say about that one). On the other hand, pointing out that your manager is probably not actually incompetent or malicious is valid, although it’s a little sad that we have to point that out.

Tuesday evening was the opening extravaganza. I was kinda bored by Mark Shuttleworth’s talk, but r0ml and Damian Conway were excellent as usual. Although, knowing enough physics to be bothered by the inaccuracies in Conway’s talk was a little annoying.

Wednesday I skipped the keynotes in favor of a proper breakfast and another practice run before giving my talk. Seemed to go well — good attendance, questions that indicated people were interested in the subject matter. Bummer that they scheduled both Erlang-related talks at the same time, though. A talk about Hypertable was interesting, although a few too many questions were answered by “we’re still working on that”. Wrapped up the day with Paul Fenwick’s An Illustrated History of Failure, a fun overview of some of the more spectacular (mostly software) failures the world has seen. In the evening, went by the MySQL / Zend / Sun party, but spent most of the time worrying that drunk people on tricycles were going to run into me and complicate my injuries. Also, giant rabbits are creepy.

Thursday. Nat’s keynote on teaching kids to program was inspirational. I downloaded the programs he recommended to try out with my daughter. Processing Large Data with Hadoop and EC2 was quite interesting. The New York Times used Hadoop and EC2 to do a pile of image processing related to releasing the archives of the first 100 or so years of the paper onto the web. Tim Bunce talked about Ultimate Perl Code Profiling using Devel::NYTProf v2. To quote an audience member, “I have only one question: why are you so awesome?”. I also went to a talk about Meebo’s Interview Process which was pretty interesting. I’m not sure that I picked up anything new, but it did validate a number of opinions I’ve formed about what works and what doesn’t over the years. In particular, having candidates spend a couple hours actually doing the sort of work they will on the job is a really valuable technique. She’s stressed that you have to ask people to do a task that people who don’t do that job would actually fail. I’d add that your interview process should probably be rigorous enough that some of your current employees who are doing that job would fail it.

Thursday evening I decided to go home for a proper dinner, then came back for Beerforge (kinda disappointingly sparsely attended) and the O’Reilly speakers’ party (full of interesting people, unsurprisingly).

On Friday, Tim Bray’s keynote was an amusingly frantic discussion of language trends, with a nice shout-out given to Erlang, and a general leaning towards functional languages as a promising developing direction. The Twilight Perl was a little less scary than average Conway, because he limited himself to no source filters or even outside modules for the most part. As a result, most of the tricks weren’t too difficult to figure out if you’re familiar with all the corners of the base language. But, it was still quite entertaining. After his talk, I bailed on the closing keynotes (a repeat of “An Illustrated History of Failure” for some reason) in favor of lunch and trying to replace my missing phone (lost somewhere Weds evening).

Two fan-boy moments: Spent a few minutes talking with Larry Wall about Erlang, and asked Tim Bray for advice about hats (specifically traveling with them).

Overall, it was a good week as usual. Once again, I was struck by how much I felt that living in town actually detracted from the conference experience. If I submit a talk for next year, I’ll make sure to actually start writing it more than a week in advance; that was stress I didn’t need.

On Friday I climbed Mount St. Helens with a Mazamas crew. It was a little rough; between vacation and moving, my training regimen sort of fell apart and I hadn’t done a serious hike in over a month and a half. But, at a certain level, you just keep putting one foot in front of the other and eventually you end up at the top. (Well, that’s not true for all mountains, but it’s pretty much the case for St. Helens. Just don’t keep walking into the crater.)

In case you hadn’t noticed, we got a lot of snow this year. Most of the roads in the Mount St. Helens National Monument have been closed until the last week. As a result, all the earlier Mazamas climbs of the mountain have been canceled. We lucked out on timing, as they cleared the road to Marble Mtn Sno-Park just a couple days before our climb. We drove up and camped there Thursday night, rising at about 4AM to hit the trail at 5. We cleared the treeline around 6:30, and by 8 we were well above all the neighboring terrain. But, still only about halfway up.

Three hours later, we made the summit, and plopped down for lunch.

The trip down was a bit faster. We were able to glissade all the way to treeline. That’s a technical term that means “sliding down the mountain on your butt”.

(Photo by Travis Schweitzer)

More photos on Flickr