« Archives in March, 2008

Not as trashy as I remembered

March 30, 2008 / Debo posted in Books / 3 Comments

I don’t remember how I heard about Roger Zelazny, but I know that I found him after I had ‘outgrown’ fantasy and sci-fi. I read the Amber series in the space of a week in 2003 after a generous friend (read: saboteur) lent me the Great Book of Amber, which has all ten volumes in a single giant book. Do the math there — that’s more than one volume a day. Two weeks before final exams, no less. Can you imagine what would have happened if I’d tried to read the series as it was being written? I think I would have immolated myself to distract me from the pain of waiting in between books.

I read them so fast that all that remained with me afterwards was a recollection of awe, with no memory of the details that contributed to the feeling. (You know, after you’re done with christmas dinner, you’re riding an insulin high, you’re fuzzy and happy, but you can’t possibly remember all the different things you gorged yourself with that made you feel that way? So too, with Amber.) Whenever I spoke of Zelazny and Amber after that, I would tell people that it was a trashy read, but an incredibly enjoyable trashy read; much more worthwhile than almost any other trashy read out there. I mean, I knew it was good. But it read mostly like fantasy, and so, my reasoning went, it had to be trashy by default.

My sister recently borrowed the first two volumes from the library. I re-read them, and was astonished.

These things are almost like, well, real literature. (Yes, I am a snob. I do say things like “this is not real literature”.)

There are so many scenes and so many descriptions in the first two books that are just so downright inspired that I often stopped and said aloud to myself, “Oh man, that was awesome!” But I won’t talk about them here. Shoo, you. Go read them on your own. Sooner, rather than later.

Go!

(P.S. I later tried some of his other work, and found it unreadable. Lord of Light, in particular, triggered my gag reflex so many times within the first three chapters that I had to put it down. I’ve heard that some of Zelazny’s short stories eclipse any of the novels that he wrote, so I’ll have to check those out sometime.)

Notes on Test-Driven Development: A Practical Guide by David Astels

March 29, 2008 / Debo posted in Books, Software, Thoughts / No Comments

Dave Astels is a very unlikely character. He’s a software consultant that specializes in agile methods, and he’s from Nova Scotia, of all places. I attended one of his talks, which is where I got a free copy of this book, and the talk was a lot like the book — practical, easy to understand, and delivered in extremely basic language.

I learned more from this book than I expected. One of the most valuable lessons I took away was on how to write test fixtures. Astels recommends that you organize test methods into fixtures that allow you to get the maximum reuse out of that fixture. So, instead of having one to hold all the tests for one class, you might instead have a error-situation fixture for that class, and others for various specific purposes. You might also have one that allows you to test a variety of classes or modules in a single common situation. I’ve found that since I started doing this, my test fixtures are harder to name effectively, but are much shorter and much easier to understand. I don’t often dread writing setUp() anymore, because the specific focus of the fixture usually results in it being quite short. Also, because the tests in the fixture are so similar (by virtue of all leveraging the same fixture), you can often refactor the heck out of them, so that they all end up calling a single ‘checker method’ that has most of the asserts in it.

I also picked up a few things on mocks. It turns out that I’ve been calling a lot of things ‘mocks’ that are actually ‘fakes’. What’s the difference? A fake just pretends to be something else. A mock does the same, but it also ensures that it is being used correctly. Astels goes over a bunch of neat ways for generating mocks and stubbing out dependencies in ways that are low-effort and thus extremely useful. The section on writing programmer tests to do basic load testing was very helpful as well. Finally, some of the ideas presented for using TDD to develop a GUI were definitely things I hadn’t thought myself before, and they don’t even look like they’d be all that much work to use.

There are a bunch of things about this book that made me sort of glad that I didn’t actually spend money on it, though. First, it’s way longer than it needs to be. Astels works his way through an entire Java project and includes all the code, which in my opinion is overkill, mostly since the project is so simple. You end up seeing a lot of techniques repeated over and over again. I would have liked to have seen a more complex project presented, with a focus on some of the parts that are traditionally difficult to decouple (e.g. database connections, network connections, other external resource dependencies). Then, there are a whole whack of sections at the end that present unit testing frameworks for other languages, but these are almost completely useless because they only present the aspects of each framework that are similar to JUnit. I would have been much more curious to see the ways in which your approach should differ when working outside of Javaland. For instance, the article on pyunit doesn’t mention that you can’t have a static setUpTestCase or tearDownTestCase like you can in Java, and it certainly doesn’t talk about the added challenges that you encounter in trying to write tests for modules written in a dynamic language. There are important differences, though, when you have a whole extra class of potential errors made possible by the absence of static type checking. This is especially galling because the first unit testing frameworks were written for SmallTalk, which also lacks static type checking.

Finally, Astels’ language and reasoning in most sections errs a little too much on the side of basic. It’s cool at first to feel like you’re chatting with one of your buddies at the bar about hacking together a project using TDD, but it starts to get distracting in parts because it feels like he’s explaining things away instead of just explaining them.

One last complaint. One of the decisions that Astel made in order to make this book ‘relentlessly practical’ was to stick solely to a single platform (Java + JUnit + other associated tools), and many of the specialized tools that he discusses, such as utilities for GUI testing, mock generation, etc., are really only available for Java at the current time (as far as I could tell). So, it’s hard to port the experience you pick up from this book into other languages. (If you happen to be using a language that lends itself to rapid development, though, it certainly might give you the incentive to reimplement some of these ideas in that language, which is always cool!)

So, a fantastic book for people who develop in Java and nothing else, poor souls that they are. Definitely worth a read for the rest of us, but I skimmed heavily and, if you’re not 100% invested in Java, you’ll probably end up doing the same.

This is what happens…

March 28, 2008 / Debo posted in Short Stories / No Comments

… when I switch my writing practice out of my moleskine and into the spartan green-and-black landscape that is WriteRoom.

—

He stood in the space between two doors. Two minutes remained. Mounted on one wall was the twisted effigy of some stitched demon. Pushed up against the other wall was an old dead stove. Scrapemarks were on the floor before it, from when it had been roughly pushed there.

Max pulled a bright orange capsule from his pocket and threw it roughly in his mouth, snapping it with his molars. Just caffeine, this one. It was all he had left. A snarl grew unevenly in his throat.

He sprinted into the building, the floors warping strangely as he turned. Shades of fleeing people sputtered to life around him, wailing and then dissolving again. He pushed himself off walls as he took sharp corners, trading his momentum for more time. One minute and thirty seconds remained.

Before him was an open space from some other dream. The air was partitioned by slats of blue, venetian blinds fashioned from dust motes and tricks of the light. More of the wailing shades. A blur of stairs, a solid thing. A man.

After seven years, here was Dreyfus. The pursuit had been so long that it was inseparable from the man. A manifestation of over 2500 hundred days, subtract a minute and twenty-two seconds.

“It’s too late, you know,” Dreyfus said wryly. “But here I am.”

Max killed him, then. Had he the time, he would have bowed to convention and played out the drama, would have indulged in the escalation of words and feelings into posturing and, finally, action.

But this was now. And Dreyfus hadn’t a reason to defend himself, really.

Max searched through stiff pockets that rested on stiff flesh. He withdrew the first edged thing that he touched. His need was sufficient that anything he found would be enough. He spun and ran and the thing grew into an angry turquoise djinn in his hands. It was a roar in every dimension: Its distorted mouth screamed, its eyes shrieked violently, the flames of its flesh howled as they consumed the air. The slats broke, the motes were no more.

Back now, through the hallways again, but different this time. Darker, less defined, fewer turns to navigate. Ahead, now, the doors once more. Glass, all of them, but the outermost ones could not be shattered and could not be pushed open.

The stitched demon was alive. It was Max’s task to make it otherwise. He hurled himself at it, stabbing with the djinn, working with calm intent. The demon swung negligently at him. It was not overly concerned with its own survival. Forty-eight seconds remained.

As the periphery faded and Max worked at the demon, his thoughts proceeded with absent whimsy, failing to settle long enough on any particular concept that could be assigned a name. Time passed quickly. And then, it ran out.

The demon still lived. Max lowered his arms. He turned. The stove was gone.

He walked out of the doors, and some of the shades followed. The ones that appeared did not re-disappear. Things moved forward.

A feeling of insubstance, and then white blossomed. The blot washed all contrast away.

Thoughts on Steve McConnell’s Software Project Survival Guide

March 24, 2008 / Debo posted in Books, Software, Thoughts / No Comments

Some time ago, I ranted joyously about Steve McConnell’s legendary tome Code Complete (the second edition). I received a copy of his “Software Project Survival Guide” in the same bundle with Code Complete, but only just now got around to reading it. (That 35 minute walk on the treadmill every morning is great for this — I’m wired from my morning espresso, and I have over half-an-hour of mind-numbingly boring walking time to kill. Topics that would knock me out in the evening — such as software project management — are much more interesting in this context.)

I know that he is a teensy, geeky-looking software engineer, but I will readily admit that I love Steve McConnell. The man is both a scientist and a realist, which is probably what makes him a fantastic engineer. If he had chosen something like physics or molecular biology as his discipline, his publication record would probably be even more staggering than it is now.

This book is no-nonsense. There’s a lot to take from it, but in my opinion, the three most valuable contributions of this book are:

(a) Unlike most agile planning and management books, it deals directly with teams that are fairly large (upwards of 20 developers). The agile answer is usually “That’s too big. Make smaller teams.” I’m not saying that the agile approach is bad, but it’s refreshing to see discussion of the issue in a different light here.

(b) The checklists of all the stuff you need to prepare for a release. I’ve been on teams where this hurt us badly, because you just don’t want to admit to yourselves that all this “extra stuff” is going to take a significant amount of time. McConnell says it perfectly: He notes that when you itemize things like this, some people object and say “The project is going to take us forever if we plan to do all of these things.” His response is that it will take even longer if you don’t plan to do them, because you’re going to end up doing them all one way or another.

(c) He provides a number of strategies for gathering project data, and a handful of metrics (and more importantly, references to many other metrics) that you can use to compute size, scheduling, and cost estimates. I’ve become really interested in estimation and planning recently, after reading another book on the subject (this one from an agile camp — review forthcoming), and also after observing the astonishing estimation features present in FogBugz, the flagship product of yet another software icon’s company (Joel Spolsky’s Fog Creek Software).

The book is admittedly quite dry in parts. For this reason, I recommend that you read it while on the treadmill. (Ha ha.) It’s not quite as bad as that software engineering textbook I blogged about a few months ago, but some of the chapters come close. Also, while I imagine the techniques described in this book are still quite relevant today, it is starting to suffer from age. Notably absent is any reference to agile methods, because the agile manifesto and its associated marketing machine hadn’t been assembled yet. I actually found this refreshing, because it gets tiresome to see how every single other methodology out there has to somehow relate itself to or distinguish itself from agile methods. Also, it’s interesting to see how some of McConnell’s suggestions actually fall squarely into the agile manifesto. For instance, he warns that developers should always have significant input into project estimates. He also emphasizes that you should split your production release into short iterations, and deliver production-quality software at the end of every iteration to keep overall quality high throughout the span of the project. His overall approach has more structure and requires more up-front planning than agile methods do, though.

I also liked how he dealt squarely with the issue of making a good preliminary estimate for the purposes of budgeting, so that the people in charge can make a rational decision about whether to go ahead with the project or not before anyone has started writing any software.

Even if you’re going to use a looser methodology than the one that McConnell describes in this book, it’s worth having around as a reference simply for the checklists. There is a ton of supporting material that goes into a software product, and it’s really nice to have an itemized list of these products at hand right from the beginning so that you don’t forget anything.

Now I have to get my hands on the rest of his books!

Rethinking parallel computing

March 22, 2008 / Debo posted in Software, Thoughts / No Comments

When I was in my third year at the University of Waterloo, I took a course on concurrency. It started off a little slow, but by the end we were architecting some pretty large systems composed of chunks that communicated asynchronously with each other.

And I was hooked. This was damned cool stuff.

Ever since then, I’ve been fascinated by parallel programming. I started to get really stoked when I saw how much press parallel computing was getting, as a result of the power wall that processor designers started to run into some years ago now. A computer architecture course that I took at the University of British Columbia focused on this problem, and throughout the term I devoured all of the literature that I could find on engineering software systems for parallel architectures. The whole class was forced to read the 2006 Berkeley landscape paper, but I careened off into all sorts of other things from there, such as skeleton languages (ocamlp3l ftw), elite hacks like RapidMind, and even some really old lisp machines with arrays of 1-bit processors. My final project involved writing some middleware in Java for transparently parallelizing some patterns of parallel computation (or ‘dwarves’) described in the aforementioned Berkeley paper. I started by trying to use aspect-oriented programming to tease out these patterns, but decided that this would only work if I could declare join-points at the granularity of program statements (like you could in the original lisp conception of AOP, described in the original paper — except that lisp has expressions, not statements, but close enough). It took some hacking, but I was pretty happy with how it turned out, and I was excited to be working in an area that was really starting to heat up.

The parallel computing scene is getting even noisier now that serious money is being poured into it. Berkeley and the University of Illinois both received hefty grants this week, with the aim of “reinventing modern computing” to cope with these parallel architectures.

But you know, the more that I think about this ‘crisis’, the more I start to wonder if it is really a problem at all.

Here’s an experiment to try. Go find a computer running Windows 3.1. Play around with it for a while. Now, compare what you’re seeing to, say, a modern Mac running OS/X. I don’t know about you, but I feel like the software on my MacBook is a fair sight better than what was available on Windows 3.1. You can get the same experience by using one of the desktop packages for linux (like KDE) from circa 2000, and comparing it to what those puppies can do now. It’s pretty wild to see how far they’ve come.

So, let’s assume for the moment that the software that I’m using now is quite a lot better than the software I was using then.

How much of that quality increase can actually be attributed to Moore’s law?

This isn’t me trying to goad you, honest. I really can’t answer this question. The common wisdom is that software shops used to be able to add more features and more complexity to their products, comfortable in the knowledge that hardware improvements would supply performance increases without having to worry too much about software optimizations. The thing is, though, that for most user applications that I can think of, the real limiting factor in the production of this software must certainly be the engineering and design process that is required to produce it, not the hardware it is running on. OS/X is a fantastic piece of software, and it’s much better today than it was when it was first released. However, the hardware it’s running on today isn’t really a heck of a lot better than when it launched.

Also, when I look at the software I use in my daily life now, the trend seems to be moving towards using a ton of small, lean, simple applications that are targeted at specific tasks, instead of trying to find one product with a million features that can do everything. I hinted at this in my recent post about WriteBoard (which I’m using to write this post right now). Witness: I now edit documents in more editors than I can count on one hand. I use vim for writing programs in languages that don’t have good IDEs. I use eclipse for writing Java. Emacs is for lisp. WriteBoard has suddenly become my blogging tool of choice. Pages is for when I need to write something that I’m going to be printing. OmniOutliner is for maintaining my gtd documents. And so on.

This trend is getting even bigger on the web. The profusion of solid web frameworks is encouraging developers and designers to cut wildly at the long tail of niche services that have long been considered unprofitable to pursue. With something like Django, all of a sudden it takes me a day to write a web application that used to take weeks, and it’s more robust to boot. So now you’re seeing web applications for summertime Hindu wedding registries on odd-numbered years, and other things of this ilk, because it costs almost nothing in terms of equipment and time to build such an application.

Do these sorts of programs really need fine-grained parallelism?

Of course, one could argue that I’ve got my causation all messed up, and that the reason we’re seeing a shift towards simplicity is because we’ve hit this computing wall and simple is all we can do. But I don’t think so. I think we’re moving towards simplicity because simple apps are fun and satisfying to use.

There’s a third reason why I think all of this might be an academic point. A little while ago, one of my teammates brought an Asus eePC to one of our meetings and let me play with it. I was really stoked about this, first because I love the idea of hacking some ocaml on a wee deck with only 12 rows and 50 columns on the display, and second because I was researching this sort of thing in an operating systems course project when I was in grad school (which resulted in the stuff I posted here). While I was tinkering with it, I was astonished at how snappy the response times were. I was trying to open a terminal and not succeeding, so finally I hit the key sequence that lets you escape to the console in KDE, and I borked the machine. So, I powered the little guy off, and switched him back on.

The machine booted in less than 5 seconds.

The reason for this was immediately obvious — the eePCs use flash memory as permanent storage. Flash drives are a heck of a lot faster than hard drives.

Now, think about how much time you spend on a Windows machine listening to it thrash the swap. Imagine if that process was orders of magnitude faster. Would you perceive this as a monumental performance increase? I think so.

So maybe peripheral technology advances are actually going to be more important than core processor technology advances from hereon in. I think I’m cool with that, mostly because I’m not much interested in fields where writing cpu-bound programs is the norm. And you know what — maybe most of us aren’t.

Of course, this bodes poorly for chipmakers. Since the dawn of the computing age, their formula for making money was pretty static: Double the numbers every 18 months, and then have the marketers convince everyone that they need all this extra processing power to keep up with software advances. Now, young avid gamers in the 90s who were strapped for cash quickly figured out that this was untrue, because you could get away for an astonishingly long time without upgrading your processor as long as you maxed out on RAM and had a decent video card. While it was true that it usually didn’t cost that much to buy a new processor, usually if you wanted a decent boost in clock rate you’d need to buy another motherboard as well, since your current board would not likely support the new round of processors. So you found ways around it.

So maybe this “emergency” isn’t really such a big threat to the future of computing at large, but is rather an immediate and very dangerous threat to a handful of very big companies — which would explain all of the racket being made. Of course, if Intel and AMD were suddenly to become catastrophically devalued because no one had a reason to buy new processors anymore, we’d all be profoundly affected in one way or another. At least for a little while.

So yeah, I think parallel computing is cool. And I think it’s probably kinda important too. But will the discipline of computing implode if we can’t find a good way to write parallel programs? Will the programs we use now stagnate and cease to evolve into more elegant and more useful tools?

My suspicion is no.

But what do I know, really.

« Older Posts