penitent, turbulent, multiplex!

This should go without saying, but it’s surprising how long I went without actually doing it.

A few years ago, I read Code Complete, which I’ve talked about at length here before. One of the first things that impressed me about that book was that it laid out a very precise reading plan that all newcomers to McConnell’ company had to follow.

Until that point, I’d mostly picked up new books and tried new techniques whenever I felt like it. If I heard of some reference or thought of some activity that would be beneficial to my learning (e.g. reading book X, or doing programming exercise Y) I would jot it down in a looong to-do list, and hope to get to it eventually.

After I saw McConnell’s suggested reading list, I went back to my own bucket of learning items and loosely organized them into a sequence. I also defined what it would take to actually get the desired benefit out of each activity. For books, this meant anything from skimming the text to doing all of the exercises contained therein. For extended exercises or activities, I gave a bit of thought to the scope that would be required to actually learn anything from the attempt.

This has ultimately resulted in me getting more out of the time I spend learning about my craft. For example, I’d previously read the Structure and Interpretation of Computer Programs. It only took me a week, but I’d be hard-pressed to tell you anything about the last four chapters of the book.

I then read it again, about a year later. This time it took me significantly longer to finish, but I hadn’t retained much of it. I decided that, in order for this activity to be worthwhile, I’d have to do a significant portion of the exercises in the book.

So now I’ve been working on SICP for about a year, and I’m not even halfway done the work I believe I will have to do in order to consider myself done with it. Yes, that’s a very long time. But, the value I get from each session is much higher than it would be if I was just skimming the same material. That is, every time I sit down with SICP and a stack of paper, I come away an hour or two later feeling a whole lot wiser than when I sat down.

I feel that the few hours that it took me to organize my list of “things I’d like to learn about” into a loosely annotated and ordered plan has helped me a lot. Hopefully it might help you as well.

When your average software developer goes to break a subsystem or module into implementable pieces, generally her first instinct is to think “What are the classes and behaviours in my problem domain?” Whether you’re driving this design with TDD, or taking a more top-down approach, this question tends to be the linchpin of the design.

This is unsurprising. Object-orientation is (ostensibly) taught to most of us in school as the one-true-way of doing things. It’s rare to find people who do not equivocate software-design-in-the-small to object-oriented design.

A couple of years ago, I decided that I wanted to learn “functional programming”. My first serious attempt was in OCaml. However, once the wheels hit the ground, I felt like I had returned to highschool. At that time, I knew the syntax and vocabulary surrounding objects, but I had no idea how I was supposed to use them.

So, I reasoned, I needed to learn how to design functional programs. I needed to learn “functional design”, for lack of a better term.

After a bit of consideration, I remembered that there are a lot of very famous universities that teach the basics of program design using Scheme. (This is also true for OCaml, but the Scheme ones are arguably more famous .) This was perfect — I needed to learn how to design programs all over again, in a “functional style”. So, why not go back to basics? And thus did I seek out MIT’s CS1 old-style curriculum and the Wizard Book — also known as The Structure and Interpretation of Computer Programs (SICP). (They’ve since switched to Python, I think. I imagine the spirit is the same.)

You may notice that I’ve been putting “functional design” and “functional style” in quotations. The reason for this is that I quickly discovered that such things do not actually exist.

Let me explain.

The fundamental exploration that jumped out at me in SICP was this: Let’s say you have a programming model that supports first-class functions, and does a pretty good job of supporting immutable data. What reasonable methods of designing and implementing programs can we arrive at using this model?

And the answers are legion. A program can be viewed as a mathematical function. It can be viewed as a signal-processing device over finite sequences of discrete signal elements. It can be viewed as a signal-processing device over infinite sequences of discrete signal elements. It can be viewed as a processor of a domain-specific language. And so on.

What SICP really shows you is a variety of methods that you can use to design programs-in-the-small, with object orientation being only one of them. So now, when I’m working on a new chunk of some big project, I find myself thinking not “where are the objects”, but instead “what happens if I view this problem as, say, a signal processing problem?” Even more importantly, when I read someone else’s code, I can ask myself the same thing.

The reflex of explicitly asking myself this question when faced with a new design problem really helps to understand it from multiple angles, and to identify the things that are going to be the hardest to model or extend in any feasible design. And that lets me focus on clearing up the things that matter.

I won’t pretend to be a master of any design-in-the-small methodology. However, I’m now aware of a whole set of possibilities that I was once largely ignorant of, and this knowledge helps me in a bunch of little ways every day.

In the last short, I used several analogies. I likened the craft of a software developer to that of a contract artist, or a shoemaker.

Analogies are useful things. When I have the pleasure of sitting down with a group of people who are really excited about what they do, and who are discussing a very challenging problem, what happens almost immediately is that someone will draw an analogy to help the discussion move forward.

This definitely gets the conversation moving. It’s the direction that is the rub. It’s really easy to let an analogy run wild, to the point that it gives you almost no insight on the problem you were initially trying to understand.

As an example: Just the other day, I was at a great lunch meeting where the discussion was centered around software testing. Someone mentioned that developers often write software with the assumption that the user is usually going to do things in a very predictable fashion, and that ‘weird’ cases are exactly that — weird. In this sense, the developer is living with the same set of assumptions as a traditional performance artist, where interactions are scripted, and mess-ups are assumed to be few and far-between.

It was mentioned that perhaps a better mindset would be that of the improv performer, where the fundamental assumption is that everything is going to be unexpected. The analogy was a good one — it clarified that there were two sets of axioms we could be working against, and perhaps that one of them was more appropriate than the other.

Within twenty minutes, we were talking about this one time that someone had been to a play where the lights had gone out, and the crew decided finish the performance by candlelight. The analogy had run amok. We were certainly still having a stimulating conversation, and possibly there was a lot of value in what was being said. But we’d completely lost the original point.

Usually when I’m involved in a conversation that is centered around an analogy, I try to bind the concepts within it to concepts in the topic that we’re actually trying to understand. When I hear a lot of those threads breaking, that’s how I know the analogy is taking on a life of its own.

Why do I think this is so important to being a good developer? I have two reasons. The first: When I encounter software that I think is particularly pleasant to work with, it is because the high-level metaphors that are being used to describe things are obvious and are used consistently throughout the product. If you look at a lot of patterns in software design, they’re essentially dressed-up analogies. (A “bridge” is sort of like a physical bridge because etc… an adapter is sort of like a power adapter because etc…)

The second is that I’ve observed that many smart people that I like to learn from tend to use analogies heavily. I don’t often get a lot of time with those people. And so, to really make sure that I’m getting the most out of my time with them, I like to carefully monitor the evolution of their analogies. After all, I can spend my afternoon talking about botched performances with pretty much anyone, really

(Steve McConnell does a good job of dissecting several analogies used to describe software development as a whole in his massive Code Complete, 2nd edition. I can’t say that I particularly like the one he finally arrives at (“it’s like construction”), but at least he is thorough in itemizing the components of the analogy that work, and the ones that don’t quite mesh well. It’s that sort of exploration that really helps you understand the original problem, in my opinion.)

If you’re getting paid to write software, then it’s pretty likely that the majority of the people around you understand you to be an engineer. That is, it’s your job to understand and create a practical solution to a stated problem.

I struggled with this for a while. This makes it really easy to cut corners. It makes it really easy to think of your job not as a craft, but as a requirement that just needs to be fulfilled as cheaply as possible. At least, that’s what it does for me.

I’ve managed to reconcile this within myself by the analogy of the contract artist. While in Italy, I read and heard countless stories about great works of art that were commissioned by the state. These works had very real and often very challenging engineering requirements that had to be met. And yet, there were myriad details that went into the creation of those works that had nothing to do with explicitly fulfilling the requirement, but that essentially gave each work its own life.

That’s how I think about what I do. And if you look around, there are a ton of other professions that are essentially the same. A cobbler just has to fix your shoes, but it’s the material he uses, the lessons he has learned, the attention to detail that he takes, that affects how well the job is done — even if it’s in the details that only another cobbler would notice.

When I worked in a restaurant as a teen, I learned that being called a ‘shoemaker’ was an insult. It meant you’d paid no attention to what you were doing. Now, years later, I see ‘shoemaker’ as kindred spirit.