Developer Efficiency

Much has been written about the productivity of software developers. The term “10x programmer” has been used and abused. It can be traced back to the paper by Sackman et al. that first described the variations in ability of individuals to program and debug. Even though I’m not writing about individual differences here I’m mentioning this publication for another reason: It set out to compare how well people do when they should debug a problem and have or have not access to a computer: Online and offline debugging¹.

Now using pen and paper every once in a while can be really helpful to facilitate thinking (or to whiteboard with a colleague) but in general the computer itself can guide and support a human programmer in programming and debugging tasks very well. This functionality was either very new (and available to only few programmers) or non-existing in 1968 when the paper above was written. But a lot changed in the decade or so thereafter. Smalltalk and Lisp developers know all about that. And yet, it sometimes seems to me that the rest of the world hasn’t really caught on yet.

Fundamentally computer programs are extremely dynamic processes. Understanding them requires not only to know about the abstractions and their static structure and interdependencies but also how they work together, how control flows, what data and objects are used. Programming then means to nudge both the abstractions as well as their dynamic relationships towards achieving a particular goal, step by step.

Imagine building a web shop and adding the “add item to shopping cart” feature: There is some UI in which items to buy are listed. Clicking on one should modify the state of a users cart and be reflected in the UI as well. Even in this overly simplistic and contrived example we have a couple of static objects and dynamic interactions:

• Items to buy
• A user
• A user’s cart
• A user interface rendering all of the above
• A user’s interactions (clicks) modifying some or all of the above

Now let’s consider what likely needs to happen:

1. The UI will likely provide some on-click handling so we need to add an handler that gets triggered when a click on an item occurs
2. The handler needs to get hold of the user or maybe a user’s cart
3. The user or the cart needs to expose a method like addItem
4. The addItem method needs to modify the cart state (maybe add the item as a reference to a list internally) and this needs to be reflected in the rendering logic of the cart, e.g. by rendering the list length as a number label somewhere.

So how could a task like this be done in a programming environment like Smalltalk?

We would start the app and then locate the on-click handler which might be a method or a block (an anonymous function). The first thing we do is to add a breakpoint to it (the statement self halt. in Smalltalk parlance). We then click on an item, a debugger window appears. We have now halted the execution in mid-stride and have access to the item, the click event. Let’s say the item has a reference to the user’s cart. We modify the handler code so that the addItem method gets called /right inside the debugger/, while having direct access (for inspection or code completion) to all the objects involved. Let’s assume the addItem method does not exist but we continue the execution anyway. The debugger will pop open again telling us that the item has no such method but also offering us to create it. We choose the latter action and end up in the debugger being able to fill the body of the addItem method similarly as before with access to all the objects involved.

Compare that to the typical edit-compile-run approach. Consider that the incremental changes map nicely to the “plan” we made before. In particular there was no need for multiple program restarts including getting to a program state we could use for testing the changes. Apart from not having to wait for compilation we were placed right into the control flow with /concrete/ access to all data and objects. This directness and immediacy lowers the abstraction level we have to deal with: Instead of thinking in (abstract) classes we have concrete objects at our fingertips, decreasing the cognitive load we have to deal with. David Buck described this much more beautifully in his post Swimming with the Fish.

Why am I writing about this? Because even though computers are able to guide us in this manner since at least 40 years, the state of the art of development tools has not caught up. Even more, this topic does not even seem to be on the radar of most people working on programming languages, tools and frameworks. If it does than often as reminiscent memories to the “good old days”.

I consider this to be an extremely important issue for a number of reasons. Two of the more important ones: I believe that the average efficiency of software developers working in the industry is only a fraction of what it could be. I find it at least reasonable to think that there could be huge potential cost savings with more efficient tooling.

Much more important than that, though: Dynamic, incremental, and explorative interaction with programs is just a stepping stone for creating computers that really are “bicycles for the mind” — personal assistants that facilitate thinking, learning, and simulating. But how can we build truly flexible assistants for everyone if we can’t even get the tooling for professional developers caught up to decades old insights?