Friday, September 07, 2007

Science and engineering

Joel Spolsky has a post about the phases of the software development cycle that's remarkably close to my own observations. In Joel's view, the first phase is art (i.e., design phase); the second is engineering (construction); and the third is science (debugging and testing).

Joel's interest is in project management and management tools, but mine is more in development tools. Once you recognize the divide between the engineering and science aspects of software development, you can better understand one of the tensions in the approach to development, a tension which leads to plenty of heated debate. This tension comes about because the Fundamental Immutable and Inviolable (Not to Mention Sacred, Holy, and Good) Laws of Engineering are sometimes at odds with the practice of science.

To wit: abstraction and modularity are the heart and lungs of software engineering. Rules #1 , 2 and 3 are "Localize concerns, i.e. Don't Repeat Yourself, separate concerns and enforce their orthogonality." More simply: use abstractions and don't violate them. By making one concern completely oblivious to (i.e., parametric in) another, you maximize your freedom to change one without affecting the other. This allows for local reasoning which in turn leads to separable development and maintenance. Disciplined developers create layered abstractions and fastidiously respect their boundaries.

But what happens when you start debugging? Dogmatically adhering to abstraction boundaries is like wearing blinders; when a bug first arises, you never know which abstraction layer it's hiding in, or if it's in the interaction between layers. Another common consequence of thinking inside the abstraction box is impulsively assuming the bug is someone else's fault. ("The compiler must be broken!") I'm reminded of Knuth's quote about computer scientists:
Such people are especially good at dealing with situations where different rules apply in different cases; they are individuals who can rapidly change levels of abstraction, simultaneously seeing things "in the large" and "in the small."
          -- quoted in Hartmanis's Turing Award lecture
I think this is describing more the science and perhaps also the design aspects--but not the engineering aspect--of software development.

Because debugging and testing are about observing and understanding an existing system, rather than constructing or modifying a system, the barriers we construct to enforce our engineering principles become obstacles. Debugging tools, IDE's, testing frameworks, etc. are all characterized by a need to violate abstraction boundaries.

As a result, the Cleans and Dirties (as Mitch calls them) fight tooth and nail about whether our software development frameworks should be absolutely strict in their adherence to the FIaI(NtMSHaG)LoE (ML) or absolutely lax (Smalltalk). I wonder if we couldn't do better by building software frameworks that were aware of these different modes of development.


Anonymous said...

Joel's characterization is apt, I think. It certainly rings true to me.

Building things is certainly easier when you don't have to know all the intricacies, but maintaining that ignorance makes it harder to fix when the inevitable happens. My gut feeling is that you would benefit from a lax approach that allows you to tighten things up when you know more.

What if the abstractions did some of the observation? That is, observational abilities are built in to the abstractions themselves?

Anonymous said...

What the heck does FIaI(NtMSHaG)LoE mean?!

Dave Herman said...

Anonymous: see paragraph 2.

shapr said...

How would you build such a mode-aware framework, or how would you alter existing frameworks to be mode-aware?

Garrett said...

I mostly disagree with Joel's post.

Programming, in its entirety, is art. The process itself requires dynamics, reflection.

Design evolves, and with it, so do the tests and the code.

Testing is not the "science" phase.

Fowler's words ring truer in my mind.

Garrett said...

(tried to preview but posted)


For a great analysis on problems with BUFD, try reading Beck's XP or Eric Evans's Domain Driven Design.