Software and complexity

Over the past few weeks, I have been conducting interviews for the position of a software developer. I must have conducted more than a score of interviews of people with about 1 to 4 years of experience in the industry. Many can’t answer what the size of an integer is or what the largest representable integer value with a given number of bits is. One person said he couldn’t recall the formula. Of those who get past that, not one person so far has been able to write a 10 line function with confidence. And this is after some initial filters applied by the HR agency that supplies us with resumes – a minimum salary and some development experience. These people have salaries in the range of 3 to 6 lacs. Somehow the industry is able to extract a commensurate amount of value from them.

The product that we develop has over 200,000 lines of code. How will someone who finds it difficult to write a 10 line function work on a codebase of this size? And yet, 200,000 lines of code would be classified as a small to medium project. There are so many software solutions that have millions of lines of code. Presumably these solutions are being maintained by people with the same levels of skills/abilities as the ones I have been interviewing.

The only way I can make sense of this is to hypothesize that the people who wrote those solutions with multi-million lines of code designed them to be maintained by people far below their own levels of ability. That is an aspect of programming that I have never paid sufficient attention to. I have always worried about how to make my code flexible in terms of the requirements it can support and scalable in the data it can handle. My consideration has always been how to minimize my effort when requirements change and load increases. Often this has meant an increase in complexity. I have never worried too much about it confident in amy ability to handle the complexity.

As I look to delegate more and more of my work, do I need to consider this additional factor – how do I write code that can be maintained by someone who barely knows programming? I am beginning to believe that the answer to that is yes.

Complexity in Software

Software entities are more complex for their size than perhaps any other human construct because no two parts are alike (at least above the statement level). If they are, we make the two similar parts into a subroutine — open or closed. In this respect, software systems differ profoundly from computers, buildings, or automobiles, where repeated elements abound.

— Frederick P. Brooks

Encountered in a book (warning: large pdf file – 53 MB) found at Mentofacturing

Interpreting History and Sceptcism

In an email exchange regarding an article in The Hindu regarding secularization and modernization, a friend (call him X) commented: “As far as the article goes ….. I didn’t like it as much. More like the author already has some conclusions and wants to write something to highlight those conclusions.”

Indeed the author already has some conclusions or rather an interpretation of history. The same could be said of my post regarding the same article. Why is that bad? Could it even be otherwise? Anyone who is sufficiently interested in a subject to write about it will and should have an interpretation of the relevant historical events. Having an opinion/conclusion/interpretation is not bad. Not having the honesty to revise ones ideas if one finds facts that contradict them is. In fact, forming tentative hypotheses and refining/correcting them as one encounters new facts is the proper method to deal with anything that has a large scope. 

As an example of this, I am a software application developer. My work involves building upon an enormous amount of previous work of which I know very little. I do not know much about how the hardware on which my application runs. I do not know much about the network over which it communicates. I do not know about all the intricacies and design details of the software libraries that I use. Yet I need and have a mental model for all these things. The model is better in some areas than in others but it is not complete and will probably never be. But the fact that I might never be able to have a complete mental model does not mean that I should try not to have a model at all.

Of course, the example is not fully analogous. It is not possible to test an interpretation of history in the way that it is possible to test a model of computer hardware or software. But the necessity for interpreting history remains. History is the only place where we can actually see ideas in action. It is the only empirical source for validating ideas.

The attitude of discounting something because the author seems to have firm opinions smacks of scepticism. In the same email, X also wrote “I think that religion in not the center of the universe  for most”. Scepticism seems to me to be very common among the non-religious. But scepticism is an intellectual dead end. It transforms philosophy from a tool for living well to a game of no consequence. If one believes that one should never form firm ideas, one cannot take ideas (especially moral ideas) seriously. (I will provide an example of this in my next post.) And that is not conducive for a good life.

Philosophy is difficult – Applied Philosophy – 1

Background – I recently started a technical blog. It has just 4 posts so far, 3 of which are about WPF¹. I have not actively tried to promote it. Yet it gets some steady traffic mostly from search engines, something that has started to happen with this blog after 30 posts, despite the fact that 2 of the posts are on highly controversial topics (here in India) and despite the numerous comments I have posted at other blogs. 

Although this is not particularly surprising, the reason is important in the context of this series.

Consider my technical blog. What is its target audience? Software developers working on the same combination of platform, application type and framework as I am. How do people find the blog? By searching for specific keywords, chosen to lead them to a solution to a very specific problem. How do they evaluate what they find? By testing it directly (usually a matter of minutes, or at most of hours) and seeing if it works. Why do they return to the blog (if they do)? Because they have found working solutions in the past and expect to find them again or get alerted to issues they have not encountered yet.

Consider this blog. What is its target audience? Everyone (i.e, no one in particular). How do people find the blog? By any number of ways but mostly by chance. How do they evaluate what they find? By reasoning about it or by their emotional reactions or by some mixture of both. Certainly not by testing it (That would take weeks, if not months or years). Why do they return to the blog? Most of the returning visitors are those who already agree with the content.

This difference is inherent in the nature of the two fields – software and philosophy (and more generally any technical field and philosophy). Technical content is narrow in scope, specific in its application and indisputable in its validity. Philosophy deals with everything that is. Technical content (with the necessary background) is easy to understand, easy to apply and its benefits are immediate and tangible. Philosophical ideas are difficult to understand, difficult to apply to real life situations even after they are understood and their benefits are rarely immediate or tangible. Philosophy is difficult.

Notes:
1) A not-yet-mainstream framework for writing desktop software applications for Windows.