I have recently been investigating the complexities and challenges posed by large scale software development processes. In doing so I wanted to find other activities that have similar structural challenges.
In this blog post I define large scale software development as an activity that encompasses a group of skilled software developers working on a shared construction (the codebase) requiring flexible methodology over a significant period of time with potentially changing requirements. I will go through each of these characteristics to give a little flavour.
A group of skilled software developers is an example of a group of skilled professionals. Small scale software projects can be achieved by a single individual. As such some of the challenges of collaboration do not exist. A handyman or a plumber working on a single small job is similar. Professionalism may be required, but not collaborative professionalism.
A shared construction means that not only are these developers working to the same objective, they are also working on the same piece of work. A gardening company that hires out gardeners to work on different sites can be seen as a group of professionals working together. But the fact that they are working on separate sites removes the complexity that would exist if they worked on the same site.
A requirement for flexible methodology means that the actions and choices of each professional may need to adjust to the scenario they encounter within the codebase. A production line pumping out cars might have apparently similar challenges, but methods and process get honed, standardised and scripted to a level where they are automatable or precisely teachable. This enables cars to be created in a predictable time and with consistent quality. Greenfield software projects might require little flexibility in the methods of developers, but every greenfield project turns into a brownfield project quite rapidly.
The endurance of the activity over a significant period of time introduces complexity. Projects that are just too large to complete in a short time period require enduring effort. This enduring effort means that personnel, best practices and individual familiarity with the work will vary over time. This opens the door for misinterpretation of existing work and obsolescence of work in progress requiring rework during the delivery phase. Refactoring may be unavoidable but it is always a challenge to communicate to an outsider that views the product as a single piece of work.
Changing requirements are a fact of life for most software developers. These can be prompted by a change in request from customers, a change in priorities of the business, a discovery of complexity during the implementation, a clarification of previously ambiguous requirements or other sources. Discovery of complexity is also a challenge to communicate to stakeholders. It is akin to finding a structurally unsafe wall when working on building an extension. It increases the scope of the work without delivering any increase in externally visible benefit.
Many of these characteristics interplay with each other. For example when working over an extended period of time the competitive landscape may change, leading to a change of requirements.
It occurs to me that Brexit negotiations (for both sides) have a high similarity to large scale software development (with some differences.)
The procedure of Brexit can be seen as encompassing a group of skilled professional (lawyers, negotiators, civil servants) working on a shared construction (the Brexit agreement) requiring a flexible methodology (the structure and detail of each part of the agreement being distinct) over a significant period of time (an initial timescale of two years from article 50) with potentially changing requirements (realpolitik is at the whim of the populous and influenced by the media.)
As such it is likely to encounter many of the challenges that large scale software development processes encounter.
So are there any techniques that can be translated from software development to the Brexit negotiation process? Given that far more software projects have run (successful or not) than Brexit negotiations.
For me two things jump out.
First – trying to deliver the project as a single piece of work carries very significant risk. Individual details can balloon to take up far more resource than expected. In an all-or-nothing scenario details that are not significant can delay work on details of far greater significance. They can also push back the timeline of the project, with the accrual of such delay likely to be significant. Also pieces worked on at a later time can impact earlier work that is considered complete – either making it obsolete or requiring additional work to bring it into line with the new work.
Software development has evolved towards an approach of early and frequent releases of small product increments to address these issues. After each release the most important pieces can be worked on next. Functional (production ready) product is delivered at each release. Reworking / refactoring is absorbed piecemeal into subsequent releases.
Second – there will be bugs. In software these impact through a variety of scenarios ranging from the product not working to users having to adopt work arounds to achieve goals that they want. For Brexit these may emerge an unintended or unforseen adaptations of business to the new regime, or arbitration and courts reaching decisions that are valid and even required by the framework but not in line with the intended spirit of the agreement.
As with software, don’t expect the first release of the agreement to be the only release of the agreement. Provision should be made to allow subsequent modifications to the agreement to address bugs that only become apparent after the event.
Another software rule of thumb may also prove useful – estimate how long it will take, then double it.
Playlist
Come together / The Beatles