A short introduction to agile software development
The concept of agile software development has existed for over 20 years. There is a manifesto for agile software development and various processes, for example Scrum and Extreme Programming, have been developed around the concept.
Almost universally, when reading about the creation of agile software development, articles will refer to the "Waterfall model" as the predecessor of agile processes and the motivation for their development. In waterfall, the project will be structured so as to perform the following phases one after another: requirements capture, software design, software build, testing, deployment and then maintenance. The theory behind the model is that errors discovered in later phases are far more costly to fix than if they are discovered earlier.
The agile processes differ a little in implementation but they have common themes around iteratively doing smaller chunks of work and testing the validity of that work as soon as possible. This is manifested by having cross-functional teams, including customers, work across an entire feature in a small amount of time so as to design, build and release a new feature as quickly as possible, leading to earlier feedback and increased value.
So what's the point?
Building software is (nearly) always a means to an end. There's usually a reason for creating the software that lives outside of the software itself: it may be a way to improve customer service, or a new product that increases revenue or a way of automating an existing process that leads to a reduction in cost.
The benefits that the creation of the software produces are obviously offset by the cost of the development of the software. This benefit cost ratio allows us to understand what our likely gains are to be from the software. In purely financial terms, the benefit cost ratio is simply the ratio between the present value of the expected gains and the present value of the expected costs. So, the job of software engineers should be to maximise this benefit cost ratio through the work that they do.
Achieving that, though, requires the appreciation that these projects take place over a period of time and so, the present value of the benefit or cost can change as a function of both nominal value but also when the benefit or cost happens.
The result of this (at least under normal economic conditions of rising interest rates) is that we would like to deliver benefit earlier, as the same benefit is worth less over the longer term, and we want to delay incurring cost, assuming the nominal value does not change.
Coupled with this, though, is the realisation that at the start of a project the error in the estimates of benefit and cost can be very large. It may be that over the course of a products lifetime the benefit cost ration is not as clear cut as we would like. We would like to remove this risk so that our decision to proceed is, in fact, a good one.
The job of a software team is to deliver benefit earlier, delay cost and to remove estimation risk as soon as possible.
While this does sound like a soundbite from Captain Obvious, it's nice to derive based on soundsome principles. Thankfully, this also sounds a lot like what all of the agile processes try to promise.
Reducing nominal cost value
A key assumption in the previous section is that the nominal value of cost stays the same over time. That is, the cost to do some piece of work in 3 years time is the same as it would cost to do today. Unfortunately, it's almost impossible for that to be the case in development projects of any non-trivial size.
As software develops over time it (usually) gets larger and it's this increase in size that results in future costs being increased. Future changes in functionality can impact more of the existing functionality and require changes in those. Obviously, the more you have, the bigger the potential change.
It is in these situations where the agility of a software development process is really put to the test.
Unfortunately, a lot of agile development is done under the false assumption that the next sprint is all that matters. That getting the next feature released as quickly as possible is of the utmost performance, regardless of the long term effect on the software as a whole.
In our opinion, true business agility depends on true software agility and that means an adherence to high quality development that, sure, delivers benefit as early as possible but not at the expense of future productivity.
The interplay between realising benefit sooner and delaying costs whilst looking to mitigate estimation risk is a complex practice and, of course, depends on the context at any point in time. We certainly would not be adverse to resolving an emergency situation through a "quick fix" but over time these quick fixes can lead to a big increase in technical debt.
How do we keep the software agile?
Entire books can and have been written on the topic of writing flexible and maintainable software. Where Scrum does not provide any guidance on implementation details, Extreme Programming does offer some advice. Drawing upon that, as well as other sources, including Domain Driven Design, we start to discover techniques for maximising agility.
These techniques target different levels of the application, from high level infrastructure down to how to write individual functions.
The rules from XP tell us we should favour simplicity (KISS), not add functionality early (YAGNI) and to refactor whenever and wherever possible.
Domain Driven Design provides great direction on how to build maintainable systems. The strategic patterns help with the high level architecture of the system while the tactical patterns help with the design of the code at the level of functions. The key takeaway from Domain Driven Design, though, is the development of the ubiquitous language which ensures that the code is written to match the language of the business. This removes the need for cognitive translations between the business and the software. DDD mandates that changes to the language and domain require refactoring of the code to keep the alignment perfect.
