Let’s face it, we all make mistakes. This is true for all humans and especially Software Engineers. We as engineers managed to shut down the F-22 Raptor fighter jet when it crossed the date line in mid flight and crashed a 370 million dollar rocket.
Most of us build software which doesn’t destroy rockets or planes. Your work, however, is still very important. We build all manner of things: From data processing pipes to web shops. What’s the most important thing to get right here? Most people would say the Software Architecture.
But let’s be serious: In most software projects you have so many unknowns that it is next to impossible to nail it right away. What’s the feature set you need to support after 3 years of service? How many users do access the system simultaneously? What kind of legal framework do you need to support when the European Union strengthens the privacy of your users? We just don’t know.
This doesn’t mean of course that you shouldn’t be diligent about your architectural choices. Use the best effort principle and build the best possible design for the requirements you know and the skill set you have. However, also minimize the cost of errors in judgement and unknowns by designing your architecture so it can be easily adjusted to new circumstances.
In recent history we as a community have achieved to build frameworks which make this plausible. Monolithic systems have been replaced by Microservices. Small execution units with a well known API and their individual tech stack. Technology has enabled us to decouple data from the persistence layer, for example by using Event Sourcing as a core building principle.
These modern tech stacks allow you to design a system which makes it comparatively easy to handle change and thus fix design issues in your architecture. If you realize that your time series database is consuming too much memory you can fix this without any downtime:
First, create a new service which handles the data and use another time series database. Make sure that the service has the same API as the one relying on the memory hungry previous database. After you’ve done that, run both services at the same time and populate the new database from your event bus via the newly created service. Finally, reroute all traffic to the new service, remove the old one, and be happy that you have designed your architecture to not be set in stone.
My point is this: Be aware that stuff might change, and do never think that your first shot on a problem is the best and correct one. Be aware that you make mistakes instead and let this be the core principle when you design Software Architectures which are planned to support applications which need to run and be maintained for more than three months. Build and design your software for change.
