The Numerous Pains of Programming: Death by a Thousand Cuts

I recently started building a small Python application, less than 1000 lines of code. However, it’s been a real challenge to get it to the point where it can be deployed and run in a production environment. I’ve done this before, but it’s always difficult, and each time I’m reminded of how much effort it takes, in part because I always have to start anew.

As a part-time developer…which of these extra things do I actually have to do? And to what extent? The code, sure, but all the other stuff?

The Pain of Programming

For most applications, writing the code is relatively straightforward, though it can still be challenging. However, there are tons of other things around the code, all the extra work you have to do to make it a quote unquote “real application”.

I’m not even including things like choosing a language or framework.

NOTE: This is not a perfect, comprehensive list–it’s a brainstorm of things I can think of that I’ve had to do, or might have to do to make an application “real” and somewhat professional.

1. Writing and maintaining comprehensive test suites, unit tests, integration tests, etc.
2. Setting up testing infrastructure and testing frameworks
3. Designing and implementing CI/CD pipelines
4. Creating and maintaining distributable packages
5. Managing production environment deployment and monitoring
6. Implementing release management
7. Writing and maintaining documentation
8. Creating and updating system architecture diagrams
9. Managing version control workflows and git complexity
10. Supporting and responding to user needs and feedback
11. Implementing basic security best practices
12. Conducting stress testing and performance optimization
13. Containerizing the application and creating Kubernetes manifests
14. Setting up vulnerability scanning
15. Managing security issues in dependencies
16. Managing databases, data models, and migrations
17. Configuring IDE settings and extensions
18. Setting up development environments (e.g., virtual environments)
19. Managing dependency trees and version conflicts
20. Configuring and maintaining linters and formatters
21. Setting up logging and monitoring infrastructure
22. What kind of logging…structured, unstructured, etc.
23. Dealing with debugging tools…more than print statements?
24. Testing with SSL certificates
25. Deploying on different operating systems and user environments
26. Dealing with authentication, perhaps RBAC too
27. Do I need different environments for development, testing, staging, and production?
28. Keeping track of issues, tickets, and other project management tasks
29. How to get secrets and configuration into the application safely and securely
30. Dealing with updating the software, from configuration to removing features and functionality
31. Understanding resource requirements, memory, cpu, etc. and what to set limits to in production
32. Threat modelling–how would an attacker get in, what would be the impact, etc.
33. What license should I use for the software? What does it mean if I choose the wrong one?
34. What metrics should it output, and how?
35. Configuring…the config file. What should it look like? How to validate it? What’s a good layout?
36. Finding dead and unused code

I could go on, and I’m sure there are many more.

All of the above is an enormous amount of work, work that goes far beyond the few hundred or thousand lines of code you write for the core of a simple application. For a small application, it doesn’t seem worth it. There doesn’t seem to be much in the way of easy-to-use automation or other tools to help with all this–it’s just a big old mess of tasks that add up to a lot of work.

Table Formatted Pain

Here’s a table view of the list above.

Again, this is not a comprehensive list, but a brainstorm of things I can think of that I’ve had to do, or might have to do, to make an application “real” and somewhat professional. I’m sure there are better lists out there somewhere…

What is a real application?

I keep using the term “real application”, but what does it mean? What makes an application real? I suppose what I mean is an enterprise, production-grade application of some kind. But that doesn’t mean much–I’m not sure anyone knows what makes an application production-grade. It’s kind of a vague term, and I think we could probably do better. The reality is that programming is more of an art, a dedicated craft, with hundreds of things that need to be done and hundreds of other tools, technologies, and techniques that can be used to do them. Some of this is optional, some of it is required, and some of it is just good practice. It’s not always clear which is which.

Tedious work

This list is a decomposition of what a developer has to do. I find all this extra work absolutely exhausting and, frankly, quite tedious. For every single one of these steps, it seems to me that there must be a better way. And for some of them, I’m sure there is, maybe I just don’t know about it yet. But I don’t think there’s a single tool, technique, or technology that can help with all of them.

Ultimately, I would love to have a magic wand that I could wave that would do all of that, so I could just focus on the code that delivers the value, but I don’t think that’s possible. Generative AI isn’t going to help that much, you can imagine it tying it all together somehow, but the problems are still there, they’re just hidden, like a river of lava under the surface.

Imagine what we could do if we just got rid of all that boredom?

An application can never be done, but it must be run.

I think the reality is that an application can never be finished. There is always more work to be done. It can never be 100% finished - there is always something missing, which is an incredible place to be when you consider that one of the main outputs of humanity at this point in time, one of our main economic drivers, is the writing and running of software. What a bizarre situation.