Unless you're a very particular kind of person, project management and system infrastructure are the last things on your mind when you undertake a new side project or startup. For a side project, the fun is in taking the motivation from having an idea, prototyping an MVP as quickly as possible, and following the shiny thoughts as much as you can. In a startup environment, the impetus for change is now customer feedback/demand, a pivoting idea, or features that were previously unaccounted for. In both cases however, the priority is speed. Speed of development and ability to dynamically adjust is what will make your side project fulfilling to work on (or even lead to you actually feeling you completed it!), or let your startup gain an edge over your competitors.
Most people will actively agree with this as a concept, but will then barrel straight into working as hard as possible leading to frustration and burn out. Inevitably when you couple rapidly changing goals with trying to keep pace with them, you're in for a bad time for the simple asymmetry of the work. It takes 30 seconds to point out an improvement that could take 3 days to implement! Once you think about this asymmetry and how frequently it occurs and the imbalances it causes, you'll see it everywhere and will try temper it with a renewed desire for planning and prioritization. Maybe even some of those dreaded a g i l e t e c h n i q u e s useful...
I'm not there yet, don't worry. I'm not going to turn into some agile-scrum-coach-master, but all I want to do is preach the virtue of having an infrastructure-first work focus. When rapidly changing goals are made the top priority, things get out of hand very quickly, especially if there's no reconciliation period to deal with the issues. Tech debt racks up, work is frequently duplicated as quickly developed, non-refactored code is hard to reuse, and ultimately you end up with something that while potentially fulfilling the goals, is near-impossible to sustainably work with. I suggest treating your infrastructure management almost as a pre-requisite of doing the work.
Given all the issues I pointed out above, the quickest antidote I can see is optimizing your common tasks. You will have to engage with rapidly changing goals if you're doing interesting things, it's almost a fact of doing them! You can however, in many cases predict the specific infrastructure you're going to be working with as you have a finite set of primary skills. For me (currently) they revolve around Vue for front-end, Python for back-end, and a mix of Github Actions, Docker, AWS Lambda, SQL-like databases, and some more bits for the glue. I have a set of infrastructure tooling built around these technologies that will let me spin up an end-to-end of a Vue static site, calling a REST API interacting with a MySQL database or reaching out to some serverless methods all running on a custom domain with prod/test environments set up all in less than half a day. That's just not possible for someone sitting down to manually do all this work and then worse again, what happens when they want to add a few new endpoints!
This realization of the outsized impact of infrastructure tooling is new to me, but when I get the chance I will be pushing all of these to Github to share with everyone, but for now a list will have to suffice! I suggest as a minimum the following:
- A Github Action to publish a Python package to PIP.
- A Github Action to publish a Docker image to a hub.
- A Github Action to publish a serverless method live.
- A Github Action to run tests on PRs.
- A few cookiecutter templates to package combinations of these together with dummy code to get up and running with development (e.g. a Makefile that has rules for running tests and building a virtualenv, a
srcandtestdirectory, and a basicmain.pythat will give a simple HTTP 200 return code for a serverless method cookiecutter). - A logging framework that incorporates invocation IDs and ideally logs to some central infrastructure. Invocation IDs are simply an ID that you reuse through one execution of the code path. This allows you to grep your logs for one ID and see all relevant logs to just that execution without guessing which logs are related from a group of random logs!
- And as an extra, a mindset change. If there are tasks that you forsee being time consuming or frequently disruptive (sending notification texts/emails, creating new users, etc etc) then try find a way to automate them. Having the ability to start a long running command then just pipe it to a generic notifier is incredibly useful!