I was initially stubborn when it came to personally using "AI" powered tools. I still am, but I have now accepted that these tools are here to stay. The cat is out of the bag, Pandora's Box has been opened, etc etc. For better or worse, we should all be operating under the assumption that these tools are currently in use, and will become more relied upon in future. This is why naming in software development has become even more important.

These tools rely, at least partially, on the semantic meaning of your variable and function names. That means if your names are inconsistent or poorly conceived, you're going to get worse output. Some examples in regular code block: (Because I also reject Jupyter Notebooks)

import openai
import os
import sys
import time

from pydantic import BaseModel, Field


openai.api_key = os.getenv("OPENAI_API_KEY")


class GermanTranslationResponse(BaseModel):
    message_in_german: str = Field(
        description="The user message translated into German"
    )

class FrenchTranslationResponse(BaseModel):
    message_in_french: str = Field(
        description="The user message translated into French"
    )

class FrenchTranslationResponse2(BaseModel):
    message_in_german: str = Field(
        description="The user message translated into French"
    )

class FrenchTranslationResponse3(BaseModel):
    message_in_french: str = Field(
        description="The user message translated into German"
    )


def get_response(message: str, language: str, model) -> str:
    context = [{
        "role": "user",
        "content": message
    }]

    chat_completion = openai.ChatCompletion.create(
        model="gpt-3.5-turbo-0613",
        messages=context,
        functions=[
            {
                "name": f"get_{language}_translation",
                "description": f"get the message translated to {language}",
                "parameters": model.schema(),
            },
        ],
        function_call={"name": f"get_{language}_translation"},
        temperature=0.0,
    )

    return chat_completion.choices[0].message.function_call.arguments

if __name__ == "__main__":
    print(f"Input message: {sys.argv[1]}")

    # Translates to German
    print("German")
    print(get_response(sys.argv[1], "German", GermanTranslationResponse))
    time.sleep(1)

    # Translates to French
    print("French")
    print(get_response(sys.argv[1], "French", FrenchTranslationResponse))
    time.sleep(1)

    # Translates to English
    print("French and German")
    print(get_response(sys.argv[1], "French", GermanTranslationResponse))
    time.sleep(1)

    # Translates to English
    print("German and French")
    print(get_response(sys.argv[1], "German", FrenchTranslationResponse))
    time.sleep(1)

    # Translates to German
    print("German and French2")
    print(get_response(sys.argv[1], "German", FrenchTranslationResponse2))
    time.sleep(1)

    # Translates to French
    print("German and French3")
    print(get_response(sys.argv[1], "French", FrenchTranslationResponse3))

$ python3 ex.py "what's up"
Input message: what's up
German
{
  "message_in_german": "Was gibt's Neues?"
}
French
{
  "message_in_french": "Quoi de neuf"
}
French and German
{
  "message_in_german": "what's up"
}
German and French
{
  "message_in_french": "what's up"
}
German and French2
{
  "message_in_german": "Was gibt's Neues?"
}
German and French3
{
  "message_in_french": "Quoi de neuf"
}

From having lights turn on a little dim before the morning alarm on my phone, to making sure all devices are off whenever there is nobody home, the depth of what you can do with automation is immense. However, there is one UX issue I have with these automations - I just don't quite trust them. Too many times I've come home and realized that my automation didn't realize I had left the house and never turned off my devices. Similar to alarms and logs in software development, automations are only worthwhile if they are reliable and you can trust them. I set out to find a way to increase my trust in the automations I have at home, and also have an easier path to debugging them if they don't run when I expected them to.

My solution was to have a simple notification on my phone whenever an automation runs on Home Assistant. The notification medium was obvious for my workflow/style - use ntfy.sh to trigger a notification on my phone so that whenever I was concerned if the automation ran or not, I would already have the information available in a zero click manner. I thought this might be a little difficult, but as it turns out, there is a built in Apprise notification service built into Home Assistant (which I think was a great idea!). My second issue was making sure that this notification would fire for all of my automations and that I wouldn't have to manually add it to each and every one. This turned out to be easier than expected, as there is an event you can trigger on called AUTOMATION_TRIGGERED which fires on every single automation run. The plan was set!

The first thing I had to do was add the Apprise integraion to my configuration.yaml for Home Assistant with the URLs I wanted to trigger notifications on: ^[1]

notify:
  - platform: apprise
    url: ntfy://ntfy.sh/home-assistant-notification-topic

After that, I created an automation that would trigger whenever an automation ran, and would then notify me with the name of the automation that ran:

That was the sum total of what I had to do! This has made me feel much more at ease and trusting of when my automations have run, and if they don't run when expected, I can easily debug what didn't happen when I expected it to. One interesting thing to note is that the automation that sends a notification doesn't end up triggering the automation to fire again. There is no recursive behavior from it. I hope this helps anyone looking to set up automations on Home Assistant using Apprise, ntfy.sh, or just trying to trigger an automation to run whenever a different automation runs! If you have any fun automations for Home Assistant, or any thoughts/comments, please let me know through whatever means you found this post, and thanks for reading!

[1] I self-host ntfy.sh so this URL is just a dummy! If you want another blog post on self-hosting ntfy.sh, let me know, but those docs are great and it's one of the most stable services I have ever ran.

Introduction

 Everybody is a little vain. Vanity frequently spurs people to action where other, more pure, motives fail.  There's an interesting relationship between vanity and perfectionism. Perfectionism is often driven by vanity, the wish to be perceived as being a master in whatever actions you are currently undertaking. This is clearly an impossible task, but striving towards an impossible ideal can often lead to successful results along the way.

 Recognizing that perfectionist tendencies could have a different root cause than you previously thought leads to a different perspective on them. For me, perfectionist tendencies have been counterproductive to my desire for acknowledgement and recognition in the things I (at least think!) am knowledgeable about. However, you need to be visible in order to be recognized, so perfectionism really gets in the way. If my perfectionist tendencies were allowed to run free, I would rarely, if ever, release anything. The concept of "Building in Public" has been a great perspective for me in this regard, even though almost all it ever leads to is me writing half-baked blog posts and creating things like my most recent project - git-vain.

Buildlog

 Gitvain is a simple Python project that sends you a notification whenever the follower/stargazers on a Github change. It supports notifications using Apprise which is a fantastic tool that allows you to specify some simple configuration and then send notifications using many, many services. PyGithub is used to fetch changes on the watched repositories. There are many outstanding issues, that will most likely never be completed. This is by design, however, as I set out a plan to get to a point where I could be satisfied enough to leave it alone, say I sufficiently completed it, and thereby prevent my perfectionism from getting in the way.

 A major part of my recent work in software development has focused around best practices and ops-adjacent work. Recognizing that setting up CI/CD at the start of a project is very easy compared to getting it set up midway through, and that actually defining the feature you want to work on and writing short functions with no side-effects to get it done makes you progress quickly. Being realistic with the scope of what you're working on allows you to keep motivation and leave your work in a state that's trivial to restart whenever you next get the chance. To achieve this, I simply started by writing out a minimum set of features I wanted to have before being happy to cut a v1.0.0. I then broke these features down into small components and added them to Vikunja my favorite todo list application.

 I started by getting a basic Python project set up with continuous deployment using Github Actions to build a Dockerfile that would deploy to my home server. This is so simple that it literally just prints a Hello World statement in the Dockerfile! Even though this seems insignificant, from here you have the perfect jumping off point; a stable point you can solidly build on.

 After this, I prioritised one type of event I wanted to be notified on - stargazers - and wrote a basic script to star/unstar a repo, and another to use ntfy to send a notification when this list changed. Already it was starting to take shape enough to keep my motivation going.

 Once this was complete, I used the simple Python stdlib shelve to add some persistence, and integrated with Apprise to support a wider variety of channels for notification.

 There's no point in going into any further detail, because A) it's not very interesting, and B) it didn't progress much further! But this is a feature, not a bug - I set out with the intention of cutting a v1.0.0 and this was almost there already!

 But what next?

Next Steps

 I realised I had a few things I wanted to add as I was going through the development process. This is typical, and it's the source of all feature creep. I curtailed this by putting anything I wanted to do into Vikunja, and deciding to convert them to issues once v1.0.0 had been completed. If I didn't, I would certainly still be adding little bits and pieces here and there now as opposed to enjoying the feeling of having "finished" a piece of work. Even if it only lasts as long as it takes to go back to working on v2.0.0!

 Some things I want to improve in v2.0.0 or in the next side project I work on are as follows:

• Use pre-commit to add more stability and standardisation to the work
  • This feels particularly powerful and would be a good way to finally upgrade my development process which still consists of a pretty vanilla vim setup. Who knows? Maybe I'll even switch to NeoVim with this new-fangled LSP!
• There are some bugs such as not accounting for pagination of the response from Github.
  • I deliberately didn't try to address these during this effort as I could easily have gotten bogged down in it and it's a small, contained piece of work that should be easy to add in the future if I leave the project and come back to it after a period of time.
• SemVer and Conventional Commits seem to be a fantastic way to completely automate the release cycle.
  • If there's no Github Action for it, it shouldn't be too difficult to make either as I didn't find a suitable service for my needs so far - next project maybe??
• Python has a good few issues when it comes to naming conventions and project setup.
  • This is potentially exacerbated by the fact that the tooling I'm familiar with is outdated by now.
    • Using setup.py works nicely (especially if you have experience with it) for a small project like this, but it's not a good idea for more serious work.
  • The naming conventions I was having issues with surrounded small things like - vs _ vs camelCase. Package names vs module names vs binary names. Small things, that actually ended up adding some annoying decision fatigue and cognitive load to keep consistent.

Conclusion

 The Python standards and conventions are a bit of a mess. Dev Ops/best practices actually do provide great help when used in conjunction with common sense, and the "perfect" project template likely cannot exist due to the subtle variations in what "perfect" looks like for different purposes.

 Nonetheless, that's what my vanity is currently driving me towards finding, so for now, I'll keep looking.

  Hopefully not in vain!

The Tools

 Grafana

   Lets start with the main one - what is Grafana? Grafana is at its core a web-based data visualization platform. It acts as a front end to many time-series databases, and uses plugins to consume data from different sources and support custom dashboard visualizations.  It also has a simple graphical tool to help you craft queries on the data. The best place to try the Grafana platform out is at play.grafana.org.

 Prometheus

   Prometheus is a time-series database which operates on a pull model. You configure exporters which will have metrics requested from them by Prometheus on a regular schedule. There is a suite of components it can make use of, but one core feature we will be using is PromQL - the Prometheus Query Language. We will use this through Grafana to aggregate metrics collected by Prometheus. One thing that is important to note is that Prometheus is designed to work with numeric information only. This means it cannot be used to search through textual logs like you might do in Splunk or Sumo Logic.

 Loki

   Being restricted to just working with metrics is quite a limitation, so we will also be using Loki. Loki encompasses a set of tools/services, but my working model of it doesn't extend much further than Prometheus for log lines. It accepts data in any format, and similar to Prometheus, it allows you to build metrics and alerts based on them.

 Promtail

   Promtail is responsible for delivering log lines from log files to Loki. It is the roughly equivalent component in the Loki stack as Node-Exporter is in the Prometheus stack. This is confusing as Promtail looks like it should be part of the Prometheus stack, but alas naming of open source tooling is never great!

   Promtail will be used to collect log lines from containers of my own services, or of services being debugged.

 Node-Exporter

   Node Exporter monitors and exports hardware and kernel level metrics to Prometheus. It is highly configurable with a long list of metrics it can collect if you desire. Despite the warnings, we will be running node-exporter from a Docker container for now. This is just for ease of encapsulation until I can move my personal home server to using NixOS or similar.

   This will provide the host-level metrics we need, such as CPU usage, RAM usage, free space, etc.

 cAdvisor

   From the cAdvisor Github page:

[cAdvisor] is a running daemon that collects, aggregates, processes, and exports information about running containers.

   These metrics can be exposed for Prometheus, and will provide the per-container resource usage metrics we need.

Stack

   So now we have all the components explained, it's worthwhile visualizing the stack we will have. One crucial thing to remember is that while this seems like a large number of services, each one is very small and modular, so it won't be consuming a huge amount of resources.

The Data

   Now we know the how of observability, we need to get to the what. Honestly, I spent a long time putting this off, probably because this was the largest gap in my knowledge! However, I think an iterative approach works best here anyways - both in iteratively building up to "complete" observability/insight, and iteratively building up my knowledge of the Grafana stack.
   I suppose it makes sense to think about why I'm making some monitoring on these services. Primarily it's to see what my server is capable of. i.e. do I need to add some RAM/storage/replace the entire CPU? How many additional containers can I run? Has there been a large spike in usage? If so by what containers/services? How much network input/output is each service going through? As a percentage of the whole input/output? How much storage is each container using? Secondly, I want to have insights into actual logs of my own services (or for others if I really want I guess? But primarily my homemade services). This should be all logs for debug purposes and usage metrics in general.
   Lets make a list:

• Host
  • Metrics
    • CPU Usage
    • RAM Usage
    • Storage Usage %
    • Load (1 min, 5 min, 15 min seems standard)
    • Network Throughput (Input/Output volume)
  • Logs
    • syslog
    • auth.log
• Per-Service
  • Metrics
    • CPU Usage %
    • RAM Usage
    • Storage Usage %
    • Network Throughput
  • Logs
    • For specific containers

The Implementation

   Now we know what we're observing, and how we're going to ingest it, we just need to do it!

   Since we painstakingly mapped out the different component services, we can tell instantly that we need cAdvisor for the per-service metrics, NodeExporter for the host metrics, and loki for all the log lines. Lets start with the metrics.
   The metrics all need to feed into prometheus in order to end up in Grafana, so we need to edit the prometheus config file in order to do that. For getting all our container metrics from cAdvisor, we just need a few lines. For NodeExporter, just a few more:

scrape_configs:
  - job_name: "cadvisor"
    scrape_interval: 15s
    static_configs:
      - targets: ["cadvisor:8080"]
  - job_name: "node_exporter"
    scrape_interval: 15s
    static_configs:
      - targets: ["node-exporter:9100"]

   Then Loki needs to be configured for syslog and auth.log. This was achieved by a simple promtail config and mapping /var/log:/var/log/host_logs in docker-compose:

scrape_configs:
- job_name: hostlogs_job
  static_configs:
  - targets:
      - localhost
    labels:
      job: hostlogs
      __path__: /var/hostlogs/*log
- job_name: docker_container_logs
  docker_sd_configs:
  - host: unix:///var/run/docker.sock
    refresh_interval: 5s
  relabel_configs:
    - source_labels: ['__meta_docker_container_name']
      regex: '/(.*)'
      target_label: 'container'

Alerting

 Finally we have the full stack set up. Last remaining thing to get a semi-professional (emphasis on the semi!) is to get some alerting going. For alerting, I'm going to use ntfy and a small Grafana integration I found called grafana-to-ntfy. This took a little more work than expected, but eventually I got it all working. Firstly, I set up a personal ntfy instance, then added the grafana-ntfy container to my docker-compose along with a simple env file as explained in the README. I then integrated it with Grafana alerting. One of the key things to note here is that I just used plain http for communication with the grafana-ntfy container as I couldn't get it set up with SSL! I kept getting invalid cert errors with reference to a cert only valid for Traefik. Also not fully documented, but the BAUTH variables need to be passed too although they should be made optional. May submit a PR for that... Follow the README to do a test notification

Set up a query to make sure notifications are coming through and then just get on with standard alarming!

Conclusion

 So, as you've probably realized, I really lost steam towards the end of this post. I have been working on this post/stack setup for about three months and it has been frustrating me to no end and stopping me from writing things I would prefer to write about, and follow my current tech interests. I try to balance doing things I feel I should do with things that I have a strong (but usually fleeting) motivation to do as these rarely overlap. This time however, even though I can see the huge benefit of having a well set up monitoring stack for my home server and how all aspects of this will improve my quality of life when debugging/doing basic admin, the balance has just tipped to being more stressful than beneficial to me.

 I will update my stack in the future, and hopefully write a more concise post on setting up a home server monitoring stack, but for now, this is all you get!

DNS is infuriating to me. It exists in the uncanny valley of technologies where on paper I know a decent amount about it, but in reality I just flounder aimlessly and ultimately end up in an IRC channel somewhere spilling my woes to any (seemingly mystically intelligent) kind soul who will help me debug my issues. This does not pair well with trying to host your own server in your living room.

I had all my DNS issues tidied up enough to forget about them (the ideal state of affairs for DNS in my opinion) while I was living in Ireland and my ISP by default gave out static IPv4 addresses, or if not, changed them so infrequently that I didn't experience a change in over a year. However, with a new ISP in Berlin, I had quite a different experience. After congratulating myself on the quality of the infrastructure-as-code I had developed, so that even after 5 months of downtime, I could just plug in my server again, connect it to the WiFi, and it was back online, I went to bed happy. The following morning, however, I woke up to find that my IP address had changed overnight. I gave the DNS provider a call, and they informed me that a static IP address can easily be provided, but it will cost an additional ten euro per month. They reassign IP addresses every 24 hours too, just to be annoying about it. This means you can't rely on a rarely updating dynamic IP address either.

I felt personally attacked at the thought of paying for a static IP address, and for some inexplicable reason, I didn't want to use existing Dynamic DNS solutions. Maybe some vague notion of not wanting another external resource that I'd have to pay for or could break due to a misconfiguration, but most likely it was just stubbornness. Enter DIYnDNS! A simple Python script that will check if my IP address has changed in the last N minutes, and if so, connect to Cloudflare, update my DNS records to point at the new IP address, then go back to sleep until it notices the IP address has changed again.

This is all pretty straightforward, but there are a few nice aspects and things I learned along the way.

• First of all, this is all packaged up in a nice and small Docker container, which allows me to also make a simple docker-compose file to allow it to be installed with ease on any reverse proxy setup like Caddy or Traefik.
• Secondly, it uses a plain .ini file for configuration, which I never understood why people used before. I think I'll be using .ini files for my configuration in future Python projects, primarily due to the inbuilt configparser library and how easy it was to use.
• Thirdly, there's a watchtower container defined in the docker-compose file too, which will automatically update the diyndns container whenever there's a new image pushed to docker hub.
• Finally, this container is updated on every push to the repository's main branch using some straightforward Github Actions CI.

The fact that all of this is possible with just some glorified configuration files and one easy to throw together python script is amazing to me. I really feel like anyone who isn't on the boat of self-hosting simple services, and understanding even the basics of how to put together a full CI/CD system like this, is missing out.

Finally, there are a good few areas that need to be improved. For starters, I should be able to configure the cron from the config file somehow, but I had horrific trials trying to get the crontab to respect env vars, follow my PATH etc. Best solution for this is likely to put it in a systemd service file, or else some baked in Python threading tool. I also think my usage of configparser could be better. I want to explore the library more so I can use it for other projects in the near future.

Thank you for reading, and please get in touch if you have any comments, advice, or suggestions. My next post will be about the hassle I had and the effort I put in to try getting this setup working with Namecheap, my original DNS provider. It was significantly more involved than this and ultimately was ridiculous enough that I moved my DNS to Cloudflare!

Photoprism is the self-hosted photo/video library that does it all for me. It has labeling of images so you can search by picture content, it has calendar and map features for more natural finding of the pictures you're trying to find and it lets you have public/private content as you choose. To be clear, I want our reliance on huge mega-corps to be gotten rid of. I don't think that necessarily implies some form of neo-luddism or that we should have to deal with worse products. We get what we create and support, and this project is definitely worthy of both. Pictures are some of the most personal things we routinely create so we should try protect these moments from prying eyes. Moreover, with Google Photos continuing to degrade their product by reneging on their promise of free unlimited storage and by silently degrading your picture quality an alternative should be welcomed!

From self-hosting Photoprism, I've found it to be quite a smooth process relative to the size and scale of its features. The one process I found difficult to get working was one of the most crucial however; syncing with my Android phone. I mostly take 35mm analog pictures (because I'm insufferable) but I appreciate being able to scroll back through my screenshots for conversations with friends and taking a quick picture on the go. I found Photoprism's guide however lacking to solve the use case of my phone automatically uploading new content to Photoprism and having it imported ready for viewing, so I tried a few different apps and settled on FolderSync.

FolderSync allows you to sync files between different locations on your phone or use WebDAV to sync with an external location. Photoprism supports WebDAV so this should be a cinch! I decided to set up one folder on my device that would one-way upload to Photoprism and then locally sync any files in other media folders on my phone to that folder. This might be a needless step, but I think it's useful for supporting multiple different ingestion locations and also optionally not indiscriminately syncing all media from your phone with a potentially public library! From screenshots, my FolderSync setup looks something like this for SD Card Sync:

And for WebDAV we have the following:

Using these, you should be able to set up automatic photo syncing from your Android phone to Photoprism! If you have any issues, refer back to the Photoprism guide for this topic as it's definitely more up to date,  thorough, and useful than this short post. However, I strongly recommend FolderSync over their suggestions of PhotoSync and SMBSync2!

I love Python. It's easy to read, even easier to write, and best of all, the easiest language (I've used) to interact with other people's code. I spent too long being only on one end of that equation though, only consuming code other people have written as opposed to taking the things I was making and making them public so the cycle could continue. I felt it was too much hassle to get started with and for no reason because who would be using the libraries I write anyways? After finally learning how to create and publish packages to PyPi, I'm glad to report it's super easy and definitely worthwhile. Even if it's just so you can easily import code you wrote for a previous project without going into the horrors of relative imports, I feel it's worth the five minutes investment to get up and running with it.

First of all, you need to make a PyPi account and then add your credentials to ~/.pypirc. Your file should look something like this:

[distutils]
    index-servers = pypi
    
[pypi]
    repository: https://upload.pypi.org/legacy/
    username: <Your PyPi username>
    password: <Your PyPi password>

Realistically, you should set up API keys, but I am just trying to show the bare minimum to get up and running with PyPi. I am in no way an authority  on this and most often don't even follow best practices.

Next take a simple project or tool you've been working on and impose some structure. Personally, I am a fan of simple tools that do things I understand. Consequentially, I end up using tools that started falling out of fashion in the 80s. I find Makefile to be just the right level of build tool for me. I understand the file at a glance, I can hack it to get what I want done, and it is going to be the topic of another blog post so I'm not going to dwell on it too much here, suffice to say you should add a Makefile to the root of your repository/project and have these key rules:

PYTHON=python3.8

ENV_DIR=.env_$(PYTHON)
IN_ENV=. $(ENV_DIR)/bin/activate &&

upload_pip: build_dist
    twine upload --repository pypi dist/*

build:
    $(IN_ENV) $(PYTHON) -m pip install --editable .
    rm -fr dist/
    $(IN_ENV) $(PYTHON) setup.py sdist bdist_wheel

build_dist:
    rm -fr dist/
    $(IN_ENV) python setup.py sdist

setup:
    $(PYTHON) -m pip install --upgrade virtualenv
    $(PYTHON) -m virtualenv -p $(PYTHON) $(ENV_DIR)
    $(IN_ENV) $(PYTHON) -m pip install --upgrade -r requirements.txt
    $(IN_ENV) $(PYTHON) -m pip install --editable .

This will let you run make upload_pip and magically have your package arrive on PyPi once you install python3 -m pip install twine. But wait, what is it going to upload?? That's defined in your setup.py:

from setuptools import (
    find_packages,
    setup
)

INSTALL_REQUIRES = []

setup(
    name='my-first-pip-package-name',
    description='This is my first Pip package!',
    version=0.0.1,
    url='https://github.com/link/to/your/repo',
    python_requires='>=3.6',
    packages=find_packages('src'),
    package_dir={'': 'src'},
    install_requires=INSTALL_REQUIRES,
    entry_points={
        'console_scripts': []
    }
)

This should be pretty self evident for what should be where,  but examples are the best way to grok something and here are two personal repos where I use these structures for Makefile and setup.py to publish Pip packages.

I know this is quite a low quality introduction to making a Pip package, so please leave me feedback. I want to improve at my writing and hearing responses is the best way I can get there!

First of all, I would not suggest doing this for anything you want to remain stable or have any decent uptime on. This is to keep up to date with the latest bleeding edge of a project and can be useful for development and side projects when you merge your code you can show it to people live a few minutes later. If you do this with containers you're looking to rely on (such as Matrix, Vikunja or Home Assistant (ohh, foreshadowing?)), you can easily see how you could accidentally deploy to a breaking version and have to do some messy sysadmin work! However, there are valid reasons to automatically update to the latest tag, and to do this, we will be using Watchtower which is a Docker container that monitors other containers and if their image updates, then will SIGTERM it, pull down the newest version and restart it. Pretty meta! Onto the code:

version: '3.3'

services:
  via-web:
    container_name: "via-web"
    image: conorjf/via-web:latest
    restart: unless-stopped

    volumes:
      - /var/log/instance_logs/via:/var/log

  via-watchtower:
    container_name: "via-watchtower"
    image: containrrr/watchtower
    restart: unless-stopped

    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

    command: via-web --interval 30

This relatively simple docker-compose.yaml shows how easy this is. We define two services, Via and Watchtower. Via's image is set to latest from Docker Hub and it uses a simple trick to easily view logs explained here. This is clearly something under development and so it's safe to set it up to deploy latest blindly. The via-watchtower service is also very simple, really only requiring 2 statements apart from the image. The volumes is so Watchtower can be informed about Docker updates/changes in state, etc and the command is of the format <container to watch for updates on> --interval <seconds to check for updates>. It's that easy! Using this and some continuous integration to build and deploy the Docker image, Via is kept constantly up to date for development and beta testing.

I hope this was instructive and please reach out if you have any questions, suggestions or comments in general :)

A Dockerfile is a simple, declarative file that contains the commands for how to create a Docker image. I find the easiest way to work with them is to consider them the sequence of commands you'd have to run on a fresh machine in order to get your project up and running. Lets take a look at this simple example from Via:

FROM ubuntu:20.04

WORKDIR /code

RUN apt -y update && apt -y install build-essential python-numpy python-setuptools python3-scipy libatlas-base-dev libatlas3-base python3-pip

RUN python3 -m pip install virtualenv

COPY requirements.txt requirements.txt
RUN python3 -m pip install -r requirements.txt

EXPOSE 8080
COPY . .

CMD ["make", "run_bottle"]

FROM ubuntu:20.04 Most (if not all... I'm not experienced with this, really you should be finding a better source of information!) Dockerfiles start with a FROM statement. This defines what base image you want to used and is published and maintained by people who really know what they're doing. In this case we are basing our environment on ubuntu:20.04. Generally speaking this is a bad idea as it is a big image and you should use something like Alpine. Using a big image like Ubuntu is helpful however for bootstrapping or for dealing with difficult requirements.

WORKDIR /code simply defines the current working directory within the container. I consider this to be like cd from the container context.

RUN ... This is where things start looking bash-y. The RUN command very simply runs whatever commands you give it within the container context. Here we're installing updates through apt and creating a Python virtual environment.

COPY is more interesting as it is passing information between the host system and the container context. Here we are copying the requirements.txt file from the root of the repository to /code/requirements.txt (as the last WORKDIR statement placed us within that directory!).

EXPOSE is used to open ports on the container. It wraps up what I'm sure are some disgusting networking hassles and lets you just say what ports you want to map from the machine running the container to the container itself. Here we're picking 8080 because that's the port we have configured Bottle to run on.

Another COPY statement in order to copy all our code in our repo to the /code dir on the container and then finally...

CMD. This is more nuanced and should probably be just used to specify arguments to an ENTRYPOINT statement, but we're just looking at getting up and running. This can be considered to be the command run when the container is started. Here we're using a make rule to start our server.

Now our Dockerfile is all made, lets build and run it!

docker build -t conorjf/via-web .
docker run conorjf/via-web:latest

Simple! the docker build -t <tag_name> . uses the Dockerfile in the current directory (and the context of the current directory) to build your image with the tag tag_name. docker run will then run the image and tag you specify. All going well, your image should now be accessible on your localhost and you can drop into a shell on it by running docker exec -it conorjf/via-web bash and do whatever looking around you'd like to see what kind of a container you've built!

I am a big advocate of learning just enough to get started with something and when you are using it, you can begin to discover the nuance and best practices in a breadth-first way wherever your curiosity points. I highly recommend checking out the Dockerfile Best Practices. Please leave me any feedback at all as I have never written content like this and would like to make it as informative as possible!