How do you keep docker/podman containers up-to-date?

Here is how I did it:

docker run -d \
  --name watchtower \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -e WATCHTOWER_NOTIFICATION_URL=matrix://username:[email protected]/?rooms=!ROOMID:domain.org \
  -e WATCHTOWER_NOTIFICATION_TEMPLATE="{{range .}}[WatchTower] ({{.Level}}): {{.Message}}{{println}}{{end}}" \
  containrrr/watchtower

Edit: I created a pull request to the WatchTower documentation, here: https://github.com/containrrr/watchtower/pull/1690

I use watchtower and hope nothing will break. I never read breaking changes.

When an issue happens, I just search the internet or change the tag to a known working version until the issue is resolved.

I can afford to have my server down for a few days. It’s not critical to me.

It kind of depends on what are your priorities. In my experience it's usually much easier to upgrade to latest version from previous version, than to jump couple versions ahead, because you didn't have time doing upgrades recently..

When you think about it, from the development point of view, the upgrade from previous to last version is the most tested path. The developers of the service probably did exactly this upgrade themselves. Many users probably did the same and reported bugs. When you're upgrading from version released many months ago to the current stable, you might be the only one with such combination of versions. The devs are also much more likely to consider all the changes that were introduced between the latest versions.

If you encounter issue upgrading, how many people will experience the same problem with your specific versions combination? How likely are you to see issue on GitHub compared to a bunch of people that are always upgrading to latest?

Also moving between latest versions, there's only limited set of changes to consider if you encounter issues. If you jumped 30 versions ahead, you might end up spending quite some time figuring out which version introduced the breaking change.

Also no matter how carefully you look at it, there's always a chance that the upgrade fails and you'll have to rollback. So if you don't mind a little downtime, you can just let the automation do the job and at worst you'll do the rollback from backup.

It's also pretty good litmus test. If service regularly breaks when upgrading to latest without any good reason, perhaps it isn't mature enough yet.

We're obviously talking about home lab where time is sometimes limited, but some downtime usually not a problem.

They're usually clearly documented in support forums by people saying "MY STUFF WON'T BOOT PLESE HALP"

It depends on the project. If the project doesn't make an effort to highlight them I would consider using a different one.

But any decent OSS will make a good change log for their updates that you can read.

There’s plenty of tutorials out there for it. A quick DuckDuckGo search turned up this as one of the first results, but the theory is the same if you wanted to bundle ‘arr containers instead of nginx/whatever. https://www.digitalocean.com/community/tutorials/workflow-multiple-containers-docker-compose

Essentially you create docker compose file for services, within which you have as many containers as you want set up like you would any other compose file. You ‘docker compose pull’ and ‘docker compose up -d’ to update/install just like you would for individual docker container, but it does them all together. It sounds like others in the thread have more automated someone with services dedicated to watching for updates and running those automatically but I just look for a flag in the app saying there’s an update available and pull/ up -d whenever it’s convenient/I realize there’s an update.

Well, most projects publish their dockerfiles so you could take ans rebuild them with the tags you want. And all the building can be built into a CI/CD pipeline so you just have to make a new push with the latest versions.

I should make something like that.

Last time it happened was with one of the newer Nextcloud updates. It was a bit of trial and error, but I eventually went back to a version that worked and I could fix the underlying issue. There should be a list of version tags either on dockerhub or GitHub that list all versions that have been pushed to live and are available to pull

You weren't asking me, but I've used K8s professionally and my take is that K8s is only suited for business environments, ones with a good number of devs and users and complex deployment/runtime needs. You're not finding that "aha!" with K8s for self-hosting at home because, simply put, you are not the target market. It's way overkill for your needs. The one exception is if you're trying to learn it at home so you can use it in a corporate environment. In that case, go wild. But just don't expect it to make sense for most modest home lab or self-hosting needs.

For sure, just stacking turtles all the way down... 🐢 It's definitely overkill for a home lab, but I'm an infra engineer, and it's what I use daily, so setting it up was worth it because I'm already really familiar with the stack. That said, I do absolutely love having declarative setup at home because I'll sometimes go months without touching things. Before I spent the time to make it declarative, I'd frequently forget how I set certain things up and waste time redoing, or figuring out where I left off. Now I just check commit history and I'm always moving forward.