What do y’all use to monitor many linux servers?

I use my family. It has a simple volume based alert for when services are offline.

Until the UPS battery gets low and it beeps, and they look for a way to turn it off vs calling you. Yup.
It'll even automatically configured variable alert volumes corresponding to the importance of the service!

Node exporter, Prometheus and grafana

this is the way

Beszel. Probably the easiest tool of all the mentioned in this thread.

Seconded. My only complaint (which this might already be a feature I haven't found yet) is it doesn't seem to support multiple drives. But yes, it is shit easy to set up and has a beautiful UI
- Totally possible:
  
  https://beszel.dev/guide/additional-disks

uptime-kuma is what I use

We just recently started using zabbix. Open source and has a web interface to get a central view that can be accessed from wherever we allow it.

So far it's been great but er have had little time and so far have used only 1% of what it can do

Still, I'd recommend it. Super easy to install, seems light weight, has clients for any os you'd need, can send out alerts (we currently use pushover for that)

I personally use CheckMK.

Offer a free "Raw" version.
Can be deployed with docker.
OSS

One thing is that it can be a lot to take in at first and took me a while to get used to it.

CheckMk user here via omd.

I'm looking for something else after the upgrade.

Black interface isn't pretty for me and the old interface was "meh too hard so we ditched it".

One half of the project split has a shit supply chain and just doesn't meet the bar for upgrade requirements.

The other half of the project split is a mess to config in an automated desired-state setup. It's all edge-triggered manual bullshit. NO. ENOUGH.

I miss 1.2 .
checkmk user here. i can second the adjustment phase. i tend to ignore my servers but when something goes sideways it's awesome to have checkmk's structure in place.

Zabbix is pretty quick and easy. Many different services built in for sending notifications, along with your own custom (including webhooks). Fully customizable dashboard as well so you can add whatever you want/need at a glance.

I stopped by to say the same thing. I use Zabbix to monitor everything

Base ansible role installs Prometheus node exporter, configured with the text file collector
VM automations push DNS records so that the Prometheus dns-sd automatically discovers them
Ansible roles for add Cron jobs that generate metrics for specific systems and dump them for the text file collector
Grafana for dashboards
Karma as a UI in front of Prometheus alert manager

Any chance you'd be willing to share playbooks or point me toward any resources you used?

I use Ansible to manage config across all my workstations/servers but I haven't gotten around to automating log shipping yet or aggregating system metrics.
Cron jobs that generate metrics for specific systems and dump them for the text file collector

Details please
- https://github.com/prometheus/node_exporter?tab=readme-ov-file#textfile-collector - which makes node exporter watch a specific directory for files that contain metrics, then re-export them back to the central Prometheus server
  
  Some systems have their own metrics endpoints - instead of getting Prometheus to scrape these directly I set up a Cron job to curl these into files for node exporter - this means I don't need extra config in Prometheus to find the endpoints, and don't need to mess with firewall rules
  
  Other systems don't directly expose metrics in a format Prometheus can use - in this case I will write/find a script that can do the conversation, then either set it up to write the metrics file directly and run it on a Cron, or run it as a service and another Cron job to do the scrape

Cockpit.

I’ve been really enjoying Cockpit as well.
My cockpit experience has been unilaterally dreadful. I'm glad you're getting value out of it.
- How comes?
is cockpit on a server by server basis or can you monitor multiple servers with it?
- You can monitor multiple machines via the host switcher menu at the top-left of the screen: Multiple Machines

send alerts via http request

On this specifically you might want to check ntfy as it's quite easy to setup and can give you notifications on pretty much any device (including iOS) via your own infrastructure all the way down to basics e.g. SSE. That mean you can subscribe to a topic, e.g. servers per physical location, alert level, etc and only get the ones you need.

Node exporter, Prometheus and grafana

Otherwise much heavier but that's also what I use.
- same

Netdata is exactly what you're looking for. It's basically an all in one monitoring and and alerting suite that collects and analyzes data, and provides a gorgeous web dashboard for you to view.

You can also manually replicate this using Prometheus, Grafana and other tools, but that requires a much bigger effort to set up.

Edit: There's a public demo instance where you can try everything out: https://frankfurt.netdata.rocks/

Seconding Netdata, I've been using it for years. It's pretty great.
I think they went to 5 nodes max on the free version as of the last patch. That's damn near useless.
- Oh that sucks. I haven't used it personally in quite a while, since I switched to the Grafana stack
- Is that just for the centralized dashboard portion? I tend to use each instance of it standalone, and primarily for the email alerts.
- The five node limit is a dealbreaker for me too. I'm also annoyed the free version doesn't have any real built in options to secure data by default. I followed a TechnoTim tutorial to get the NetData/Prometheus/Grafana stuff setup but it was too limited and required too much manual effort.

Check out Netdata or Zabbix.

Nagios. It does depend on what you mean by monitor though. Nagios is good at telling you that "service A on host B" is down" but less useful for looking at things like performance trends. I particularly like being able to setup dependencies between services, so I get the alert for the root cause, and not all of the services that have gone down because of it.

telegraf, influxdb, grafana, and gatus

I just see if it works when I need it. If I’m at home it works. If I’m at work it may work. If I’ve left to travel it’s 95% definitely down and cannot be fixed. This works well!

Ages ago I used to use Webmin. I have no clue as how it stacks up to others nowadays.

Serverbox. https://github.com/lollipopkit/flutter_server_box

While I use LibreNMS as it uses SNMP for monitoring (which is pretty much available everywhere), I don't believe it has http alerts, but I know for a fact that it can send Telegram messages.

Not exactly what you're looking for, but I like using proxmox

I remember liking Sensu. We used it a little bit at my previous job, but I didn't get a chance to work with it much. I can't remember what we specifically used it for though. Sorry, wish I had more info for you.

Telegraf+influxdb+grafana is what I use at work, it is a multi purpose tool though, can be used to monitor EVERYTHING though

Hello,

I'm still using Nagios here. And for the availability of the services I'm using uptime-kuma (in a docker).

https://en.m.wikipedia.org/wiki/Nagios