Selfhosted @lemmy.world MTK @lemmy.world 3d ago

Rant! 100GB Log file in Nextcloud.

I set it to debug at somepoint and forgot maybe? Idk, but why the heck does the default config of the official Docker is to keep all logs, forever, in a single file woth no rotation?

Feels like 101 of log files. Anyway, this explains why my storage recipt grew slowly but unexpectedly.

74 comments

You should always setup logrotate. Yes the good old Linux logrotate...
- I don't disagree that logrotate is a sensible answer here, but making that the responsibility of the user is silly.
  
  Are you crazy? I understand that we are used to dumbed down stuff, but come on...
  
  Rotating logs is in the ABC of any sysadmin, even before backups.
  
  First, secure your ssh logins, then secure your logs, then your fail2ban then your backups...
  
  To me, that's in the basic stuff you must always ensure.
- We should each not have to configure log rotation for every individual service. That would require identify what and how it logs data in the first place, then implementing a logrotate config. Services should include a reasonable default in logrotate.d as part of their install package.
  
  Docker services should let docker handle it, and the user could then manage it through Docker or forward to some other logging service (syslog, systemd, etc). Processes in containers shouldn't touch rotation or anything, just log levels and maybe which types of logs go to stdout vs stderr.
  
  Ideally yes, but I've had to do this regularly for many services developed both in-house and out of house.
  
  Solve problems, and maybe share your work if you like, I think we all appreciate it.
  
  Agreed, but going container route those nice basic practices are dead.
  
  And also, being mextcloud a php service, of can't by definition ship with a logrotate config too, because its never packaged by your repo.
for some helpful config, the below is the logging config I have and logs have never been an issue.

You can even add 'logfile' => '/some/location/nextcloud.log', to get the logs in a different place

'logtimezone' => 'UTC', 'logdateformat' => 'Y-m-d H:i:s', 'loglevel' => 2, 'log_rotate_size' => 52428800,
Imho it’s because docker does away with (abstracts?) many years of sane system administration principles (like managing logfile rotations) that you are used to when you deploy bare metal on a Debian box. It’s a brave new world.
- It's because with docker you don't need to do log files. Logging should be to stdout, and you let the host, orchestration framework, or whoever is running the container so logs however they want to. The container should not be writing log files in the first place, containers should be immutable except for core application logic.
  
  At worst it saves in the config folder/volume where persistent stuff should be.
  
  Docker stores that stdout per default in a log file in var/lib/docker/containers/...
  
  Good point!
- I disagree with this, container runtimes are a software like all others where logging needs to be configured. You can do so in the config of the container runtime environment.
  
  Containers actually make this significantly easier because you only need to configure it once and it will be applied to all containers.
  
  You are right and as others have pointed out correctly it’s Nextcloud not handling logging correctly in a containerized environment. I was ranting more about my dislike of containers in general, even though I use the technology (correctly) myself. It’s because I am already old on the scale of technology timelines.
  
  Or you can forward to your system logger, like syslog or systemd.
  
  But then projects like NextCloud do it all wrong by using a file. Just log to stdout and I'll manage the rest.
- Or you can use Podman, which integrates nicely with Systemd and also utilizes all the regular system means to deal with log files and so on.
  
  Good suggestion, although I do feel it always comes back to this “many ways to do kind of the same thing” that surrounds the Linux ecosystem. Docker, podman, … some claim it’s better, I hear others say it’s not 100% compatible all the time. My point being more fragmentation.
  
  Does podman do the Docker networking thing where I can link containers together without exposing ports to the rest of the system? I like my docker compose setup where I only expose caddy (TLS trunking) and Jellyfin (because my TV fails connecting w/ TLS).
Everything I hear about Nextcloud scares me away from messing with it.
- Yes. And then I read press announcements like this https://nextcloud.com/blog/press_releases/nextcloud-procolix-partner-netherlands/
- I'm considering switching to Seafile. I just need documents to sync and Collabora integration, and it seems to do both without dealing with PHP nonsense.
- Just use the official Docker AIO and it is very, very little trouble. It's by far the easiest way to use Nextcloud and the related services like Collabora and Talk.
  
  The ~~price rboem~~ problem is that the log file is inside the container in the www folder.
  
  Edit: typo
- If you only use it for files, the only thing it's good for imho. it's awesome! :)
- I stopped using Nextcloud a couple of years ago after it corrupted my encrypted storage. I'm giving it a try again because of political emergency. But we sure need a long term replacement. Written in Rust or some other sane language.
  
  Nc is great, it really is amazing that it is foss. Sure it isn't the slickest or fastest, and it does need more maintenance than most foss services, but it is also more complex and has so many great features.
  
  I really recommend nc, 99% of the time it just works for me. It just seems that their docker was done pretty poorly imo, but still it just works most of the time.
  
  I've considered writing my own, but it's a ton of work. Even for my very basic use case of a file browser that offloads all edits to Collabora CODE. I had a basic system started in Go some years back, but bailed when I got a basic setup working (just file ops).
  
  Maybe I'll give it a shot again. I mostly use Rust now, and I'm kind of stalling on my P2P Lemmy idea anyway. I really don't like PHP and I don't use many of the Nextcloud features anyway. I just want Google Drive w/ LibreOffice or OnlyOffice.
  
  My NC setup "just works" though. So I'm not super motivated to replace it.
  
  Edit: looks like Seafile may do the trick.
Wow, thanks for the heads up! I use Nextcloud AIO and backups take VERY long. I need to check about those logs!

Don't know if I'm just lucky or what, but it's been working really well for me and takes good care of itself for the most part. I'm a little shocked seeing so many complaints in this thread because elsewhere on the Internet that's the go-to method.
- It can be fidgety, especially if you stray from the main instructions, generally I do think it's okay, but also updates break it a bit every now and again.
  
  Yeah, anything that involves a bunch of ~~complicated relationship interaction between~~ PHP scripts I just don't mess with too much.
  
  Right now I'm hosting it through Docker on top of OpenMediaVault which is hosted on Proxmox.
  
  If an update absolutely borks NextCloud and for some reason its BorgBackup function doesn't work, I can at least hope to count on the ProxMox snapshot of the whole volume!
  
  And besides that, I don't actually store anything essential in NextCloud's volume itself. It's all an external mount that I could browse with any file explorer, so worst case, I'd just lose a lot of convenience. :p
Reminds me of when my Jellyfin container kept growing its log because of something watchtower related. Think it ended up at 100GB before I noticed. Not even debug, just failed updates I think. It's been a couple of months.
- Well that's not jellyfins faults but rather watchtower...
101 of log files

is to configure it yourself
- Look, defaults are a thing and if your defaults suck then you've made a mistake and if your default is to save a 100GB of log file in one file then something is wrong. The default in Dockers should just be not to save any log files on the persistent volumes.
  
  Exactly. It should just write to stdout and let whatever is running it manage it.
Feels like blaming others for not paying attention.
- Persistent storage should never be used for logging in docker. Nextcloud is one of the worst offenders of breaking docker conventions I've found, this is just one of the many ways they prove they don't understand docker.
  
  Logs should simply be logged to stdout, which will be read by docker or by a logging framework. There should never be "log files" for a container, as it should be immutable, with persistent volumes only being used for configuration or application state.
  
  The AIO container is so terrible, like, that's not how you're supposed to use Docker.
  It's unclear whether OP was using that or saner community containers, might just be the AIO one.

74 comments