How can I make my Lemmy server delete cached images?
How can I make my Lemmy server delete cached images?
The storage usage is at 340GB currently, which is a lot and it's rapidly increasing. I use Backblaze B2 for my storage. Here is my docker compose file: x-logging: &default-logging driver: "json-file" options: max-size: "50m" max-file: "4"
services:
proxy:
image: docker.io/library/nginx
volumes:
- ./nginx_internal.conf:/etc/nginx/nginx.conf:ro,Z
- ./proxy_params:/etc/nginx/proxy_params:ro,Z
restart: always
logging: *default-logging
depends_on:
- pictrs
- lemmy-ui
labels:
- traefik.enable=true
- traefik.http.routers.http-lemmy.entryPoints=http
- traefik.http.routers.http-lemmy.rule=Host(`gregtech.eu`)
- traefik.http.middlewares.https_redirect.redirectscheme.scheme=https
- traefik.http.middlewares.https_redirect.redirectscheme.permanent=true
- traefik.http.routers.http-lemmy.middlewares=https_redirect
- traefik.http.routers.https-lemmy.entryPoints=https
- traefik.http.routers.https-lemmy.rule=Host(`gregtech.eu`)
- traefik.http.routers.https-lemmy.service=lemmy
- traefik.http.routers.https-lemmy.tls=true
- traefik.http.services.lemmy.loadbalancer.server.port=8536
- traefik.http.routers.https-lemmy.tls.certResolver=le-ssl
lemmy:
image: dessalines/lemmy:0.19.8
hostname: lemmy
restart: always
logging: *default-logging
volumes:
- ./lemmy.hjson:/config/config.hjson:Z
depends_on:
- postgres
- pictrs
networks:
- default
- database
lemmy-ui:
image: dessalines/lemmy-ui:0.19.8
volumes:
- ./volumes/lemmy-ui/extra_themes:/app/extra_themes:Z
depends_on:
- lemmy
restart: always
logging: *default-logging
environment:
- LEMMY_UI_LEMMY_INTERNAL_HOST=lemmy:8536
- LEMMY_UI_LEMMY_EXTERNAL_HOST=gregtech.eu
- LEMMY_UI_HTTPS=true
pictrs:
image: docker.io/asonix/pictrs:0.5
# this needs to match the pictrs url in lemmy.hjson
hostname: pictrs
# we can set options to pictrs like this, here we set max. image size and forced format for conversion
# entrypoint: /sbin/tini -- /usr/local/bin/pict-rs -p /mnt -m 4 --image-format webp
#entrypoint: /sbin/tini -- /usr/local/bin/pict-rs run --max-file-count 10 --media-max-file-size 500 --media-retention-proxy 10d --media-retention-variants 10d filesystem sled -p /mnt
user: 991:991
environment:
- PICTRS__STORE__TYPE=object_storage
- PICTRS__STORE__ENDPOINT=https://s3.eu-central-003.backblazeb2.com/
- PICTRS__STORE__BUCKET_NAME=gregtech-lemmy
- PICTRS__STORE__REGION=eu-central
- PICTRS__STORE__USE_PATH_STYLE=false
- PICTRS__STORE__ACCESS_KEY=redacted
- PICTRS__STORE__SECRET_KEY=redacted
- MEDIA__RETENTION__VARIANTS=4d
- MEDIA__RETENTION__PROXY=4d
#- PICTRS__MEDIA__IMAGE__FORMAT=webp
#- PICTRS__MEDIA__IMAGE__QUALITY__WEBP=50
#- PICTRS__MEDIA__ANIMATION__QUALITY=50
volumes:
- ./volumes/pictrs:/mnt:Z
restart: always
logging: *default-logging
postgres:
image: docker.io/postgres:16-alpine
hostname: postgres
volumes:
- ./volumes/postgres:/var/lib/postgresql/data:Z
#- ./customPostgresql.conf:/etc/postgresql.conf:Z
restart: always
#command: postgres -c config_file=/etc/postgresql.conf
shm_size: 256M
logging: *default-logging
environment:
- POSTGRES_PASSWORD=password
- POSTGRES_USER=lemmy
- POSTGRES_DB=lemmy
networks:
- database
postfix:
image: docker.io/mwader/postfix-relay
restart: "always"
logging: *default-logging
#pictrs-safety:
# image: ghcr.io/db0/pictrs-safety:v1.2.2
# hostname: pictrs-safety
# environment:
# ports:
# - "14051:14051"
# user: 991:991
# restart: always
# logging: *default-logging
# depends_on:
# - pictrs
networks:
default:
name: traefik_access
external: true
database:
Lemmy doesn't so much cache the images so much as "store them forever".
Regular post thumbnails are just "internal" images. I'm not sure if newer Lemmy versions log these internally like user uploads are, but assuming not, there's no direct way to deal with them.
Pict-rs keeps a true cache of image variants (alternate versions of images that were requested at different resolutions / formats). Those can be cleared programmatically:
curl -XDELETE -H "X-Api-token:YOUR_API_KEY" http://127.0.0.1:8080/internal/variantsWhere your pict-rs API port is exposed to localhost on port 8080. Also note that those variants will be re-created on demand.
One easy-ish way to get rid of images is to select from the
posttable for anythumbnail_urlthat starts with your instance. e.g.https://your-instance/pictrs/image/%s. You can filter that bypublisheddate if you only want to wipe out thumbnails for posts older than a year, for example.With that list, you can strip off the filename/alias (the uuid and extension) and pass each alias filename to a script that tells pict-rs to delete it (you need at least pict-rs 0.5 for this to work):
#!/bin/bash ALIAS="$1" API_KEY=FooBarAPIKeyIsFake curl -XPOST -H "X-Api-token:$API_KEY" http://127.0.0.1:8080/internal/delete?alias=$ALIASBelow 0.5, the only way to delete images fro pict-rs without the delete token was to use the purge endpoint. But that deletes all aliases and not just the one you want to delete since it does de-duplication under the hood.
If you want to get fancy so that image posts still render correctly, when building your list of thumbnails to delete, you can check if the value of the
urlcolumn is an image. If it is, grab the current thumbnail image to pass to the delete function and then update the thumbnail URL value to that of the post URL.Wrong community
Why? It seems on topic to me
Try [email protected] and the matrix channel mentioned there.