[BE] [Bug]: 0.18.0; 0.18.1-rc.9 - Timeouts when submitting images to pictrs container #3447
[BE] [Bug]: 0.18.0; 0.18.1-rc.9 - Timeouts when submitting images to pictrs container #3447
Requirements Is this a bug report? For questions or discussions use https://lemmy.ml/c/lemmy_support Did you check to see if this issue already exists? Is this only a single bug? Do not put multipl...
Requirements
- [X] Is this a bug report? For questions or discussions use https://lemmy.ml/c/lemmy_support
- [X] Did you check to see if this issue already exists?
- [X] Is this only a single bug? Do not put multiple bugs in one issue.
- [X] Is this a backend issue? Use the lemmy-ui repo for UI / frontend issues.
Summary
When attempting to submit images on a self-hosted Lemmy instance to the attached pictrs container it consistently times out. I have once successfully managed to get a small image successfully uploaded to post.
Of note: This is related to my attempt to build up a traefik-based reverse proxy configuration that can be contributed; See:
- https://gitlab.com/Matt.Jolly/lemmy-traefik-docker - Lemmy for Traefik
- https://gitlab.com/Matt.Jolly/traefik-grafana-prometheus-docker/ - traefik turnkey service
Steps to Reproduce
Attempt to submit any image to any community on a lemmy instance provisioned as below:
Docker Stack:
version: "3.9" x-logging: &default-logging driver: journald options: tag: "{{.Name}}[{{.ID}}]" services: web: image: dessalines/lemmy:0.18.0 restart: always logging: *default-logging environment: - RUST_LOG="warn,lemmy_server=info,lemmy_api=info,lemmy_api_common=info,lemmy_api_crud=info,lemmy_apub=info,lemmy_db_schema=info,lemmy_db_views=info,lemmy_db_views_actor=info,lemmy_db_views_moderator=info,lemmy_routes=info,lemmy_utils=info,lemmy_websocket=info" volumes: - ./lemmy.hjson:/config/config.hjson depends_on: - db networks: - internal - traefik-network deploy: labels: - traefik.enable=true - traefik.http.routers.lemmy-http.rule=Host(`lemmy.srcfiles.zip`) && (PathPrefix(`/api`) || PathPrefix(`/pictrs`) || PathPrefix(`/feeds`) || PathPrefix(`/nodeinfo`) || PathPrefix(`/.well-known`) || Method(`POST`) || HeaderRegexp(`Accept`, `^[Aa]pplication/.*`)) - traefik.http.routers.lemmy-https.rule=Host(`lemmy.srcfiles.zip`) && (PathPrefix(`/api`) || PathPrefix(`/pictrs`) || PathPrefix(`/feeds`) || PathPrefix(`/nodeinfo`) || PathPrefix(`/.well-known`) || Method(`POST`) || HeaderRegexp(`Accept`, `^[Aa]pplication/.*`)) - traefik.http.routers.lemmy-http.entrypoints=web - traefik.http.routers.lemmy-https.entrypoints=websecure - traefik.http.routers.lemmy-http.middlewares=redirect-https@file - traefik.http.routers.lemmy-https.tls.certresolver=letsencrypt - traefik.http.routers.lemmy-https.service=lemmy # Define the Traefik service for this docker container - traefik.http.services.lemmy.loadbalancer.server.port=8536 web-frontend: image: dessalines/lemmy-ui:0.18.0 environment: - LEMMY_UI_LEMMY_INTERNAL_HOST=web:8536 - LEMMY_UI_LEMMY_EXTERNAL_HOST=lemmy.srcfiles.zip - LEMMY_HTTPS=true depends_on: - web restart: always logging: *default-logging networks: - internal - traefik-network deploy: labels: - traefik.enable=true - traefik.http.routers.lemmy-static-http.rule=Host(`lemmy.srcfiles.zip`) - traefik.http.routers.lemmy-static-https.rule=Host(`lemmy.srcfiles.zip`) - traefik.http.routers.lemmy-static-http.entrypoints=web - traefik.http.routers.lemmy-static-https.entrypoints=websecure - traefik.http.routers.lemmy-static-http.middlewares=redirect-https@file - traefik.http.routers.lemmy-static-https.tls.certresolver=letsencrypt - traefik.http.routers.lemmy-static-https.service=lemmy-static # Define the Traefik service for this docker container - traefik.http.services.lemmy-static.loadbalancer.server.port=1234 db: image: postgres:15-alpine hostname: db environment: - POSTGRES_USER=lemmy - POSTGRES_PASSWORD=DB_PASS - POSTGRES_DB=lemmy volumes: - db:/var/lib/postgresql/data restart: always logging: *default-logging networks: - internal pictrs: image: asonix/pictrs:0.4.0-rc.9 # this needs to match the pictrs url in lemmy.hjson hostname: pictrs # we can set options to pictrs like this, here we set max. image size and forced format for conversion # entrypoint: /sbin/tini -- /usr/local/bin/pict-rs -p /mnt -m 4 --image-format webp environment: #- PICTRS_OPENTELEMETRY_URL=http://otel:4137 - PICTRS__API_KEY=API_KEY - RUST_LOG=debug - RUST_BACKTRACE=full - PICTRS__MEDIA__VIDEO_CODEC=vp9 - PICTRS__MEDIA__GIF__MAX_WIDTH=256 - PICTRS__MEDIA__GIF__MAX_HEIGHT=256 - PICTRS__MEDIA__GIF__MAX_AREA=65536 - PICTRS__MEDIA__GIF__MAX_FRAME_COUNT=400 user: 991:991 volumes: - ./volumes/pictrs:/mnt:Z restart: always logging: *default-logging networks: - internal networks: traefik-network: external: true internal: volumes: db: ``` Lemmy Config: ```json { # settings related to the postgresql database database: { # Configure the database by specifying parts of a URI # Username to connect to postgres user: "lemmy" # Password to connect to postgres password: "DB_PASSWORD" # Host where postgres is running host: "db" # Port where postgres can be accessed port: 5432 # Name of the postgres database for lemmy database: "lemmy" # Maximum number of active sql connections pool_size: 10 } # Settings related to activitypub federation # Pictrs image server configuration. pictrs: { # Address where pictrs is available (for image hosting) url: "http://pictrs:8080/" # Set a custom pictrs API key. ( Required for deleting images ) api_key: "API_KEY" } # Email sending configuration. All options except login/password are mandatory email: { # Hostname and port of the smtp server smtp_server: "localhost:25" # Login name for smtp server smtp_login: "string" # Password to login to the smtp server smtp_password: "string" # Address to send emails from, eg "[email protected]" smtp_from_address: "[email protected]" # Whether or not smtp connections should use tls. Can be none, tls, or starttls tls_type: "none" } # Parameters for automatic configuration of new instance (only used at first start) setup: { # Username for the admin user admin_username: "Admin_Username" # Password for the admin user. It must be at least 10 characters. admin_password: "Admin_Password" # Name of the site (can be changed later) site_name: "Srcfiles.zip" # Email for the admin user (optional, can be omitted and set later through the website) admin_email: "[email protected]" } # the domain name of your instance (mandatory) hostname: "lemmy.srcfiles.zip" # Address where lemmy should listen for incoming requests bind: "0.0.0.0" # Port where lemmy should listen for incoming requests port: 8536 # Whether the site is available over TLS. Needs to be true for federation to work. tls_enabled: true # The number of activitypub federation workers that can be in-flight concurrently worker_count: 0 # The number of activitypub federation retry workers that can be in-flight concurrently retry_count: 0 } ``` ### Technical Details Logs for all containers redirected to journald: ``` Jul 02 00:24:25 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal dockerd[1529]: time="2023-07-02T00:24:25.413244565Z" level=info msg="initialized VXLAN UDP port to 4789 " Jul 02 00:24:27 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.5qgz5t3iv4u7knbysp8mutc5x[78b59192fec9][1529]: thread 'main' panicked at 'Error connecting to postgres://lemmy:8KLH%266zh%25Pk%25RfnNs%23o6X@db:5432/lemmy: could not translate host name "db" to address: Name does not resolve Jul 02 00:24:27 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.5qgz5t3iv4u7knbysp8mutc5x[78b59192fec9][1529]: ', crates/db_schema/src/utils.rs:161:56 Jul 02 00:24:27 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.5qgz5t3iv4u7knbysp8mutc5x[78b59192fec9][1529]: note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace Jul 02 00:24:27 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal dockerd[1529]: time="2023-07-02T00:24:27.223705239Z" level=info msg="ignoring event" container=78b59192fec975b15eb5bd56495428191d43c5e681aeaf1650f8f6649d923c6e module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" Jul 02 00:24:28 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal dockerd[1529]: time="2023-07-02T00:24:28.067529555Z" level=error msg="fatal task error" error="task: non-zero exit (101)" module=node/agent/taskmanager node.id=eqmkexmxt9n5sykvr336cgcxn service.id=ly7nzeslpzoldz6x4ka1w7q8n task.id=5qgz5t3iv4u7knbysp8mutc5x Jul 02 00:24:29 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web-frontend.1.voxru8b5oq76e8wxvtqxooayb[74ab5638fae6][1529]: Inferno is in development mode. Jul 02 00:24:29 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web-frontend.1.voxru8b5oq76e8wxvtqxooayb[74ab5638fae6][1529]: http://0.0.0.0:1234 Jul 02 00:24:31 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_db.1.gcomi8rfl05c01d2wsckvdf7l[7e492452ab59][1529]: Jul 02 00:24:31 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_db.1.gcomi8rfl05c01d2wsckvdf7l[7e492452ab59][1529]: PostgreSQL Database directory appears to contain a database; Skipping initialization Jul 02 00:24:31 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_db.1.gcomi8rfl05c01d2wsckvdf7l[7e492452ab59][1529]: Jul 02 00:24:31 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_db.1.gcomi8rfl05c01d2wsckvdf7l[7e492452ab59][1529]: 2023-07-02 00:24:31.517 UTC [1] LOG: starting PostgreSQL 15.3 on x86_64-pc-linux-musl, compiled by gcc (Alpine 12.2.1_git20220924-r10) 12.2.1 20220924, 64-bit Jul 02 00:24:31 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_db.1.gcomi8rfl05c01d2wsckvdf7l[7e492452ab59][1529]: 2023-07-02 00:24:31.527 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432 Jul 02 00:24:31 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_db.1.gcomi8rfl05c01d2wsckvdf7l[7e492452ab59][1529]: 2023-07-02 00:24:31.528 UTC [1] LOG: listening on IPv6 address "::", port 5432 Jul 02 00:24:31 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_db.1.gcomi8rfl05c01d2wsckvdf7l[7e492452ab59][1529]: 2023-07-02 00:24:31.531 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" Jul 02 00:24:31 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_db.1.gcomi8rfl05c01d2wsckvdf7l[7e492452ab59][1529]: 2023-07-02 00:24:31.538 UTC [25] LOG: database system was shut down at 2023-07-02 00:23:47 UTC Jul 02 00:24:31 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_db.1.gcomi8rfl05c01d2wsckvdf7l[7e492452ab59][1529]: 2023-07-02 00:24:31.549 UTC [1] LOG: database system is ready to accept connections Jul 02 00:24:33 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_pictrs.1.rqbyzp7wpl85mrvssos32xv08[2483c9bdda3a][1529]: 2023-07-02T00:24:33.996938Z INFO actix_server::builder: starting 1 workers Jul 02 00:24:33 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_pictrs.1.rqbyzp7wpl85mrvssos32xv08[2483c9bdda3a][1529]: 2023-07-02T00:24:33.997291Z INFO actix_server::server: Actix runtime found; starting in Actix runtime Jul 02 00:24:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:24:34.421929Z INFO lemmy_db_schema::utils: Running Database migrations (This may take a long time)... Jul 02 00:24:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:24:34.430160Z INFO lemmy_db_schema::utils: Database migrations complete. Jul 02 00:24:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:24:34.460890Z INFO lemmy_server::code_migrations: Running user_updates_2020_04_02 Jul 02 00:24:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:24:34.471034Z INFO lemmy_server::code_migrations: 0 person rows updated. Jul 02 00:24:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:24:34.472553Z INFO lemmy_server::code_migrations: Running community_updates_2020_04_02 Jul 02 00:24:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:24:34.478290Z INFO lemmy_server::code_migrations: 0 community rows updated. Jul 02 00:24:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:24:34.479144Z INFO lemmy_server::code_migrations: Running post_updates_2020_04_03 Jul 02 00:24:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:24:34.482847Z INFO lemmy_server::code_migrations: 0 post rows updated. Jul 02 00:24:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:24:34.483502Z INFO lemmy_server::code_migrations: Running comment_updates_2020_04_03 Jul 02 00:24:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:24:34.493459Z INFO lemmy_server::code_migrations: 0 comment rows updated. Jul 02 00:24:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:24:34.494267Z INFO lemmy_server::code_migrations: Running private_message_updates_2020_05_05 Jul 02 00:24:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:24:34.496764Z INFO lemmy_server::code_migrations: 0 private message rows updated. Jul 02 00:24:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:24:34.497262Z INFO lemmy_server::code_migrations: Running post_thumbnail_url_updates_2020_07_27 Jul 02 00:24:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:24:34.498811Z INFO lemmy_server::code_migrations: 0 Post thumbnail_url rows updated. Jul 02 00:24:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:24:34.499369Z INFO lemmy_server::code_migrations: Running apub_columns_2021_02_02 Jul 02 00:24:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:24:34.502009Z INFO lemmy_server::code_migrations: Running instance_actor_2021_09_29 Jul 02 00:24:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:24:34.519404Z INFO lemmy_server::code_migrations: Running regenerate_public_keys_2022_07_05 Jul 02 00:24:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:24:34.532065Z INFO lemmy_server::code_migrations: Running initialize_local_site_2022_10_10 Jul 02 00:24:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: federation enabled, host is lemmy.srcfiles.zip Jul 02 00:24:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: Starting http server at 0.0.0.0:8536 Jul 02 00:24:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:24:34.582458Z INFO lemmy_server::scheduled_tasks: Updating active site and community aggregates ... Jul 02 00:24:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:24:34.629545Z INFO lemmy_server::scheduled_tasks: Done. Jul 02 00:24:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:24:34.629578Z INFO lemmy_server::scheduled_tasks: Updating hot ranks for all history... Jul 02 00:24:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:24:34.641287Z INFO lemmy_server::scheduled_tasks: Done. Jul 02 00:24:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:24:34.641314Z INFO lemmy_server::scheduled_tasks: Updating banned column if it expires ... Jul 02 00:24:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:24:34.643321Z INFO lemmy_server::scheduled_tasks: Clearing old activities... Jul 02 00:24:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:24:34.650950Z INFO lemmy_server::scheduled_tasks: Done. Jul 02 00:25:00 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:25:00.666966Z INFO lemmy_server::scheduled_tasks: Updating hot ranks for last week... Jul 02 00:25:00 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:25:00.678157Z INFO lemmy_server::scheduled_tasks: Done. Jul 02 00:25:08 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_pictrs.1.rqbyzp7wpl85mrvssos32xv08[2483c9bdda3a][1529]: 2023-07-02T00:25:08.573500Z INFO HTTP request{http.method=POST http.route=/image http.flavor=1.1 http.scheme=https http.host=lemmy.srcfiles.zip http.client_ip=119.xxx.xxx.251 http.user_agent=Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/114.0 http.target=/image otel.name=HTTP POST /image otel.kind="server" request_id=62733fb4-d8a0-467a-84ba-17b8d6817792}: tracing_actix_web::root_span_builder: new Jul 02 00:25:18 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:25:18.573847Z WARN Error encountered while processing the incoming HTTP request: lemmy_server::root_span_builder: Request error: error sending request for url (http://pictrs:8080/image): operation timed out Jul 02 00:25:18 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: Reqwest(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("pictrs")), port: Some(8080), path: "/image", query: None, fragment: None }, source: TimedOut }) Jul 02 00:25:31 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_pictrs.1.rqbyzp7wpl85mrvssos32xv08[2483c9bdda3a][1529]: 2023-07-02T00:25:31.753707Z INFO HTTP request{http.method=POST http.route=/image http.flavor=1.1 http.scheme=https http.host=lemmy.srcfiles.zip http.client_ip=119.xxx.xxx.251 http.user_agent=Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/114.0 http.target=/image otel.name=HTTP POST /image otel.kind="server" request_id=c0ad7192-c565-41df-9117-a7c11158e02d}: tracing_actix_web::root_span_builder: new Jul 02 00:25:41 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:25:41.751831Z WARN Error encountered while processing the incoming HTTP request: lemmy_server::root_span_builder: Request error: error sending request for url (http://pictrs:8080/image): operation timed out Jul 02 00:25:41 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: Reqwest(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("pictrs")), port: Some(8080), path: "/image", query: None, fragment: None }, source: TimedOut }) Jul 02 00:26:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal dockerd[1529]: time="2023-07-02T00:26:34.072053549Z" level=info msg="NetworkDB stats ip-172-xxx-xxx-210.ap-southeast-2.compute.internal(b4b857b480fa) - netID:q52fvnhhdx2bsx58a1up9eejd leaving:false netPeers:1 entries:2 Queue qLen:0 netMsg/s:0" Jul 02 00:26:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal dockerd[1529]: time="2023-07-02T00:26:34.072124607Z" level=info msg="NetworkDB stats ip-172-xxx-xxx-210.ap-southeast-2.compute.internal(b4b857b480fa) - netID:z41ch1tirqlsl7ewldco9yr48 leaving:true netPeers:0 entries:9 Queue qLen:0 netMsg/s:0" Jul 02 00:26:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal dockerd[1529]: time="2023-07-02T00:26:34.072140102Z" level=info msg="NetworkDB stats ip-172-xxx-xxx-210.ap-southeast-2.compute.internal(b4b857b480fa) - netID:z0mkh59efda2n8owy2k0abtem leaving:false netPeers:1 entries:11 Queue qLen:0 netMsg/s:0" Jul 02 00:26:34 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal dockerd[1529]: time="2023-07-02T00:26:34.072152674Z" level=info msg="NetworkDB stats ip-172-xxx-xxx-210.ap-southeast-2.compute.internal(b4b857b480fa) - netID:y3r7f2sf7ztu3kol77irag4c9 leaving:false netPeers:1 entries:17 Queue qLen:0 netMsg/s:0" Jul 02 00:27:55 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_pictrs.1.rqbyzp7wpl85mrvssos32xv08[2483c9bdda3a][1529]: 2023-07-02T00:27:55.387654Z INFO HTTP request{http.method=POST http.route=/image http.flavor=1.1 http.scheme=https http.host=lemmy.srcfiles.zip http.client_ip=119.xxx.xxx.251 http.user_agent=Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/114.0 http.target=/image otel.name=HTTP POST /image otel.kind="server" request_id=b722387b-a05f-4577-a5fd-b9149a1ad2e0}: tracing_actix_web::root_span_builder: new Jul 02 00:28:05 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:28:05.388191Z WARN Error encountered while processing the incoming HTTP request: lemmy_server::root_span_builder: Request error: error sending request for url (http://pictrs:8080/image): operation timed out Jul 02 00:28:05 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: Reqwest(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("pictrs")), port: Some(8080), path: "/image", query: None, fragment: None }, source: TimedOut }) Jul 02 00:28:12 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_pictrs.1.rqbyzp7wpl85mrvssos32xv08[2483c9bdda3a][1529]: 2023-07-02T00:28:12.300053Z INFO HTTP request{http.method=POST http.route=/image http.flavor=1.1 http.scheme=https http.host=lemmy.srcfiles.zip http.client_ip=119.xxx.xxx.251 http.user_agent=Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/114.0 http.target=/image otel.name=HTTP POST /image otel.kind="server" request_id=31a00146-27cc-4818-aefe-e106fe20e4b1}: tracing_actix_web::root_span_builder: new Jul 02 00:28:22 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:28:22.301154Z WARN Error encountered while processing the incoming HTTP request: lemmy_server::root_span_builder: Request error: error sending request for url (http://pictrs:8080/image): operation timed out Jul 02 00:28:22 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: Reqwest(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("pictrs")), port: Some(8080), path: "/image", query: None, fragment: None }, source: TimedOut }) Jul 02 00:28:31 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_pictrs.1.rqbyzp7wpl85mrvssos32xv08[2483c9bdda3a][1529]: 2023-07-02T00:28:31.002835Z INFO HTTP request{http.method=POST http.route=/image http.flavor=1.1 http.scheme=https http.host=lemmy.srcfiles.zip http.client_ip=119.xxx.xxx.251 http.user_agent=Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/114.0 http.target=/image otel.name=HTTP POST /image otel.kind="server" request_id=00d20eec-49d0-4e97-b143-89d6cccecd0b}: tracing_actix_web::root_span_builder: new Jul 02 00:28:41 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: 2023-07-02T00:28:41.002705Z WARN Error encountered while processing the incoming HTTP request: lemmy_server::root_span_builder: Request error: error sending request for url (http://pictrs:8080/image): operation timed out Jul 02 00:28:41 ip-172-xxx-xxx-210.ap-southeast-2.compute.internal lemmy_web.1.3zd37bj4pr2pgycd9ph4obgnz[5e22b6ec01a7][1529]: Reqwest(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("pictrs")), port: Some(8080), path: "/image", query: None, fragment: None }, source: TimedOut }) ``` ### Version BE 0.18.0 ### Lemmy Instance URL lemmy.srcfiles.zip *Originally posted by [Kangie](https://github.com/Kangie) in [#3447](https://github.com/LemmyNet/lemmy/issues/3447)*
I'm seeing this with a similar set-up behind nginx when uploading a small (~1.5MB) video. Static images up to 25MB seem fine (at least that was what I was able to test).
It looks like pict-rs is transcoding video to vp9 (even when the video is already encoded in vp9). The very short timeout lemmy has when forwarding to pict-rs seems to cause this to fail.
pict-rs does not appear to have an option to not transcode video files (which I have attempted to raise on the pict-rs repo) however larger files (depending on the instance max upload size) could still cause timeouts.
This causes all video uploads to fail.
Looking around it is possibly intended for lemmy to not allow video uploads, however if this is case it really wasn't obvious to me.
1 0 Reply