Help building a Docker Compose for Nextcloud + aio-talk (HPB/Turn/STUN) + Traefik reverse proxy

ℹ️ Support 💬 Talk (spreed)
, , , , ,
1

Hi everyone,

I’m trying to set up a Nextcloud instance with Nextcloud Talk HPB (High Performance Backend) using the aio-talk all-in-one image, along with Traefik v2 as a reverse proxy. My goal is to have a production-ready setup with:

  • Nextcloud for files, users, and apps

  • Talk using aio-talk as HPB, TURN, and STUN server

  • Traefik handling HTTPS with Let’s Encrypt and proper WebSocket routing

  • All containers running in Docker Compose

I already have a working Docker Compose with separate nextcloud and mysql containers, plus a nextcloud-talk-hpb service using the aio-talk image, and Traefik configured. I also have scripts for:

  • Adding trusted domains dynamically (10-dynamic-app-config.sh)

  • Installing the Talk app (05-install-talk.sh)

  • Apache config (000-default.conf)

Here’s what I want guidance on:

  1. The cleanest way to connect my main Nextcloud container to the aio-talk HPB container, considering secrets (SIGNALING_SECRET, INTERNAL_SECRET, TURN_SECRET) and avoiding hairpin NAT / loopback issues.

  2. Whether it’s better to use aio-talk alone as a self-contained Nextcloud + Talk + HPB setup, or keep my main Nextcloud separate.

  3. How to properly configure Traefik for WebSocket support (wss://) and TURN/STUN ports for external clients.

  4. Best practices for environment variables and Docker networking to ensure Nextcloud can talk to HPB internally without relying on public DNS.

  5. Any example Docker Compose layout for a production-ready setup combining Nextcloud, HPB (aio-talk), TURN/STUN, and Traefik.

Current issues I’ve encountered:

  • Setting the Talk backend to https://signal.nextcloud-dev.example.com gives this error in Nextcloud logs:
stream_socket_client(): Unable to connect to ssl://signal.nextcloud-dev.avalcloud.com:443 (Connection timed out) at .../CertificateService.php#90

This indicates Nextcloud container cannot reach the public domain from inside Docker.

  • Trying to use ws://talk-hpb:8081 from the browser triggers CSP violation errors.

  • Even after setting internal/external URLs, I still hit timeouts or 500 errors.

  • WebSocket and HPB connections don’t seem to generate logs in Traefik or Nextcloud, because the request never reaches the containers due to browser CSP or network issues.

What I have in my Docker Compose:

services:
  mysql:
    image: mysql:8.0 
    container_name: nextcloud-mysql
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD} 
      MYSQL_DATABASE: ${MYSQL_DATABASE}
      MYSQL_USER: ${MYSQL_USER}
      MYSQL_PASSWORD: ${MYSQL_PASSWORD}
    volumes:
      - mysql-data:/var/lib/mysql 

  nextcloud:
    image: nextcloud:latest 
    container_name: nextcloud-app
    restart: always
    ports:
      - "8080:8080"
    depends_on:
      - mysql
    labels:
      # Enable Traefik for this service
      - "traefik.enable=true"
      
      # 1. HTTP Router (Redirect HTTP to HTTPS)
      - "traefik.http.routers.nextcloud-http.entrypoints=web"
      - "traefik.http.routers.nextcloud-http.rule=Host(`nextcloud-dev.example.com`)"
      - "traefik.http.routers.nextcloud-http.middlewares=nextcloud-redirect"
      
      # 2. HTTPS Router (Secure Access)
      - "traefik.http.routers.nextcloud-https.entrypoints=websecure"
      - "traefik.http.routers.nextcloud-https.rule=Host(`nextcloud-dev.example.com`)"
      # Use the Let's Encrypt resolver to get a certificate
      - "traefik.http.routers.nextcloud-https.tls=true"
      - "traefik.http.routers.nextcloud-https.tls.certresolver=le"
      # The Nextcloud service is available internally on port 8080
      - "traefik.http.services.nextcloud.loadbalancer.server.port=8080"
      
      # 3. Middleware for HTTP-to-HTTPS Redirection
      - "traefik.http.middlewares.nextcloud-redirect.redirectscheme.scheme=https"
    environment:
      NEXTCLOUD_ADMIN_USER: ${NEXTCLOUD_ADMIN_USER}
      NEXTCLOUD_ADMIN_PASSWORD: ${NEXTCLOUD_ADMIN_PASSWORD}
      
      MYSQL_HOST: mysql  
      MYSQL_DATABASE: ${MYSQL_DATABASE}
      MYSQL_USER: ${MYSQL_USER}
      MYSQL_PASSWORD: ${MYSQL_PASSWORD}
      
      # Trusted Domains (Replicating ConfigMap)
      # Replace 'localhost' with your desired domain if testing remotely.
      NEXTCLOUD_TRUSTED_DOMAINS: "nextcloud-dev.avalcloud.com"

      NC_DOMAIN: ${NC_DOMAIN}
      TALK_PORT: ${TALK_PORT}
      TURN_SECRET: ${TURN_SECRET}
      SIGNALING_SECRET: ${SIGNALING_SECRET}
      INTERNAL_SECRET: ${INTERNAL_SECRET}

      OVERWRITEPROTOCOL: https      
      APACHE_PORT: 8080
    volumes:
      - nextcloud-data:/var/www/html 
      - ./nextcloud-config/000-default.conf:/etc/apache2/sites-enabled/000-default.conf:ro
      - ./nextcloud-config/ports.conf:/etc/apache2/ports.conf:ro
      - ./nextcloud-config/10-dynamic-app-config.sh:/docker-entrypoint-hooks.d/before-starting/10-dynamic-app-config.sh:ro
      - ./nextcloud-config/05-install-talk.sh:/docker-entrypoint-hooks.d/before-starting/05-install-talk.sh:ro
      # - ./nextcloud-config/20-talk-config.sh:/docker-entrypoint-hooks.d/before-starting/20-talk-config.sh:ro

  # Add this service block to your existing docker-compose.yaml
  talk-hpb:
    image: ghcr.io/nextcloud-releases/aio-talk:latest
    container_name: nextcloud-talk-hpb
    restart: always
    labels:
      - "traefik.enable=true"
      
      # 1. HTTP Router (Redirect HTTP to HTTPS)
      - "traefik.http.routers.talk-http.entrypoints=web"
      - "traefik.http.routers.talk-http.rule=Host(`signal.nextcloud-dev.example.com`)"
      - "traefik.http.routers.talk-http.middlewares=nextcloud-redirect"
      
      # 2. HTTPS Router (Secure WebSocket Access)
      - "traefik.http.routers.talk-https.entrypoints=websecure"
      - "traefik.http.routers.talk-https.rule=Host(`signal.nextcloud-dev.example.com`)"
      - "traefik.http.routers.talk-https.tls=true"
      - "traefik.http.routers.talk-https.tls.certresolver=le"
      
      # The Talk service is available internally on port 8081
      - "traefik.http.services.talk-hpb.loadbalancer.server.port=8081"
    environment:
      - NC_DOMAIN=nextcloud-dev.avalcloud.com
      - TALK_HOST=talk-hpb
      - TALK_PORT=${TALK_PORT}
      - TURN_SECRET=${TURN_SECRET}
      - SIGNALING_SECRET=${SIGNALING_SECRET}
      - INTERNAL_SECRET=${INTERNAL_SECRET}
    ports:
      # Expose the TURN ports (required for external clients)
      - "3478:3478/tcp"
      - "3478:3478/udp"
      # Expose the internal signaling port for the reverse proxy
      - "8081:8081"
    depends_on:
      - nextcloud
  traefik:
    image: traefik:v2.11 # Use a stable Traefik v2 image
    container_name: traefik
    restart: always
    ports:
      - "80:80"    # The default HTTP port
      - "443:443"  # The default HTTPS port
      # - "8080:8080" # Optional: Traefik dashboard (for debugging)
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro # So Traefik can discover services
      - ./traefik-data:/etc/traefik/acme/ # Persistent storage for certificates
    command:
      # Entry Points (where Traefik listens)
      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443
      
      # Enable Docker Provider (to read service labels)
      - --providers.docker=true
      - --providers.docker.exposedbydefault=false
      
      # Configure Let's Encrypt (le = Let's Encrypt)
      - --certificatesresolvers.le.acme.email=someemail@example.com
      - --certificatesresolvers.le.acme.storage=/etc/traefik/acme/acme.json
      - --certificatesresolvers.le.acme.tlschallenge=true # Use TLS challenge for validation   
      # Global logs/debug (optional)
      - --log.level=WARN
### Volumes
volumes:
  mysql-data:
  nextcloud-data:

Scripts:
000-default.conf

<VirtualHost *:8080> 
    ServerAdmin webmaster@localhost
    DocumentRoot /var/www/html

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

10-dynamic-app-config.sh

#!/bin/bash
    
# Check if nextcloud is properly installed by checking for the status.php file
if [ ! -f /var/www/html/status.php ]; then
  echo "Nextcloud is not installed, skipping configuration"
  exit 0
fi

if [ -f /var/www/html/config/config.php ]; then
  echo "Checking for trusted domains changes..."
  
  # Get current and new trusted domains
  IFS=',' read -ra NEW_DOMAINS <<< "$NEXTCLOUD_TRUSTED_DOMAINS"
  CURRENT_DOMAINS=$(php /var/www/html/occ config:system:get trusted_domains | tr -d '[],"' | tr '\n' ',' | sed 's/localhost,//g' | sed 's/localhost//g')
  
  # Remove trailing comma if present
  CURRENT_DOMAINS=${CURRENT_DOMAINS%,}
  
  if [ "$CURRENT_DOMAINS" != "$NEXTCLOUD_TRUSTED_DOMAINS" ]; then
    echo "Trusted domains have changed, updating configuration..."
    
    # Loop through domains and add them to config
    COUNT=1
    # First remove all existing trusted domains
    php /var/www/html/occ config:system:delete trusted_domains
    
    # Then add new domains
    for DOMAIN in "${NEW_DOMAINS[@]}"; do
      DOMAIN=$(echo "$DOMAIN" | xargs) # Trim whitespace
      if [ ! -z "$DOMAIN" ]; then
        php /var/www/html/occ config:system:set trusted_domains $COUNT --value="$DOMAIN"
        COUNT=$((COUNT + 1))
      fi
    done
    
    echo "Trusted domains configuration updated"
  else
    echo "No changes to trusted domains detected"
  fi
else
  echo "Nextcloud not yet installed, skipping trusted domains configuration"
fi

05-install-talk.sh

#!/bin/bash
# Script to install the Nextcloud Talk app (spreed)

echo "--- Checking for Nextcloud Talk (spreed) app installation ---"

# Check if the Talk app is installed and skip installation if already present
if [ "$(php /var/www/html/occ app:list | grep -c 'spreed')" -eq 0 ]; then
  echo "Talk app not installed. Installing now..."
  
  # Install the app (downloads and enables it)
  php /var/www/html/occ app:install spreed
  
  if [ $? -eq 0 ]; then
    echo "Talk app installed successfully."
  else
    echo "ERROR: Talk app installation failed. Check internet connection or Nextcloud logs."
    exit 1
  fi
else
  # If the app is installed, ensure it is enabled
  if [ "$(php /var/www/html/occ app:list --shipped | grep -c 'spreed: enabled')" -eq 0 ]; then
    echo "Talk app found but not enabled. Enabling now..."
    php /var/www/html/occ app:enable spreed
  else
    echo "Talk app is already installed and enabled."
  fi
fi

echo "--- Talk app status verified ---"

What I’m asking the community:

Can someone provide:

  1. A clean Docker Compose example combining Nextcloud + aio-talk HPB/Turn/STUN + Traefik with WebSocket support

  2. Guidance on internal vs external URLs so Nextcloud can communicate with HPB without hitting public DNS

  3. Correct Traefik label configuration for HTTPS, WSS, and TURN ports

  4. Any gotchas with secrets, trusted_domains, or dynamic config scripts for Talk

I’m aiming for a production-ready setup where:

  • Nextcloud UI works

  • Talk works with HPB backend

  • TURN/STUN works for external clients

  • WebSocket connections are stable

  • No internal Docker hairpin/NAT issues

Thanks in advance for your help!

2

Hi, you might want to check out

3

I have added this script and mounted on /docker-entrypoint-hooks.d/before-starting/20-talk-config.sh that configured HPB and TURN server:

#!/bin/bash

# Check if Nextcloud is installed
if [ ! -f /var/www/html/status.php ]; then
  echo "Nextcloud is not installed, skipping Talk configuration."
  exit 0
fi

echo "--- Configuring Nextcloud Talk HPB Settings ---"

# 1. Ensure Talk app is enabled
php /var/www/html/occ app:enable spreed
if [ $? -ne 0 ]; then
    echo "ERROR: Could not enable Talk app. Is it installed? Skipping configuration."
    exit 1
fi

# 2. High Performance Backend (HPB) Configuration
#    CRITICAL CHANGE: We use the PUBLIC URL (https://) here.
#    Because of the Traefik alias, Nextcloud connects internally to Traefik -> HPB.
#    Because it is the public URL, the Browser gets the correct WSS address.
php /var/www/html/occ config:app:set spreed signaling_servers --value="{\"servers\":[{\"server\":\"https://signal.${NC_DOMAIN}\",\"verify\":true}],\"secret\":\"${SIGNALING_SECRET}\"}"

# 3. WebSocket / External Access
#    We allow the public WSS endpoint
php /var/www/html/occ config:app:set spreed websockets_allowed_urls --value="[\"wss://signal.${NC_DOMAIN}/ws\"]"

# 4. TURN/STUN Server Configuration
php /var/www/html/occ config:app:set spreed stun_server --value="${NC_DOMAIN}:3478"
php /var/www/html/occ config:app:set spreed turn_server --value="signal.${NC_DOMAIN}:3478"
php /var/www/html/occ config:app:set spreed turn_secret --value="${TURN_SECRET}"
php /var/www/html/occ config:app:set spreed turn_protocol --value="tcp"
php /var/www/html/occ config:app:set spreed turn_rest_api_endpoint --value="https://signal.${NC_DOMAIN}/api/v1/turn?secret={secret}&session_id={session_id}&protocol={protocol}&username={username}"

# 5. Internal Secret
php /var/www/html/occ config:app:set spreed signaling_secret --value="${SIGNALING_SECRET}"
php /var/www/html/occ config:app:set spreed internal_secret --value="${INTERNAL_SECRET}"

echo "--- Nextcloud Talk HPB Configuration Complete ---"

And change the network section of Treafik service:

traefik:
    image: traefik:v2.11
    # ... existing config ...
    networks:
      nextcloud-network:
        aliases:
          # THIS IS THE KEY FIX:
          # It lets Nextcloud resolve the public domain to the internal Traefik IP
          - signal.${NC_DOMAIN}
          - ${NC_DOMAIN} # Optional: helps if you have loopback issues with the main domain too