Internal Server Error after Docker Compose Update

Haui · June 1, 2024, 1:57pm

Hi folks,
after many wonderful months using nextcloud without issue, I have encountered a new roadblock.

After I have been greeted with “update now” welcome screen accessing my nextcloud instance today, I decided to run occ upgrade but afterwards it shows an error:

Internal server error

The server could not complete the request.

If this occurs again, please send the following technical details to your server administrator.

More details can be found in the server log.
Technical details

    Remote Address: 192.168.178.65
    Inquiry ID: zK0lFnMMi6yg4uMAIz

I took the whole setup down and up again after I read a suggestion that redis might not be happy. I also looked at the logs but docker seems okay and nextcloud is insanely hard to read. Here’s some of it:

{"reqId":"ByBP57NaN5RsPwTby8NE","level":1,"time":"2024-06-01T13:39:27+00:00","remoteAddr":"192.168.178.65","user":"Haui","app":"no app in context","method":"POST","url":"/index.php/apps/bookmarks/public/rest/v2/lock","message":"Unable to generate a URL for the named route \"mediadc.page.index\" as such route does not exist.","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:126.0) Gecko/20100101 Firefox/126.0","version":"29.0.1.1","exception":{"Exception":"Symfony\\Component\\Routing\\Exception\\RouteNotFoundException","Message":"Unable to generate a URL for the named route \"mediadc.page.index\" as such route does not exist.","Code":0,"Trace":[{"file":"/var/www/html/lib/private/Route/Router.php","line":410,"function":"generate","class":"Symfony\\Component\\Routing\\Generator\\UrlGenerator","type":"->","args":["mediadc.page.index",[],1]},{"file":"/var/www/html/lib/private/Route/CachingRouter.php","line":65,"function":"generate","class":"OC\\Route\\Router","type":"->","args":["mediadc.page.index",[],false]},{"file":"/var/www/html/lib/private/URLGenerator.php","line":103,"function":"generate","class":"OC\\Route\\CachingRouter","type":"->","args":["mediadc.page.index",[]]},{"file":"/var/www/html/lib/private/NavigationManager.php","line":371,"function":"linkToRoute","class":"OC\\URLGenerator","type":"->","args":["mediadc.page.index"]},{"file":"/var/www/html/lib/private/NavigationManager.php","line":132,"function":"init","class":"OC\\NavigationManager","type":"->","args":[]},{"file":"/var/www/html/apps/theming/lib/ThemingDefaults.php","line":199,"function":"getAll","class":"OC\\NavigationManager","type":"->","args":["guest"]},{"file":"/var/www/html/lib/private/legacy/OC_Defaults.php","line":271,"function":"getShortFooter","class":"OCA\\Theming\\ThemingDefaults","type":"->","args":[]},{"file":"/var/www/html/lib/public/Defaults.php","line":176,"function":"getLongFooter","class":"OC_Defaults","type":"->","args":[]},{"file":"/var/www/html/core/templates/layout.guest.php","line":51,"function":"getLongFooter","class":"OCP\\Defaults","type":"->","args":[]},{"file":"/var/www/html/lib/private/Template/Base.php","line":180,"args":["/var/www/html/core/templates/layout.guest.php"],"function":"include"},{"file":"/var/www/html/lib/private/Template/Base.php","line":150,"function":"load","class":"OC\\Template\\Base","type":"->","args":["/var/www/html/core/templates/layout.guest.php",null]},{"file":"/var/www/html/lib/private/legacy/OC_Template.php","line":142,"function":"fetchPage","class":"OC\\Template\\Base","type":"->","args":[null]},{"file":"/var/www/html/lib/private/legacy/OC_Template.php","line":173,"function":"fetchPage","class":"OC_Template","type":"->","args":[null]},{"file":"/var/www/html/lib/private/Template/Base.php","line":132,"function":"fetchPage","class":"OC_Template","type":"->","args":[]},{"file":"/var/www/html/lib/private/legacy/OC_Template.php","line":320,"function":"printPage","class":"OC\\Template\\Base","type":"->","args":[]},{"file":"/var/www/html/index.php","line":131,"function":"printExceptionErrorPage","class":"OC_Template","type":"::","args":[["Error"],500]}],"File":"/var/www/html/3rdparty/symfony/routing/Generator/UrlGenerator.php","Line":144,"message":"Unable to generate a URL for the named route \"mediadc.page.index\" as such route does not exist.","exception":{},"CustomMessage":"Unable to generate a URL for the named route \"mediadc.page.index\" as such route does not exist."}}
{"reqId":"ByBP57NaN5RsPwTby8NE","level":1,"time":"2024-06-01T13:39:27+00:00","remoteAddr":"192.168.178.65","user":"Haui","app":"no app in context","method":"POST","url":"/index.php/apps/bookmarks/public/rest/v2/lock","message":"Unable to generate a URL for the named route \"notes.page.index\" as such route does not exist.","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:126.0) Gecko/20100101 Firefox/126.0","version":"29.0.1.1","exception":{"Exception":"Symfony\\Component\\Routing\\Exception\\RouteNotFoundException","Message":"Unable to generate a URL for the named route \"notes.page.index\" as such route does not exist.","Code":0,"Trace":[{"file":"/var/www/html/lib/private/Route/Router.php","line":410,"function":"generate","class":"Symfony\\Component\\Routing\\Generator\\UrlGenerator","type":"->","args":["notes.page.index",[],1]},{"file":"/var/www/html/lib/private/Route/CachingRouter.php","line":65,"function":"generate","class":"OC\\Route\\Router","type":"->","args":["notes.page.index",[],false]},{"file":"/var/www/html/lib/private/URLGenerator.php","line":103,"function":"generate","class":"OC\\Route\\CachingRouter","type":"->","args":["notes.page.index",[]]},{"file":"/var/www/html/lib/private/NavigationManager.php","line":371,"function":"linkToRoute","class":"OC\\URLGenerator","type":"->","args":["notes.page.index"]},{"file":"/var/www/html/lib/private/NavigationManager.php","line":132,"function":"init","class":"OC\\NavigationManager","type":"->","args":[]},{"file":"/var/www/html/apps/theming/lib/ThemingDefaults.php","line":199,"function":"getAll","class":"OC\\NavigationManager","type":"->","args":["guest"]},{"file":"/var/www/html/lib/private/legacy/OC_Defaults.php","line":271,"function":"getShortFooter","class":"OCA\\Theming\\ThemingDefaults","type":"->","args":[]},{"file":"/var/www/html/lib/public/Defaults.php","line":176,"function":"getLongFooter","class":"OC_Defaults","type":"->","args":[]},{"file":"/var/www/html/core/templates/layout.guest.php","line":51,"function":"getLongFooter","class":"OCP\\Defaults","type":"->","args":[]},{"file":"/var/www/html/lib/private/Template/Base.php","line":180,"args":["/var/www/html/core/templates/layout.guest.php"],"function":"include"},{"file":"/var/www/html/lib/private/Template/Base.php","line":150,"function":"load","class":"OC\\Template\\Base","type":"->","args":["/var/www/html/core/templates/layout.guest.php",null]},{"file":"/var/www/html/lib/private/legacy/OC_Template.php","line":142,"function":"fetchPage","class":"OC\\Template\\Base","type":"->","args":[null]},{"file":"/var/www/html/lib/private/legacy/OC_Template.php","line":173,"function":"fetchPage","class":"OC_Template","type":"->","args":[null]},{"file":"/var/www/html/lib/private/Template/Base.php","line":132,"function":"fetchPage","class":"OC_Template","type":"->","args":[]},{"file":"/var/www/html/lib/private/legacy/OC_Template.php","line":320,"function":"printPage","class":"OC\\Template\\Base","type":"->","args":[]},{"file":"/var/www/html/index.php","line":131,"function":"printExceptionErrorPage","class":"OC_Template","type":"::","args":[["Error"],500]}],"File":"/var/www/html/3rdparty/symfony/routing/Generator/UrlGenerator.php","Line":144,"message":"Unable to generate a URL for the named route \"notes.page.index\" as such route does not exist.","exception":{},"CustomMessage":"Unable to generate a URL for the named route \"notes.page.index\" as such route does not exist."}}
{"reqId":"ByBP57NaN5RsPwTby8NE","level":1,"time":"2024-06-01T13:39:27+00:00","remoteAddr":"192.168.178.65","user":"Haui","app":"no app in context","method":"POST","url":"/index.php/apps/bookmarks/public/rest/v2/lock","message":"Unable to generate a URL for the named route \"tasks.page.index\" as such route does not exist.","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:126.0) Gecko/20100101 Firefox/126.0","version":"29.0.1.1","exception":{"Exception":"Symfony\\Component\\Routing\\Exception\\RouteNotFoundException","Message":"Unable to generate a URL for the named route \"tasks.page.index\" as such route does not exist.","Code":0,"Trace":[{"file":"/var/www/html/lib/private/Route/Router.php","line":410,"function":"generate","class":"Symfony\\Component\\Routing\\Generator\\UrlGenerator","type":"->","args":["tasks.page.index",[],1]},{"file":"/var/www/html/lib/private/Route/CachingRouter.php","line":65,"function":"generate","class":"OC\\Route\\Router","type":"->","args":["tasks.page.index",[],false]},{"file":"/var/www/html/lib/private/URLGenerator.php","line":103,"function":"generate","class":"OC\\Route\\CachingRouter","type":"->","args":["tasks.page.index",[]]},{"file":"/var/www/html/lib/private/NavigationManager.php","line":371,"function":"linkToRoute","class":"OC\\URLGenerator","type":"->","args":["tasks.page.index"]},{"file":"/var/www/html/lib/private/NavigationManager.php","line":132,"function":"init","class":"OC\\NavigationManager","type":"->","args":[]},{"file":"/var/www/html/apps/theming/lib/ThemingDefaults.php","line":199,"function":"getAll","class":"OC\\NavigationManager","type":"->","args":["guest"]},{"file":"/var/www/html/lib/private/legacy/OC_Defaults.php","line":271,"function":"getShortFooter","class":"OCA\\Theming\\ThemingDefaults","type":"->","args":[]},{"file":"/var/www/html/lib/public/Defaults.php","line":176,"function":"getLongFooter","class":"OC_Defaults","type":"->","args":[]},{"file":"/var/www/html/core/templates/layout.guest.php","line":51,"function":"getLongFooter","class":"OCP\\Defaults","type":"->","args":[]},{"file":"/var/www/html/lib/private/Template/Base.php","line":180,"args":["/var/www/html/core/templates/layout.guest.php"],"function":"include"},{"file":"/var/www/html/lib/private/Template/Base.php","line":150,"function":"load","class":"OC\\Template\\Base","type":"->","args":["/var/www/html/core/templates/layout.guest.php",null]},{"file":"/var/www/html/lib/private/legacy/OC_Template.php","line":142,"function":"fetchPage","class":"OC\\Template\\Base","type":"->","args":[null]},{"file":"/var/www/html/lib/private/legacy/OC_Template.php","line":173,"function":"fetchPage","class":"OC_Template","type":"->","args":[null]},{"file":"/var/www/html/lib/private/Template/Base.php","line":132,"function":"fetchPage","class":"OC_Template","type":"->","args":[]},{"file":"/var/www/html/lib/private/legacy/OC_Template.php","line":320,"function":"printPage","class":"OC\\Template\\Base","type":"->","args":[]},{"file":"/var/www/html/index.php","line":131,"function":"printExceptionErrorPage","class":"OC_Template","type":"::","args":[["Error"],500]}],"File":"/var/www/html/3rdparty/symfony/routing/Generator/UrlGenerator.php","Line":144,"message":"Unable to generate a URL for the named route \"tasks.page.index\" as such route does not exist.","exception":{},"CustomMessage":"Unable to generate a URL for the named route \"tasks.page.index\" as such route does not exist."}}

<?php
$CONFIG = array (
  'htaccess.RewriteBase' => '/',
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'apps_paths' => 
  array (
    0 => 
    array (
      'path' => '/var/www/html/apps',
      'url' => '/apps',
      'writable' => false,
    ),
    1 => 
    array (
      'path' => '/var/www/html/custom_apps',
      'url' => '/custom_apps',
      'writable' => true,
    ),
  ),
  'instanceid' => '***',
  'passwordsalt' => '***',
  'secret' => '***',
  'trusted_domains' => 
  array (
    0 => '192.168.178.***',
    1 => '192.168.178.***',
    2 => '192.168.178.***',
    3 => 'cloud.hs.***',
    4 => 'nextcloud-app-1',
    5 => 'localhost',
    6 => '172.27.0.***',
    7 => '127.0.0.1',
    8 => '127.0.1.1',
  ),
  'datadirectory' => '/var/www/html/data',
  'dbtype' => 'mysql',
  'version' => '29.0.1.1',
  'dbname' => 'nextcloud',
  'dbhost' => 'db',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => '***',
  'dbpassword' => '***',
  'installed' => true,
  'maintenance' => false,
  'theme' => '',
  'loglevel' => 0,
  'overwrite.cli.url' => 'https://cloud.hs.***',
  'trusted_proxies' => 
  array (
    0 => '172.27.0.11/16',
  ),
  'mail_smtpmode' => 'smtp',
  'mail_smtpsecure' => 'ssl',
  'mail_sendmailmode' => 'smtp',
  'mail_from_address' => '***',
  'mail_domain' => '***',
  'mail_smtphost' => '***',
  'mail_smtpport' => '465',
  'mail_smtpauth' => 1,
  'mail_smtpname' => '***',
  'mail_smtppassword' => '***',
  'default_phone_region' => 'DE',
  'memcache.distributed' => '\\OC\\Memcache\\Redis',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'redis' => 
  array (
    'host' => 'redis',
    'password' => '***',
    'port' => 6379,
  ),
  'maintenance_window_start' => 1,
);

jtr · June 1, 2024, 2:01pm

What Docker image are you using that requires a manual upgrade?

Please post your Docker Compose file.

EDIT: You appear to be using the micro-services Docker image. That image doesn’t use the Nextcloud Updater (and you typically would not use occ upgrade manually in it either). Can you detail a bit more how you typically are updating your Nextcloud container? That may provide some clues as to what went wrong here. GitHub - nextcloud/docker: ⛴ Docker image of Nextcloud

The entire upgrade will be visible in your Docker logs for the container + Nextcloud’s log during that time period. It may provide some hints if the container upgrade failed.

seligmanns · June 1, 2024, 2:24pm

Same problem here after upgrading from 28.0.5 to 28.0.6 (docker compose)
My Desktop client told me that there is a problem with the circles app .
So I disabled it via occ app:disable circles.
This fixed it.

Stefan

Haui · June 1, 2024, 2:39pm

Thanks for answering. The container has always prompted me to update. No idea why thats the case. I asked around but never found a real answer and since it worked flawlessly I didnt bother breaking a working system. Here’s the compose file:

version: '2'

services:
  db:
    env_file:
      - .env
    image: mariadb:10.5
    restart: always
    command: --transaction-isolation=READ-COMMITTED --binlog-format=ROW
    volumes:
      - /media/containers/Maria/db:/var/lib/mysql
      - /etc/localtime:/etc/localtime
    environment:
      - MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}
      - MYSQL_PASSWORD=${MYSQL_PASSWORD}
      - MYSQL_DATABASE=${MYSQL_DATABASE}
      - MYSQL_USER=${MYSQL_USER}

  redis:
    image: redis
    restart: always
    ports:
      - 6379:6379
    command: redis-server --appendonly yes --requirepass ${REDIS_PASSWD}
    volumes:
      - /etc/localtime:/etc/localtime
  app:
    image: nextcloud
    restart: always
    links:
      - db
      - redis
    volumes:
      - /mnt/md0/sambashare/Backups/Nextcloud/files:/var/www/html/data
      - /mnt/md0/sambashare/Backups/Nextcloud/config:/var/www/html/config
      - /mnt/md0/sambashare/Backups/Nextcloud/custom_apps:/var/www/html/custom_apps
      - /mnt/md0/sambashare/Backups/Nextcloud/themes:/var/www/html/themes
      - /etc/localtime:/etc/localtime
    environment:
      - MYSQL_PASSWORD=${MYSQL_PASSWORD}
      - MYSQL_DATABASE=${MYSQL_DATABASE}
      - MYSQL_USER=${MYSQL_USER}
      - MYSQL_HOST=${MYSQL_HOST}
      - NEXTCLOUD_HOSTNAME=${NEXTCLOUD_HOSTNAME}
      - NEXTCLOUD_TRUSTED_DOMAINS=${NEXTCLOUD_TRUSTED_DOMAINS}
      - APACHE_DISABLE_REWRITE_IP=1
      - TRUSTED_PROXIES=${TRUSTED_PROXIES}
      - OVERWRITECLIURL=${OVERWRITECLIURL}
      - REDIS_HOST=redis
      - REDIS_PORT=6379
      - REDIS_HOST_PASSWORD=${REDIS_PASSWD}

networks:
  default:
    external: true
    name: testnetwork

I built a script that takes care of my backups for all my containers/services (around 50 in total, all different compose files). I put in a function that makes a separate backup before an update and then runs a docker exec occ upgrade.

There is a possible problem now that I am describing it. I read somewhere that some apps need to be loaded for the upgrade and if I use maintenance mode for the update and then go back it might not have loaded all the apps?

Haui · June 1, 2024, 2:42pm

Holy crap! That did it! Thank you very much! do you have any idea why that happened? Is it possible that me having maintenance mode on while updating is causing the issue?

seligmanns · June 1, 2024, 3:32pm

meanwhile I visited the website of the circles app on github and had a look at the open issues.
I assume that the app doesn’t work any longer with the latest nc versions.

tullsta · June 3, 2024, 9:15pm

Hello Stefan,

thanks for the info - helped me out here!

For curiosity: are you also using the bookmarks-app? This one got an update on June 1st to 14.1.0 and rather immediately a hotfix to 14.1.1 with the comment: “14.1.1 Fixed => Fix circle usage in code to avoid dependency hell”.

thanks again + kind regards - tullsta