Appdata in object store CephS3 with problems/bug/issue:

hi all

issue summary

“CustomMessage”: “Could not get object urn:oid:78 for file appdata_ocyyyyyyyyy/appstore/appapi_apps.json”

This object is on the CephS3 bucket and I can see its content (with an S3 cli)

Question: Am I doing some configuration error or wrong procedure, or is it some bug?
Now all details

nextcloud 31.0.7
ubuntu 24.04 LTS
php8.3 fpm
apache2
redis cluster (3 masters, 3 slaves)
mariadb 10.11.13
primary storage Ceph

After unziping nextcloud I do the initial installation with

sudo -E -u www-data php /var/www/nextcloud/occ maintenance:install --database "mysql" --database-host "db_priv_IP" --database-name "nextcloud" --database-user "nextcloud" --database-pass "xxxx" --admin-user "admin" --admin-pass "yyyy"

Creates a config.php with minimal config

<?php
$CONFIG = array (
  'passwordsalt' => '...',
  'secret' => '...',
  'trusted_domains' => 
  array (
    0 => 'localhost',
  ),
  'datadirectory' => '/var/www/nextcloud/data',
  'dbtype' => 'mysql',
  'version' => '31.0.7.1',
  'overwrite.cli.url' => 'http://localhost',
  'dbname' => 'nextcloud',
  'dbhost' => 'db_priv_IP',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'nextcloud',
  'dbpassword' => 'xxx',
  'installed' => true,
  'instanceid' => 'xxxx',
);

I immediately execute a ansible playbook to set all other configuration options to get the following config.php

<?php
$CONFIG = array (
  'passwordsalt' => '...',
  'secret' => '...',
  'trusted_domains' => 
  array (
    0 => 'myserver',
    2 => 'ip1',
    5 => 'ip2',
  ),
  'datadirectory' => '/var/www/nextcloud/data',
  'dbtype' => 'mysql',
  'version' => '31.0.7.1',
  'overwrite.cli.url' => 'https://mynextcloud_url',
  'dbname' => 'nextcloud',
  'dbhost' => 'db_ip',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'nextcloud',
  'dbpassword' => 'xxx',
  'installed' => true,
  'instanceid' => 'xxx',
  'trusted_proxies' => 
  array (
    0 => 'trustedIP1',
    1 => 'trustedIP2',
  ),
  'allowed_admin_ranges' => 
  array (
    0 => 'IPRange1',
    1 => 'IPRange2',
  ),
  'redis.cluster' => 
  array (
    'seeds' => 
    array (
      0 => 'redisIP1:7000',
      1 => 'redisIP1:7001',
      2 => 'redisIP2:7000',
      3 => 'redisIP2:7001',
      4 => 'redisIP3:7000',
      5 => 'redisIP3:7001',
    ),
    'failover_mode' => '\\RedisCluster::FAILOVER_ERROR',
  ),
  'default_phone_region' => 'PT',
  'default_timezone' => 'Europe/Lisbon',
  'enable_previews' => false,
  'log_type' => 'file',
  'logfile' => '/var/log/nextcloud/nextcloud.log',
  'loglevel' => '3',
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'memcache.distributed' => '\\OC\\Memcache\\Redis',
  'objectstore' => 
  array (
    'class' => '\\OC\\Files\\ObjectStore\\S3',
    'arguments' => 
    array (
      'bucket' => 'mybucket-a',
      'hostname' => 'myceph.domain',
      'key' => 'mykey',
      'secret' => 'mysecret',
      'port' => '8080',
      'use_path_style' => true,
    ),
  ),
  'updater.release.channel' => 'production',
);

I can upload files, and they go to CephS3 bucket as objects,
the problem is with the appdata dir, I am able to list the apps with occ

sudo -E -u www-data php /var/www/nextcloud/occ app:list
Enabled:
  - activity: 4.0.0
  - app_api: 5.0.2
  - bruteforcesettings: 4.0.0
  - circles: 31.0.0

But in the web admin interface it gets stuck when I select “+ Apps”

The logs show

{
  "reqId": "som_req",
  "level": 3,
  "time": "2025-07-30T14:21:14+00:00",
  "remoteAddr": "ip",
  "user": "admin",
  "app": "objectstore",
  "method": "GET",
  "url": "/index.php/settings/apps",
  "message": "Could not get object urn:oid:78 for file appdata_ocyyyyyyyyy/appstore/appapi_apps.json",
  "userAgent": "Mozilla/5.0 (X11; Linux x86_64; rv:141.0) Gecko/20100101 Firefox/141.0",
  "version": "31.0.7.1",
  "exception": {
    "Exception": "Exception",
    "Message": "Failed to read object urn:oid:78",
    "Code": 0,
    "Trace": [
      {
        "file": "/var/www/nextcloud/lib/private/Files/ObjectStore/ObjectStoreStorage.php",
        "line": 303,
        "function": "readObject",
        "class": "OC\\Files\\ObjectStore\\S3",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/lib/private/Files/Storage/Common.php",
        "line": 163,
        "function": "fopen",
        "class": "OC\\Files\\ObjectStore\\ObjectStoreStorage",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/lib/private/Files/View.php",
        "line": 1211,
        "function": "file_get_contents",
        "class": "OC\\Files\\Storage\\Common",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/lib/private/Files/View.php",
        "line": 555,
        "function": "basicOperation",
        "class": "OC\\Files\\View",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/lib/private/Files/Node/File.php",
        "line": 33,
        "function": "file_get_contents",
        "class": "OC\\Files\\View",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/lib/private/Files/SimpleFS/SimpleFile.php",
        "line": 55,
        "function": "getContent",
        "class": "OC\\Files\\Node\\File",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/apps/app_api/lib/Fetcher/AppAPIFetcher.php",
        "line": 126,
        "function": "getContent",
        "class": "OC\\Files\\SimpleFS\\SimpleFile",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/apps/app_api/lib/Fetcher/ExAppFetcher.php",
        "line": 151,
        "function": "get",
        "class": "OCA\\AppAPI\\Fetcher\\AppAPIFetcher",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/apps/app_api/lib/Fetcher/ExAppFetcher.php",
        "line": 168,
        "function": "get",
        "class": "OCA\\AppAPI\\Fetcher\\ExAppFetcher",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/apps/app_api/lib/Service/ExAppsPageService.php",
        "line": 45,
        "function": "getExAppsWithUpdates",
        "class": "OCA\\AppAPI\\Fetcher\\ExAppFetcher",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/apps/settings/lib/Controller/AppSettingsController.php",
        "line": 97,
        "function": "provideAppApiState",
        "class": "OCA\\AppAPI\\Service\\ExAppsPageService",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php",
        "line": 200,
        "function": "viewApps",
        "class": "OCA\\Settings\\Controller\\AppSettingsController",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php",
        "line": 114,
        "function": "executeController",
        "class": "OC\\AppFramework\\Http\\Dispatcher",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/lib/private/AppFramework/App.php",
        "line": 161,
        "function": "dispatch",
        "class": "OC\\AppFramework\\Http\\Dispatcher",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/lib/private/Route/Router.php",
        "line": 315,
        "function": "main",
        "class": "OC\\AppFramework\\App",
        "type": "::"
      },
      {
        "file": "/var/www/nextcloud/lib/base.php",
        "line": 1040,
        "function": "match",
        "class": "OC\\Route\\Router",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/index.php",
        "line": 24,
        "function": "handleRequest",
        "class": "OC",
        "type": "::"
      }
    ],
    "File": "/var/www/nextcloud/lib/private/Files/ObjectStore/S3ObjectTrait.php",
    "Line": 75,
    "message": "Could not get object urn:oid:78 for file appdata_ocyyyyyyyyy/appstore/appapi_apps.json",
    "exception": {},
    "CustomMessage": "Could not get object urn:oid:78 for file appdata_ocyyyyyyyyy/appstore/appapi_apps.json"
  }
}

Thanks in advance
with kind regards
Mario