Error after system crash: occ does not work - urgent

Hello everyone,

after a linux crash (UBUNTU 20.04 LTS) that deleted the logs folder (/ var / log), I can’t start Nextcloud anymore.

I try to run OCC to repair, and it gives me this error, without telling me which file or directory is missing:

sudo -u www-data php /var/www/nextcloud/occ

An unhandled exception has been thrown:
RedisException: No such file or directory in /var/www/nextcloud/lib/private/RedisFactory.php:124
Stack trace:
#0 /var/www/nextcloud/lib/private/RedisFactory.php(124): Redis->connect()
#1 /var/www/nextcloud/lib/private/RedisFactory.php(164): OC\RedisFactory->create()
#2 /var/www/nextcloud/lib/private/Memcache/Redis.php(43): OC\RedisFactory->getInstance()
#3 /var/www/nextcloud/lib/private/Memcache/Factory.php(115): OC\Memcache\Redis->__construct()
#4 /var/www/nextcloud/lib/private/Server.php(1065): OC\Memcache\Factory->createLocking()
#5 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(160): OC\Server->OC{closure}()
#6 /var/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php(118): OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility{closure}()
#7 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(127): Pimple\Container->offsetGet()
#8 /var/www/nextcloud/lib/private/ServerContainer.php(136): OC\AppFramework\Utility\SimpleContainer->query()
#9 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(56): OC\ServerContainer->query()
#10 /var/www/nextcloud/lib/private/Server.php(2024): OC\AppFramework\Utility\SimpleContainer->get()
#11 /var/www/nextcloud/lib/private/Files/View.php(118): OC\Server->getLockingProvider()
#12 /var/www/nextcloud/lib/private/Server.php(435): OC\Files\View->__construct()
#13 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(160): OC\Server->OC{closure}()
#14 /var/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php(118): OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility{closure}()
#15 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(127): Pimple\Container->offsetGet()
#16 /var/www/nextcloud/lib/private/ServerContainer.php(136): OC\AppFramework\Utility\SimpleContainer->query()
#17 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(56): OC\ServerContainer->query()
#18 /var/www/nextcloud/lib/private/Server.php(1387): OC\AppFramework\Utility\SimpleContainer->get()
#19 /var/www/nextcloud/lib/base.php(594): OC\Server->boot()
#20 /var/www/nextcloud/lib/base.php(1083): OC::init()
#21 /var/www/nextcloud/console.php(48): require_once(’/var/www/nextcl…’)
#22 /var/www/nextcloud/occ(11): require_once(’/var/www/nextcl…’)
#23 {main}root@cloudio:/var/log/nextcloud# ^C

Furthermore, despite having set the LogLevels on config.php to a value of 0, they always appear at level 3 in the log .

I have nextcloud totally blocked and I don’t know which way to turn

Thanks in advance for your help

Did you checked your Redis server is up and running?
E.g.

sudo service redis-server status

sudo service redis-server status

● redis-server.service - Advanced key-value store
Loaded: loaded (/lib/systemd/system/redis-server.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2021-10-27 12:40:56 CEST; 2h 10min ago
Docs: Redis,
man:redis-server(1)
Main PID: 1256 (redis-server)
Tasks: 4 (limit: 9445)
Memory: 20.1M
CGroup: /system.slice/redis-server.service
└─1256 /usr/bin/redis-server 127.0.0.1:6379

Oct 27 12:40:55 cloudio.tsfnet.com systemd[1]: Starting Advanced key-value store…
Oct 27 12:40:56 cloudio.tsfnet.com systemd[1]: redis-server.service: Can’t open PID file /run/redis/redis-server.pid (yet?) after start-post: Operation not permitted
Oct 27 12:40:56 cloudio.tsfnet.com systemd[1]: Started Advanced key-value store.

ANYWAY, if I restart the service, the message of PID file went away

sudo service redis-server restart

sudo service redis-server status

● redis-server.service - Advanced key-value store
Loaded: loaded (/lib/systemd/system/redis-server.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2021-10-27 14:52:14 CEST; 1s ago
Docs: http://redis.io/documentation,
man:redis-server(1)
Process: 7322 ExecStart=/usr/bin/redis-server /etc/redis/redis.conf (code=exited, status=0/SUCCESS)
Process: 7325 ExecStartPost=/bin/sh -c echo $MAINPID > /var/run/redis/redis.pid (code=exited, status=0/SUCCESS)
Main PID: 7324 (redis-server)
Tasks: 4 (limit: 9445)
Memory: 12.2M
CGroup: /system.slice/redis-server.service
└─7324 /usr/bin/redis-server 127.0.0.1:6379

Could be something wrong with memcache, could you please post you configuration, Env. parameters and logs according to the template for a new issue?

Here the infos:

Nextcloud version: 22.2.0.2
Operating system and version: Ubuntu 20.04.3 LTS
Apache or nginx version: Apache/2.4.41
PHP version: 7.4.3

CONFIG.PHP

<?php
$CONFIG = array (
  'passwordsalt' => '[sensitive Data REMOVED by Mod/JK]',
  'secret' => '[sensitive Data REMOVED by Mod/JK]',
  'trusted_domains' => 
  array (
    0 => 'localhost',
    1 => '172.16.0.8',
    2 => 'cloudio.tsfnet.com',
    3 => 'cloudio.tsfnet.com',
  ),
  'datadirectory' => '/mnt/ncdata',
  'dbtype' => 'pgsql',
  'version' => '22.2.0.2',
  'overwrite.cli.url' => 'https://cloudio.tsfnet.com/',
  'dbname' => 'nextcloud_db',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'dbuser' => '[sensitive Data REMOVED by Mod/JK]',
  'dbpassword' => '[sensitive Data REMOVED by Mod/JK]',
  'installed' => true,
  'instanceid' => '[sensitive Data REMOVED by Mod/JK]',
  'upgrade.disable-web' => true,
  'log_type' => 'file',
  'logfile' => '/var/log/nextcloud/nextcloud.log',
  'loglevel' => '0',
  'log.condition' => 
  array (
    'apps' => 
    array (
      0 => 'admin_audit',
    ),
  ),
  'mail_smtpmode' => 'smtp',
  'remember_login_cookie_lifetime' => '1800',
  'log_rotate_size' => '10485760',
  'trashbin_retention_obligation' => 'auto, 60',
  'versions_retention_obligation' => 'auto, 180',
  'activity_expire_days' => '120',
  'simpleSignUpLink.shown' => false,
  'memcache.local' => '\\OC\\Memcache\\Redis',
  'filelocking.enabled' => true,
  'memcache.distributed' => '\\OC\\Memcache\\Redis',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'redis' => 
  array (
    'host' => '/var/run/redis/redis-server.sock',
    'port' => 0,
    'timeout' => 0.5,
    'dbindex' => 0,
    'password' => '[sensitive Data REMOVED by Mod/JK]',
  ),
  'ldapProviderFactory' => 'OCA\\User_LDAP\\LDAPProviderFactory',
  'app_install_overwrite' => 
  array (
    0 => 'twainwebscan',
  ),
  'maintenance' => false,
  'default_phone_region' => 'it',
  'htaccess.RewriteBase' => '/',
  'logtimezone' => 'Etc/UTC',
);

NEXTCLOUD LOG:

{"reqId":"i3l2DV8PXPTtGMo7kHVw","level":3,"time":"2021-10-27T07:45:01+00:00","remoteAddr":"","user":"--","app":"cron","method":"","url":"--","message":"No such file or directory","userAgent":"--","version":"22.2.0.2","exception":{"Exception":"RedisException","Message":"No such file or directory","Code":0,"Trace":[{"file":"/var/www/nextcloud/lib/private/RedisFactory.php","line":124,"function":"connect","class":"Redis","type":"->"},{"file":"/var/www/nextcloud/lib/private/RedisFactory.php","line":164,"function":"create","class":"OC\\RedisFactory","type":"->"},{"file":"/var/www/nextcloud/lib/private/Memcache/Redis.php","line":43,"function":"getInstance","class":"OC\\RedisFactory","type":"->"},{"file":"/var/www/nextcloud/lib/private/Memcache/Factory.php","line":115,"function":"__construct","class":"OC\\Memcache\\Redis","type":"->"},{"file":"/var/www/nextcloud/lib/private/Server.php","line":1065,"function":"createLocking","class":"OC\\Memcache\\Factory","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php","line":160,"function":"OC\\{closure}","class":"OC\\Server","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php","line":118,"function":"OC\\AppFramework\\Utility\\{closure}","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php","line":127,"function":"offsetGet","class":"Pimple\\Container","type":"->"},{"file":"/var/www/nextcloud/lib/private/ServerContainer.php","line":136,"function":"query","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php","line":56,"function":"query","class":"OC\\ServerContainer","type":"->"},{"file":"/var/www/nextcloud/lib/private/Server.php","line":2024,"function":"get","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/View.php","line":118,"function":"getLockingProvider","class":"OC\\Server","type":"->"},{"file":"/var/www/nextcloud/lib/private/Server.php","line":435,"function":"__construct","class":"OC\\Files\\View","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php","line":160,"function":"OC\\{closure}","class":"OC\\Server","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php","line":118,"function":"OC\\AppFramework\\Utility\\{closure}","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php","line":127,"function":"offsetGet","class":"Pimple\\Container","type":"->"},{"file":"/var/www/nextcloud/lib/private/ServerContainer.php","line":136,"function":"query","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php","line":56,"function":"query","class":"OC\\ServerContainer","type":"->"},{"file":"/var/www/nextcloud/lib/private/Server.php","line":1387,"function":"get","class":"OC\\AppFramework\\Utility\\SimpleContainer","type":"->"},{"file":"/var/www/nextcloud/lib/base.php","line":594,"function":"boot","class":"OC\\Server","type":"->"},{"file":"/var/www/nextcloud/lib/base.php","line":1083,"function":"init","class":"OC","type":"::"},{"file":"/var/www/nextcloud/cron.php","line":42,"args":["/var/www/nextcloud/lib/base.php"],"function":"require_once"}],"File":"/var/www/nextcloud/lib/private/RedisFactory.php","Line":124,"CustomMessage":"--"}}

APACHE2 LOG:

[Wed Oct 27 16:15:02.053124 2021] [access_compat:error] [pid 1359:tid 139789760190208] [client 172.16.0.59:49852] AH01797: client denied by server configuration: /var/www/nextcloud/status.php

@mbassan i removed some sensitive Data from your posting. You should consider your instance as being compromized, though.

thanks a lot @JimmyKater

1 Like

Hello, any news ? :sob:

ummm referring to your nextcloud log it might be possible that not only your logfiles were removed but apparently some more folders/directories/files as there’s some errormessage that a file/directory wasn’t found. But I can’t see which one is missing exactly

is there a way to refresh the installation without losing data and configurations?
is there a “DEBUG” way to figure out what’s missing?

do you have a backup?

database and data disk (separate from the OS), yes

perfect. so you won’t lose anything if you’d perfom a reinstallation from scratch

do you mean nextcloud or all OS included?

as it seems to be a VM-installation you better do it anew completely…
just make sure that you save your config.php additional to the backup.

I’ll try to do this, then is there a tutorial where I can learn how to restore the DB and data/files/folders?

i’d say look around here on the forum… I’m sure there is.

anyway, it’s amazing that no one can figure out which file / folder is missing.
We were recommending this product to one of our primary customers (we are software companies) in the Enterprise version, but we avoid because if a small problem like this stops everything and there is no possibility of debugging, we have some doubts

This looks a bit offensive to be an Enterprise customer and use open community to solve an issue “yesterday”, by not providing any data. There is Problem Template that was even not used and nobody have mind reading here. How should I guess what is missing by not seen it?

As was mentioned above, you have backup and can simply restore it and check if all works.

P.S. have same positive experience with 5 users installation and 50K users in a Free Version.