Php occ files:scan results un segmentation error on specific path

ℹ️ Support
#1
Support intro

Sorry to hear you’re facing problems :slightly_frowning_face:

help.nextcloud.com is for home/non-enterprise users. If you’re running a business, paid support can be accessed via portal.nextcloud.com where we can ensure your business keeps running smoothly.

In order to help you as quickly as possible, before clicking Create Topic please provide as much of the below as you can. Feel free to use a pastebin service for logs, otherwise either indent short log examples with four spaces:

example

Or for longer, use three backticks above and below the code snippet:

longer
example
here

Some or all of the below information will be requested if it isn’t supplied; for fastest response please provide as much as you can :heart:

Nextcloud version (eg, 20.0.5): 21.0.1
Operating system and version (eg, Ubuntu 20.04): CentOS 7.9
Apache or nginx version (eg, Apache 2.4.25): Nginx 1.16.1
PHP version (eg, 7.4): 7.4

The issue you are facing:

When running a file:scan on a particular account, I am getting this error…

[root@cloudnine php.d]# sudo -u nginx php /var/www/nextcloud/occ files:scan --path <user>
Starting scan for user 1 out of 1 (<user>)
Speicherzugriffsfehler

Along with this error, the only other mention of this issue is in the syslog:

cloudnine kernel: php[32274]: segfault at 7ffc49ee9f68 ip 00007f955fc0e2d0 sp 00007ffc49ee9f60 error 6 in json.so[7f955fc09000+9000]

We recently updated this NC instance from 19 to 20 to 21.0.1 and we did not have any other apparent issue, until someone complained about their files not being shown, when logging into their account.

Is this the first time you’ve seen this error? (Y/N): Y

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'instanceid' => 'xxxxxxxxx',
  'passwordsalt' => 'yyyyyyyyyyyyy',
  'secret' => 'zzzzzzzzzzz',
  'trusted_domains' =>
  array (
    0 => 'cloudnine.foo.bar',
    1 => 'collabora.foo.bar',
  ),
  'datadirectory' => '/mnt/nextcloud/data',
  'overwrite.cli.url' => 'https://cloudnine.foo.bar',
  'overwritehost' => 'cloudnine.foo.bar',
  'dbtype' => 'mysql',
  'version' => '21.0.1.1',
  'dbname' => 'nextcloud',
  'dbhost' => 'localhost',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'dbuser',
  'dbpassword' => 'dbpass',
  'logtimezone' => 'UTC',
  'installed' => true,
  'ldapIgnoreNamingRules' => false,
  'mail_from_address' => 'cloudnine',
  'mail_smtpmode' => 'smtp',
  'mail_domain' => 'noreply.foo.bar',
  'mail_smtphost' => 'relay.foo.bar',
  'mail_smtpport' => '25',
  'preview_libreoffice_path' => '/opt/libreoffice4.4/program/soffice',
  'enable_previews' => true,
  'enabledPreviewProviders' =>
  array (
    0 => 'OC\\Preview\\Image',
    1 => 'OC\\Preview\\MP3',
    2 => 'OC\\Preview\\TXT',
    3 => 'OC\\Preview\\PDF',
    4 => 'OC\\Preview\\Movie',
    5 => 'OC\\Preview\\Photoshop',
    6 => 'OC\\Preview\\TIFF',
    7 => 'OC\\Preview\\SVG',
    8 => 'OC\\Preview\\OpenDocument',
  ),
  'user_sso_auth_endpoint' => 'https://id.foo.bar/_id/auth/authorize',
  'user_sso_token_endpoint' => 'http://openidconnect.foo.bar:pppp/c2id/token',
  'user_sso_userinfo_endpoint' => 'http://openidconnect.foo.bar:pppp/c2id/userinfo',
  'user_sso_client_id' => 'xxxxxxxxxxx',
  'user_sso_client_secret' => 'yyyyyyyyyyyyyyy',
  'user_sso_callback_uri' => 'https://cloudnine.foo.bar/index.php/apps/user_sso/callback',
  'loglevel' => 2,
  'logfile' => '/var/log/nextcloud.log',
  'theme' => '',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'memcache.local' => '\\OC\\Memcache\\Redis',
  'redis' =>
  array (
    'host' => '/var/run/redis/redis.sock',
    'port' => '0',
    'timeout' => '0.0',
  ),
  'trusted_proxies' =>
  array (
    0 => '127.0.0.1',
  ),
  'forwarded_for_headers' =>
  array (
    0 => 'HTTP_X_FORWARDED_FOR',
    1 => 'HTTP_FORWARDED_FOR',
  ),
  'trashbin_retention_obligation' => '7,30',
  'maintenance' => false,
  'ldapProviderFactory' => '\\OCA\\User_LDAP\\LDAPProviderFactory',
  'updater.release.channel' => 'stable',
  'mail_smtpauthtype' => 'LOGIN',
  'mysql.utf8mb4' => true,
  'app_install_overwrite' =>
  array (
    0 => 'passman',
  ),
  'encryption.legacy_format_support' => false,
  'encryption.key_storage_migrated' => false,
  'default_phone_region' => 'DE',
);

I am suspecting some php-related config issue, but I can’t seem to find the real culprit. I checked the basic PHP settings, but couldn’t come up with anything useful. Also, I seem to be unable to get php more chatty, about what is going on, so any advice on how to dig deeper into this issue is much appreciated.

#2

Are there any possibly related errors in the server’s syslog?

#3

Well, this is my main concern - there is actually nothing about this in the nextcloud log. Since php simply seems to crash whem running occ from the cli, I am out of ideas on how to actually debug that. The entries in /var/log/messages and dmesg don’t give a lot away.

#4

One thing I can provide though is, when running php occ files:scan -vvv it will output the list of folders it scans and then, after the last one, php will segfault.

#5

Okay, so to wrap this up… we finally managed to get to the root cause of this issue. There had obviously been a problem, when two old accounts were due for removal. For reasons, that I am not able to uncover, two accounts remained on the data store and also in the oc_ldap_user_mapping table, as well as the oc_accounts table.

Along with their stale shares, this all mixed up into a situation, where any occ command, which queried the of_filecache and oc_shares went nuts and issued 16k requests to the database bevore the connection to the database was shutdown by MariaDB.

I was only able to uncover that, by tweaking the MariaDB loglevels, including the sql queries, to log each and every sql request. Then I checked the sessions, which were aborted and finally arrived that the accounts, which should have already long been gone.

I have no idea, why this didn’t surface when we were still running NC19, since these accounts should have been deleted way longer than that. The take away is: sometimes you’ll have to dig really deep into stuff, to get to the nuggets. :wink: