Issue when running occ files:scan with external storage

Hello,

I am running Nextcloud Hub 5 (27.0.0) on Ubuntu minimal 22.04.2 latest patched.
Beside that:
Server version: Apache/2.4.52 (Ubuntu)
PHP: 8.2.7 (cli)
MariaDB: 15.1

My configuration is as follows:
A bare metal virtual machine, dedicated to Nextcloud. Basically followed this post with some deviations how Apache was set up, additional packages etc.
The application is located at /var/www/html directly, Apache setup is without virtual host, just direct access via 000-default.conf. Port has been changed though. Access to Nextcloud is done via OPNsense HAproxy, externally, even if I am in my own network.
Datadisk is a 2nd disk (sdb1), which is an iSCSI target on my TrueNAS, which is running on the same host (ESXi), but a different VM. It is mounted as /mnt/nextcloud/nextcloud-data.
And finally, why I mainly built Nextcloud, was to share my photos over multiple devices, Windows PC, Macbook, iPhone, iPad, basically replacing iCloud.
The photos are also located on TrueNAS, shared via SMB. (better said, there is one share, and /Photos is a subdir there).
In Nextcloud, I connect that share via “External Storage”.

Issue:
After connecting the share, I observe some weird behavior.
First, files are not shown, until I click into the folder. And some folders don’t show anything, although the files are there.
I checked and compared permissions on various folders inside Photos, and couldn’t see any differences. I even reset all ACL recursively, to make sure, there are no differences.
Also, some folders are showing Size “Pending”, some show the size, latter is I think for those that I accessed and Nextcloud was able to enumerate.

Through troubleshooting I was able to determine this behavior:
Let’s say I have a folder 2012, which doesn’t show my any folders or files, if I rename the folder to 2012x in windows, and wait 1-2 seconds for it to update in Nextcloud, when I open the folder 2012x, I suddenly see the files. If I rename it back to 2012, I don’t see any files. Rename it back to 2012x, files are visible.
Then I run “sudo -u www-data php occ files:scan --all” from the shell", check the folder 2012x, everything is gone. If I rename that folder back to 2012, files are visible. If I re-run the command, files are gone. I could repeat this all day…

I checked the www-data permissions on both /mnt/nextcloud/nextcloud-data and /var/www/html, both show for all folders www-data. Not that it would matter in my opinion.

Bear in mind, I installed Nextcloud just yesterday, so it is a completely fresh install.

I also installed Preview Generator, and ran the command to first generate previews. It stopped at a certain folder. I found out that it did that, because the folder contained .DS-Store file that were previously generated by Synology (yes, the Photos-Folder was copied over from Synology). Removing those files, allowed Preview Generator to continue.
However, it does not create previews for all the folders, only those that I access via Nextcloud interface.

So I need to solve two things:

  1. how do I recursively browse all the folders on the external drive (how do I even access that drive from the shell?), so that preview generator can create previews for ALL photos (and/or files)?
  2. how do I fix the issue with files:scan command, that apparently does something so that files become hidden?

And finally, Memories also appears to have problems, because it doesn’t “import” everything, but I believe if I solve the previous problems, Memories should work fine too.

I checked Admin → Logging, nothing there but those errors generated by preview generator:

This is my config.php file:

‘htaccess.RewriteBase’ => ‘/’,
‘overwriteprotocol’ => ‘https’,
‘instanceid’ => ‘xxxxxxxxxxxx’,
‘passwordsalt’ => ‘xxxxxxxxxxxxxxxx’,
‘secret’ => ‘xxxxxxxxxxxxxxxxxxxxx’,
‘trusted_domains’ =>
array (
0 => ‘xxxxxxxxxxxxxx’,
1 => ‘xxxxxxxxxxxxxxxxxxxxxx’,
),
‘datadirectory’ => ‘/mnt/nextcloud/nextcloud-data’,
‘dbtype’ => ‘mysql’,
‘version’ => ‘27.0.0.8’,
‘overwrite.cli.url’ => ‘xxxxxxxxxxxxxxxx’,
‘dbname’ => ‘xxxxxxxxxxxxxxx’,
‘dbhost’ => ‘localhost’,
‘dbport’ => ‘’,
‘dbtableprefix’ => ‘oc_’,
‘mysql.utf8mb4’ => true,
‘dbuser’ => ‘xxxxxxxxxxxx’,
‘dbpassword’ => ‘xxxxxxxxxxxxxxxxxxxx’,
‘installed’ => true,
‘mail_smtpmode’ => ‘smtp’,
‘mail_smtpsecure’ => ‘tls’,
‘mail_sendmailmode’ => ‘smtp’,
‘mail_from_address’ => ‘xxxxxxxxxxxxx’,
‘mail_domain’ => ‘xxxxxxxxxxxxx’,
‘mail_smtpauthtype’ => ‘LOGIN’,
‘mail_smtpauth’ => 1,
‘mail_smtphost’ => ‘xxxxxxxxxxxxx’,
‘mail_smtpport’ => ‘587’,
‘mail_smtpname’ => ‘xxxxxxxxxxxxxxxxxx’,
‘mail_smtppassword’ => ‘xxxxxxxxxxxxxxxx’,
‘default_phone_region’ => ‘xx’,
‘loglevel’ => 2,
‘maintenance’ => false,
‘memcache.local’ => ‘\OC\Memcache\APCu’,
‘memcache.distributed’ => ‘\OC\Memcache\Redis’,
‘memcache.locking’ => ‘\OC\Memcache\Redis’,
‘redis’ =>
array (
‘host’ => ‘localhost’,
‘port’ => 0,
‘timeout’ => 1.5,
‘dbindex’ => 0,
),
‘app_install_overwrite’ =>
array (
0 => ‘memories’,
),
‘memories.exiftool’ => ‘/var/www/html/apps/memories/exiftool-bin/exiftool-amd64-glibc’,
‘memories.vod.path’ => ‘/var/www/html/apps/memories/exiftool-bin/go-vod-amd64’,
‘memories.gis_type’ => 1,

nextcloud.log in /var/www/html/data doesn’t really have much in it:
{“reqId”:“2t1LVBU4oPqONmLK2IXP”,“level”:3,“time”:“2023-06-22T12:22:48+00:00”,“remoteAddr”:“xxxxxxxxxxxxxxx”,“user”:“–”,“app”:“PHP”,“method”:“GET”,“url”:“/ocs/v2.php/apps/notifications/api/v2/notifications”,“message”:“OCP\HintException: C>
{“reqId”:“nreFc4yQvXE6ws1L7QwF”,“level”:3,“time”:“2023-06-22T12:53:52+00:00”,“remoteAddr”:“xxxxxxxxxxxxxx”,“user”:”–“,“app”:“remote”,“method”:“GET”,“url”:”/status.php",“message”:"Configuration was not read or initialized correctly, not o>

I really really hope someone can help. If any additional info is needed, anytime.

Thank you
Kosta

Hello,

I think this topic can be closed. Due to various issues with this configuration, and with the fact that another configuration seems to work flawlessly, I decided to abandon this configuration.

I have mounted the appropriate Photos share on TrueNAS into /mnt and then used External Storage “Local” connection to bind it to all users. This configuration seems to work much better.