Folder doesn't open after server copy and occ files:scan

lawrence · May 17, 2022, 9:22am

[/details]
Nextcloud version : 24.0.0
Operating system and version : Debian 10
Apache or nginx version : 2.4.38
PHP version : 8.1.5

The issue you are facing:
Folders copied to the NextCloud data directory, following by a php occ files:scan as advised in the Administration Manual, shows the folder in the correct place on the NextCloud desktop. The problem is that I can’t then open the directory and see the files, even though the findividual files are shown as “new uploads” on the NextCloud desktop.

This is my first installation of NextCloud server, on a virtual server on my VPS running Debian 10.

When I upload a folder of files by dragging into the browser (Chrome on Windows 10) the process works smoothly, and I can access the folder as expected. No problem at all.

I was then looking for a way to avoid downloading the files from my FTP server, which is on the same VPS, and uploading them again. I read in a post here that it is possible to copy files into the correct location, change permissions and ownership to the correct settings, and then run occ files:scan. When I did this I could not access the folder (via NextCloud) that was copied on the server, although the folder was actually visible on the desktop.

I then renamed the folder (successfully) but it was still not accessible.

I then deleted the folder completely, and uploaded the same folder via the normal method through my Chrome browser (on Windows 10). The problem persists with the normally uploaded folder, and I still can’t access it, although I can see it. I have also tried to rename this folder, which works, but still can’t access the folder.

I have checked (via SSH) the ownership of both the directory and all the files within it, and all the permissions of directory and files, and they appear to be all correct.

I have re-run php occ files:scan --all as well as trying php occ files:cleanup and php occ files:scan-app-data and none of these commands make any difference.

Is this the first time you’ve seen this error? : Y

Steps to replicate it:

Copy a directory of files into the NextCloud data directory on the host server
Run php occ files:scan
Copied directory appears in NextCloud, but is inaccessible

The output of your Nextcloud log in Admin > Logging:

[PHP] Error: Error: unlink(/home/cloud/public_html/data/appdata_ochhhj09d6pe/css/icons/icons-list.template): No such file or directory at /home/cloud/public_html/lib/private/Files/Storage/Local.php#305 at <<closure>>

 0. <<closure>>
    OC\Log\ErrorHandler::onError()
 1. /home/cloud/public_html/lib/private/Files/Storage/Local.php line 305
    unlink()
 2. <<closure>>
    OC\Files\Storage\Local->unlink()
 3. /home/cloud/public_html/apps/files_trashbin/lib/Storage.php line 193
    call_user_func()
 4. /home/cloud/public_html/apps/files_trashbin/lib/Storage.php line 102
    OCA\Files_Trashbin\Storage->doDelete()
 5. /home/cloud/public_html/lib/private/Files/View.php line 1177
    OCA\Files_Trashbin\Storage->unlink()
 6. /home/cloud/public_html/lib/private/Files/View.php line 733
    OC\Files\View->basicOperation()
 7. /home/cloud/public_html/lib/private/Files/Node/File.php line 131
    OC\Files\View->unlink()
 8. /home/cloud/public_html/lib/private/Files/SimpleFS/SimpleFile.php line 149
    OC\Files\Node\File->delete()
 9. /home/cloud/public_html/lib/private/Template/SCSSCacher.php line 386
    OC\Files\SimpleFS\SimpleFile->delete()
10. /home/cloud/public_html/lib/private/Template/SCSSCacher.php line 278
    OC\Template\SCSSCacher->resetCache()
11. /home/cloud/public_html/lib/private/Template/SCSSCacher.php line 142
    OC\Template\SCSSCacher->variablesChanged()
12. /home/cloud/public_html/lib/private/Template/CSSResourceLocator.php line 109
    OC\Template\SCSSCacher->process()
13. /home/cloud/public_html/lib/private/Template/CSSResourceLocator.php line 61
    OC\Template\CSSResourceLocator->cacheAndAppendScssIfExist()
14. /home/cloud/public_html/lib/private/Template/ResourceLocator.php line 78
    OC\Template\CSSResourceLocator->doFind()
15. /home/cloud/public_html/lib/private/TemplateLayout.php line 342
    OC\Template\ResourceLocator->find()
16. /home/cloud/public_html/lib/private/TemplateLayout.php line 250
    OC\TemplateLayout::findStylesheetFiles()
17. /home/cloud/public_html/lib/private/legacy/OC_Template.php line 185
    OC\TemplateLayout->__construct()
18. /home/cloud/public_html/lib/public/AppFramework/Http/TemplateResponse.php line 204
    OC_Template->fetchPage()
19. /home/cloud/public_html/lib/private/AppFramework/Http/Dispatcher.php line 178
    OCP\AppFramework\Http\TemplateResponse->render()
20. /home/cloud/public_html/lib/private/AppFramework/App.php line 172
    OC\AppFramework\Http\Dispatcher->dispatch()
21. /home/cloud/public_html/lib/private/Route/Router.php line 298
    OC\AppFramework\App::main()
22. /home/cloud/public_html/lib/base.php line 1023
    OC\Route\Router->match()
23. /home/cloud/public_html/index.php line 36
    OC::handleRequest()

GET /index.php/apps/dashboard/
from 146.212.254.35 by cloudmin at 2022-05-17T13:09:46+00:00

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

?php
$CONFIG = array (
  'instanceid' => 'xxx-redacted-xxx',
  'passwordsalt' => 'xxx-redacted-xxx',
  'secret' => 'xxx-redacted-xxx',
  'trusted_domains' => 
  array (
    0 => 'cloud.xxxxxxxx.xxx',
  ),
  'datadirectory' => '/home/cloud/public_html/data',
  'dbtype' => 'mysql',
  'version' => '24.0.0.12',
  'overwrite.cli.url' => 'https://cloud.xxxxxxxx.xxx',
  'dbname' => 'cloud',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'cloud',
  'dbpassword' => 'xxxxxxxxxxxx',
  'installed' => true,
);

The output of your Apache/nginx/system log in /var/log/____:

[Tue May 17 00:00:00.592085 2022] [mpm_prefork:notice] [pid 16556] AH00163: Apache/2.4.38 (Debian) mod_fcgid/2.3.9 OpenSSL/1.1.1n configured -- resuming normal operations
[Tue May 17 00:00:00.592145 2022] [core:notice] [pid 16556] AH00094: Command line: '/usr/sbin/apache2'

mritzmann · May 17, 2022, 10:50am

You have to enable the app “Log Reader” so that you can view the log in the frontend.

lawrence · May 17, 2022, 11:48am

Thanks for the advice @mritzmann. I have checked the list of “Active Apps” and the Log Reader seems to be enabled.

I just don’t know how to access the app. I can’t find it anywhere on the dashboard. Do I have to add it as a widget or something? There doesn’t seem to be a menu with Apps in anywhere that I can see…

mritzmann · May 17, 2022, 1:30pm

As admin User:

Open Settings (top right)
“Logging”

/index.php/settings/admin/logging

Alternatively, you can find the same log in the file system. In your case:

/home/cloud/public_html/data/nextcloud.log

lawrence · May 17, 2022, 4:12pm

Thanks again @mritzmann ! I have found the Logging item in the settings, but it (logically) only displays the last ‘X’ number of entries.

I have also found the nextcloud.log file and managed to download it. I did find it before but it is over 250kb and quite a mess to sift through, and I don’t have the knowledge to filter the most relevant entries in any automated way. So, I have taken one recent entry that is related to PHP, and pasted that into my post above. I will have a look through for others as I have time and energy. In the meantime, if you know of a good way to filter the entries then please do let me know - a simple grep of the file is just not cutting it!!

lawrence · May 17, 2022, 4:14pm

I would appreciate knowing if anyone can identify this as a known issue, or is it likely that there is something wrong with my installation of NextCloud?

What else can I check to identify the problem, rather than sifting through logs? I will do that also of course, but if I can avoid it…

mritzmann · May 17, 2022, 4:47pm

I know you’ve already ruled that out, but to me it sounds exactly like an permissions problem.

Change owner of files:

chown -R <user> /home/cloud/public_html/data

Please note that the files do not have to belong to your Linux user but to the user under which PHP is running (this can of course be cloud or something else, e.g www-data).

To check with which user php is running, you can create a new PHP file, write the following into it and access the PHP file in the browser.

<?php echo exec('whoami'); ?>

mritzmann · May 17, 2022, 4:48pm

This issue sounds like a configuration error to me.

lawrence · May 18, 2022, 6:00am

Thanks again Markus (@mritzmann)

I have just now:

checked again the Apache / PHP user and it is www-data as expected (thanks for the scriptlet!)
as you suggested, I have again chown -R www-data:www-data * in the public_html directory
then finally I ran again the php occ files:scan as the www-data user

After clearing the browser cookies and cache, the problem still persists with this one directory.

It does sound like a config problem to me also, but the thing is I haven’t configured anything beyond the plain vanilla installation.

The only different factor related to this specific folder is that it was originally created by copying the folder and files from my seedbox (which runs on the same VPS), changing ownership / permissions and then running the php occ files:scan --all using the www-data user. Despite having deleted the original folder and files that were uploaded that way, and then uploading them the “normal” way, the problem still persists, while other folders and files are working fine.

lawrence · May 18, 2022, 8:55am

Hello again (@mritzmann & all… is anyone else out there??! he he)

I have now tried the following:

deleted the “offending” folder through the NextCloud dashboard
checked via SSH session that the folder and files are indeed no longer present (they were not)
re-copied the content again from another directory on the same VPS
changed owner and group of the replaced folder and files to www-data recursively
checked if the folder and files are present in the NC dashboard (they were not, as expected)
renamed the folder to remove any potential “problematic” characters
run the command sudo -u www-data php occ files:scan --all via SSH while in the /home/cloud/public_html directory
checked that the folder then is present in the NC dashboard (it is)
checked if I can open the folder and see the contained files (I can’t)

So, it seems that the problem is still present. The strange thing is that my brother, who is my “test user” on the system, can download the “offending” folder and contained files after I shared it with him.

Does anyone have any thoughts on this? Is there a bug in the files:scan script? Is it because of the size of the files (they are over 50Mb each)?

I have in the meantime tried the same process with another folder of music files, and it worked perfectly. The folder name and the filenames have an identical structure to the one that isn’t openable. The only difference is that the files in this new folder are “only” around 20-30Mb each.

mritzmann · May 18, 2022, 9:52am

Can you run a tail and post the output here again?

tail -f -n0 nextcloud.log
Open folder
end tail -f -n0 nextcloud.log
Post output

lawrence · May 18, 2022, 10:29am

This is what I get:
{"reqId":"ZGTZa2JeTu3PTslPZwDZ","level":2,"time":"2022-05-18T10:28:04+00:00","remoteAddr":"198.235.24.139","user":"--","app":"core","method":"GET","url":"/","message":"Trusted domain error. \"198.235.24.139\" tried to access using \"villabeon.com\" as host.","userAgent":"Expanse, a Palo Alto Networks company, searches across the global IPv4 space multiple times per day to identify customers' presences on the Internet. If you would like to be excluded from our scans, please send IP addresses/domains to: scaninfo@paloaltonetworks.com","version":"24.0.0.12"}

Nothing seems to happen related to the folder in question.

mritzmann · May 18, 2022, 11:47am

I’m sorry, but I don’t know what else to do. Maybe someone else can help you.

lawrence · May 18, 2022, 12:29pm

OK, well thanks so much for all the help so far! You have helped my understanding a lot.

Let’s see if anyone else can comment. I am a little suprised that you are the only one that has replied so far, and that so many of the topics on this forum remain unsolved (from a casual browse through it seems that <10% are solved). Is that normal with the NextCloud software, even when it claims to be at v24??

mritzmann · May 18, 2022, 12:57pm

I think most people just don’t feel like providing support for free, which I can understand. The vast majority of questions here are setup issues that Nextcloud has nothing to do with.

On the other hand, yes, there are bugs. But these are often reported on GitHub and not here.

even when it claims to be at v24

Personally, I never used the latest major version because I experienced bugs too often (the old major version still gets security patches). Therefore: I have not yet tested Nextcloud v24 and therefore cannot say anything about its stability.

lawrence · May 20, 2022, 10:04am

Is there anyone on here that works for NextCloud or develops the product?

Or are you guys just relying on customers to support each other?

Thanks again Markus for the help so far!