Files:scan does not find all files uploaded via ftp

grafsimeon · April 8, 2020, 9:35am

My installation:

Nextcloud version 17.0.0.9
Operating system and version: webhostet, I don’t have full root access.
Apache or nginx version: apache unknown version
PHP version: 7.2.28

serverinfo:

The output of your Nextcloud log in Admin > Logging:

nextclou.log bei dropbox

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'instanceid' => '...',
  'passwordsalt' => '...',
  'secret' => '...',
  'trusted_domains' => 
  array (
    0 => 'domain.tld',
  ),
  'datadirectory' => '/var/www/vhosts/hostingxxxx.netcup.net/path/data',
  'dbtype' => 'mysql',
  'version' => '17.0.0.9',
  'overwrite.cli.url' => 'https://domain.tld',
  'dbname' => '',
  'dbhost' => '',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'admin',
  'dbpassword' => '....',
  'installed' => true,
  'mail_domain' => 'domain.tld',
  'mail_from_address' => 'mail',
  'mail_smtpmode' => 'smtp',
  'mail_smtpsecure' => 'tls',
  'mail_sendmailmode' => 'smtp',
  'mail_smtphost' => 'xxxx.netcup.net',
  'mail_smtpauthtype' => 'LOGIN',
  'mail_smtpauth' => 1,
  'mail_smtpname' => 'mail@domain.tld',
  'mail_smtppassword' => '....',
  'maintenance' => false,
  'updater.release.channel' => 'stable',
  'theme' => '',
  'loglevel' => 2,
  'app_install_overwrite' => 
  array (
    0 => 'occweb',
  ),
);

The output of your Apache/nginx/system log in /var/log/____:

error_log bei dropbox

As my Nextcloud instance cannot handle big files sizes in upload because of my webhosted server. So I upload bigger files via ftp and then scan them with occ app command files:scan --all. But it is not finding all files.
The strange thing about it is, that it finds some files, but not all.
I tried adding an external storage (ftp) which points to my webspace. In a seperate folder than nextcloud is running but it is still the same problem.

Is this the first time you’ve seen this error? (Y/N): No, but the first time I have time to care about it.

I would be very happy do get a little help here :). Please let me know if you need any more information.
THANKS!

devnull · April 8, 2020, 10:28am

Perhaps it is better to use external storage “local”.

And what about the app “Flow Upload” for big files?
https://apps.nextcloud.com/apps/flowupload

grafsimeon · April 8, 2020, 11:02am

Hi!
thanks for the answer.
I tried flowupload a while ago already and it did not work correctly. Maybe I can give it another try.

But I can’t geht the “local” option with external storage to work. What kind of path do I have to enter in the web gui?

devnull · April 8, 2020, 11:06am

Look a screenshot:

grafsimeon · April 8, 2020, 11:11am

This is the php error:

stat(): stat failed for external-nc-data/ at /var/www/vhosts/hostingxxxx.xxxx.netcup.net/path/domain.tld/lib/private/Files/Storage/Local.php#143

devnull · April 8, 2020, 11:14am

I think you must use as “Konfiguration” the full path.

/var/www/vhosts/.../external-nc-data

If you use a vps or similar post:
ls -al /external-nc-data
(only dirs . and …)

You can also switch in the directory external-nc-data and post
pwd

grafsimeon · April 8, 2020, 11:20am

sorry, what is vps?

when I use the full path the nextcloud log spams:

realpath(): open_basedir restriction in effect. File(/var/www/vhosts/hostingxxxx.xxxx.netcup.net/external-nc-data) is not within the allowed path(s): (/var/www/vhosts/hostingxxxx.xxxx.netcup.net/path/domain.tld/:/tmp/:/var/lib/php5/sessions:/var/lib/php/sessions:/var/www/vhosts/hostingxxxx.xxxx.netcup.net/tmp) at /var/www/vhosts/hostingxxxx.xxxx.netcup.net/path/domain.tld/lib/private/Files/Storage/Local.php#69

Nemskiller · April 8, 2020, 11:20am

Did you modify chown and chmod of the files you uploaded by ftp ?
In my case when i uploaded files with scp, i had to do chmod/chown on these files then occ files:scan user to make them visible in Nextcloud

devnull · April 8, 2020, 11:23am

You must find the full path for your directory “external-nc-data”.

/var/www/vhosts/hostingxxxx.xxxx.netcup.net/path/domain.tld/external-nc-data

???

Can you use a terminal (SSH).
Go to the directory with “cd” and
pwd

Do you use ftp?
Compare the dirs in ftp and in www.
Add /var/www/vhosts/…/ to the dir-name “external-nc-data”.

grafsimeon · April 8, 2020, 11:25am

I did check the write/read rights wihtin my ftp program and set it to the same which other files had. in my case 644.

devnull · April 8, 2020, 11:26am

Yes. But what about the paths?
If you only use Web and FTP you have no directory /
All starts with /var/www/vhosts/hostingxxxx.xxxx.netcup.net

Upload in an admin-dir this php-filemanager:

Only one index.php in one directory.
Start it and look at the path.

Nemskiller · April 8, 2020, 11:38am

And the owner is : www-data or httpd ?

grafsimeon · April 8, 2020, 12:43pm

when i do this i get the ouptut: /

how do i check the owner? and how do i edit it?

Nemskiller · April 8, 2020, 1:02pm

Into the server
Go in the folder before your ftp uploaded folder.
Type
: ls -ll
Then return
You will for exemple :
Ftpuset:root 2020/04/03 Myfolder
In this case the user owner is ftpuser and the group owner is root.

If you are on Debian/Ubuntu you have to do chown -R www-data:www-data Myfolder

Look at the Nextcloud documentation about Checking permission you will see examples.

grafsimeon · April 8, 2020, 1:07pm

okay … haha this is super difficult for me. Dont think i can fix this.

maybe I just use something else than nextcloud. but I like the idea having the files on my own webspace.

Nemskiller · April 8, 2020, 1:37pm

Don’t give up.
I learnt how to use Linux thanks to Nextcloud.

Look on the web about videos or “how to“ on use Linux / Nextcloud

devnull · April 8, 2020, 1:40pm

Please upload this file (file manager) to filemanager.php or directory/index.php and start it in the browser.

https://raw.githubusercontent.com/dulldusk/phpfm/master/index.php
(save as and then upload with ftp)

Screenshot:

https://camo.githubusercontent.com/db2fbd735329db17ee4ac60703d05b9505efb44a/68747470733a2f2f7777772e64756c6c6475736b2e636f6d2f6173736574732f66696c65732f706870666d2f7373312e372e322e706e67

Look at the path for external data directory.

grafsimeon · April 8, 2020, 1:48pm

Okay guys, I found my solution:

I had files with special german letters like “ä” in the name. I renamed them and suddenly occ command found them!

thank you for the help everyone!

the path above is correct. but i think netcup hosting provider is kind of blocking nextcloud to access it? can that be?

devnull · April 8, 2020, 1:51pm

Tja. Deutsche Umlaute sollten abgeschafft werden
Hast du denn den Filemanager noch ausprobiert?

grafsimeon · April 8, 2020, 2:08pm

Ja, auf den ersten blick sieht der echt gut aus. Aber muss mich mal in ruhe damit beschäftigen. Eventuell kann ich den für andere Sachen gut gebrauchen.