Occ files:scan --all Produces errors

Hi.
I have a Nextcloud (11.0.3 stable) on a Raspi3 with lighttpd, after having a bunch of locked files that i deleted i ran “sudo -u www-data php occ files:scan --all”, but accidentally the server got rebooted while the scan ran and now it can’t complete and produces these errors. What do i do?
(I had to remove hashtags otherwise the formatting were affected)

Exception while scanning: “files/dc9e587002a9d3d7bfd9cf3865984567” is locked
0 /var/www/html/nextcloud/lib/private/Files/Storage/Common.php(680): OC\Lock\DBLockingProvider->acquireLock(‘files/dc9e58700…’, 2)
1 /var/www/html/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php(588): OC\Files\Storage\Common->acquireLock(‘scanner::’, 2, Object(OC\Lock\DBLockingProvider))
2 /var/www/html/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php(588): OC\Files\Storage\Wrapper\Wrapper->acquireLock(‘scanner::’, 2, Object(OC\Lock\DBLockingProvider))
3 /var/www/html/nextcloud/lib/private/Files/Cache/Scanner.php(316): OC\Files\Storage\Wrapper\Wrapper->acquireLock(‘scanner::’, 2, Object(OC\Lock\DBLockingProvider))
4 /var/www/html/nextcloud/lib/private/Files/Utils/Scanner.php(196): OC\Files\Cache\Scanner->scan(’’, true, 3)
5 /var/www/html/nextcloud/apps/files/lib/Command/Scan.php(159): OC\Files\Utils\Scanner->scan(’/mads’)
6 /var/www/html/nextcloud/apps/files/lib/Command/Scan.php(227): OCA\Files\Command\Scan->scanFiles(‘mads’, ‘/mads’, false, Object(Symfony\Component\Console\Output\ConsoleOutput), false)#7 /var/www/html/nextcloud/3rdparty/symfony/console/Command/Command.php(256): OCA\Files\Command\Scan->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
8 /var/www/html/nextcloud/core/Command/Base.php(161): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
9 /var/www/html/nextcloud/3rdparty/symfony/console/Application.php(818): OC\Core\Command\Base->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
10 /var/www/html/nextcloud/3rdparty/symfony/console/Application.php(186): Symfony\Component\Console\Application->doRunCommand(Object(OCA\Files\Command\Scan), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
11 /var/www/html/nextcloud/3rdparty/symfony/console/Application.php(117): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
12 /var/www/html/nextcloud/lib/private/Console/Application.php(169): Symfony\Component\Console\Application->run(NULL, NULL)
13 /var/www/html/nextcloud/console.php(90): OC\Console\Application->run()
14 /var/www/html/nextcloud/occ(11): require_once(’/var/www/html/n…’)
15 {main}

±--------±------±-------------+
| Folders | Files | Elapsed time |
±--------±------±-------------+
| 0 | 0 | 00:00:00 |
±--------±------±-------------+

As the first line of the error states there is a file still in lock state. If you manually removed the file it will still keep the lock state in its database.

Depending on what kind of lock mechanism you use, you will need to clear it, and then do a rescan.

If you are using Redis you can run the following command (as root):

redis-cli flushall

Which will clear its cache of locked files.

@X4LD1M0 Thank you for your quick reply.
The file “files/dc9e587002a9d3d7bfd9cf3865984567” does not exist in the nextcloud data/user/flies folder.
I haven’t got much knowledge of different commands, i only installed nextcloud as it came out of the box. The only thing i changed was to put the data on an USB disk and changed the web server to lighttpd from apache, that seemed to lose connection and timed out all the time.
I was trying to rescan the files to the database with the occ command.

So you haven’t configured any locking mechanism in config.php, which means the default (built-in) is used.

You can clear it by empty the file_locks table in the database.

  1. Login to your database (if you use MySql use mysql -u root -p and type the root password to login
  2. Select the NC database use nextcloud;
  3. Truncate the file_locks table TRUNCATE TABLE <prefix>_file_locks;

Then try to scan again.

@X4LD1M0
I’m very sorry that i didn’t reply earlier.
The server seems to have fixed the error during some of it’s internal cleaning or scan.
Thank you for your help.

@X4LD1M0
And i was completely wrong. A couple of other errors related to the locked file got fixed (it could not get uploaded because it was to big. - 23Gb).
When i login to the mysql and try “use nextloud” it tells ne it’s not there:
copy pasted it
plus the status from mysql

And under it my config.php from the server. (xxx the stuff i’m not sure if should be posted)

I really appreciate your help, DB and webserver are very new to me.
Thanks you

MariaDB [(none)]> use nextcloud;
ERROR 1049 (42000): Unknown database 'nextcloud’
MariaDB [(none)]> \s

mysql Ver 15.1 Distrib 10.0.30-MariaDB, for debian-linux-gnueabihf (armv7l) using readline 5.2

Connection id: 38
Current database:
Current user: root@localhost
SSL: Not in use
Current pager: stdout
Using outfile: ''
Using delimiter: ;
Server: MariaDB
Server version: 10.0.30-MariaDB-0+deb8u1 (Raspbian)
Protocol version: 10
Connection: Localhost via UNIX socket
Server characterset: latin1
Db characterset: latin1
Client characterset: utf8
Conn. characterset: utf8
UNIX socket: /var/run/mysqld/mysqld.sock
Uptime: 2 days 3 hours 8 min 22 sec
Threads: 1 Questions: 123 Slow queries: 0 Opens: 15 Flush tables: 1 Open tables: 78 Queries per second avg: 0.000

$CONFIG = array (
‘instanceid’ => ‘ocogvukx1dj1’,
‘passwordsalt’ => 'xxxxxxxxxxxxxxx,
‘secret’ => 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxx,
‘trusted_domains’ =>
array (
0 => ‘192.168.1.102’,
),
‘datadirectory’ => ‘/mnt/usb-WD_My_Passport_0748_575855314343313931373832-0:0-part1/nextcloud/data’,
‘overwrite.cli.url’ => ‘http://192.168.1.102/nextcloud’,
‘dbtype’ => ‘sqlite3’,
‘version’ => ‘11.0.3.2’,
‘logtimezone’ => ‘UTC’,
‘installed’ => true,
‘maintenance’ => false,
‘theme’ => ‘’,
‘loglevel’ => 2,
‘updater.secret’ => '$xxxxxxxxxxxxxxxxxxxxxxxxxxxx,
);

From your NC config it shows that it is using sqlite, not mysql. So the commands I provided will be useless. I am not so experienced with sqlite, so will need to do a bit of digging into how to access the database.

sqlite has a very bad performance so it should only be used when there is not way to use a real database (some hosting environments). So I would recommend to switch to mysql first. File locking is then much faster with redis as locking cache.

On a raspberry, you don’t have too many resources but it should be enough to use mysql and redis. You might review the cache settings after some time that they are large enough for your database (but not too large).

I thought i was using mysql?
Here’s a screenshot of htop:
[I feel i bit stupid now] :-I

The mysql process might be running but it doesn’t mean it is actively used. In your config.php you wrote:[quote=“moteprime, post:6, topic:11861”]
‘dbtype’ => ‘sqlite3’,
[/quote]

Which says to NC to use sqlite. Also there are no details for username/password which would be required for MySQL.

but you can switch to mysql:
https://docs.nextcloud.com/server/11/admin_manual/configuration_database/db_conversion.html?highlight=mysql

StephanW Yes you are right. All those mysql processes in top had me convinced otherwise.

So what i want to do are, moving/converting the data from the sqlite3 data over to the mysql databade?

What are the username i should use in mysql. the raspi user er »pi«, the nextcloud user are »mads«, the webservers user are »www-data«?

The username/password to use can be defined by yourself, by adding the details to mysql. This goes more into the mysql management. I will give some examples.

First you need to login to mysql (on command line) to run the following commands. Normally this is done as root user, and depending if you have set a password, you will also need to provide it to be able to login. (Recommendation is to always have root set with a password for mysql for security purposes).

Then you can run the following commands:

CREATE DATABASE 'nextcloud' CHARACTER SET utf8 COLLATE utf8_bin; GRANT ALL PRIVILEGES ON nextcloud.* 'nextcloud_user'@'localhost' IDENTIFIED BY 'nextcloud_pass'; FLUSH PRIVILEGES;

What this does is the following:

  1. Create the (clear new) database
  2. Set the user privileges on the new database
  3. Create the new user with password

Once this is done you can run the conversion command:

php occ db:convert-type --all-apps mysql nextcloud_user 127.0.0.1 nextcloud

It will ask for the password, which is the one defined in the GRANT command as ‘nextcloud_pass’.

If course you can change the user and password to what every you want, as long you make sure you define them also in the conversion command.

@X4LD1M0 Thank you, that’s exactly what i needed, especially the nextcloud user.
Is that user defined in the nextcloud serverscripts?
Are the character specification necessary?

@X4LD1M0
I’m sorry to ask again, but the thing a cannot figure out are what i put in i the “”
‘nextcloud’ Isn’t it just nextcloud. I haven’t changed anything from standard installation.
And the ‘nextcloud_user’@‘localhost’, Is it the web user, the linux user or the nextcloud user, and ‘localhost’ are the 127.0.0.1 ip number or can i just leave locahost?

It is my first time working with a database myself.

That is OK, we all learn new things every day :slight_smile:

The syntax for the lines are:

CREATE DATABASE <database name> CHARACTER SET utf8 COLLATE utf8_bin; GRANT ALL PRIVILEGES ON <database name>.* '<db username>'@'localhost' IDENTIFIED BY '<db password>';

  • <database name> is the new database which you assign to store all the NextCloud data.
  • <db username> is the unique username which the NC application will use to identify itself to mysql (which will give it access to the <database name> only
  • <db password> is the password associated with the <db username>

So the username and password are separate details only for mysql login. Using this process you can have unique usernames which give only access to specific databases, so you can have multiple applications talking to the same mysql process, but each can only access its own information due to the login process.

You can leave localhost, since this is by default set in the /etc/hosts file to 127.0.0.1. If you want you can replace it with 127.0.0.1 just to bypass the DNS process.

Hope this helps you further.

The user will also need to be set in the config.php so NextCloud knows the login details to mysql.

More details can be found in:

https://docs.nextcloud.com/server/11/admin_manual/configuration_server/config_sample_php_parameters.html

@X4LD1M0 Thank you so much.
I’m sorry i do not reply faster, but i’m working on this on the side. I’m am very grateful for your help, and will get back with results and more appreciation as soon as I can. :wink: