During update from 13.0.5 to 13.06 database error: SQLSTATE[23000]: Integrity constraint violation: 19 UNIQUE constraint failed: oc_filecache.fileid

Support intro

Sorry to hear you’re facing problems :slightly_frowning_face:

help.nextcloud.com is for home/non-enterprise users. If you’re running a business, paid support can be accessed via portal.nextcloud.com where we can ensure your business keeps running smoothly.

The following error is reported during the update from 13.0.5 to 13.0.6
Using the same update procedure with previous releases, that normally works fine.

# occ upgrade
Nextcloud or one of the apps require upgrade - only a limited number of commands are available
You may use your browser or the occ upgrade command to do the upgrade
Set log level to debug
Doctrine\DBAL\Exception\UniqueConstraintViolationException: An exception occurred while executing 'INSERT INTO oc_filecache (fileid, storage, parent, name, mimetype, mimepart, mtime, encrypted, path, path_hash, etag, storage_mtime, permissions, size, unencrypted_size, checksum) SELECT fileid, storage, parent, name, mimetype, mimepart, mtime, encrypted, path, path_hash, etag, storage_mtime, permissions, size, unencrypted_size, checksum FROM __temp__oc_filecache':                                                                                          

SQLSTATE[23000]: Integrity constraint violation: 19 UNIQUE constraint failed: oc_filecache.fileid                                          
Update failed
Maintenance mode is kept active
Reset log level

Some or all of the below information will be requested if it isn’t supplied; for fastest response please provide as much as you can :heart:

Nextcloud version (eg, 12.0.2): 13.0.5 -> 13.0.6
Operating system and version (eg, Ubuntu 17.04): openSUSE_42.2
Apache or nginx version (eg, Apache 2.4.25): apache2 2.4.23
PHP version (eg, 7.1): 7.0.7

The issue you are facing:
Upgrade fails

Is this the first time you’ve seen this error? (Y/N): Y

Steps to replicate it:

  1. install nextcloud 13.0.6
  2. occ upgrade

The update procedure is the same as all preceding updates. Those were succesfull.

The output of your Nextcloud log in Admin > Logging:

{"reqId":"84o4cpC4DpTk9erpjvc8","level":0,"time":"2018-09-08T15:32:35+02:00","remoteAddr":"","user":"--","app":"core","method":"--","url":"--","message":"starting upgrade from 13.0.5.2 to 13.0.6.1","userAgent":"--","version":"13.0.5.2"}
{"reqId":"84o4cpC4DpTk9erpjvc8","level":3,"time":"2018-09-08T15:32:39+02:00","remoteAddr":"","user":"--","app":"core","method":"--","url":"--","message":"Exception: {\"Exception\":\"Doctrine\\\\DBAL\\\\Exception\\\\UniqueConstraintViolationException\",\"Message\":\"An exception occurred while executing 'INSERT INTO oc_filecache (fileid, storage, parent, name, mimetype, mimepart, mtime, encrypted, path, path_hash, etag, storage_mtime, permissions, size, unencrypted_size, checksum) SELECT fileid, storage, parent, name, mimetype, mimepart, mtime, encrypted, path, path_hash, etag, storage_mtime, permissions, size, unencrypted_size, checksum FROM __temp__oc_filecache':\\n\\nSQLSTATE[23000]: Integrity constraint violation: 19 UNIQUE constraint failed: oc_filecache.fileid\",\"Code\":0,\"Trace\":\"#0 \\\/srv\\\/www\\\/vhosts\\\/owncloud.server-fqdn\\\/nextcloud-13.0.6\\\/3rdparty\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/DBALException.php(128): Doctrine\\\\DBAL\\\\Driver\\\\AbstractSQLiteDriver->convertException('An exception oc...', Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOException))\\n#1 \\\/srv\\\/www\\\/vhosts\\\/owncloud.server-fqdn\\\/nextcloud-13.0.6\\\/3rdparty\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Connection.php(964): Doctrine\\\\DBAL\\\\DBALException::driverExceptionDuringQuery(Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOSqlite\\\\Driver), Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOException), 'INSERT INTO oc_...')\\n#2 \\\/srv\\\/www\\\/vhosts\\\/owncloud.server-fqdn\\\/nextcloud-13.0.6\\\/lib\\\/private\\\/Repair\\\/SqliteAutoincrement.php(96): Doctrine\\\\DBAL\\\\Connection->query('INSERT INTO oc_...')\\n#3 \\\/srv\\\/www\\\/vhosts\\\/owncloud.server-fqdn\\\/nextcloud-13.0.6\\\/lib\\\/private\\\/Repair.php(93): OC\\\\Repair\\\\SqliteAutoincrement->run(Object(OC\\\\Repair))\\n#4 \\\/srv\\\/www\\\/vhosts\\\/owncloud.server-fqdn\\\/nextcloud-13.0.6\\\/lib\\\/private\\\/Updater.php(258): OC\\\\Repair->run()\\n#5 \\\/srv\\\/www\\\/vhosts\\\/owncloud.server-fqdn\\\/nextcloud-13.0.6\\\/lib\\\/private\\\/Updater.php(139): OC\\\\Updater->doUpgrade('13.0.6.1', '13.0.5.2')\\n#6 \\\/srv\\\/www\\\/vhosts\\\/owncloud.server-fqdn\\\/nextcloud-13.0.6\\\/core\\\/Command\\\/Upgrade.php(270): OC\\\\Updater->upgrade()\\n#7 \\\/srv\\\/www\\\/vhosts\\\/owncloud.server-fqdn\\\/nextcloud-13.0.6\\\/3rdparty\\\/symfony\\\/console\\\/Command\\\/Command.php(264): OC\\\\Core\\\\Command\\\\Upgrade->execute(Object(Symfony\\\\Component\\\\Console\\\\Input\\\\ArgvInput), Object(Symfony\\\\Component\\\\Console\\\\Output\\\\ConsoleOutput))\\n#8 \\\/srv\\\/www\\\/vhosts\\\/owncloud.server-fqdn\\\/nextcloud-13.0.6\\\/3rdparty\\\/symfony\\\/console\\\/Application.php(874): Symfony\\\\Component\\\\Console\\\\Command\\\\Command->run(Object(Symfony\\\\Component\\\\Console\\\\Input\\\\ArgvInput), Object(Symfony\\\\Component\\\\Console\\\\Output\\\\ConsoleOutput))\\n#9 \\\/srv\\\/www\\\/vhosts\\\/owncloud.server-fqdn\\\/nextcloud-13.0.6\\\/3rdparty\\\/symfony\\\/console\\\/Application.php(228): Symfony\\\\Component\\\\Console\\\\Application->doRunCommand(Object(OC\\\\Core\\\\Command\\\\Upgrade), Object(Symfony\\\\Component\\\\Console\\\\Input\\\\ArgvInput), Object(Symfony\\\\Component\\\\Console\\\\Output\\\\ConsoleOutput))\\n#10 \\\/srv\\\/www\\\/vhosts\\\/owncloud.server-fqdn\\\/nextcloud-13.0.6\\\/3rdparty\\\/symfony\\\/console\\\/Application.php(130): Symfony\\\\Component\\\\Console\\\\Application->doRun(Object(Symfony\\\\Component\\\\Console\\\\Input\\\\ArgvInput), Object(Symfony\\\\Component\\\\Console\\\\Output\\\\ConsoleOutput))\\n#11 \\\/srv\\\/www\\\/vhosts\\\/owncloud.server-fqdn\\\/nextcloud-13.0.6\\\/lib\\\/private\\\/Console\\\/Application.php(173): Symfony\\\\Component\\\\Console\\\\Application->run(Object(Symfony\\\\Component\\\\Console\\\\Input\\\\ArgvInput), Object(Symfony\\\\Component\\\\Console\\\\Output\\\\ConsoleOutput))\\n#12 \\\/srv\\\/www\\\/vhosts\\\/owncloud.server-fqdn\\\/nextcloud-13.0.6\\\/console.php(90): OC\\\\Console\\\\Application->run()\\n#13 \\\/srv\\\/www\\\/vhosts\\\/owncloud.server-fqdn\\\/nextcloud-13.0.6\\\/occ(11): require_once('\\\/srv\\\/www\\\/vhosts...')\\n#14 {main}\",\"File\":\"\\\/srv\\\/www\\\/vhosts\\\/owncloud.server-fqdn\\\/nextcloud-13.0.6\\\/3rdparty\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Driver\\\/AbstractSQLiteDriver.php\",\"Line\":48}","userAgent":"--","version":"13.0.5.2"}

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'datadirectory' => '/srv/owncloud',
  'dbtype' => 'sqlite3',
  'defaultapp' => 'bookmarks',
  'version' => '13.0.5.2',
  'installed' => true,
  'logfile' => '/var/log/nextcloud/nextcloud.log',
  'log_rotate_size' => 1048576,
  'loglevel' => 0,
  'theme' => '',
  'trusted_domains' => 
  array (
  ),
  'maintenance' => false,
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'forcessl' => true,
  'forceSSLforSubdomains' => true,
  'htaccess.RewriteBase' => '/',
  'trashbin_retention_obligation' => 'auto, 30',
);

The output of your Apache/nginx/system log in /var/log/____:

webserve is down during the upgrade

note: removed not applicable screenshot

Related to this bug report?

however the configuration looks different…

@tflidd: looks similar, but I don’t whether it is the same.

I’ve an sqlite3 database. It’s not a heavy nextcloud instance.
I’ve none of the four locking options enabled in config.php

filelocking.enabled’ => true,
Default is true, so filelocking is enabled

‘filelocking.ttl’ => 60*60,
1 hour, the whole thing completes in a few seconds.

‘memcache.locking’ =>
Not set -> defaults to none.

‘filelocking.debug’ => false,
This might be useful. Shall I enabled this one?

What is the SQL to execute. As this uses temporarely data it is hard to replicate…

You could try to check the filecache first, there is an option for the occ command:

files:cleanup

In case you have duplicates in your database, it should remove this. If that does not help, please open a new topic on the bug tracker. You can refer to the above topic but with the very different setup, it is better to create a new one.

Thanks.

For reference:

Before cleanup:

$ echo “.headers on
.mode csv
select name, path_hash from oc_filecache;” | sqlite3 owncloud.db | wc -l
8633

$ occ files:cleanup
15 orphaned file cache entries deleted

Files is no reduced to 8621

$ echo “.headers on      
.mode csv
select name, path_hash from oc_filecache;” | sqlite3 owncloud.db | sort | uniq -c | sort -nr | grep -v "      1" | head -10
    20 “”,d41d8cd98f00b204e9800998ecf8427e
    17 files,45b963397aa40d4a0063e0d85e4fe7a1
    17 cache,0fea6a13c52b4d4725368f24b045ca84
    10 v0.6.0,f72c70196746fc6912257146fcd84fcf
    10 lucene_index,a0c55384eae17768d8f9fe6696efed6e

Not much changed in this respect.

Will do the upgrade later this week.

Would it be possible to completely empty the file_cache table? It’s a cache after all, as such the entries will be created once accessed again, or is this not the case?

occ files:cleanup

didn’t help, still the same error :frowning:

Opened bug report https://github.com/nextcloud/server/issues/11221

When I emptied the oc_filecache table the update succeeded succesfully,
however after loging in, the was very much noticable that the cache had to be build again.
So initially the response from the server is slow.