PHP Fatal error: Allowed memory size of (2048M still get the error)

Nextcloud version (11.0.3.2) upgraded from ownCloud 9

Operating system and version (Ubuntu 16.04)

Apache version (Apache 2.4.18)

PHP version (7.0)
Is this the first time you’ve seen this error? yes, with Nextcloud, not happend with ownCloud

Can you reliably replicate it?
It happens all the time you run files:scan --all
Also, if I ran cron.php from command line or automatically when cron job runs, it works and exits without error, but sometimes I get a server alarm I set and tells me memory is almost full, I look for processes running and cron.php is consuming about 14GB ram, so I kill it. Run it manually… and it goes good.
The other thing is the oc_filecache table about 17GB… something might be wrong? I have only my user and my files, I am a normal user, movies, games, programs… everything is about 2TB.
I have configured external storages from my nextcloud user interface, and connected 5 folders (inside my sata disk, ie /mnt/disk1/folder1, folder2, folder3, music, movies) with SFTP, and also my complete data disk mounted as /mnt/disk1.

The issue you are facing:
When I run “occ scan:files --all” it starts scanning, some time later you get: PHP Fatal error: Allowed memory size of 536870912 bytes exhausted

I also noticed one time, running cron.php consumes up to 14GB ram memory and mysql oc_filecache up to 17GB disk space, but this should have something to do with the previous error (php loop?)

The output of your Nextcloud log
its my nextcloud.log file, not the complete file but almost, if it is needed I can re-paste it fully.
https://pastebin.com/wXUBNPT4

The output of your config.php file in /path/to/nextcloud:
> <?php
> $CONFIG = array (
> ‘updatechecker’ => false,
> ‘instanceid’ => ‘xxxxxxxxxx’,
> ‘passwordsalt’ => ‘xxxxxxxxxxxxxxxxxxx’,
> ‘secret’ => ‘xxxxxxxxxxxxx’,
> ‘trusted_domains’ =>
> array (
> 0 => ‘20.0.0.100’,
> 1 => ‘myexternalip’,
> 2 => ‘localhost’,
> ),
> ‘datadirectory’ => ‘/mnt/DISCO1/home/nextcloud/data’,
> ‘overwrite.cli.url’ => ‘/nextcloud’,
> ‘htaccess.RewriteBase’ => ‘/nextcloud’,
> ‘dbtype’ => ‘mysql’,
> ‘version’ => ‘11.0.3.2’,
> ‘dbname’ => ‘owncloud’,
> ‘dbhost’ => ‘localhost’,
> ‘dbtableprefix’ => ‘oc_’,
> ‘dbuser’ => ‘user’,
> ‘dbpassword’ => ‘xxxxxxxxxxxx’,
> ‘logtimezone’ => ‘UTC’,
> ‘installed’ => true,
> ‘loglevel’ => 3,
> ‘mail_from_address’ => ‘myemailname’,
> ‘mail_smtpmode’ => ‘smtp’,
> ‘mail_domain’ => ‘gmail.com’,
> ‘mail_smtpauthtype’ => ‘LOGIN’,
> ‘mail_smtphost’ => ‘smtp.gmail.com’,
> ‘mail_smtpport’ => ‘465’,
> ‘mail_smtpauth’ => 1,
> ‘mail_smtpsecure’ => ‘ssl’,
> ‘mail_smtpname’ => ‘myemail@gmail.com’,
> ‘mail_smtppassword’ => ‘pass’,
> ‘maintenance’ => true,
> ‘data-fingerprint’ => ‘xxxxxxxxxxxxxxxx’,
> ‘theme’ => ‘’,
> ‘memcache.local’ => ‘\OC\Memcache\APCu’,
> ‘default_language’ => ‘en’,
> ‘ldapIgnoreNamingRules’ => false,
> ‘ldapProviderFactory’ => ‘\OCA\User_LDAP\LDAPProviderFactory’,
> ‘updater.release.channel’ => ‘stable’,
> );

The output of your Apache log:
(I can paste it fully if asked)
https://pastebin.com/snBFWDub

I have backups, but for now I think I will try to empty the oc_filecache table and try a occ files:scan --all again.

I would also like to add the last files:scan --all output just after the server gets almost out of ram:
> Scanning files for 2 users
> Starting scan for user 1 out of 2 (admin)
> Starting scan for user 2 out of 2 (myusername)
> PHP Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 20480 bytes) in /mnt/DISCO1/home/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php on line 91
> PHP Fatal error: Uncaught PDOException: SQLSTATE[HY000]: General error: 2014 Cannot execute queries while other unbuffered queries are active. Consider using PDOStatement::fetchAll(). Alternatively, if your code is only ever going to run against mysql, you may enable query buffering by setting the PDO::MYSQL_ATTR_USE_BUFFERED_QUERY attribute. in /mnt/DISCO1/home/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php:91
> Stack trace:
> /mnt/DISCO1/home/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php(91): PDOStatement->execute(Array)
> /mnt/DISCO1/home/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php(989): Doctrine\DBAL\Driver\PDOStatement->execute(Array)
> mnt/DISCO1/home/nextcloud/lib/private/DB/Connection.php(215): Doctrine\DBAL\Connection->executeUpdate(‘UPDATE oc_file...', Array, Array) > /mnt/DISCO1/home/nextcloud/lib/private/Lock/DBLockingProvider.php(203): OC\DB\Connection->executeUpdate('UPDATEoc_file…’, Array)
> /mnt/DISCO1/home/nextcloud/lib/ in /mnt/DISCO1/home/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractMySQLDriver.php on line 116

Thank you very much!

S.


OK, I truncated the oc_filecache table (I read somewhere that I should not touch that table… but :-x), firstly I set nextcloud in maintenance mode just in case it helps. Restarted mysql and apache.

Now the table is about 250mb, seems ok… I ran again the files:scan --all command and this is the output:

> sudo -u www-data php occ files:scan --all

> Scanning files for 2 users
> Starting scan for user 1 out of 2 (admin)
> Starting scan for user 2 out of 2 (sebadamus)
> PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 18874368 bytes) in /mnt/DISCO1/home/nextcloud/lib/private/Lock/DBLockingProvider.php on line 76
> PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 8388616 bytes) in /mnt/DISCO1/home/nextcloud/lib/private/Lock/DBLockingProvider.php on line 263 

Searching in San Google, the PHP Fatal error: Allowed memory… (normaly is 128mb as default, but I give it more just in case, about 512mb to test so thats why the value incresed from normal default value. But the error apears again. Seems PHP code is going crazy or looping somewhere.

The line 76 in file DBLockingProvider.php is if ($type === self::LOCK_SHARED) {
> /**
> * Mark a locally acquired lock
> *
> * @param string $path
> * @param int $type self::LOCK_SHARED or self::LOCK_EXCLUSIVE
> */
> protected function markAcquire($path, $type) {
> parent::markAcquire($path, $type);
> if ($type === self::LOCK_SHARED) {
> $this->sharedLocks[$path] = true;
> }
> }

And the 263 is: $lockedPaths = array_keys($this->sharedLocks);
> /**
> * release all lock acquired by this instance which were marked using the mark* methods
> */
> public function releaseAll() {
> parent::releaseAll();
> // since we keep shared locks we need to manually clean those
> $lockedPaths = array_keys($this->sharedLocks);
> $lockedPaths = array_filter($lockedPaths, function ($path) {
> return $this->sharedLocks[$path];
> });

Hope someone give some light on this…

Thanks!

I do have a similar problem. I did update from nextcloud version 11 to 12 on my centos server, but from there on when I try to access my owncloud instance I do get an internel server error 500.
nextcloud.log says:

{“reqId”:“id”,“level”:3,“time”:“2017-05-23T16:23:43+00:00”,“remoteAddr”:“ip”,“user”:“username”,“app”:“PHP”,“method”:“GET”,“url”:"/nextcloud/index.php/settings/admin",“message”:“Allowed memory size of 1073741824 bytes exhausted (tried to allocate 7795568645519978368 bytes) at /var/www/html/nextcloud/lib/private/Template/SCSSCacher.php#264”,“userAgent”:“Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36”,“version”:“12.0.0.29”}

I did change the memory_limit variable in the php.ini up to 4096M, but it didn’t change anything.
Also, when I try to access my nextcloud instance and get the error 500, a process called “setroubleshootd” appears on my server and takes 100% cpu usage.

SCSSCacher.php line 264 looks normal.
Can someone pls help to get my nextcloud working again? I can provide information if needed.
Thanks

I have just upgraded from nextcloud 11 to 12, but the error continues :-/

Maxris, have you come from ownCloud 9 to Nextcloud 11 like me? if it is so, maybe it has something to do with bad values in database after upgrading from ownCloud. Please tell.

No, I did a fresh install of nextcloud 11, because I came from owncloud.
By the way, it seems like only my web interface doesn’t work. owncloud apps (android, windows, linux) do still sync.

I fixed it now, by solving some issues regarding the code integrity scan and “resetting up” opcache… however it does work now

Excellent Maxris!, but can you explain a little more so I can try similar way to fix it?

I am now testing on a fresh new install ubuntu 16.04 nextcloud 11, from a friend installation, he gets the same: PHP Fatal error: Allowed memory size of xxxxxxx bytes exhausted. We both use external files app and we have connected with SFTP. I have a very very large directory with music, but my friend has a new server very small amout of files.

Now I tried to increase the php memory_limit size (both apache and cli php.ini) to 2048M, since default apache php is 128M in Ubuntu 16.04, putting 2048M I thought would be enough… but nop… same error when running files:scan -all

What could be the problem?

> sebadamus@mintstation /var/www/nextcloud $ sudo -u www-data php occ files:scan --all
> Scanning files for 2 users
> Starting scan for user 1 out of 2 (admin)
> Starting scan for user 2 out of 2 (sebadamus)
> PHP Fatal error:  Allowed memory size of 2147483648 bytes exhausted (tried to allocate 20480 bytes) in /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php on line 91
> PHP Fatal error:  Uncaught PDOException: SQLSTATE[HY000]: General error: 2014 Cannot execute queries while other unbuffered queries are active.  Consider using PDOStatement::fetchAll().  Alternatively, if your code is only ever going to run against mysql, you may enable query buffering by setting the PDO::MYSQL_ATTR_USE_BUFFERED_QUERY attribute. in /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php:91
> Stack trace:
> #0 /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php(91): PDOStatement->execute(Array)
> #1 /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php(1008): Doctrine\DBAL\Driver\PDOStatement->execute(Array)
> #2 /var/www/nextcloud/lib/private/DB/Connection.php(213): Doctrine\DBAL\Connection->executeUpdate('UPDATE `oc_file...', Array, Array)
> #3 /var/www/nextcloud/lib/private/Lock/DBLockingProvider.php(203): OC\DB\Connection->executeUpdate('UPDATE `oc_file...', Array)
> #4 /var/www/nextcloud/lib/private/Lock/AbstractLockingProvider.php(120):  in /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractMySQLDriver.php on line 116

OK, I had to remove the external storage SFTP configured in Nextcloud (the whole disk), seems there was the problem for now. I have just left the external storage SFTP I create for each folder, but I guess the problem will happen again when the directories get bigger.

But cant be sure without more help :frowning:

Sorry, I cant help you… I am not that deep in all that nextcloud stuff. Good luck anyway

You might want to check your “memory_limit” configuration in php.ini.

1 Like

I am having the same issue updating to 18.0.2.
I set the did memory_limit=-1, because I was not able to start nextcloud anymore any other way.

2 Likes

The issue repeats with 19.0.0.12. Has any of you reported the bug in github?..