Server not responding after Nextcloud upgrade (20.0.0.9 to 21.0.1)

ℹ️ Support
,
11

From the issue on Github: There is a (crude) workaround for the out-of-memory issue. Enable apcu for CLI PHP processes. That “solves” the issue for most users when they’re using apcu caching.

Add this to your PHP config
apc.enable_cli = 1

Why “a crude workaround”? Under normal circumstances it is not recommended to enable apcu for PHP CLI processes. It is a setting for testing and debugging. So I don’t understand why Nextcloud is recommending this setting and making it now a hard requirement for some setups.

See: PHP: Runtime Configuration - Manual

12

Just so you (and devs) know that this is a real issue with real people experiencing it, I have exactly the same problem here with Nextcloud 21 updates. It all goes well until cron jobs run or if I set up a manual occ files:scan job whereupon the server freezes at 100% cpu and ram use and becomes inaccessible.
I take a backup image file before attempting any of this so I was able to revert to 20.08 quite quickly and without fuss.
Would really like to get 21 installed though given that the main benefits are supposed to be a speed up of the web GUI :slight_smile:
I’m using Ubuntu server 20.10 with Apache2 and php7.4 and currently nextcloud 20.08

I joined another thread about this here:
https://help.nextcloud.com/t/freezes-entire-server-computer-caused-by-var-www-nextcloud-cron-php/112359
James

13

Current situation is that if you have configured APCU properly (on CLI as well, you don’t have this problem. Is it the right way? Github is the right place to discuss this with the developers: https://github.com/nextcloud/server/issues/25742

On the forum, we can just share the workarounds and hope for a nicer fix that not all people fall into this trap.

1 Like
14

Agree. This is a disastrous upgrade, and i hope a fix will be issued asap

1 Like
15

WOW Nextcloud 21 upgrade is very bad nothing but issues i’ve reported so many issues all the way to encryption :frowning:

1 Like
16

Tried to install the latest upgrade to Nexctcloud 21 today, hopeful that this issue would have been resolved, that is version 21.0.2 RC1
Same old same old, server freezes after about two minutes of use and becomes inaccessible to all.
At this point it is impossible to do any work on the server, including attempting to edit the config.php so, I reinstated the server image backup I took and edited the config.php first.
Nextcloud 20 will not run with that lne added t my config.php unless I am putting it in the wrong place?
My config.php looks very like a script rather than a config file, is this correct? If so, where should that line be added please?
Bottom line is, still no upgrade to 21 for me

====config.php=============(
?php
$CONFIG = array (
‘instanceid’ => ‘oc4repx2mfk4’,
‘passwordsalt’ => 'redacted,
‘secret’ => ‘redacted’,
‘trusted_domains’ =>
array (
0 => ‘192.168.1.4’,
1 => ‘redacted’,
2 => ‘broncloud.sasstaff.co.uk’,
),
‘datadirectory’ => ‘/mnt/Bronstore/nextcloud_data’,
‘dbtype’ => ‘mysql’,
‘version’ => ‘20.0.10.0’,
‘overwrite.cli.url’ => ‘https://192.168.1.4/nextcloud’,
‘dbname’ => ‘nextcloud’,
‘dbhost’ => ‘localhost’,
‘dbport’ => ‘’,
‘dbtableprefix’ => ‘oc_’,
‘dbuser’ => ‘nextclouduser’,
‘dbpassword’ => ‘removed’,
‘installed’ => true,
‘mail_smtpmode’ => ‘sendmail’,
‘mail_smtpauthtype’ => ‘LOGIN’,
‘mail_from_address’ => ‘jkp’,
‘mail_domain’ => ‘bronyaur.co.uk’,
‘maintenance’ => false,
‘theme’ => ‘’,
‘loglevel’ => 0,
‘memcache.local’ => ‘\OC\Memcache\APCu’,
‘mysql.utf8mb4’ => true,
‘updater.release.channel’ => ‘beta’,
‘app_install_overwrite’ =>
array (
0 => ‘fulltextsearch’,
1 => ‘calendar’,
2 => ‘spreed’,
3 => ‘ransomware_protection’,
4 => ‘maps’,
),
‘preview_max_x’ => ‘1080’,
‘preview_max_y’ => ‘1920’,
‘data-fingerprint’ => ‘557ec2843a35aa888a855e1715d9482f’,
);

17

Sadly I have the same issue: nextcloud just stops responding after a while, and I receive bad gateway errors/timeouts/etc. Works fine again after a restart of php7.4-fpm it seems. Sometimes a mysql restart helps as well.

Any idea? Any updates from developers?

18

This is my (and others) issue. The upgrade goes smoothly enough but, as soon as the cron job runs and invokes occ the server resources max out and the whole thing freezes.

Very sad. I have tried twice, once upgrading using the web updater and the other I uninstalled NC20 and started with a fresh installation. Same thing happened.

I seem to be stuck for now. But, I will hope that jumping to 22 will be a goer :slight_smile:
James

19

In my config, I have opcache.enable_cli=1 - I guess it does the same thing?!?! I didn’t have any issues upgrading, at least.

For people who missed it, this is the solution for the problem - and a newer version of Nextcloud will simply warn when you don’t have this enabled. Alfred pointed out this is against the recommended settings for PHP - I’ll ask the devs to figure out if that’s so and if our solution (enforcing a non-recommended option) is a good idea.

Looking at the second answer here, it seems that having an opcache for the CLI is actually fine, recommended even if you want the performance benefit:

20

Good morning from Scotland.
I have tried this fix, as I mentioned in one of posts above where I reproduced my config.php.
Sadly this completely destroyed my NC setup which wouldn’t run until I removed the entry apc.enable_cli = 1
Incidentally, this entry didn’t previously exist so I was not sure where to put it and what syntax to use since the config.php in my setup looks rather more lie a scripting language than a simple config file.

Some assistance with this might get me to the point I have been aiming for since the release of NC21 :slight_smile:

My config.php again for ease of reference:

====config.php=============(
?php
$CONFIG = array (
‘instanceid’ => ‘oc4repx2mfk4’,
‘passwordsalt’ => 'redacted,
‘secret’ => ‘redacted’,
‘trusted_domains’ =>
array (
0 => ‘192.168.1.4’,
1 => ‘redacted’,
2 => ‘broncloud.sasstaff.co.uk’,
),
‘datadirectory’ => ‘/mnt/Bronstore/nextcloud_data’,
‘dbtype’ => ‘mysql’,
‘version’ => ‘20.0.10.0’,
‘overwrite.cli.url’ => ‘https://192.168.1.4/nextcloud’,
‘dbname’ => ‘nextcloud’,
‘dbhost’ => ‘localhost’,
‘dbport’ => ‘’,
‘dbtableprefix’ => ‘oc_’,
‘dbuser’ => ‘nextclouduser’,
‘dbpassword’ => ‘removed’,
‘installed’ => true,
‘mail_smtpmode’ => ‘sendmail’,
‘mail_smtpauthtype’ => ‘LOGIN’,
‘mail_from_address’ => ‘jkp’,
‘mail_domain’ => ‘bronyaur.co.uk’,
‘maintenance’ => false,
‘theme’ => ‘’,
‘loglevel’ => 0,
‘memcache.local’ => ‘\OC\Memcache\APCu’,
‘mysql.utf8mb4’ => true,
‘updater.release.channel’ => ‘beta’,
‘app_install_overwrite’ =>
array (
0 => ‘fulltextsearch’,
1 => ‘calendar’,
2 => ‘spreed’,
3 => ‘ransomware_protection’,
4 => ‘maps’,
),
‘preview_max_x’ => ‘1080’,
‘preview_max_y’ => ‘1920’,
‘data-fingerprint’ => ‘557ec2843a35aa888a855e1715d9482f’,
);

21

Hi Jos
Sorry posting noob questions in Github! , I did wonder but I posted there due to a comment from someone on the Forum would you believe :slight_smile: They told me I should ask where the real developers reside! Otherwise I would never have posted there.
I apologise and thank you for letting me know so politely :slight_smile:

James

22

Please note that I am not an expert.

From what I understand the apc.enable_cli = 1 setting is meant to be in
the server’s /etc/php.ini file.

P.V.Anthony

23

Hi pvanthony,
Well that would explain why I can’t see it in the config.php!
I’ll have a look and report back :slight_smile: Thanks, and, for the record, I’m no expert either

J

24

Ok, I had a scan of /etc/php/7.4/apache2/php.ini
for the required setting to change and it’s not there either? I did find an entry for CLI-server though:

[CLI Server]
; Whether the CLI web server uses ANSI color coding in its terminal output.
cli_server.color = On
apc.enable_cli = 1

Originally it only had the entry for server.color so I added the famous setting underneath :slight_smile:
This time, when I saved the file, Nextcloud didn’t collapse in a puddle of sweat so already it looks better. WOn’t really know unless I try the upgrade again but, based on the flood of mail on this subject today I feel I should hold off until the powers that be sort this out between them :slight_smile:
J

1 Like
25

ok so that’s it, neat! I was already told that opcache.enable_cli=1 is something different, I still don’t get then why it worked fine for me as I don’t have apc.enable_cli = 1 in my settings. But glad it works now for you!

26

Well, I wouldn’t go so far as say it works for me as I am still using NC 20!
I have not been able to update to 21 as yet. I am spending some tie on the issue today and have made a server backup first of course.
The only change is that I have added the above line to to correct file this time! Last time I edited config.php which trashed the server. Now I am editing the php.ini file in my php folder/s ( did the php7.4 one and the fpm one also.
Neither of these changes seem to have affected NC20 in any detrimental way. Today there was an update to 20.0.11 RC1 which went perfectly using the web updater. Following this, all was well so I went on to the offered update 21.0.3 RC1 which, so far, has completely failed to download either using the updater or trying manually so I do not yet know whether this ‘Solution’ has worked :slight_smile:
I’ll keep trying and report back.

J

27

Hello again,
I finally got the NC21 zip downloaded and installed. Web updater failed so I had to do it manually by editing the step file.
Sadly though, something is badly wrong. NC runs fine but OCC does not. I also have a lot of errors on the security checkup page:

There are some errors regarding your setup.

** Last background job execution ran 1 hour ago. Something seems wrong. Check the background job settings*

** A background job is pending that checks for user imported SSL certificates. Please check back later.*

** Your installation has no default phone region set. This is required to validate phone numbers in the profile settings without a country code. To allow numbers without a country code, please add “default_phone_region” with the respective ISO 3166-1 code :arrow_upper_right: of the region to your config file.*
** The database is missing some indexes. Due to the fact that adding indexes on big tables could take some time they were not added automatically. By running “occ db:add-missing-indices” those missing indexes could be added manually while the instance keeps running. Once the indexes are added queries to those tables are usually much faster.*

    • Missing index “fs_size” in table “oc_filecache”.*

Please double check the installation guides :arrow_upper_right:, and check for any errors or warnings in the log.

Check the security of your Nextcloud over our security scan :arrow_upper_right:.

I tried running occ db:add-missing-indices but I get an error:
OC\HintException: [0]: Memcache \OC\Memcache\APCu not available for local cache (Is the matching PHP module installed and enabled?)
I have checked the APCu installation and it is definitely present and correct and, since I didn’t change anything else, I’m assuming this is a bug elsewhere in my setup.

I’m going to revert to my backup which works and see if anyone can suggest what might be going wrong here.

J

28

Hi,
Exactly what’s happened to me.
Cron job is broken since 9 days now and Thunderbird can’t access to calendar app anymore.

Tryed to go to 21.0.3 RC1 but still broken.
Running on Raspberry Pi 4

1 Like
29

Is anyone willing to help me tackle this problem? Why is my system claiming it cannot find the Memcache when it is clearly installed and running? This has to be an easy fix for someone in the know :slight_smile:
I have fixed the default phone region issue and am assuming that, because occ cannot run, that is the cause of the background job issue.
Nextcloud seems to be running ok but these errors make me nervous.

Help would be appreciated.

James

30

After some in depth research and poking around I found this solution which worked for me:

Specifically this part:

bruno666 commented 26 days ago •

Fix:
If you add apc.enable_cli=1 at the end of your php.ini

Thanks a lot. It works fine on my Debian buster by adding this in /etc/php/7.3/mods-available/apcu.ini (global setting for both php-fpm and php-cli)

So, for now, I am sorted and have my green tick back :slight_smile:

Shouldn’t have taken so long though, all I wanted to do was update to NC21 and we are heading toward 22 now ?

J