Internal Server Error since 14.0.4

ℹ️ Support
#1

Dear all,

i just updatet from 14.0.3 to 14.0.4 and since then i’m encountering an internal server error when trying to access my nextcloud instrance.
The Update process was looking just fine, but when finalizing the update in the web panel, it said

Internal Server Error

The server encountered an internal error and was unable to complete your request.
Please contact the server administrator if this error reappears multiple times, please include the technical details below in your report.
More details can be found in the webserver log.

I’m running Nextcloud on Debian Stretch with Nginx and PHP7.2.

What could have caused the problem and how may i fix this?
Where are the corresponding logs located, since i cannot access the web-panel?

Thanks in advance!

Philipp

#2

Hello,

Your log should be located in the data folder, “nextcloud.log”.

Without it, it’ll be difficult to help you.

Could you also please describe your instance in more details pls.

Regards

#3

This is what /var/www/nextcloud/data/nextcloud.log says:

{“reqId”:“GXajsRAPitf2TJqr6VrA”,“level”:3,“time”:“2018-10-06T15:33:58+00:00”,“remoteAddr”:“162.158.90.209”,“user”:“philipp”,“app”:“core”,“method”:“POST”,“url”:"/index.php",“message”:“Following symlinks is not allowed (’/var/nextcloud_data/philipp/files’ -> ‘/mnt/hdd/nextcloud_data/philipp/files/’ not inside ‘/var/nextcloud_data/philipp/’)”,“userAgent”:“Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36”,“version”:“14.0.1.1”}
{“reqId”:“GXajsRAPitf2TJqr6VrA”,“level”:3,“time”:“2018-10-06T15:33:58+00:00”,“remoteAddr”:“162.158.90.209”,“user”:“philipp”,“app”:“core”,“method”:“POST”,“url”:"/index.php",“message”:“Following symlinks is not allowed (’/var/nextcloud_data/philipp/.’ -> ‘/mnt/hdd/nextcloud_data/philipp/’ not inside ‘/var/nextcloud_data/philipp/’)”,“userAgent”:“Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36”,“version”:“14.0.1.1”}
{“reqId”:“GXajsRAPitf2TJqr6VrA”,“level”:3,“time”:“2018-10-06T15:33:58+00:00”,“remoteAddr”:“162.158.90.209”,“user”:“philipp”,“app”:“index”,“method”:“POST”,“url”:"/index.php",“message”:{“Exception”:“OCP\Files\ForbiddenException”,“Message”:“Following symlinks is not allowed”,“Code”:0,“Trace”:[{“file”:"/var/www/nextcloud/lib/private/Files/Storage/Local.php",“line”:184,“function”:“getSourcePath”,“class”:“OC\Files\Storage\Local”,“type”:"->",“args”:["."]},{“file”:"/var/www/nextcloud/lib/private/Files/Storage/Wrapper/Wrapper.php",“line”:234,“function”:“filemtime”,“class”:“OC\Files\Storage\Local”,“type”:"->",“args”:["."]},{“file”:"/var/www/nextcloud/lib/private/Files/Cache/Updater.php",“line”:245,“function”:“filemtime”,“class”:“OC\Files\Storage\Wrapper\Wrapper”,“type”:"->",“args”:["."]},{“file”:"/var/www/nextcloud/lib/private/Files/Cache/Updater.php",“line”:137,“function”:“correctParentStorageMtime”,“class”:“OC\Files\Cache\Updater”,“type”:"->",“args”:[“files”]},{“file”:"/var/www/nextcloud/lib/private/Files/View.php",“line”:320,“function”:“update”,“class”:“OC\Files\Cache\Updater”,“type”:"->",“args”:[“files”,1538840038]},{“file”:"/var/www/nextcloud/lib/private/Files/View.php",“line”:1161,“function”:“writeUpdate”,“class”:“OC\Files\View”,“type”:"->",“args”:[{“cache”:null,“scanner”:{“class”:“OC\Files\Cache\Scanner”},“watcher”:null,“propagator”:null,“updater”:{“class”:“OC\Files\Cache\Updater”},“class”:“OCA\Files_Trashbin\Storage”},“files”]},{“file”:"/var/www/nextcloud/lib/private/Files/View.php",“line”:268,“function”:“basicOperation”,“class”:“OC\Files\View”,“type”:"->",“args”:[“mkdir”,"/philipp/files",[“create”,“write”]]},{“file”:"/var/www/nextcloud/lib/private/Files/Node/Folder.php",“line”:160,“function”:“mkdir”,“class”:“OC\Files\View”,“type”:"->",“args”:["/philipp/files"]},{“file”:"/var/www/nextcloud/lib/private/Files/Node/Root.php",“line”:382,“function”:“newFolder”,“class”:“OC\Files\Node\Folder”,“type”:"->",“args”:["/philipp/files"]},{“function”:“getUserFolder”,“class”:“OC\Files\Node\Root”,“type”:"->",“args”:["*** sensitive parameter replaced "]},{“file”:"/var/www/nextcloud/lib/private/Files/Node/LazyRoot.php",“line”:64,“function”:“call_user_func_array”,“args”:[[{“class”:“OC\Files\Node\Root”},“getUserFolder”],[" sensitive parameter replaced "]]},{“file”:"/var/www/nextcloud/lib/private/Files/Node/LazyRoot.php",“line”:281,“function”:"__call",“class”:“OC\Files\Node\LazyRoot”,“type”:"->",“args”:[“getUserFolder”,[" sensitive parameter replaced "]]},{“file”:"/var/www/nextcloud/lib/private/Server.php",“line”:1370,“function”:“getUserFolder”,“class”:“OC\Files\Node\LazyRoot”,“type”:"->",“args”:[" sensitive parameter replaced "]},{“file”:"/var/www/nextcloud/lib/private/User/Session.php",“line”:501,“function”:“getUserFolder”,“class”:“OC\Server”,“type”:"->",“args”:[" sensitive parameter replaced "]},{“file”:"/var/www/nextcloud/lib/private/User/Session.php",“line”:372,“function”:“prepareUserLogin”,“class”:“OC\User\Session”,“type”:"->",“args”:[true," sensitive parameter replaced "]},{“file”:"/var/www/nextcloud/lib/private/User/Session.php",“line”:566,“function”:“completeLogin”,“class”:“OC\User\Session”,“type”:"->",“args”:[" sensitive parameters replaced "]},{“file”:"/var/www/nextcloud/lib/private/User/Session.php",“line”:337,“function”:“loginWithPassword”,“class”:“OC\User\Session”,“type”:"->",“args”:[" sensitive parameters replaced "]},{“file”:"/var/www/nextcloud/lib/private/Setup.php",“line”:412,“function”:“login”,“class”:“OC\User\Session”,“type”:"->",“args”:[" sensitive parameters replaced "]},{“file”:"/var/www/nextcloud/core/Controller/SetupController.php",“line”:67,“function”:“install”,“class”:“OC\Setup”,“type”:"->",“args”:[{“install”:“true”,“adminlogin”:" sensitive parameter replaced ",“adminpass”:" sensitive parameter replaced ",“adminpass-clone”:" sensitive parameter replaced ",“directory”:"/var/nextcloud_data",“dbtype”:“mysql”,“dbuser”:“nextcloud_db_user”,“dbpass”:"***",“dbpass-clone”:"***",“dbname”:“nextcloud_db”,“dbhost”:“localhost”,“dbIsSet”:true,“directoryIsSet”:true}]},{“file”:"/var/www/nextcloud/lib/base.php",“line”:925,“function”:“run”,“class”:“OC\Core\Controller\SetupController”,“type”:"->",“args”:[{“install”:“true”,“adminlogin”:" sensitive parameter replaced ",“adminpass”:" sensitive parameter replaced ",“adminpass-clone”:" sensitive parameter replaced ***",“directory”:"/var/nextcloud_data",“dbtype”:“mysql”,“dbuser”:“nextcloud_db_user”,“dbpass”:"***",“dbpass-clone”:"***",“dbname”:“nextcloud_db”,“dbhost”:“localhost”,“dbIsSet”:true,“directoryIsSet”:true}]},{“file”:"/var/www/nextcloud/index.php",“line”:42,“function”:“handleRequest”,“class”:“OC”,“type”:"::",“args”:[]}],“File”:"/var/www/nextcloud/lib/private/Files/Storage/Local.php",“Line”:387,“CustomMessage”:"–"},“userAgent”:“Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36”,“version”:“14.0.1.1”}

#4

Normally this signals that your web server does not own all of the folders in the NC install path.

Following symlinks is not allowed (’/var/nextcloud_data/philipp/.’ -> ‘/mnt/hdd/nextcloud_data/philipp/’ not inside ‘/var/nextcloud_data/philipp/’)

That is what pointed me in that direction.

Have a look here as well:

https://docs.nextcloud.com/server/12/admin_manual/installation/nginx.html

#5

But that is nothing i changed recently. This configuration was working until updating to 14.0.4

#6

True. But I have seen that sometime upgrades re-writes the permissions, that is why I asked whether or not your folder’s permissions are still what they should be for nginx?

#7

I tried a chmod -cR www-data /mnt/hdd/nextcloud_data
The error stays the same :confused:

#8

May be a stupid question, but what about chown -R www-data:www-data /mnt/hdd/nextcloud_data ?

#9

No difference…
Are there maybe other logs i could check?

#10

That tells me that the rule in nginx is not following symlinks like it should. I know apache has a +FollowSymLinks directive in its virtualhost file. I know you said nothing changed in your nginx, but it may be something between nginx and the new .htaccess file in the updated NC installation. If you have your old backup of NC available, could you perhaps check for a difference between the 2 .htaccess files?

you could check nginx logs for general nginx errors, but no other Nextcloud logs. Is the loglevel set to verbose in Nextcloud?

#11

Thanks for that hint, but i added

disable_symlinks off;

in the server-part of my nextcloud-virtual-host and still nothing changed

#12

You restarted nginx after the change, right?

If yes, then I am sorry, I am running out of ideas man! So sorry!

#13

Quick question, could you explain how you mounted that HDD? Did you create local mount point in mnt? If so, why not chown -R the /mnt/WHATEVER and use that directly, why are you specifically using Symlinks?

I DO NOT KNOW WHAT HAPPENS IF YOU CHANGE A DATA DIRECTORY, SO BE CAREFUL PLEASE!

#14

I’ve just tried to start the update process again from CLI.
Everything looks good, till i occ upgrade

Should the "occ upgrade" command be executed? [Y/n] y

An unhandled exception has been thrown:

Error: Class ‘InterfaSys\LogNormalizer\Normalizer’ not found in /var/www/nextcloud/lib/private/Log.php:88

Stack trace:

#0 /var/www/nextcloud/lib/private/Server.php(552): OC\Log->__construct(Object(OC\Log\File), Object(OC\SystemConfig), NULL, Object(OC\Support\CrashReport\Registry))

#1 /var/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php(118): OC\Server->OC{closure}(Object(OC\Server))

#2 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(117): Pimple\Container->offsetGet(‘OCP\ILogger’)

#3 /var/www/nextcloud/lib/private/ServerContainer.php(132): OC\AppFramework\Utility\SimpleContainer->query(‘OCP\ILogger’)

#4 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(165): OC\ServerContainer->query(‘OCP\ILogger’)

#5 /var/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php(114): OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility{closure}(Object(OC\Server))

#6 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(117): Pimple\Container->offsetGet(‘Logger’)

#7 /var/www/nextcloud/lib/private/ServerContainer.php(132): OC\AppFramework\Utility\SimpleContainer->query(‘Logger’)

#8 /var/www/nextcloud/lib/private/Server.php(1560): OC\ServerContainer->query(‘Logger’)

#9 /var/www/nextcloud/lib/private/Server.php(483): OC\Server->getLogger()

#10 /var/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php(118): OC\Server->OC{closure}(Object(OC\Server))

#11 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(117): Pimple\Container->offsetGet(‘OC\Memcache\Fac…’)

#12 /var/www/nextcloud/lib/private/ServerContainer.php(132): OC\AppFramework\Utility\SimpleContainer->query(‘OC\Memcache\Fac…’)

#13 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(165): OC\ServerContainer->query(‘OC\Memcache\Fac…’)

#14 /var/www/nextcloud/3rdparty/pimple/pimple/src/Pimple/Container.php(114): OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility{closure}(Object(OC\Server))

#15 /var/www/nextcloud/lib/private/AppFramework/Utility/SimpleContainer.php(117): Pimple\Container->offsetGet(‘MemCacheFactory’)

#16 /var/www/nextcloud/lib/private/ServerContainer.php(132): OC\AppFramework\Utility\SimpleContainer->query(‘MemCacheFactory’)

#17 /var/www/nextcloud/lib/private/Server.php(1514): OC\ServerContainer->query(‘MemCacheFactory’)

#18 /var/www/nextcloud/lib/base.php(894): OC\Server->getMemCacheFactory()

#19 /var/www/nextcloud/lib/base.php(641): OC::registerAutoloaderCache()

#20 /var/www/nextcloud/lib/base.php(1068): OC::init()

#21 /var/www/nextcloud/console.php(46): require_once(’/var/www/nextcl…’)

#22 /var/www/nextcloud/occ(11): require_once(’/var/www/nextcl…’)

#23 {main}

Keep maintenance mode active? [y/N] y

Maintenance mode kept active

#15

I used the web updater now on a small 14.0.3 to 14.0.4, and it worked without issues. So I am stumped with yours, and I think you will need to log a bug for this maybe? Sorry man!

#16

I now removed the symlinks and mounted the hdd via bind in fstab.
I also set up a new instance of NC14 and maybe i now know, what’s the issue.
When unpacking the zip-file, obviously my watchdog jumped in and restarted the server. Maybe the update was corrupted by this and didn’t install correctly :man_shrugging:t4:

#17

That is possible. I do not know what watchdog is, assuming some auto maintainence thing, but that is possible. If you want to upgrade via web, you will have t9 find a way around that. If you want to upgrade via occ command, you could always stop apache, upgrade, check everything, then enable apache again.

Glad it is fixed now. Please mark your topic as solved for future users coming to this forum.

#18

Yes watchdog is a daemon restarting the system in case of hung-ups, high loads etc.
Deactivating the daemon during the install/update process solved my problem.

1 Like
#19

Coolness. Glad you figured it out! Will help future forum users for sure! Thanks for marking it as a solution!

1 Like