Apps missing after server migration from ncp

[droid001]

I’ve made a migration from a previous server following the Migrating to a different server instructions. Now I’m faced with the problem that the apps used in the previous installation don’t show up. I have only

• Dashboard
• Files
• Photos
• Activity
  The contacts show up in the navbar’s quick contacts view

but going to the page /apps/contacts gives me an error
The page could not be found on the server or you may not be allowed to view it.

I recreated the ncp database user ´ncadmin´ and granted it all privileges over the nextcloud db with
GRANT ALL PRIVILEGES ON nextcloud.* TO 'username'@'localhost'; FLUSH PRIVILEGES;
so I can’t see there being any access problems there.

The server logs do show an error with a missing table:
Doctrine\DBAL\Exception\TableNotFoundException: An exception occurred while executing a query: SQLSTATE[42S02]: Base table or view not found: 1146 Table 'nextcloud.oc_storages' doesn't exist. I haven’t found any answers here on the forum though. I attempted to recreate the missing indices with the occ command

sudo -u www-data php occ db:add-missing-indices
with no luck. A repairing attempt didn’t help either:
sudo -u www-data php occ maintenance:repair

Debian Buster 11
Nextcloud 27.1.1
PHP 8.1

[kjathome]

The error indicates that the table is missing. If you ran a backup then there should also be a backup of the database. It might be the easiest step to recover the nextcloud database, grant all permissions to the respective user an perform a scan of the files on the storage. This updates and synchronizes the catalog of nextcloud files in the database.
Alternativly you can run the create table and related statement from within the database backup.

[droid001]

@kjathome Thanks for the pointers.
The data database is restored with the backup of the previous installation and that’s why I can for instance see the contacts in the contacts quickview in the navbar. I just don’t understand why the nextcloud.oc_storages didn’t follow along the backup or isn’t detected in the new installation. Feels like a permissions problem but I have no idea what it could be.
I’m currently scanning the files and see if this helps…
sudo -u www-data php occ files:scan --all produced:

| Folders | Files  | Errors | Elapsed time |
+---------+--------+--------+--------------+
| 12770   | 132670 | 0      | 00:07:33     |
+---------+--------+--------+--------------+

sudo -u www-data php occ files:scan-app-data produced:

| Folders | Files  | Elapsed time |
+---------+--------+--------------+
| 129764  | 181895 | 00:29:31     |
+---------+--------+--------------+

but no change in getting the apps back into the navbar and the app pages are still not viewable.

[droid001]

i had a look at the nextcloud db and found that oc_storages indeed exists:

MariaDB [nextcloud]> SHOW TABLES;
+--------------------------------+
| Tables_in_nextcloud            |
+--------------------------------+
...
| oc_share                       |
| oc_share_external              |
| oc_storages                    |
| oc_storages_credentials        |
| oc_systemtag                   |
| oc_systemtag_group             |
...

[kjathome]

Can you run the following command
sudo mysql --database nextcloud --execute "select * from oc_storages; select * from oc_storages_credentials; check table oc_storages, oc_storages_credentials extended; show grants"
to see the content of the tables, the check result as well as the permissions.

[droid001]

This is what I get:

+------------+--------------------------------------+-----------+--------------+
| numeric_id | id                                   | available | last_checked |
+------------+--------------------------------------+-----------+--------------+
|          1 | home::ncp                            |         1 |         NULL |
|          2 | local::/var/www/nextcloud/data/      |         1 |         NULL |
|          3 | home::Linus                          |         1 |         NULL |
|          4 | home::[redacted]                     |         1 |         NULL |
|          5 | home::[redacted]                     |         1 |         NULL |
|          6 | home::[redacted]                     |         1 |         NULL |
|          7 | home::[redacted]                     |         1 |         NULL |
|          8 | local::/var/www/nextcloud-data/data/ |         1 |         NULL |
+------------+--------------------------------------+-----------+--------------+
+----+-------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id | user  | identifier       | credentials                                                                                                                                                                                                                                                                                                        |
+----+-------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|  3 | Linus | password::global | [redacted HEX string] |
|  6 | ncp   | password::global | [redacted HEX string] |
+----+-------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+-----------------------------------+-------+----------+----------+
| Table                             | Op    | Msg_type | Msg_text |
+-----------------------------------+-------+----------+----------+
| nextcloud.oc_storages             | check | status   | OK       |
| nextcloud.oc_storages_credentials | check | status   | OK       |
+-----------------------------------+-------+----------+----------+
+----------------------------------------------------------------------------------------------------------------+
| Grants for ncadmin@localhost                                                                                   |
+----------------------------------------------------------------------------------------------------------------+
| GRANT USAGE ON *.* TO `ncadmin`@`localhost` IDENTIFIED BY PASSWORD '*redactedHEXstring' |
| GRANT ALL PRIVILEGES ON `nextcloud`.* TO `ncadmin`@`localhost`                                                 |
+----------------------------------------------------------------------------------------------------------------+

[kjathome]

This looks pretty similar to my outout:

+------------+------------------------------------------+-----------+--------------+
| numeric_id | id                                       | available | last_checked |
+------------+------------------------------------------+-----------+--------------+
|          1 | home::ncp                                |         1 |         NULL |
|          2 | local::/var/www/nextcloud/data/          |         1 |         NULL |
|          3 | home::XXXXXXX                            |         1 |         NULL |
|          4 | local::/media/myCloudDrive1/ncdata/      |         1 |         NULL |
|          5 | home::XXXXXXXXXX                         |         1 |         NULL |
|          6 | local::/media/myCloudDrive1/ncdata/data/ |         1 |         NULL |
+------------+------------------------------------------+-----------+--------------+
+-----------------------------------+-------+----------+----------+
| Table                             | Op    | Msg_type | Msg_text |
+-----------------------------------+-------+----------+----------+
| nextcloud.oc_storages             | check | status   | OK       |
| nextcloud.oc_storages_credentials | check | status   | OK       |
+-----------------------------------+-------+----------+----------+
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Grants for root@localhost                                                                                                                                                 |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| GRANT ALL PRIVILEGES ON *.* TO `root`@`localhost` IDENTIFIED VIA mysql_native_password USING 'HEXSTRING' OR unix_socket WITH GRANT OPTION |
| GRANT PROXY ON ''@'%' TO 'root'@'localhost' WITH GRANT OPTION                                                                                                             |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

So your config refers to ncadmin whereas mine uses root. When migrating from 32 to 64 bit I did a fresh installation and as far as I remember it was straight forward.

Another approch could be to grep for those tables or other hints in all logs files to see where the issue originates.

sudo find /etc/log /var/www/nextcloud/ -type f -iname "*log*" -print0 | xargs -0 egrep -i "oc_storage|error|denied"

In grep’s search expression you might add some other search terms.

[kjathome]

I got back to your original post which states that table oc_storages could not be found. But as we can it exists. Nextcloud runs with user www-data, ncadmin is used to connect to the database. ncadmin has all permissions to access that table (could be checked by log in to mysql with user ncadmin and running select * from nextcloud.oc_storages).

I’m not sure but very often it’s due the cache that something stopped to work (=> clear opcache).
And I remeber that once I had an issue with permissions on the tempdirectory (sudo ncc config:system:get “tempdirectory”). Just get the value and check the permissions.
Otherwise check all configs with

sudo ncc

to get all options,

sudo ncc app:list

to get all apps, etc…

[droid001]

The error of oc_storages not being found is gone in the process of addressing errors in the admin overview check. The latest one though is

[webdav] Error: Doctrine\DBAL\Exception\DeadlockException: An exception occurred while executing a query: SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction at <<closure>>
...

Caused by:

Doctrine\DBAL\Driver\PDO\Exception: SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction at <<closure>>
...

The system temp directory which happens to be /var/www/nextcloud-data/data/tmp is owned by www-data as I used the -R handle when changing user ownership: sudo chown -R www-data:www-data /var/www/nextcloud.

The list of apps I get from the sudo ncc app:list command is:

Enabled:
  - activity: 2.19.0
  - circles: 27.0.1
  - cloud_federation_api: 1.10.0
  - comments: 1.17.0
  - contactsinteraction: 1.8.0
  - dashboard: 7.7.0
  - dav: 1.27.0
  - federatedfilesharing: 1.17.0
  - federation: 1.17.0
  - files: 1.22.0
  - files_external: 1.19.0
  - files_pdfviewer: 2.8.0
  - files_reminders: 1.0.0
  - files_rightclick: 1.6.0
  - files_sharing: 1.19.0
  - files_trashbin: 1.17.0
  - files_versions: 1.20.0
  - firstrunwizard: 2.16.0
  - logreader: 2.12.0
  - lookup_server_connector: 1.15.0
  - nextcloud_announcements: 1.16.0
  - notifications: 2.15.0
  - oauth2: 1.15.1
  - password_policy: 1.17.0
  - photos: 2.3.0
  - privacy: 1.11.0
  - provisioning_api: 1.17.0
  - recommendations: 1.6.0
  - related_resources: 1.2.0
  - serverinfo: 1.17.0
  - settings: 1.9.0
  - sharebymail: 1.17.0
  - support: 1.10.0
  - survey_client: 1.15.0
  - systemtags: 1.17.0
  - text: 3.8.0
  - theming: 2.2.0
  - twofactor_backupcodes: 1.16.0
  - user_status: 1.7.0
  - viewer: 2.1.0
  - weather_status: 1.7.0
  - workflowengine: 2.9.0
Disabled:
  - admin_audit: 1.17.0
  - bruteforcesettings: 2.7.0
  - encryption: 2.15.0
  - suspicious_login: 5.0.0
  - twofactor_totp: 9.0.0
  - updatenotification: 1.17.0 (installed 1.17.0)
  - user_ldap: 1.17.0

…which feels wrong as there is quite a bunch missing. I’m a little bit hesitant to reinstalling them from the app store as this might wipe out the current app data clean. Maybe I have one to sacrifice… which I’ll do next.

[droid001]

In the Apps grid view, for example the GpxPod app is marked as in used which isn’t the case my user apps view. Disabling it and reloading it brings the app into the app list and icon in the nav bar. Will be testing with other…
… like the Contacts app. Reinstalling it brought back the contacts.
There must be a better way of getting back the apps though. I’m wondering now how this happened. Is it because I didn’t put the previous installment into maintenance mode before taking a db dump? @kjathome Did you have to reinstall your apps during your migration?

[kjathome]

That’s my app list (default plus some I added):

Enabled:
  - activity: 2.19.0
  - admin_audit: 1.17.0
  - audioplayer: 3.4.0
  - calendar: 4.5.2
  - circles: 27.0.1
  - cloud_federation_api: 1.10.0
  - comments: 1.17.0
  - contacts: 5.4.2
  - contactsinteraction: 1.8.0
  - dashboard: 7.7.0
  - dav: 1.27.0
  - federatedfilesharing: 1.17.0
  - federation: 1.17.0
  - files: 1.22.0
  - files_accesscontrol: 1.17.1
  - files_automatedtagging: 1.17.0
  - files_external: 1.19.0
  - files_pdfviewer: 2.8.0
  - files_reminders: 1.0.0
  - files_rightclick: 1.6.0
  - files_scripts: 3.0.0
  - files_sharing: 1.19.0
  - files_trashbin: 1.17.0
  - files_versions: 1.20.0
  - fileslibreofficeedit: 1.1.0
  - firstrunwizard: 2.16.0
  - logreader: 2.12.0
  - lookup_server_connector: 1.15.0
  - news: 24.0.0
  - nextcloud_announcements: 1.16.0
  - notes: 4.8.1
  - notifications: 2.15.0
  - notify_push: 0.6.3
  - oauth2: 1.15.1
  - password_policy: 1.17.0
  - photos: 2.3.0
  - privacy: 1.11.0
  - provisioning_api: 1.17.0
  - recommendations: 1.6.0
  - related_resources: 1.2.0
  - serverinfo: 1.17.0
  - settings: 1.9.0
  - sharebymail: 1.17.0
  - support: 1.10.0
  - survey_client: 1.15.0
  - suspicious_login: 5.0.0
  - systemtags: 1.17.0
  - tasks: 0.15.0
  - text: 3.8.0
  - theming: 2.2.0
  - twofactor_backupcodes: 1.16.0
  - twofactor_totp: 9.0.0
  - updatenotification: 1.17.0
  - user_status: 1.7.0
  - viewer: 2.1.0
  - weather_status: 1.7.0
  - workflow_ocr: 1.27.5
  - workflow_pdf_converter: 1.12.0
  - workflowengine: 2.9.0
Disabled:
  - bruteforcesettings: 2.7.0
  - encryption: 2.15.0 (installed 2.4.0)
  - files_downloadactivity: 1.16.0 (installed 1.16.0)
  - nextcloudpi: 0.0.1 (installed 0.0.1)
  - previewgenerator: 99.99.99 (installed 99.99.99)
  - user_ldap: 1.17.0

[kjathome]

Contact and calendar data are stored in mariadb/mysql in corresponding tables. That’s good design separating logic and data. Hence it’s possible to (re)install apps.

The apps are stored in a directory tree below /var/www/nextcloud/apps. For this reason the nextcloud directory is also part of the backup process. You could examine the corresponding backup files on our backup device.

[droid001]

That begins to make sense then. I guess the Apps grid looks at the database entries for displaying if the installation has activated/enabled an app and the user apps builds up the list from entries in the nextcloud/apps directory.

[kjathome]

That’s true, there is a table named oc_appconfig that reflects the app’s configuration and the content should be consistent with all apps in /var/www/nextcloud/apps. I don’t know if there is command to sync both (similar to nc-scan)

[kjathome]

No, what I’ve seen that some apps not yet be supported by a newer nextcloud version. These are disabled during the upgrade process and sometimes I had to do this before manually. But it’s good to read that you finally found a solution

[droid001]

Thanks for hanging in there and for the help along the way. I definitely had a myopic view that everything app related was in the database and that one only needed the data and config folder for fulfilling a migration. Maybe the documentation needs a paragraph on reinstating the apps when migrating to a different server/hardware.