Update from 17.0 to 18.0.1.3 fails because 'flow_operations_scope' is missing

[jonsger]

Nextcloud version (eg, 18.0.1.3):
Operating system and version (GNU Guix 1.0.1-ba7ead2):
Apache or nginx version (Nginx 1.17.8):
PHP version (7.3):

The issue you are facing: Update from 17.0.2 to 18.0.1 fails

Is this the first time you’ve seen this error? (Y):

Steps to replicate it:

1. Install 17.0.2
2. Upgrade to 18.0.1

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'passwordsalt' => 'BLA',
  'secret' => 'BLA',
  'trusted_domains' => 
  array (
    0 => 'localhost',
    1 => '127.0.0.1',
    2 => 'wolke7.brielmaier.net',
  ),
  'datadirectory' => '/srv/http/wolke7.brielmaier.net/data/',
  'dbtype' => 'mysql',
  'version' => '17.0.2.1',
  'overwrite.cli.url' => 'http://localhost',
  'dbname' => 'nextcloud',
  'dbhost' => '127.0.0.1',
  'dbtableprefix' => '',
  'dbuser' => 'nextcloud',
  'dbpassword' => 'BLA',
  'installed' => true,
  'instanceid' => 'ocqpw3dgcm0m',
  'maintenance' => true,
  'data-fingerprint' => '338866d8af15ad1ee4243cfb99fa7348',
  'updater.secret' => 'BLA',
  'updater.release.channel' => 'stable',
  'loglevel' => 2,
  'theme' => '',
);

occ status

sudo -u nginx php occ status
Nextcloud or one of the apps require upgrade - only a limited number of commands are available
You may use your browser or the occ upgrade command to do the upgrade
  - installed: true
  - version: 18.0.1.3
  - versionstring: 18.0.1
  - edition: 

sudo -u nginx php occ upgrade -vvv
...
2020-02-19T15:09:29+00:00 Updating database schema
2020-02-19T15:09:29+00:00 Updated database
2020-02-19T15:09:29+00:00 Updating <workflowengine> ...
2020-02-19T15:09:29+00:00 Repair step: Populating added database structures for workflows
2020-02-19T15:09:29+00:00 Doctrine\DBAL\Exception\TableNotFoundException: An exception occurred while executing 'SELECT `o`.`id` FROM `flow_operations` `o` LEFT JOIN `flow_operations_scope` `s` ON `o`.`id` = `s`.`operation_id` WHERE `s`.`operation_id` IS NULL': SQLSTATE[42S02]: Base table or view not found: 1146 Table 'nextcloud.flow_operations_scope' doesn't exist
2020-02-19T15:09:29+00:00 Update failed
2020-02-19T15:09:29+00:00 Maintenance mode is kept active
2020-02-19T15:09:29+00:00 Reset log level

sudo -u nginx php occ maintenance:repair -vvv
 - Fix the share type of guest shares when migrating from ownCloud
 - Copy the share password into the dedicated column
 - Set existing shares as accepted
 - Update OAuth token expiration times
 - Add background job to check for backup codes
 - Populating added database structures for workflows

In AbstractMySQLDriver.php line 42:
                                                                                            
  [Doctrine\DBAL\Exception\TableNotFoundException] An exception occurred while executing 'SELECT `o`.`id` FROM `flow_operations` `o` LEFT JOIN `flow_operations_scope` `s` ON `o`.`id` = `s`.`operation_id` WHERE `s`.`operation_id` IS NULL':                                                                               
                                                                                            
  SQLSTATE[42S02]: Base table or view not found: 1146 Table 'nextcloud.flow_operations_scope' doesn't exist                                                                                                                                                     

Exception trace:
  at /srv/http/wolke7.brielmaier.net/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractMySQLDriver.php:42
 Doctrine\DBAL\Driver\AbstractMySQLDriver->convertException() at /srv/http/wolke7.brielmaier.net/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php:169
 Doctrine\DBAL\DBALException::wrapException() at /srv/http/wolke7.brielmaier.net/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php:145
 Doctrine\DBAL\DBALException::driverExceptionDuringQuery() at /srv/http/wolke7.brielmaier.net/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php:911
 Doctrine\DBAL\Connection->executeQuery() at /srv/http/wolke7.brielmaier.net/lib/private/DB/Connection.php:199
 OC\DB\Connection->executeQuery() at /srv/http/wolke7.brielmaier.net/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php:200
 Doctrine\DBAL\Query\QueryBuilder->execute() at /srv/http/wolke7.brielmaier.net/lib/private/DB/QueryBuilder/QueryBuilder.php:215
 OC\DB\QueryBuilder\QueryBuilder->execute() at /srv/http/wolke7.brielmaier.net/apps/workflowengine/lib/Migration/PopulateNewlyIntroducedDatabaseFields.php:86
 OCA\WorkflowEngine\Migration\PopulateNewlyIntroducedDatabaseFields->getIdsWithoutScope() at /srv/http/wolke7.brielmaier.net/apps/workflowengine/lib/Migration/PopulateNewlyIntroducedDatabaseFields.php:48
 OCA\WorkflowEngine\Migration\PopulateNewlyIntroducedDatabaseFields->run() at /srv/http/wolke7.brielmaier.net/lib/private/Repair.php:104
 OC\Repair->run() at /srv/http/wolke7.brielmaier.net/core/Command/Maintenance/Repair.php:126
 OC\Core\Command\Maintenance\Repair->execute() at /srv/http/wolke7.brielmaier.net/3rdparty/symfony/console/Command/Command.php:255
 Symfony\Component\Console\Command\Command->run() at /srv/http/wolke7.brielmaier.net/3rdparty/symfony/console/Application.php:915
 Symfony\Component\Console\Application->doRunCommand() at /srv/http/wolke7.brielmaier.net/3rdparty/symfony/console/Application.php:272
 Symfony\Component\Console\Application->doRun() at /srv/http/wolke7.brielmaier.net/3rdparty/symfony/console/Application.php:148
 Symfony\Component\Console\Application->run() at /srv/http/wolke7.brielmaier.net/lib/private/Console/Application.php:214
 OC\Console\Application->run() at /srv/http/wolke7.brielmaier.net/console.php:99
 require_once() at /srv/http/wolke7.brielmaier.net/occ:11

In PDOConnection.php line 90: [Doctrine\DBAL\Driver\PDOException (42S02)] SQLSTATE[42S02]: Base table or view not found: 1146 Table 'nextcloud.flow_operations_scope' doesn't exist                                                                                                                                   

Exception trace:
  at /srv/http/wolke7.brielmaier.net/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnection.php:90
 Doctrine\DBAL\Driver\PDOConnection->query() at /srv/http/wolke7.brielmaier.net/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php:908
 Doctrine\DBAL\Connection->executeQuery() at /srv/http/wolke7.brielmaier.net/lib/private/DB/Connection.php:199
 OC\DB\Connection->executeQuery() at /srv/http/wolke7.brielmaier.net/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php:200
 Doctrine\DBAL\Query\QueryBuilder->execute() at /srv/http/wolke7.brielmaier.net/lib/private/DB/QueryBuilder/QueryBuilder.php:215
 OC\DB\QueryBuilder\QueryBuilder->execute() at /srv/http/wolke7.brielmaier.net/apps/workflowengine/lib/Migration/PopulateNewlyIntroducedDatabaseFields.php:86
 OCA\WorkflowEngine\Migration\PopulateNewlyIntroducedDatabaseFields->getIdsWithoutScope() at /srv/http/wolke7.brielmaier.net/apps/workflowengine/lib/Migration/PopulateNewlyIntroducedDatabaseFields.php:48
 OCA\WorkflowEngine\Migration\PopulateNewlyIntroducedDatabaseFields->run() at /srv/http/wolke7.brielmaier.net/lib/private/Repair.php:104
 OC\Repair->run() at /srv/http/wolke7.brielmaier.net/core/Command/Maintenance/Repair.php:126
 OC\Core\Command\Maintenance\Repair->execute() at /srv/http/wolke7.brielmaier.net/3rdparty/symfony/console/Command/Command.php:255
 Symfony\Component\Console\Command\Command->run() at /srv/http/wolke7.brielmaier.net/3rdparty/symfony/console/Application.php:915
 Symfony\Component\Console\Application->doRunCommand() at /srv/http/wolke7.brielmaier.net/3rdparty/symfony/console/Application.php:272
 Symfony\Component\Console\Application->doRun() at /srv/http/wolke7.brielmaier.net/3rdparty/symfony/console/Application.php:148
 Symfony\Component\Console\Application->run() at /srv/http/wolke7.brielmaier.net/lib/private/Console/Application.php:214
 OC\Console\Application->run() at /srv/http/wolke7.brielmaier.net/console.php:99
 require_once() at /srv/http/wolke7.brielmaier.net/occ:11

In PDOConnection.php line 88: [PDOException (42S02)] SQLSTATE[42S02]: Base table or view not found: 1146 Table 'nextcloud.flow_operations_scope' doesn't exist                                                                         

Exception trace:
  at /srv/http/wolke7.brielmaier.net/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnection.php:88
 PDO->query() at /srv/http/wolke7.brielmaier.net/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnection.php:88
 Doctrine\DBAL\Driver\PDOConnection->query() at /srv/http/wolke7.brielmaier.net/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php:908
 Doctrine\DBAL\Connection->executeQuery() at /srv/http/wolke7.brielmaier.net/lib/private/DB/Connection.php:199
 OC\DB\Connection->executeQuery() at /srv/http/wolke7.brielmaier.net/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php:200
 Doctrine\DBAL\Query\QueryBuilder->execute() at /srv/http/wolke7.brielmaier.net/lib/private/DB/QueryBuilder/QueryBuilder.php:215
 OC\DB\QueryBuilder\QueryBuilder->execute() at /srv/http/wolke7.brielmaier.net/apps/workflowengine/lib/Migration/PopulateNewlyIntroducedDatabaseFields.php:86
 OCA\WorkflowEngine\Migration\PopulateNewlyIntroducedDatabaseFields->getIdsWithoutScope() at /srv/http/wolke7.brielmaier.net/apps/workflowengine/lib/Migration/PopulateNewlyIntroducedDatabaseFields.php:48
 OCA\WorkflowEngine\Migration\PopulateNewlyIntroducedDatabaseFields->run() at /srv/http/wolke7.brielmaier.net/lib/private/Repair.php:104
 OC\Repair->run() at /srv/http/wolke7.brielmaier.net/core/Command/Maintenance/Repair.php:126
 OC\Core\Command\Maintenance\Repair->execute() at /srv/http/wolke7.brielmaier.net/3rdparty/symfony/console/Command/Command.php:255
 Symfony\Component\Console\Command\Command->run() at /srv/http/wolke7.brielmaier.net/3rdparty/symfony/console/Application.php:915
 Symfony\Component\Console\Application->doRunCommand() at /srv/http/wolke7.brielmaier.net/3rdparty/symfony/console/Application.php:272
 Symfony\Component\Console\Application->doRun() at /srv/http/wolke7.brielmaier.net/3rdparty/symfony/console/Application.php:148
 Symfony\Component\Console\Application->run() at /srv/http/wolke7.brielmaier.net/lib/private/Console/Application.php:214
 OC\Console\Application->run() at /srv/http/wolke7.brielmaier.net/console.php:99
 require_once() at /srv/http/wolke7.brielmaier.net/occ:11

There is no flow_operations_scope table in the database. Only flow_checks and flow_operations.

How can I resolve the issue? Can I create the missing table by hand?

[j-ed]

You can try to create the mssing table with the following parameters:

MariaDB [nextcloud]> describe oc_flow_operations_scope;
+--------------+-------------+------+-----+---------+----------------+
| Field        | Type        | Null | Key | Default | Extra          |
+--------------+-------------+------+-----+---------+----------------+
| id           | bigint(20)  | NO   | PRI | NULL    | auto_increment |
| operation_id | int(11)     | NO   | MUL | 0       |                |
| type         | int(11)     | NO   |     | 0       |                |
| value        | varchar(64) | YES  |     |         |                |
+--------------+-------------+------+-----+---------+----------------+
4 rows in set (0.01 sec)

[jonsger]

Thanks. Can you also show me the result of DESCRIBE flow_operations?

[j-ed]

MariaDB [nextcloud]> describe oc_flow_operations;
+-----------+--------------+------+-----+---------+----------------+
| Field     | Type         | Null | Key | Default | Extra          |
+-----------+--------------+------+-----+---------+----------------+
| id        | int(11)      | NO   | PRI | NULL    | auto_increment |
| class     | varchar(256) | NO   |     | NULL    |                |
| name      | varchar(256) | NO   |     | NULL    |                |
| checks    | longtext     | YES  |     | NULL    |                |
| operation | longtext     | YES  |     | NULL    |                |
| entity    | varchar(256) | NO   |     |         |                |
| events    | longtext     | NO   |     | NULL    |                |
+-----------+--------------+------+-----+---------+----------------+
7 rows in set (0.01 sec)

[n2020]

We get the same error when we updating from version 17.0.3 to 18.0.1.

SQLSTATE[42S02]: Base table or view not found: 1146 Table ‘nextcloud.oc_flow_operations_scope’ doesn’t exist
Update failed

[RalfM]

I had the same problem here, couldn’t insert the missing tables manually with the given information. I did a rollback to 17.0.3.

[efftee]

Hi RalfM, how did you make your rollback to 17.0.3 ?

[RalfM]

I simply replaced all files from the backup folder. The database was ok.

[kesselb]

Read this before you update to Nextcloud 18

Please use the right way to update your installation: https://docs.nextcloud.com/server/18/admin_manual/maintenance/manual_upgrade.html Some people just extract to update into the current nextcloud folder. This will not remove files from older version and can lead to unexpected results. For example: https://github.com/nextcloud/server/issues/18265 I understand that moving around those files is annoying. We know that apps/workflowengine/appinfo/database.xml is causing the issues. At least delet…

[efftee]

Thank you RalfM !
For those who may read this, after some research it seems this is the procedure here
https://docs.nextcloud.com/server/18/admin_manual/maintenance/restore.html