How to reset the complete NC core during testing?

Hello,

I am trying to avoid a problem with testing the DB connection during integration tests. Can you help me?

TL;DR

How can the complete NC core be restarted in order to enforce a new DB connection to be created?

Motivation

For integration testing, the DB is always a hard thing to deal with. This is the more true, as the DB is partly abstracted from the app’s code by the NC core, e.g. when accessing files in the NC instance from the files app. There is a - more or less complete - documentation of the API but the internals might change anytime.

Therefore, it makes sense to use the fresh fixture pattern (ref) to start all tests with a clean and well-defined state. So, it is necessary to reset the complete DB content for each test.

General idea

As long as the DB is not used yet, the server does not cache anything. So, one can think of a general structure of a test case like

// Namespace, use statements etc ...
class MyTestCase extends TestCase {
  protected function setUp() {
    // Load fixture into DB
    exec('path/restore-system-to-fixture.sh');

    // Prepare rest of system
  }
  // ...
}

The file path/restore-system-to-fixture.sh is a shell script that allows setting the complete system to a defined state (data folder, database dump for non-SQLite DBs).

The approach to manually code the fixture in PHP was not realized. This is due to the fact, that the internals of the files app need to be taken into account accordingly. This is out of the scope of our app and this might break the test easily if there is any modification made upstream.

Setup and wiring of the classes

Typically, there is a PHPUnit test suite, that has a bootstrap.php configured. This file is something like this (for the cookbook app):

<?php

require_once __DIR__ . '/../../../tests/bootstrap.php';
\OC_App::loadApp('cookbook');

The path __DIR__ . '/../../../tests/bootstrap.php' points to the bootstrap.php of the Nextcloud server. During evaluating this inclusion, some basic things are initialized like the \OC::$server and the corresponding database connection.

So, the order of events for a single test is

  1. Startup code of PHPUnit
  2. App bootstrap using bootstrap.php as above
  3. Core bootstrap using the bootstrap.php from the server repository
  4. Initialization of the server and opening of a connection to the DB
  5. Some internals of PHPUnit to prepare the test run
  6. Setup using setUp() method of test
  7. Test run of the main test function
  8. Tear down if configured

The bold

Non-SQLite database systems

For all database types except for SQLite this is working. The reason is that all these are capable of multiple simultaneous incoming connections.

So, the connection by the server core is already open but the test class can still trigger an external script to connect to the DB server. The DB might be rebuilt completely but this is all right at this point.

Only after the core has started accessing the data it might have started caching the data and this approach will no longer work. One way to accomplish this is to use PHPUnit’s feature of process separation.

Usage of SQLite

In general, SQLite only accepts one connection at a time. One can easily backup the database by simply copying the single file to another location. When done manually/asynchronously, this works well.

Problem description

When taking the approach from above, with SQL, the following problem arises:

The test is started and the list above starts to run. The core in an early stage opens a connection to the existing SQLite file. In the setUp the old file is unlinked and replaced by a new one (from the fixture).

The problem is that the server has the file already opened and the inode/file descriptor is saved within the PHP process. So, the old DB content is used during the test run. For the next test run, the current fixture will be used.

Obviously, this time shift by one test execution is all but obvious when looking at the PHP test classes and might be very surprising.

Ideas for a solution

There are a few ideas on how to tackle the problem but non was found to be working or it was unclear how to implement the solution.

Restart of the core

When it was possible to trigger/force a restart of the complete core or at least the database connection, one could do this after the SQLite file was recovered in the setUp() method.

The additional benefit of a complete restart would be that any cached data should be gone as well. That might allow avoiding the time-consuming process isolation.

Delaying of the bootstrap

Another approach is to not include the bootstrap.php of the NC core within the app’s bootstrap.php. As the bootstrap file of the core will trigger the DB connection, it should be called within the setUp() method to start the core only after the DB has been reset.

Usage of the PHP classes of the NC server

One could try to dump the classes involved using IDBConnection and similar classes and restore from within the NC process. This would avoid the problem with the access from different processes but introduce the problem that the complete fixture must be built in PHP. Any upstream change might break the complete test environment.

Usage of transactions

One idea is to use transactions to “undo” any DB change. While this might work well for queries against the data of the database (SELECT, INSERT, UPDATE, DELETE, CRUD operations), it will not necessarily work well with schema updates.

One of the reasons, I want to run the tests with a real database is to test all migrations (including error handling). As far as I know at least PostgreSQL is not able to rollback a update of the schema. For the other database engines I am not sure.

For testing of the usual business logic, this might be a feasible solution. There is even a package available to realize this structure.

Dummy DB as stub/mock

According to some literature, the best one could do regarding the DB was to create a stub or mock in order to avoid the runtime penalty of storing the things in the DB. As a side effect, the fixture would be only in PHP again.

I do not see how this might work here. All possible kinds of actions might be considered. In fact, I think one would have to rewrite a complete DB engine from scratch. Obviously, this is not the idea of that approach. Again the problem is there that the internals of the core should keep working but not be tested along the device-under-test.

Hello @christianlupus ,

are you aware of https://cloud.nextcloud.com/call/xs25tz5y ?

Best regards,
Mark

are you aware of backup instructions? the things you want to keep is exactly what you want to delete if you are looking to destroy the system…

I am well aware of the data structure. The thing is the timing in an automated test environment. I am doing exactly this in my restore-system-to-fixture.sh script.

Hello @rakekniven,

No, I was not aware of that room/chat. Is this some open chat, all developers or community participants can come together and have a talk?

I am not 100% sure, I get your point in the statement. Do you mean this topic was recently discussed there or that the topic might be better discussed in a interactive session?

Thanks in any case
Christian

Just want to let you know about this chat.
There you can ask development related questions or place a link to this topic.

Not many devs are in this forum regularly.

2 Likes

Hello @rakekniven, I wanted to ask if it is possible to obtain an account for that NC instance? Using the (externally shared) link, I have to manually keep track of the reading progress as I am always a “new” user to the server. This is sort of annoying.

I found this issue, so direct federation seems not possible if I get it correctly. Of course, I do not need any further functionality.

Hello @christianlupus ,

as I am not an administrator of this NC instance I would kindly ask you to contact @jospoortvliet .
As you are actively contributing to documentation (and maybe other areas) I would vote for it.

Best regards,
Mark

1 Like