Print php call stack when using occ files:scan from CLI

Hello,

I’m trying to understand the NextCloud code a bit. I would like to see the sequence of method calls and php scripts that get used/executed when I run this command from the CLI:

sudo -u www-data php /var/www/nextcloud/occ files:scan --all

I guess there are lots of ways to do this such as debugging in an IDE but from googling I figure you can capture and log a trace of the call stack quite easily with something like this:

$e = new Exception;
error_log(var_export($e->getTraceAsString(), true));

And here is the technique fleshed out a bit more:

function test() {
    throw new Exception;
}

try {
    test();
} catch(Exception $e) {
    error_log(var_export($e->getTraceAsString(), true));
}
?>

My Question:
Where do I put that bit of code so I can trigger the exception to log the call stack?
So far I found out that running the command php occ files:scan would start off in the Nextcloud install dir in the web server in a file called: occ

<?php
//$argv = $_SERVER['argv'];
require_once __DIR__ . '/console.php';

And this is an excerpt from console.php…

..
...
....
	$application = new Application(
		\OC::$server->getConfig(),
		\OC::$server->getEventDispatcher(),
		\OC::$server->getRequest(),
		\OC::$server->get(\Psr\Log\LoggerInterface::class),
		\OC::$server->query(\OC\MemoryInfo::class)
	);
	$application->loadCommands(new ArgvInput(), new ConsoleOutput());
	$application->run();
}catch (Exception $ex) {
	exceptionHandler($ex);
} catch (Error $ex) {
	exceptionHandler($ex);
}

Does anyone know where I plug in my bit of code e.g: throw new Exception; to trigger the exception then catch it so I can log the call stack when I execute the command occ files:scan ?

Thank you,

Flex

Hello.

The technique you purpose seems rather clumsy to me to be honest. It might work if you know a special location in code is triggering the problem but do not understand why it is called at all. Then, you can get a stack trace that way.

If you want to leave from live code, i highly suggest to setup a development environment and use step debugging. That way you see also the variables!

Am alternative would be to let the PHP parser create a trace of all called functions. Then you can amylase it in retrospective. But this is not easy to do (rather complex to not lose track). Maybe a combination of all suggested Melissa might work.

Christian

Just searching for “files:scan” in the Nextcloud Server repository source code yields server/Scan.php at master · nextcloud/server · GitHub which seems highly relevant as it looks like a command that on line 79 configures itself to act for the files:scan command. Isn’t this what you want?

Had that not been found I would have poked around in the Command folders under the Core and Apps directories to see what I would, since we know it’s a command you are looking for.

@christianlupus and @rawtaz
Thanks both for your suggestions but I simply want to know what sequence of methods get executed when I run the command php occ files:scan so I was hoping to use the method from my first post to get a call stack trace because that would tell me everything I need to know. Then I would not have to set up and learn how to use a php debugger in an IDE or poke around dozens of files not knowing what I’m looking for trying to make sense of what’s going on.

I really think I could get what I need if I knew where to put throw new Exception; and a matching Catch block.

Any advice, maybe from some php developers, how and where to integrate the exception handling mentioned in my first post would be really appreciated!

Cheers,

Flex

This is not accurate. It will only tell you the call stack to that specific point in the code (where you place the throw), but before that there may have been a lot of other calls going on that you won’t see. So this is simply not a good method.

That said, you should probably establish what you are actually looking for or wanting to see. If you are looking for something specific, your approach can be useful. But if you are more looking around to learn how things (e.g. the files:scan command) work, then a proper debugger is much better, e.g. for the reasons mentioned above.

I just told you one place where you can do it. What are you missing?

I’m not a php dev so I have to take your word on this but I thought the exception handling I mention in my first post would give the call stack from throw new exception all the way back to where the php occ files:scan command began its execution path. Maybe I’m wrong about that.

OK then I’ve spent a few hours now trying to set up xdebug to remote debug Nextcloud (NC) that is installed on a Pi 4 running Ubuntu Server from Visual Studio Code (VS Code) running in Manjaro on my laptop. In my set up I have nextcloud in apache2 in /var/www/nextcloud and I moved the data dir to an external drive that is mounted to the Ubuntu Server at /media/drive.

I managed to get this remote debugging from VS Code working for another c project but in the case of NextCloud it seems more complicated. In order to run commands through VS Code in Manjaro the VS Code user needs permissions to run Nextcloud and also to access the NC data dir. In my case NC is in apache with all those php files having ownership www-data.

So now I am getting permission errors in VS Code largely because VS Code really wants the php project to be set up in the user home folder and not in the apache web server /var/www directory. I’m also getting errors such as it cannot find the debugging “client”? and it cannot write to the data dir. Information on the internet is confusing and contradictory. I am using xdebug v3.x. The NC site says: " For debugging scripts on the command line, like occ or unit tests, set the XDEBUG_CONFIG environment variable." But it is not clear how or where I set that variable.

I knew setting up NextCloud debugging in an IDE would be a pain which is why I tried to avoid this at all costs. Any help on how to debug NC from an IDE like Visual Studio Code is much appreciated. For instance should I first install NC in my user home directory and then debug it from there? Or is this only a requirement of VS Code maybe?

Thank you,

Flex

Well, you need a bit of knowledge on how the pieces work together. Then, you can set up everything as needed.

First, I want to point out that typically, you use a manual working environment on your local machine. Then, you do not have to worry much about permissions and remote file systems. Especially, you should not use a productive system for debugging for security reasons. It was not obvious from your text to me if the RPi would be your productive env or just a test installation.

In general, I have used xdebug with NC and with VS code oftenly. It is working quite nicely so far.

Debugging in PHP is working a bit differently than for example with C code. So, this might be the issue here for you.

The main documentation is to be found on the official xdebug page. To set some configuration parameters of xdebug just do something like this (extracted from the docs):

export XDEBUG_CONFIG="client_host=192.168.42.34 log=/tmp/xdebug.log"

In the very same console the env variable is set now. As soon as you restart the console or open another profile or change to another user. You just have to retype the command again if you restarted the console.
In that console you can run the PHP command in question. (This assumes, you are using the host PHP interpreter. No docker, VM, or other intermediate abstraction layer.)

Before you can start the debugging session, prepare the VS code session. Note, that the PHP interpreter will call home to the IDE. If there is no IDE listening, the PHP interpreter will just continue running without debugging enabled. The next time, it starts (be is on the console or in the web browser), it will retry the IDE connection and decide again.
You will have to install an extension in VS code to allow the code IDE interact with the PHP interpreter. You prepare that configuration. You start the debugger in the code. This will start a server listening for requests by the PHP interpreter. Optionally set the break points in the IDE. Then, you start the PHP script in question.


As you are running on a Linux machine, I think you should just run on a single machine. Install locally both a PHP interpreter and a database (unless you want to use SQLite). Set up the database. Download the NC server package into the local machine. Unzip the file as the main user to some location (e.g. to /home/felix/nextcloud-server). Then, you can cd to that folder and install the NC instance using occ as you see fit. Then, you are ready to go with the IDE.

This assumes you are only interested in the occ execution. If you want to have a real running NC server that you can browse in the browser, you will have to do a bit more work. There is a way using docker to install it in a well-contained way or you can set up everything manually in whatever machine. The more complexity you add, the more problems might arise and the harder the setup might get. It is manageable, but you need more and more administrative skills.

In all relevant aspects, that is what you will get. What I meant is that besides the function calls you see in that stack trace, there may have been a lot of other code executing as well.

For example consider index.php calling foo() which in turn calls bar() and star() one after the other, and in star() you throw an exception and get a stack trace. You will in that stack trace only see index.php calling foo() calling star(), you will not see the call to bar().

As long as you are aware of this and it’s fine, maybe you can still make use of the stack trace by throwing it somewhere :slight_smile:


Depending on what you actually want to see/know, you may be able to get it just by adding some logging in strategic places of the code. I often debug PHP code that way, and it works fine and I don’t need a proper debugger attached to the PHP processes. It really all boils down to what you are actually trying to do all of this for, which unless I’m missing something is quite unclear (and hence you can’t get very specific answers).

@christianlupus
I am trying out your ideas now and will report back thank you.

@rawtaz
There is some confusion in the NC forums about what exactly php occ files:scan actually does. I think I figured it out from alot of testing but I want to see the method calls to be sure.

Specifically: I want to see what logic is applied when scanning a file and folder from the NC data dir in to the oc_filecache table in the NC db. I want to test when that file/folder already exists in the data dir and the NC db and also test when it has just been added to the data dir but is not in the NC db yet.

I want to understand what is the purpose of the mtime and the storage_mtime columns in the oc_filecache table. From testing I think storage_mtime contains the modified timestamp of the file or folder in the data dir and mtime contains the timestamp that NC displays in the NC Web UI of that file or folder from the data dir. So every time you use occ to do a scan it compares a files os-level modified timestamp with the timestamp in storage_mtime (if it exists) and if different it updates that timestamp in mtime (and presumably storage_mtime also).

Except I still don’t know why there needs to be BOTH mtime AND storage_mtime and I would like to see that logic performed in real time when I run php occ files:scan. OK so I guess a debugger is the best way to do that now I need to see if I can make a debugger work!

Cheers,

Flex

1 Like

Thanks for the clarification! Yeah, using a debugger for that is totally the nicest way to do it, because then you have so much simpler tools at your disposal to look around and inspect things as they run.

It could be done by just looking into the code and correlating that with some logging, but meh. Either way works.

1 Like