Nextcloud and php-soap

sebastiank · August 10, 2023, 12:31am

Hey All,

This is a strange one. So I’m developing and app that uses php-soap and have the strangest thing happening when we test it on our production NC system.

So I’m developing the app on my local machine with Debian 11 (WSL) PHP 8.2, everything works as expected on this machine.

But when I transfer the app to the production system Debian 11 (LXC) PHP 8.1 is get an error while trying the service.wdsl file.

The file is stored locally on both machines and I’ve made sure its in the same place on both machines and has the same access permissions. (the location and access are NOT the problem)

The error I get is,

SOAP-ERROR: Parsing WSDL: Couldn't load from '/var/www/nextcloud/apps/integration_ews/lib/Components/EWS/Assets/services.wsdl' : failed to load external entity

Here is the strange part, if I create another one line php file without any NC code.

$client = \SoapClient('/var/www/nextcloud/apps/integration_ews/lib/Components/EWS/Assets/services.wsdl');

And I run this file as sudo www-data which is the same user as the NC app, it loads without any issue, and afterwards all the NC App code works as expected also.

Is there something in NC that could be causing this?

Here is the app if you want to look at the code.

Kerasit · August 10, 2023, 5:20am

Well as I reads it, there is difference in php version where you develops under the 8.2 library and tests it aswell, and in prod it runs under 8.1

Develop this and test it under same version as you aims to run it in production, aswell as be completely sure it is unit tested with the excact same modules enabled and configured as your production environment.

Development and unit testing tends to have dev packages and development aimed versions of modules, specifically packaged with additional stuff, for debugging and enabling unit testing more fine grained. I am sure you already know all this, however it is the most common root cause - even from experienced developers - and some details and configs is easy to “forget” when maintaining the test environment, as even here, you would like as much debug information as possible.

Your first action should be to setup a test environment as excact to your production as possible. Especially important is it to run the excact same PHP version and modules as in production.

nickvergessen · August 10, 2023, 11:06am

Hi, maybe you can try to actively enable XML entities, similar to this code in the user saml app:

The place of your code that might need adjusting is:

github.com

ksainc/integration_ews/blob/a134e08b47323d776255798286c1368e113d6933/lib/Service/CoreService.php#L161-L194


      
          	public function connectAccount(string $uid, string $account_provider, string $account_id, string $account_secret): bool {
          
          
		// construct remote data store client
          		$RemoteStore = new EWSClient($account_provider, $account_id, $account_secret, 'Exchange2007');
          		// retrieve root folder attributes
          		$rs = $this->RemoteCommonService->fetchFolder($RemoteStore, 'root', true, 'A');
          		// process response
          		if (isset($rs)) {
          			// extract server version from response
          			preg_match_all(
          				'/<ServerVersionInfo[^>]*?\sVersion=(["\'])?((?:.(?!\1|>))*.?)\1?/',
          				$RemoteStore->getClient()->__last_response,
          				$match
          			);
          
          
			// retrieve default configuration
          			$uc = $this->ConfigurationService->retrieveUser($uid);
          			// update configuration
          			$uc['account_provider'] = $account_provider;
          			$uc['account_id'] = $account_id;

This file has been truncated. show original

sebastiank · August 10, 2023, 10:37pm

Hi @Kerasit and @nickvergessen

Thank you for your feedback!

@nickvergessen I have tried using libxml_disable_entity_loader with a true and false value without success. But this seems to be a different solution which is worth trying. Thank you.

@Kerasit Yes, the PHP version difference would be an obvious issue but I tested PHP 7.4, 8.1 and 8.2 on my local machine and all worked properly. I agree I need to create some unit tests, but I am the soul developer I am the moment didn’t, as I have been doing live testing with Xdebug.

So after many hours of poking around and reading. I think I’ve narrowed down the issue, but not the cause. So from what I can tell the SoapClient() function creates a temporary binary encoded version to the WSDL file in “/tmp” and something NC is preventing it from creating it.

I have confirmed that deleting this file causes the same error. Which I can then regenerate by running the single line of from my initial post, outside of any NC code. BUT if that line is inside any class loaded by NC it fails.

Any ideas?

Sebastian

Kerasit · August 11, 2023, 6:04am

Partly got some ideas. Verify that your service account running your php (www-data) in production, is allowed to create files in /tmp

There IS a layer of interest here, as you runs in WSL in development and LXC in production.
WSL you are having a complete virtual harddrive exclusively for your WSL and in LXC you are running a none privileged container. It seems that in your production environment, your security is sane, so your PHP executeable (either PHP-FPM or the webserver using mod_php) is not allowed outside its own scope. Try the following to verify (I will use apache2 as example so the service account is www-data. Please swap with your own correct service account):

sudo -u www-data echo "#!/bin/bash" > /tmp/test.sh; sudo chmod +x /tmp/test.sh

If an executable file is created without issues in /tmp, all is fine.

sebastiank · August 11, 2023, 9:22pm

Hi @Kerasit,

So I’ve double checked and NC/PHP definitely, have access to “/tmp”.

In that screen shot you can see all the NC temp files, and the wsdl file that is created, when I run that single line out side of the NC environment. Once that file is in place the SoapClient works correctly in the NC environment.

I run this, either from the command line to by triggering it through the webserver:

<?php

$client = new \SoapClient('/var/www/nextcloud/apps/integration_ews/lib/Components/EWS/Assets/services.wsdl');

It’s something to do with the NC environment. IDK what though.

I am going to try @nickvergessen recommendation tonight and we’ll see what happens.

Sebastian

sebastiank · August 11, 2023, 10:16pm

WE HAVE A WINNER!!!

@nickvergessen thank you so much for the solution.

As per @nickvergessen post above the issue was being cause by a security feature in NC.

Sebastian