Nextcloud Talk Signaling Server - now automated installation

@onflow Thanks for the feedback! <3 Itā€™s actually not just me, but some more contributors as well - for the Talk script thatā€™s @morph027 and @szaimen. :slight_smile:

Regarding your questionā€¦ I donā€™t know if anyone have tested it with many users, since nobody have told me about it. But, thatā€™s also a rule of thumb in IT - if you donā€™t hear anything then you can expect that everything is working good. :smiley: So, I guess it works, but it would be very nice if you could test it.

Iā€™m not sure if you can use the script straight up when running behind a reverse proxy. The script is made to be installed standalone.

Please let me know the results!

I did a quick test of Hanssonā€™s implementation of the HPB, i.e. signaling server.

  1. Two identical VMs on a VMware cluster (using the free 40GB OVA file from Hansson).
    Itā€™s a 3-host Dell R740 cluster with full-flash vSAN storage with shared 4x1Gbps pipe.
  2. Both had their CPU count doubled (2 to 4) and RAM quadrupled (2GB to 8GB) .
  3. Both completely exposed to internet - no firewalls/VPN, no ports closed.
  4. Standard script was used; GoDaddyā€™s DNS service and Letā€™s Encrypt SSL certs on each.
  5. One VM had only Talk installed; the other has Talk+HPB (signaling server) installed.
  6. The Talk+HPB VM used 2 different FQDNs, each with its own SSL cert, linked to the same IP.
  7. Both VMs were automatically updated to 20.0.03 after installation.
  8. Both worked after the final reboot without any tinkeringā€¦

I also have a Nextcloud VM (latest version; Ubuntu 18.04) without Talk but using BBB app.

Nine people (3x3 grid) were invited to join 4 different video conference setups

  • Nextcloud with Talk only (first VM)
  • Nextcloud with Talk+HPB (second VM)
  • Nextcloud with BBB app
  • Zoom (commercial license)

All users were within a city, using Windows PCs (not smartphones), 6 out of 9 on home WiFi.
In every case there was at least a 25/5 Mbps (down/up) connection to the internet (HTML5 test).
Each session lasted between 5 and 10 minutesā€¦

No tools were used to measure performance and/or trace bottlenecks.
The idea was to get the overall (subjective) participantsā€™ impression of different systems.

Overall impression:

  • Zoom was the smoothest (may have to do with it being used the most by participants).
  • Talk+HPB was close to Zoom; occasional freezes of some users (could be WiFi related)
  • No serious problems with Nextcloud+BBB app; some minor freezes/blocking at times.
  • Just Talk was unusable after the 5th user joined (long freezes; no audio/video).

Sticking with BBB for now. Iā€™d switch from BBB to Talk+HPB if the latter had a recording option.

I liked the idea that HPB installed on one NC server can be used by a different NC server with just Talk installed (as an external HPB). The same as I use with coTURN, OnlyOffice, BBBā€¦ But I havenā€™t tested thatā€¦

EDIT:
The NC+BBB VM is not from Hanssonā€™s OVA file.
It started its life at v.16 a few years ago, linked to LDAP/AD directory, using nginx/MariaDB (as opposed to Apache/Postgres), uses a coTURN server for STUN/TURN and runs on Ubuntu 18.04ā€¦
The external BBB server itself is still on Ubuntu 16.04ā€¦ Hence, itsā€™ not exactly apples2apples comparisonā€¦
Would be interesting to add Jitsi to the mixā€¦ It has a nice option where you can force the resolution on users webcams (i.e. down convert) to reduce traffic. In the end (I think) many problems arise from the crappy fluctuating home router WiFi bandwidth and the upstream bandwidth (i.e. sending your audio+video streams) often being one tenth of the downstream.
But my jitsi server went through too many customizations; need to start from scratchā€¦

5 Likes

Wow, thanks @anon71540698 for that extensive testing! :+1: :+1:

Sorry for the slight off topic, but I donā€™t really know where to ask.
I managed to install the signaling server, it seems fine inside my lan.

I followed this guide: https://markus-blog.de/index.php/2020/07/30/how-to-install-nextcloud-talk-high-performance-backend-with-stun-turnserver-on-ubuntu/

But,
I want to run this behind my already existing webserver (it runs nextcloud) and I canā€™t figure out how to do the proxy correctly inside the first VM. (the second VM is the signaling server)

Right now Iā€™m trying with this code, I even made a different dDNS for it:

<VirtualHost *:443>

ServerName si******s.com

ProxyRequests on
RequestHeader set X-Forwarded-Proto "http"

ProxyPass "/" "http://192.168.1.72/"
ProxyPassReverse "/" "http://192.168.1.72/"

# Enable proxying Websocket requests to the standalone signaling server

 RewriteEngine On
# Websocket connections from the clients.
RewriteRule ^/standalone-signaling/spreed$ - [L]
# Backend connections from Nextcloud.
RewriteRule ^/standalone-signaling/api/(.*) http://192.168.1.72:8080/api/$1 [L,P]

But I tried without a different DNS only with ā€œ/standalone-signalingā€ with and without <Location>.
I only managed to get 404, 500, 5003. Right now I can reach the second VMā€™s nginx default page from outside. But Nextcloud only give a 503 at the Talk signaling setting.

Some help would be much appreaciated! Thank you!

Open a new thread please.

I have used the NC+HPB setup between just two parties for longer period of times. Three timesā€¦

Each time the hosting partyā€™s computer - every time a different one - was locking up after ~40minā€¦
It will lose the connection to the mic and webcam firstā€¦ Then lock up the app/window/tabā€¦

First, I blamed my (very) old Logitech webcam that serves both audio and video.
But then it was the same story on the latest Latitude 7410, with only Windows 10 LTSC.
And a Hackintosh (to eliminate Windows) attached to a Dell monitor with a built-in webcam.
Always Chrome browserā€¦ The connection each time was between Canada and the UK.

None of those ever created issues with the NC+BBB app setupā€¦

Thanks for the report! Maybe this should get reported to the Nextcloud Talk team since for me it sounds like a bug in Talk itself?

1 Like

This might have something to do with the SSL cert, HSTS in particularā€¦

On a (possibly) related note: when looking at the SSL cert acquired running the script, it shows

image

Not the same as I normally see from Letā€™s Encrypt (albeit 2 months old)

image

Has Letā€™s Encrypt changed their certificate issuer name?

EDIT:
Looks likeā€¦

1 Like

The Talk scripts are confirmed to work running the Nextcloud snap as well. Amazing!

I hope Iā€™m in the right place. I just learned about this built in mechanism to install and setup HPB. So I gave it try today and wow, installation was a breeze, and that is coming from a complete noob.

That said I canā€™t seem to get it working. I get an OK Response and an App Version in the NextCloud UI but Iā€™m getting this error in the NextCloud Talk logging, and of course not connecting.

GuzzleHttp\Exception\ClientException: Client error: POST https://URL.ddns.net/api/v1/room/euuxbswk resulted in a 403 Forbidden response: 403 Forbidden

Forbidden (truncatedā€¦)

/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Middleware.php - line 65:

GuzzleHttp\Exception\RequestException::create()

/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php - line 203:

GuzzleHttp\Middleware::GuzzleHttp\{closure}("*** sensiti ... *")

/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php - line 156:

GuzzleHttp\Promise\Promise::callHandler()

/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/TaskQueue.php - line 47:

GuzzleHttp\Promise\Promise::GuzzleHttp\Promise\{closure}("*** sensiti ... *")

/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php - line 246:

GuzzleHttp\Promise\TaskQueue->run()

/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php - line 223:

GuzzleHttp\Promise\Promise->invokeWaitFn()

/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php - line 267:

GuzzleHttp\Promise\Promise->waitIfPending()

/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php - line 225:

GuzzleHttp\Promise\Promise->invokeWaitList()

/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php - line 62:

GuzzleHttp\Promise\Promise->waitIfPending()

/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Client.php - line 183:

GuzzleHttp\Promise\Promise->wait()

/var/www/nextcloud/lib/private/Http/Client/Client.php - line 304:

GuzzleHttp\Client->request()

/var/www/nextcloud/apps/spreed/lib/Signaling/BackendNotifier.php - line 80:

OC\Http\Client\Client->post()

/var/www/nextcloud/apps/spreed/lib/Signaling/BackendNotifier.php - line 130:

OCA\Talk\Signaling\BackendNotifier->doRequest()

/var/www/nextcloud/apps/spreed/lib/Signaling/BackendNotifier.php - line 317:

OCA\Talk\Signaling\BackendNotifier->backendRequest()

/var/www/nextcloud/apps/spreed/lib/Signaling/Listener.php - line 203:

OCA\Talk\Signaling\BackendNotifier->roomInCallChanged()

/var/www/nextcloud/3rdparty/symfony/event-dispatcher/EventDispatcher.php - line 251:

OCA\Talk\Signaling\Listener::OCA\Talk\Signaling\{closure}("*** sensiti ... *")

/var/www/nextcloud/3rdparty/symfony/event-dispatcher/EventDispatcher.php - line 73:

Symfony\Component\EventDispatcher\EventDispatcher->callListeners()

/var/www/nextcloud/lib/private/EventDispatcher/EventDispatcher.php - line 86:

Symfony\Component\EventDispatcher\EventDispatcher->dispatch()

/var/www/nextcloud/apps/spreed/lib/Room.php - line 963:

OC\EventDispatcher\EventDispatcher->dispatch()

/var/www/nextcloud/apps/spreed/lib/Controller/CallController.php - line 116:

OCA\Talk\Room->changeInCall()

/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php - line 169:

OCA\Talk\Controller\CallController->leaveCall()

/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php - line 100:

OC\AppFramework\Http\Dispatcher->executeController()

/var/www/nextcloud/lib/private/AppFramework/App.php - line 152:

OC\AppFramework\Http\Dispatcher->dispatch()

/var/www/nextcloud/lib/private/Route/Router.php - line 308:

OC\AppFramework\App::main()

/var/www/nextcloud/ocs/v1.php - line 88:

OC\Route\Router->match()

/var/www/nextcloud/ocs/v2.php - line 24:

require_once("/var/www/nextcloud/ocs/v1.php")

Thanks for the feedback!

Hmm, I never tried with ddns.net but there shouldnā€™t be any difference.

Does you Apache config math the DNS record, and does the DNS record match your external IP?

You can check DNS with http://whatsmydns.net

It appears the A records do indeed point to my WAN IP.
I unfortunate donā€™t know how to check the Apache config.

@RJ-Make You could try: cat /etc/apache2/sites-enabled/yourconfig.conf

Post it here please.

I get this error returned.

cat: /etc/apache2/sites-enabled/yourconfig.conf:

ā€œNo such file or directoryā€

I donā€™t know what ā€œyourconfig.confā€ should be named. So after some research I found these 2 .conf files inside the ā€œsites-enabledā€ directory.

site.ddns.net.conf >> url to base nextcloud installation the VM created.
site2.ddns.net.conf >> url for HPB VM script created.

Do you need to see the content of the files?

Yes please :slight_smile:

Main NextCloud URL

<VirtualHost :80>
RewriteEngine On
RewriteRule ^(.
)$ https://%{HTTP_HOST} [R=301,L]

<VirtualHost *:443>

Header add Strict-Transport-Security: "max-age=15768000;includeSubdomains"
SSLEngine on
SSLCompression off
SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1
SSLHonorCipherOrder on
SSLCipherSuite 
SSLSessionTickets off

YOUR SERVER ADDRESS

ServerAdmin admin@site.ddns.net
ServerName site.ddns.net

SETTINGS

<FilesMatch "\.php$">
    SetHandler "proxy:unix:/run/php/php7.4-fpm.nextcloud.sock|fcgi://localhost"
</FilesMatch>

DocumentRoot /var/www/nextcloud

<Directory /var/www/nextcloud>
Options Indexes FollowSymLinks
AllowOverride All
Require all granted
Satisfy Any
</Directory>

<IfModule mod_dav.c>
Dav off
</IfModule>

SetEnv HOME /var/www/nextcloud
SetEnv HTTP_HOME /var/www/nextcloud

# The following lines prevent .htaccess and .htpasswd files from being
# viewed by Web clients.
<Files ".ht*">
Require all denied
</Files>

# Disable HTTP TRACE method.
TraceEnable off
# Disable HTTP TRACK method.
RewriteEngine On
RewriteCond %{REQUEST_METHOD} ^TRACK
RewriteRule .* - [R=405,L]

# Avoid "Sabre\DAV\Exception\BadRequest: expected filesize XXXX got XXXX"
<IfModule mod_reqtimeout.c>
RequestReadTimeout body=0
</IfModule>

LOCATION OF CERT FILES

SSLCertificateChainFile /etc/letsencrypt/live/site.ddns.net/chain.pem
SSLCertificateFile /etc/letsencrypt/live/site.ddns.net/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/site.ddns.net/privkey.pem
SSLOpenSSLConfCmd DHParameters /etc/letsencrypt/live/site.ddns.net/dhparam.pem

EXTRAS

SSLUseStapling On
SSLStaplingCache "shmcb:logs/ssl_stapling(32768)"

HPB URL

<VirtualHost *:443>
ServerName site2.ddns.net:443
SSLCertificateChainFile /etc/letsencrypt/live/site2.ddns.net/chain.pem
SSLCertificateFile /etc/letsencrypt/live/site2.ddns.net/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/site2.ddns.net/privkey.pem
SSLOpenSSLConfCmd DHParameters /etc/letsencrypt/live/site2.ddns.net/dhparam.pem

# Intermediate configuration
SSLEngine               on
SSLCompression          off
SSLProtocol             -all +TLSv1.2 +TLSv1.3
SSLCipherSuite          
SSLHonorCipherOrder     off
SSLSessionTickets       off
ServerSignature         off

# Logs
LogLevel warn
CustomLog /access.log combined
ErrorLog /error.log

# Just in case - see below
SSLProxyEngine On
SSLProxyVerify None
SSLProxyCheckPeerCN Off
SSLProxyCheckPeerName Off
# contra mixed content warnings
RequestHeader set X-Forwarded-Proto "https"
# Custom error page
ProxyErrorOverride On
DocumentRoot "/var/www/html"
ProxyPass /error/ !
ErrorDocument 404 /error/404_proxy.html
# Enable proxying Websocket requests to the standalone signaling server.
# https://httpd.apache.org/docs/2.4/mod/mod_proxy_wstunnel.html
ProxyPass / "http://127.0.0.1:8081/"
RewriteEngine on
RewriteCond %{HTTP:Upgrade} websocket [NC]
RewriteCond %{HTTP:Connection} upgrade [NC]
RewriteRule ^/?(.*) "ws://127.0.0.1:8081/$1" [P,L]
# Extra (remote) headers
RequestHeader set X-Real-IP %{REMOTE_ADDR}s
Header set X-XSS-Protection "1; mode=block"
Header set Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
Header set X-Content-Type-Options nosniff
Header set Content-Security-Policy "frame-ancestors 'self'"
1 Like

A quick update: this turned out to be a network problem (donā€™t really know all the details).
But after it was corrected, I had two more 1h+ conferences and both went without any issuesā€¦

So, no stability/reliability issues whatsoeverā€¦

3 Likes

I donā€™t see anything obvious except that these seems to be missing. Did you remove them manually?

It should look like this:

Please try to reinstall (run the script again) and see if it changes.

I did. I was not sure it that cipher was sensitive information. I will attempt a re-install this weekend and report back.

Re-installed with the same results.

I used the journalctl -lfu signaling to watch the logs and I noticed the only url I see in that of the main site, not the HPB (site2).

Perhaps this log is for the main service.