Nextcloud Talk Signaling Server - now automated installation

kyrofa · September 25, 2020, 4:26pm

Thank you @szaimen, that was helpful indeed. So it sounds like the interface between Nextcloud and the server is only network based. In that case, I see no reason you can’t point the Talk app from within the snap to your own server.

Note that the snap is not a VM, it’s just an application. You don’t need to somehow access it and install this server in there, you can install it on the same host running the snap (or even another, unrelated host), perhaps using the blog post @szaimen provided . The only difficulty you need to consider is confinement, because the application only has permission to do certain things. Thankfully, “talk on the network” is one of them, and it seems that’s all this would require.

enoch85 · September 25, 2020, 5:00pm

So it couldn’t be included in the snap itself?

In other words, you would need to install it separately from the snap which hosts Nextcloud, and then connect it with the snap?

kyrofa · September 25, 2020, 5:17pm

Sure it could be, in the same way that the snap could include openHAB if you also wanted smart home functionality out of it, but that doesn’t seem to be a proper separation of concerns . It seems better suited to be its own snap, or just traditionally installed.

onflow · November 11, 2020, 5:12pm

@enoch85 Thank you for all the work you do. I am co-hosting a medium-sized gathering (25-50 individuals) of folks coming up in 1.5 weeks and searching for a videoconference solution. I am wondering if you have had a chance to test the Nextcloud Talk Signaling Server script that you provide with groups of this size? Have you heard any discussion of anyone who has had a chance to test your script out in the wild?

I’ve got your script running beautifully presently and use it primarily as an individual user for the basic storage functions, but have all your other scripts installed and running behind a separate Nginx reverse proxy, but with also the necessary non-standard ports opened in my router firewall for the circumstances where the reverse proxy causes complications with my connection.

With gratitude. I very much appreciate all your contributions.

enoch85 · November 11, 2020, 8:14pm

@onflow Thanks for the feedback! <3 It’s actually not just me, but some more contributors as well - for the Talk script that’s @morph027 and @szaimen.

Regarding your question… I don’t know if anyone have tested it with many users, since nobody have told me about it. But, that’s also a rule of thumb in IT - if you don’t hear anything then you can expect that everything is working good. So, I guess it works, but it would be very nice if you could test it.

I’m not sure if you can use the script straight up when running behind a reverse proxy. The script is made to be installed standalone.

Please let me know the results!

anon71540698 · December 13, 2020, 3:30pm

I did a quick test of Hansson’s implementation of the HPB, i.e. signaling server.

Two identical VMs on a VMware cluster (using the free 40GB OVA file from Hansson).
It’s a 3-host Dell R740 cluster with full-flash vSAN storage with shared 4x1Gbps pipe.
Both had their CPU count doubled (2 to 4) and RAM quadrupled (2GB to 8GB) .
Both completely exposed to internet - no firewalls/VPN, no ports closed.
Standard script was used; GoDaddy’s DNS service and Let’s Encrypt SSL certs on each.
One VM had only Talk installed; the other has Talk+HPB (signaling server) installed.
The Talk+HPB VM used 2 different FQDNs, each with its own SSL cert, linked to the same IP.
Both VMs were automatically updated to 20.0.03 after installation.
Both worked after the final reboot without any tinkering…

I also have a Nextcloud VM (latest version; Ubuntu 18.04) without Talk but using BBB app.

Nine people (3x3 grid) were invited to join 4 different video conference setups

Nextcloud with Talk only (first VM)
Nextcloud with Talk+HPB (second VM)
Nextcloud with BBB app
Zoom (commercial license)

All users were within a city, using Windows PCs (not smartphones), 6 out of 9 on home WiFi.
In every case there was at least a 25/5 Mbps (down/up) connection to the internet (HTML5 test).
Each session lasted between 5 and 10 minutes…

No tools were used to measure performance and/or trace bottlenecks.
The idea was to get the overall (subjective) participants’ impression of different systems.

Overall impression:

Zoom was the smoothest (may have to do with it being used the most by participants).
Talk+HPB was close to Zoom; occasional freezes of some users (could be WiFi related)
No serious problems with Nextcloud+BBB app; some minor freezes/blocking at times.
Just Talk was unusable after the 5th user joined (long freezes; no audio/video).

Sticking with BBB for now. I’d switch from BBB to Talk+HPB if the latter had a recording option.

I liked the idea that HPB installed on one NC server can be used by a different NC server with just Talk installed (as an external HPB). The same as I use with coTURN, OnlyOffice, BBB… But I haven’t tested that…

EDIT:
The NC+BBB VM is not from Hansson’s OVA file.
It started its life at v.16 a few years ago, linked to LDAP/AD directory, using nginx/MariaDB (as opposed to Apache/Postgres), uses a coTURN server for STUN/TURN and runs on Ubuntu 18.04…
The external BBB server itself is still on Ubuntu 16.04… Hence, its’ not exactly apples2apples comparison…
Would be interesting to add Jitsi to the mix… It has a nice option where you can force the resolution on users webcams (i.e. down convert) to reduce traffic. In the end (I think) many problems arise from the crappy fluctuating home router WiFi bandwidth and the upstream bandwidth (i.e. sending your audio+video streams) often being one tenth of the downstream.
But my jitsi server went through too many customizations; need to start from scratch…

enoch85 · December 13, 2020, 9:57pm

Wow, thanks @anon71540698 for that extensive testing!

Byter · December 15, 2020, 7:47pm

Sorry for the slight off topic, but I don’t really know where to ask.
I managed to install the signaling server, it seems fine inside my lan.

I followed this guide: https://markus-blog.de/index.php/2020/07/30/how-to-install-nextcloud-talk-high-performance-backend-with-stun-turnserver-on-ubuntu/

But,
I want to run this behind my already existing webserver (it runs nextcloud) and I can’t figure out how to do the proxy correctly inside the first VM. (the second VM is the signaling server)

Right now I’m trying with this code, I even made a different dDNS for it:

<VirtualHost *:443>

ServerName si******s.com

ProxyRequests on
RequestHeader set X-Forwarded-Proto "http"

ProxyPass "/" "http://192.168.1.72/"
ProxyPassReverse "/" "http://192.168.1.72/"

# Enable proxying Websocket requests to the standalone signaling server

 RewriteEngine On
# Websocket connections from the clients.
RewriteRule ^/standalone-signaling/spreed$ - [L]
# Backend connections from Nextcloud.
RewriteRule ^/standalone-signaling/api/(.*) http://192.168.1.72:8080/api/$1 [L,P]

But I tried without a different DNS only with “/standalone-signaling” with and without <Location>.
I only managed to get 404, 500, 5003. Right now I can reach the second VM’s nginx default page from outside. But Nextcloud only give a 503 at the Talk signaling setting.

Some help would be much appreaciated! Thank you!

rakekniven · December 17, 2020, 5:32pm

Open a new thread please.

anon71540698 · December 18, 2020, 4:18pm

I have used the NC+HPB setup between just two parties for longer period of times. Three times…

Each time the hosting party’s computer - every time a different one - was locking up after ~40min…
It will lose the connection to the mic and webcam first… Then lock up the app/window/tab…

First, I blamed my (very) old Logitech webcam that serves both audio and video.
But then it was the same story on the latest Latitude 7410, with only Windows 10 LTSC.
And a Hackintosh (to eliminate Windows) attached to a Dell monitor with a built-in webcam.
Always Chrome browser… The connection each time was between Canada and the UK.

None of those ever created issues with the NC+BBB app setup…

szaimen · December 18, 2020, 4:28pm

Thanks for the report! Maybe this should get reported to the Nextcloud Talk team since for me it sounds like a bug in Talk itself?

anon71540698 · December 19, 2020, 6:51pm

This might have something to do with the SSL cert, HSTS in particular…

On a (possibly) related note: when looking at the SSL cert acquired running the script, it shows

Not the same as I normally see from Let’s Encrypt (albeit 2 months old)

Has Let’s Encrypt changed their certificate issuer name?

EDIT:
Looks like…

enoch85 · December 19, 2020, 10:03pm

The Talk scripts are confirmed to work running the Nextcloud snap as well. Amazing!

RJ-Make · December 22, 2020, 9:51pm

I hope I’m in the right place. I just learned about this built in mechanism to install and setup HPB. So I gave it try today and wow, installation was a breeze, and that is coming from a complete noob.

That said I can’t seem to get it working. I get an OK Response and an App Version in the NextCloud UI but I’m getting this error in the NextCloud Talk logging, and of course not connecting.

GuzzleHttp\Exception\ClientException: Client error: POST https://URL.ddns.net/api/v1/room/euuxbswk resulted in a 403 Forbidden response: 403 Forbidden

Forbidden (truncated…)

/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Middleware.php - line 65:

GuzzleHttp\Exception\RequestException::create()

/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php - line 203:

GuzzleHttp\Middleware::GuzzleHttp\{closure}("*** sensiti ... *")

/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php - line 156:

GuzzleHttp\Promise\Promise::callHandler()

/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/TaskQueue.php - line 47:

GuzzleHttp\Promise\Promise::GuzzleHttp\Promise\{closure}("*** sensiti ... *")

/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php - line 246:

GuzzleHttp\Promise\TaskQueue->run()

/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php - line 223:

GuzzleHttp\Promise\Promise->invokeWaitFn()

/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php - line 267:

GuzzleHttp\Promise\Promise->waitIfPending()

/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php - line 225:

GuzzleHttp\Promise\Promise->invokeWaitList()

/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php - line 62:

GuzzleHttp\Promise\Promise->waitIfPending()

/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Client.php - line 183:

GuzzleHttp\Promise\Promise->wait()

/var/www/nextcloud/lib/private/Http/Client/Client.php - line 304:

GuzzleHttp\Client->request()

/var/www/nextcloud/apps/spreed/lib/Signaling/BackendNotifier.php - line 80:

OC\Http\Client\Client->post()

/var/www/nextcloud/apps/spreed/lib/Signaling/BackendNotifier.php - line 130:

OCA\Talk\Signaling\BackendNotifier->doRequest()

/var/www/nextcloud/apps/spreed/lib/Signaling/BackendNotifier.php - line 317:

OCA\Talk\Signaling\BackendNotifier->backendRequest()

/var/www/nextcloud/apps/spreed/lib/Signaling/Listener.php - line 203:

OCA\Talk\Signaling\BackendNotifier->roomInCallChanged()

/var/www/nextcloud/3rdparty/symfony/event-dispatcher/EventDispatcher.php - line 251:

OCA\Talk\Signaling\Listener::OCA\Talk\Signaling\{closure}("*** sensiti ... *")

/var/www/nextcloud/3rdparty/symfony/event-dispatcher/EventDispatcher.php - line 73:

Symfony\Component\EventDispatcher\EventDispatcher->callListeners()

/var/www/nextcloud/lib/private/EventDispatcher/EventDispatcher.php - line 86:

Symfony\Component\EventDispatcher\EventDispatcher->dispatch()

/var/www/nextcloud/apps/spreed/lib/Room.php - line 963:

OC\EventDispatcher\EventDispatcher->dispatch()

/var/www/nextcloud/apps/spreed/lib/Controller/CallController.php - line 116:

OCA\Talk\Room->changeInCall()

/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php - line 169:

OCA\Talk\Controller\CallController->leaveCall()

/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php - line 100:

OC\AppFramework\Http\Dispatcher->executeController()

/var/www/nextcloud/lib/private/AppFramework/App.php - line 152:

OC\AppFramework\Http\Dispatcher->dispatch()

/var/www/nextcloud/lib/private/Route/Router.php - line 308:

OC\AppFramework\App::main()

/var/www/nextcloud/ocs/v1.php - line 88:

OC\Route\Router->match()

/var/www/nextcloud/ocs/v2.php - line 24:

require_once("/var/www/nextcloud/ocs/v1.php")

enoch85 · December 23, 2020, 8:59am

Thanks for the feedback!

Hmm, I never tried with ddns.net but there shouldn’t be any difference.

Does you Apache config math the DNS record, and does the DNS record match your external IP?

You can check DNS with http://whatsmydns.net

RJ-Make · December 23, 2020, 10:56am

It appears the A records do indeed point to my WAN IP.
I unfortunate don’t know how to check the Apache config.

enoch85 · December 24, 2020, 5:44am

@RJ-Make You could try: cat /etc/apache2/sites-enabled/yourconfig.conf

Post it here please.

RJ-Make · December 24, 2020, 12:37pm

I get this error returned.

cat: /etc/apache2/sites-enabled/yourconfig.conf:

“No such file or directory”

I don’t know what “yourconfig.conf” should be named. So after some research I found these 2 .conf files inside the “sites-enabled” directory.

site.ddns.net.conf >> url to base nextcloud installation the VM created.
site2.ddns.net.conf >> url for HPB VM script created.

Do you need to see the content of the files?

enoch85 · December 24, 2020, 12:57pm

Yes please

RJ-Make · December 24, 2020, 1:28pm

Main NextCloud URL

<VirtualHost :80>
RewriteEngine On
RewriteRule ^(.)$ https://%{HTTP_HOST} [R=301,L]

Header add Strict-Transport-Security: "max-age=15768000;includeSubdomains"
SSLEngine on
SSLCompression off
SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1
SSLHonorCipherOrder on
SSLCipherSuite 
SSLSessionTickets off

YOUR SERVER ADDRESS

ServerAdmin admin@site.ddns.net
ServerName site.ddns.net

SETTINGS

<FilesMatch "\.php$">
    SetHandler "proxy:unix:/run/php/php7.4-fpm.nextcloud.sock|fcgi://localhost"
</FilesMatch>

DocumentRoot /var/www/nextcloud

<Directory /var/www/nextcloud>
Options Indexes FollowSymLinks
AllowOverride All
Require all granted
Satisfy Any
</Directory>

<IfModule mod_dav.c>
Dav off
</IfModule>

SetEnv HOME /var/www/nextcloud
SetEnv HTTP_HOME /var/www/nextcloud

# The following lines prevent .htaccess and .htpasswd files from being
# viewed by Web clients.
<Files ".ht*">
Require all denied
</Files>

# Disable HTTP TRACE method.
TraceEnable off
# Disable HTTP TRACK method.
RewriteEngine On
RewriteCond %{REQUEST_METHOD} ^TRACK
RewriteRule .* - [R=405,L]

# Avoid "Sabre\DAV\Exception\BadRequest: expected filesize XXXX got XXXX"
<IfModule mod_reqtimeout.c>
RequestReadTimeout body=0
</IfModule>

LOCATION OF CERT FILES

SSLCertificateChainFile /etc/letsencrypt/live/site.ddns.net/chain.pem
SSLCertificateFile /etc/letsencrypt/live/site.ddns.net/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/site.ddns.net/privkey.pem
SSLOpenSSLConfCmd DHParameters /etc/letsencrypt/live/site.ddns.net/dhparam.pem

EXTRAS

SSLUseStapling On
SSLStaplingCache "shmcb:logs/ssl_stapling(32768)"

HPB URL

<VirtualHost *:443>
ServerName site2.ddns.net:443
SSLCertificateChainFile /etc/letsencrypt/live/site2.ddns.net/chain.pem
SSLCertificateFile /etc/letsencrypt/live/site2.ddns.net/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/site2.ddns.net/privkey.pem
SSLOpenSSLConfCmd DHParameters /etc/letsencrypt/live/site2.ddns.net/dhparam.pem

# Intermediate configuration
SSLEngine               on
SSLCompression          off
SSLProtocol             -all +TLSv1.2 +TLSv1.3
SSLCipherSuite          
SSLHonorCipherOrder     off
SSLSessionTickets       off
ServerSignature         off

# Logs
LogLevel warn
CustomLog /access.log combined
ErrorLog /error.log

# Just in case - see below
SSLProxyEngine On
SSLProxyVerify None
SSLProxyCheckPeerCN Off
SSLProxyCheckPeerName Off
# contra mixed content warnings
RequestHeader set X-Forwarded-Proto "https"
# Custom error page
ProxyErrorOverride On
DocumentRoot "/var/www/html"
ProxyPass /error/ !
ErrorDocument 404 /error/404_proxy.html
# Enable proxying Websocket requests to the standalone signaling server.
# https://httpd.apache.org/docs/2.4/mod/mod_proxy_wstunnel.html
ProxyPass / "http://127.0.0.1:8081/"
RewriteEngine on
RewriteCond %{HTTP:Upgrade} websocket [NC]
RewriteCond %{HTTP:Connection} upgrade [NC]
RewriteRule ^/?(.*) "ws://127.0.0.1:8081/$1" [P,L]
# Extra (remote) headers
RequestHeader set X-Real-IP %{REMOTE_ADDR}s
Header set X-XSS-Protection "1; mode=block"
Header set Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
Header set X-Content-Type-Options nosniff
Header set Content-Security-Policy "frame-ancestors 'self'"