Hello Thomas
I faced the same issue on Debian Stretch.
First, check that if you have openssl installed
sudo apt install openssl
Then I managed to solv this error by creating the OPENSSL_CONF variable
export OPENSSL_CONF=/etc/ssl/
and launch Nextcloud Appimage
./Nextcloud-2.3.3-x86_64.AppImage
Hope this will solve it on your system.
Loïc