UPDATED Aug 15 2018 - Script 100% auto install on Debian. Apache, MPM Event, php-fpm socket, Redis socket on both local and locking cache, LRU data eviction, LetsEncrypt SSL A+


#1

UPDATE Aug 15 2018
This script has been updated to work around a security issue that LetsEncrypt had. APUc cache has been replaced with intelligent LRU data eviction using a Redis unixsocket. The cache speed increase is incredible during benchmarking. Zend opcache is used for the PHP scripts. To keep things KISS modsecurity has been removed.

Why APCu kinda sucks….

  1. When APCu becomes completely full it simply dumps all of its data. Restarting Apache or PHP-FPM will also flush all of APCu’s data. Redis uses intelligent data eviction. Persistence and replication can also be set!

  2. APCu is old and crass. It is a hacked up version of APC leaving only the userland cache. Redis is awesome

When Redis 4.0 comes to Debian stable I will be using the new LFU Least Frequently Used, policy and adding multiple Redis sockets.


From 2017
Wrote a script that that will 100% do everything for a full install on Debian GNU/Linux. Found a lot of cargo cult programing scripts for nginx but wanted this to work for Apache using the Event MPM with php-fpm running on unixsockets. The shell comes clean in shellcheck and is in dash (not bash).

A lot of emphasis has been put into security. I still see people using the outdated way of using root passwords in the db. This script will install mariadb using a unixsocket to pass root perms when needed to the database. See https://goo.gl/14PsfB . modsecuity is also installed has a whitelist for NextCloud. Redis is on a password protect unixsocket.

Install a fresh copy of Debian 9.
Fill in your info such as name and domain name.
chmod +x nextcloud.sh && ./nextcloud.sh

#!/bin/sh


###############################################################################
## This script will install NextCloud on Debian 9 as follows:                ##
## php-fpm using unixsocket and own fpm pool for isolation                   ##
## Apache2 using MPM Event (should be a least as fast as nginx)              ##
## Redis though unixsocket  for memory local cache                           ##
## Redis though  unixsocket for memory locking cache                         ##
## Data eviction is using intellegent LRU for cache.			     ##
## SSL using Lets Encrypt with secure TLS defaluts. A+ Rating                ##
##                                                                           ##
## By Erik Adler aka onryo erik.adler@mensa.se                               ##
## gpg --keyserver pgp.mit.edu --recv-keys 0xedc3869e8fa82fc8                ##
###############################################################################



# login user name. Change this!
nc_user='tomthumb'

# login user password. Change this!
nc_pw='123456'

# Enter your domain name ie dingdong.com. Change this!
domainname='yoursite.com'

# email for vhost and cert. Change this!
email='mymail@protonmail.com'



###############################################################################

# The amount of cache used by Redis for locking and file cache.
# Will use LRU when full for intelligent data eviction. This can
# be adjusted for your systems RAM.
redis_max_mem='3G'

# Change to version of NextCloud to download if not latest.
# latest.tar.bz2 is default
nextcloudVersion='latest.tar.bz2'

# Public gpg key used to verify NextCloud
# The key can be found at https://nextcloud.com/nextcloud.asc
# D75899B9A724937A is default
gpgKey='D75899B9A724937A'

# Path to NextCloud. Feel free to change this.
nc_home='/var/www/vhosts'

# Defaut is a random 32 char pw for NextCloud db admin user.
# A backup can be found under /root/admin_pw_backup.txt if needed.
db_admin_pw="$(tr -cd '[:alnum:]' < /dev/urandom | fold -w32 | head -n1)"

###############################################################################


[ "$(id -u)" = 0 ] || { printf 'Must be root to run script\n'; exit 1; }

hostName="$(hostname)"

apt-get update -y && sudo apt-get upgrade -y

# Apache2 and php7.0-fpm stuff
apt-get install apache2 -y
apt-get install php7.0-fpm php7.0-gd php7.0-json php7.0-mysql php7.0-curl -y
apt-get install php7.0-intl php7.0-mcrypt php7.0-imagick php7.0-xml -y
apt-get install php7.0-gmp php7.0-smbclient php7.0-ldap php7.0-imap -y
apt-get install php7.0-mbstring php7.0-bz2 php7.0-zip  -y

a2enmod proxy_fcgi setenvif
a2enconf php7.0-fpm
systemctl reload apache2

# Media functions and preview.
apt-get install ffmpeg libreoffice -y

# Setup database
apt-get install mariadb-server -y

mysql -e "CREATE DATABASE nextclouddb;"
mysql -e "CREATE USER 'nextcloud'@'localhost' IDENTIFIED BY '$db_admin_pw';"
mysql -e "GRANT ALL PRIVILEGES ON nextclouddb.* TO 'nextcloud'@'localhost';"
mysql -e "FLUSH PRIVILEGES;"
mysql -e "quit"

# If for some reason you feel you need a NC pw backup. Only system root can read this.
echo "$db_admin_pw" > /root/admin_pw_backup.txt
chmod 400 /root/admin_pw_backup.txt

## By default starting with Debian 9 mariadb does not store a root
## password in the database. If your db gets owned there is no root user hash
## that can be cracked. root management is now via unixsocket by the system.
## A huge security increase! Renders SQLi -> hash cracking -> ownage obsolete.
## You can see here:  SELECT User, Host, password, plugin from mysql.user;
## For more info see https://goo.gl/14PsfB

# mysql_secure_installation. Keeping default unixsocket pw management.
apt-get install expect -y

secure_mysql=$(expect -c "
set timeout 10
spawn mysql_secure_installation
expect \"Enter current password for root (enter for none):\"
send \"$MYSQL\\r\"
expect \"Change the root password?\"
send \"n\\r\"
expect \"Remove anonymous users?\"
send \"y\\r\"
expect \"Disallow root login remotely?\"
send \"y\\r\"
expect \"Remove test database and access to it?\"
send \"y\\r\"
expect \"Reload privilege tables now?\"
send \"y\\r\"
expect eof
")

echo "$secure_mysql"
apt-get remove --purge expect -y


# Set up a php-fpm pool with a unixsocket
cat >/etc/php/7.0/fpm/pool.d/nextcloud.conf<<EOF
[Next Cloud]
user = www-data
group = www-data
listen = /run/php/php7.0-fpm.nextcloud.sock
listen.owner = www-data
listen.group = www-data
pm = dynamic
pm.max_children = 5
pm.start_servers = 3
pm.min_spare_servers = 2
pm.max_spare_servers = 4
pm.max_requests = 200
env[HOSTNAME] = $hostName
env[PATH] = /usr/local/bin:/usr/bin:/bin
env[TMP] = /tmp
env[TMPDIR] = /tmp
env[TEMP] = /tmp
security.limit_extensions = .php
php_admin_value [cgi.fix_pathinfo] = 1
EOF

# Disable the idling example pool.
mv /etc/php/7.0/fpm/pool.d/www.conf /etc/php/7.0/fpm/pool.d/www.conf.backup

# Get NextCloud and verify the gpg signature. Change to the exact nc version
cd /tmp || { printf 'There is no /tmp dir\n'; exit 1; }
apt-get install dirmngr sudo -y
gpg --recv-keys "$gpgKey" || gpg --keyserver pgp.mit.edu "$gpgKey"
wget "https://download.nextcloud.com/server/releases/$nextcloudVersion"
wget "https://download.nextcloud.com/server/releases/$nextcloudVersion.asc"

gpg --verify "$nextcloudVersion.asc" "$nextcloudVersion" 2>&1 | grep  \
    'Good signature' || { printf 'BAD GPG SIGNATURE\n'; exit 1; }

tar xjfv "$nextcloudVersion"
mkdir -p "$nc_home"
mv nextcloud "$nc_home/"
chown -R www-data:www-data "$nc_home/nextcloud"

# Sets up the vhost
cat >/etc/apache2/sites-available/nextcloud.conf<<EOF
<VirtualHost *:80>
    ServerAdmin "$email"
    DocumentRoot "$nc_home/nextcloud"
    ServerName "$domainname"
    <Directory "$nc_home/nextcloud/">
        AllowOverride All
        Options -Indexes +FollowSymlinks
        <IfModule mod_dav.c>
            Dav off
        </IfModule>
        SetEnv HOME "$nc_home/nextcloud"
        SetEnv HTTP_HOME "$nc_home/nextcloud"
    </Directory>
    <Directory "$nc_home/nextcloud/data/">
        Require all denied
    </Directory>
    <FilesMatch \\.php$>
        SetHandler "proxy:unix:/run/php/php7.0-fpm.nextcloud.sock|fcgi://localhost"
    </FilesMatch>
</virtualhost>
EOF

a2dissite 000-default
a2ensite nextcloud

systemctl reload apache2
systemctl reload php7.0-fpm.service

# Install NextCloud
cd "$nc_home/nextcloud/" || { printf 'No nextcloud dir\n'; exit 1; }

# root uses sudo -u to allow very arcain password strings.
sudo -u www-data php "$nc_home/nextcloud/occ"  maintenance:install \
    --database 'mysql' --database-name 'nextclouddb' --database-user 'nextcloud' \
    --database-pass "$db_admin_pw" --admin-user "$nc_user" --admin-pass "$nc_pw"

su -m www-data php -c "php $nc_home/nextcloud/occ config:system:set \
    trusted_domains 0 --value=$domainname"

# Enable all previews
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
  enable_previews --value=true --type=boolean"
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
  enabledPreviewProviders 0 --value='OC\\Preview\\PNG'"
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
  enabledPreviewProviders 1 --value='OC\\Preview\\JPEG'"
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
  enabledPreviewProviders 2 --value='OC\\Preview\\GIF'"
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
  enabledPreviewProviders 3 --value='OC\\Preview\\BMP'"
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
  enabledPreviewProviders 4 --value='OC\\Preview\\XBitmap'"
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
  enabledPreviewProviders 5 --value='OC\\Preview\\MarkDown'"
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
  enabledPreviewProviders 6 --value='OC\\Preview\\MP3'"
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
  enabledPreviewProviders 7 --value='OC\\Preview\\TXT'"
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
  enabledPreviewProviders 8 --value='OC\\Preview\\Illustrator'"
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
  enabledPreviewProviders 9 --value='OC\\Preview\\Movie'"
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
  enabledPreviewProviders 10 --value='OC\\Preview\\MSOffice2003'"
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
  enabledPreviewProviders 11 --value='OC\\Preview\\MSOffice2007'"
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
  enabledPreviewProviders 12 --value='OC\\Preview\\MSOfficeDoc'"
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
  enabledPreviewProviders 13 --value='OC\\Preview\\OpenDocument'"
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
  enabledPreviewProviders 14 --value='OC\\Preview\\PDF'"
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
  enabledPreviewProviders 15 --value='OC\\Preview\\Photoshop'"
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
  enabledPreviewProviders 16 --value='OC\\Preview\\Postscript'"
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
  enabledPreviewProviders 17 --value='OC\\Preview\\StarOffice'"
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
  enabledPreviewProviders 18 --value='OC\\Preview\\SVG'"
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
  enabledPreviewProviders 19 --value='OC\\Preview\\TIFF'"
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
  enabledPreviewProviders 20 --value='OC\\Preview\\Font'"

##  Redis for distributed caching on unixsocket
apt-get install php7.0-redis redis-server -y

## Generate pw for redis connection
redis_pw="$(tr -cd '[:alnum:]' < /dev/urandom | fold -w32 | head -n1)"

sed -i "s/# requirepass foobared/requirepass ${redis_pw}/g" /etc/redis/redis.conf
sed -i 's/port 6379/port 0/g' /etc/redis/redis.conf
sed -i 's/# unixsocket/unixsocket/g' /etc/redis/redis.conf
sed -i 's/unixsocketperm 700/unixsocketperm 770/g' /etc/redis/redis.conf

usermod -a -G redis www-data
chown -R redis:www-data /var/run/redis

systemctl reload apache2
systemctl reload php7.0-fpm.service
systemctl enable redis-server
systemctl start redis-server

su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
    memcache.locking --value='\\OC\\Memcache\\Redis'"
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
    filelocking.enabled --value='true' --type=boolean"
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
    redis host --value='/var/run/redis/redis.sock'"
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
    redis port --value='0' --type=integer"
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
    redis timeout --value='0' --type=integer"
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
    redis password --value=$redis_pw"

systemctl restart redis-server.service
systemctl reload apache2

## Setting local cache to use Redis.
su -m www-data -c "php $nc_home/nextcloud/occ config:system:set \
    memcache.local --value='\\OC\\Memcache\\Redis'"

## Setting up Redis data eviction policies in redis.conf for LRU allkeys
cat >>/etc/redis/redis.conf<<EOF
maxmemory "$redis_max_mem"
maxmemory-policy allkeys-lru
maxmemory-samples 5
EOF

# Tuning Redis at startup with initrams. Disable hugepages since
# they causes issues with Redis. Unix socket does not use tcp but the
# warning in the redis log is annoying.
cat >/etc/initramfs-tools/scripts/local-premount/redis<<EOF
#!/bin/sh

echo "never" > /sys/kernel/mm/transparent_hugepage/enabled
sysctl -w vm.overcommit_memory=1
sysctl -w net.core.somaxconn=1024
EOF

chmod +x /etc/initramfs-tools/scripts/local-premount/redis

update-initramfs -v -u -k "$(uname -r)"

# Zend opcache for PHP script cache
cat >>/etc/php/7.0/mods-available/opcache.ini<<EOF
opcache.enable=1
opcache.enable_cli=1
opcache.interned_strings_buffer=8
opcache.max_accelerated_files=10000
opcache.memory_consumption=128
opcache.save_comments=1
opcache.revalidate_freq=1
EOF

systemctl reload apache2
systemctl reload php7.0-fpm.service
systemctl restart redis-server.service

## Setup SSL using Lets Encrypt
apt install python-certbot-apache -y

sudo certbot --authenticator standalone --installer apache \
  --redirect -d "$domainname" --rsa-key-size 4096 --must-staple \
  --hsts --uir --staple-ocsp --strict-permissions --email "$email" \
  --agree-tos --pre-hook "service apache2 stop" \
  --post-hook "service apache2 start" -n

# Auto renew cert
crontab -l > certbot
echo '0 0 * * 0 /usr/bin/certbot renew' >> certbot
crontab certbot
rm certbot

Beim Aktivieren der App ist ein Fehler aufgetreten
How to build-client for Windows Platform _?
'memcache.local' => '\OC\Memcache\Redis', throws misconfiguration
#2

Little typo there I guess? :wink:

But thx for sharing, one can learn lot from this.


#3

Thx, the real take away beside using password protected unixsockets is the way there is no root password in the database. Back in the day I was a penetration tester and developer of BackTrack (now Kali Linux). SQLi, grabbing the hashes and then cracking them was on of my favorite methods. That wont happen when there is no root password =) The nextcloud admin pw prolly wont be broken since it is a 32 char random sting.
db_admin_pw="$(tr -cd ‘[:alnum:]’ < /dev/urandom | fold -w32 | head -n1)"

Would like to see a socket there too though.


#5

This script works btw, better even than the tech and me’s script


#7

Sorry for OT, where can I get the Tech and me Script? I have bought the Hyper-V Image, but since I am new at Linux, I can’t understand where all the pathes to find. Maybe Tech and me is very busy, but the support is awful. If I can read the script, I find out by my self.

TIA


#8

its on their github page:

Last I used it about 2 months ago it was tricky… got some errors didn’t know what was going wrong. I made my own installer for debian:

I found an even easier way to install it with yunohost. Just download the yunohost iso, install it to a vm, and then activate the nextcloud app. Worked great for me anyway.


#9

I’m not able to geht NC on my fresh Deb 9 running with this script :-/

I’m struggling with the Letsencrypt part.

Obtaining a new certificate
Performing the following challenges:
Client with the currently selected authenticator does not support any combination of challenges that will satisfy the CA.
Client with the currently selected authenticator does not support any combination of challenges that will satisfy the CA.

I have tried the update from certbot


But also not successful.
There i have a mistake - i think with the apache configuration because the certob script is not able zu check and activete my URL

Does anybody have any idear to support me?


#10

@Konrad : I have had the same problem with letsencrypt. The problem relies within the old version in debian strrtch’s standard repo. You have to enable stretch backports and update certbot and certbot-apache to the latest stretch-backport version. Best regards kamekun


#11

This is due to a security issue LetsEncrypt had with TLS-SNI challenge: https://community.letsencrypt.org/t/2018-01-11-update-regarding-acme-tls-sni-and-shared-hosting-infrastructure/50188

This method was disabled for older certbot clients than 20.X(?), thus for apache/nginx authentication modules. Solution is, if you don’t want to/ can update certbot, to use webroot authenticator, which uses http challenge by default: e.g.
certbot renew -i apache -a webroot -w /path/to/webroot (apache installer module but webroot authentication module).

But of course better to upgrade certbot client.


#12

I ended up using acme.sh to handle certbot/letsencrypt cert issuance and renewal and it works great since I don’t have a static ip. It can be found here https://github.com/Neilpang/acme.sh.

The issue I am having is that when I try to access nextcloud (both via local static ip and domain) via https - I get a 503 Service Unavailable error. Is there something additional I forgot to include - I should mention that I did enable SSL in Apache2 post script.

I did confirm I can access nextcloud via http but clearly perfer https.


#13

I use the NextcloudPi curl script for installation on Debian 9. I like that it handles SSL via the ncp-config tool. Also nice that the tool received unattended updates.


#14

I am making progress on resolving my errors but still have some questions. Since I am not using certbot to generate my let’s encrypt ssl and adjust Apache, what changes do I need to make to the Apache config & Nextcloud config files?


#15

Thank you very much for your useful script.

I used the backport-package of certbot and commented the apt install-line, but left everything else as it is.

I only receive a 403 Forbidden if I enter the URL

https://nc.mydomain.tld/index.php

In apache error log I get:

[Tue Jun 12 15:49:27.910228 2018] [authz_core:error] [pid 588:tid 140702918895360] [client 192.168.0.180:36178] AH01630: client denied by server configuration: /a/nextcloud/, referer: https://nc.mydomain.tld/nextcloud

Permissions for /a seem to be right.

Any ideas?

mrtx


#16

I have resolved all of my issues by modifying the Nextcloud.conf file to include a Virtual Host for 443 and point it to the location of my SSL certs.

When I run I check the basic settings I have 2 errors that come up; strict headers which I have corrected by modifying my nextcloud.conf file; and the other is PHP OPcache.

The specific PHP OPcache error is:

The PHP OPcache is not properly configured. For better performance it is recommended to use the following settings in the php.ini:

opcache.enable=1
opcache.enable_cli=1
opcache.interned_strings_buffer=8
opcache.max_accelerated_files=10000
opcache.memory_consumption=128
opcache.save_comments=1
opcache.revalidate_freq=1a

Any thoughts on how to modify the php files? The standard google results are not yielding any success.


#17

Have you checked the nextcloud.conf file to make sure it has both the 80 and 443 w/ssl paths virtual hosts listed? I was having a similar issue which led me to the virtual host conf.

Also have you confirmed you can access the 80 and 443 ports from outside your network? Many ISPs block them by default.


#18

@Scott This script isn’t so complex. It just installs the basics. The Tech and Me script provide tweaks and is based on a lot of arguments. In other words, it’s more complex and there might be bugs in it. If you find any, please report them to the Nextcloud VM repo on Github.

@3Dscrewer Why do you think that the support is “awful”?

In general, you can’t use the script on Debian, as it’s intended to use on Ubuntu 18.04 (atm).


#20

The way you have things set up it might be really easy to support both Debian and Ubuntu. Just have two different lists of dependencies. One for Ubuntu and one for Debian. The main script can do something like $ lsb_release -d then call the appropriate list!


#21

Are you willing to implement it? :wink:


#23

Yeah sure maybe.
When I find a little time I would like to first rewrite my script (the one above) to work around the security issue that broke LetsEncrypt with the TLS-SNI challenge: https://community.letsencrypt.org/t/2018-01-11-update-regarding-acme-tls-sni-and-shared-hosting-infrastructure/50188 and to use a tuned Redis instead of ACPu.

Ha det =)


#24

maybe this would do the trick ??

certbot --apache --rsa-key-size 4096 --must-staple --hsts --uir --staple-ocsp \
  --strict-permissions --email "$email" --agree-tos --redirect -d "$domainname"

to

certbot --rsa-key-size 4096 --authenticator standalone --installer apache  --must-staple --hsts --uir \
--staple-ocsp --strict-permissions --email "$email" --agree-tos --redirect \
  -d "$domainname" --pre-hook "apachectl -k stop" --post-hook "apachectl -k start"

and dont forget small dependency:
source.list
deb http://ftp.debian.org/debian stretch-backports main
and
apt-get install python-certbot-apache -t stretch-backports -y