Cluster:
- 19.0.1-apache
- Redis 5.0.6 for file locking and session storage
- PHP 7
- Apache 2
- Minio S3 for storage
- Percona xtradb cluster 8.0.19
The whole cluster sits on top of kubernetes cluster.
Each component has 3 instances, e.g. 3 redis(s) , 3 Nextcloud instances and so forth…
The traffic is behind a load balancer proxy (NginX)
when I enable load balancer round robin on the proxy level (NginX),
the server throws to the user 401, and tells that the user is not logged in
What Have I missing ? =S
Kind of the cluster architecture setup
Server configuration
Docker image:
Docker file
FROM nextcloud:19.0.1-apache
RUN set -ex; \
\
apt-get update; \
apt-get install -y --no-install-recommends \
ffmpeg \
libmagickcore-6.q16-6-extra \
procps \
smbclient \
supervisor \
# libreoffice \
; \
rm -rf /var/lib/apt/lists/*
RUN set -ex; \
\
savedAptMark="$(apt-mark showmanual)"; \
\
apt-get update; \
apt-get install -y --no-install-recommends \
libbz2-dev \
libc-client-dev \
libkrb5-dev \
libsmbclient-dev \
; \
\
docker-php-ext-configure imap --with-kerberos --with-imap-ssl; \
docker-php-ext-install \
bz2 \
imap \
; \
pecl install smbclient; \
docker-php-ext-enable smbclient; \
\
# reset apt-mark's "manual" list so that "purge --auto-remove" will remove all build dependencies
apt-mark auto '.*' > /dev/null; \
apt-mark manual $savedAptMark; \
ldd "$(php -r 'echo ini_get("extension_dir");')"/*.so \
| awk '/=>/ { print $3 }' \
| sort -u \
| xargs -r dpkg-query -S \
| cut -d: -f1 \
| sort -u \
| xargs -rt apt-mark manual; \
\
apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false; \
rm -rf /var/lib/apt/lists/*
RUN mkdir -p \
/var/log/supervisord \
/var/run/supervisord \
;
RUN chmod -R 777 /usr/local/etc/php/conf.d
COPY supervisord.conf /
ENV NEXTCLOUD_UPDATE=1
RUN useradd -ms /bin/bash next
USER next
CMD ["/entrypoint.sh", "apache2-foreground"]
Nextcloud configuration:
Config report
<?php
$CONFIG = array (
'htaccess.RewriteBase' => '/',
'memcache.local' => '\\OC\\Memcache\\APCu',
'apps_paths' =>
array (
0 =>
array (
'path' => '/var/www/html/apps',
'url' => '/apps',
'writable' => false,
),
1 =>
array (
'path' => '/var/www/html/custom_apps',
'url' => '/custom_apps',
'writable' => true,
),
),
'overwriteprotocol' => 'https',
'overwritehost' => '*******',
'logfile' => '/dev/stdout',
'trusted_proxies' =>
array (
0 => '*******',
),
'skeletondirectory' => '',
'activity_expire_days' => 14,
'auth.bruteforce.protection.enabled' => false,
'knowledgebaseenabled' => false,
'trashbin_retention_obligation' => 'auto, 30',
'secret' => '*******',
'passwordsalt' => '*******',
'updater.release.channel' => 'stable',
'integrity.check.disabled' => false,
'mysql.utf8mb4' => true,
'memcache.distributed' => '\\OC\\Memcache\\Redis',
'memcache.locking' => '\\OC\\Memcache\\Redis',
'filelocking.enabled' => 'true',
'redis' =>
array (
'host' => 'redis-redis-ha-haproxy',
'port' => '6379',
),
'objectstore' =>
array (
'class' => '\\OC\\Files\\ObjectStore\\S3',
'arguments' =>
array (
'bucket' => 'nextcloud',
'hostname' => 'minio',
'port' => 9000,
'autocreate' => true,
'key' => '*******',
'secret' => ''*******',',
'region' => 'optional',
'use_ssl' => false,
'use_path_style' => true,
),
),
'mail_smtpmode' => 'smtp',
'mail_smtphost' => 'smtp.sendgrid.net',
'mail_smtpport' => '465',
'mail_smtpsecure' => 'ssl',
'mail_smtpauth' => true,
'mail_smtpauthtype' => 'LOGIN',
'mail_smtpname' => 'apikey',
'mail_smtppassword' => '*******',',
'mail_from_address' => 'admin',
'mail_domain' => '*******',',
'instanceid' => '*******',
'trusted_domains' =>
array (
0 => 'files.pica.co.il',
),
'datadirectory' => '/usr/local/data',
'dbtype' => 'mysql',
'version' => '19.0.1.1',
'overwrite.cli.url' => '*******',
'dbname' => 'nextcloud',
'dbhost' => 'haproxy',
'dbport' => '',
'dbtableprefix' => 'oc_',
'dbuser' => 'oc_kfirfer',
'dbpassword' => '''*******'',',
);