Download takes a long time to start, but download speed is then very fast

nxcl · October 25, 2019, 10:59am

Now updated to NC 17 and still no improvement.
Once I click download, the CPU goes crazy

This is shown in the debugger when I start a download

Is there a problem with the apps i have installed ?!?!

anon9582441 · October 25, 2019, 11:30am

Is Apache doing heavy IO? Perhaps it’s trying to cache the whole file to some temp place before sending it. Or perhaps trying to zip it etc.

Also try redis for memcache.local.

  'memcache.local' => '\\OC\\Memcache\\Redis',
  'memcache.distributed' => '\\OC\\Memcache\\Redis',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'redis' =>                                                             
  array (
    'host' => '/tmp/redis.sock',
    'port' => 0,
    'timeout' => 0,
    'dbindex' => 0,
  ),

nxcl · October 25, 2019, 12:07pm

I have now adjusted my nextcloud/config/config.php as follows:

Server restarted but unfortunately no improvement.
Or do I misunderstand your config?

In my config
'memcache.distributed' => '\\OC\\Memcache\\Redis',
is missing ?

How can I check if the file is loaded somewhere in the Temp first?
Or is it zipped?
If the download starts, it is not a zip.

anon9582441 · October 26, 2019, 8:16am

You can use lsof -p <pid> to see what files are open for a process.
Use ps ax|grep fpm to see the pids.

# ps ax|grep fpm
 4824 ?        Ss     0:44 php-fpm: master process (/etc/php/fpm-php7.3/php-fpm.conf)
13447 ?        S      1:38 php-fpm: pool www
13840 ?        S      1:40 php-fpm: pool www
17047 ?        S      1:18 php-fpm: pool www
19311 ?        S      1:35 php-fpm: pool www
20186 ?        S      1:40 php-fpm: pool www
20784 ?        S      1:40 php-fpm: pool www
22800 ?        S      1:38 php-fpm: pool www
24707 ?        S      1:35 php-fpm: pool www
26665 ?        S      1:37 php-fpm: pool www
26924 ?        S      1:39 php-fpm: pool www

The memcache.distributed is mentioned at https://docs.nextcloud.com/server/16/admin_manual/configuration_server/caching_configuration.html

Have you checked your if your SQL database is the bottle neck? I found that when I did thumbnail previews for all files my SQL DB grew with several GB, making it very slow.

nxcl · October 29, 2019, 6:38am

Hey,

ps ax|grep fpm

I’m not sure which pid i have to use…
But the only one where something with my downloaded file is shown is 18213
I can’t post the entire issue because too many characters for a post. Therefore, just the part where the affected file appears.

root@mylinuxclient:~# lsof -p 18213

php-fpm7. 18213 www-data  DEL    REG                0,5              22355 /dev/zero
php-fpm7. 18213 www-data  mem    REG                8,2     26376  3671401 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache
php-fpm7. 18213 www-data    0u   CHR                1,3       0t0        6 /dev/null
php-fpm7. 18213 www-data    1u   CHR                1,3       0t0        6 /dev/null
php-fpm7. 18213 www-data    2u   CHR                1,3       0t0        6 /dev/null
php-fpm7. 18213 www-data    3ur  REG                8,2         0  2097172 /tmp/.ZendSem.TnhSEd (deleted)
php-fpm7. 18213 www-data    4u  unix 0xffff8a546f66bc00       0t0   299563 /run/php/php7.3-fpm.sock type=STREAM
php-fpm7. 18213 www-data    5u  unix 0xffff8a5467d6c800       0t0   299579 type=STREAM
php-fpm7. 18213 www-data    6u  unix 0xffff8a5467d6d400       0t0   299580 type=STREAM
php-fpm7. 18213 www-data    7r   REG               8,16 874512384 31457380 /var/nc_data/admin/files/ubuntu-18.04.2-live-server-am                                                  d64.iso
php-fpm7. 18213 www-data    9u  unix 0xffff8a5467de9c00       0t0    22358 /run/php/php7.3-fpm.sock type=STREAM

Have you checked your if your SQL database is the bottle neck? I found that when I did thumbnail previews for all files my SQL DB grew with several GB, making it very slow.

root@mylinuxclient:/etc/mysql# cat my.cnf
[client]
default-character-set = utf8mb4
port = 3306
socket = /var/run/mysqld/mysqld.sock

[mysqld_safe]
log_error=/var/log/mysql/mysql_error.log
nice = 0
socket = /var/run/mysqld/mysqld.sock

[mysqld]
basedir = /usr
bind-address = 127.0.0.1
binlog_format = ROW
bulk_insert_buffer_size = 16M
character-set-server = utf8mb4
collation-server = utf8mb4_general_ci
concurrent_insert = 2
connect_timeout = 5
datadir = /var/lib/mysql
default_storage_engine = InnoDB
expire_logs_days = 10
general_log_file = /var/log/mysql/mysql.log
general_log = 0
innodb_buffer_pool_size = 1024M
innodb_buffer_pool_instances = 1
innodb_flush_log_at_trx_commit = 2
innodb_log_buffer_size = 32M
innodb_max_dirty_pages_pct = 90
innodb_file_per_table = 1
innodb_open_files = 400
innodb_io_capacity = 4000
innodb_flush_method = O_DIRECT
key_buffer_size = 128M
lc_messages_dir = /usr/share/mysql
lc_messages = en_US
log_bin = /var/log/mysql/mariadb-bin
log_bin_index = /var/log/mysql/mariadb-bin.index
log_error=/var/log/mysql/mysql_error.log
log_slow_verbosity = query_plan
log_warnings = 2
long_query_time = 1
max_allowed_packet = 16M
max_binlog_size = 100M
max_connections = 200
max_heap_table_size = 64M
myisam_recover_options = BACKUP
myisam_sort_buffer_size = 512M
port = 3306
pid-file = /var/run/mysqld/mysqld.pid
query_cache_limit = 2M
query_cache_size = 64M
query_cache_type = 1
query_cache_min_res_unit = 2k
read_buffer_size = 2M
read_rnd_buffer_size = 1M
skip-external-locking
skip-name-resolve
slow_query_log_file = /var/log/mysql/mariadb-slow.log
slow-query-log = 1
socket = /var/run/mysqld/mysqld.sock
sort_buffer_size = 4M
table_open_cache = 400
thread_cache_size = 128
tmp_table_size = 64M
tmpdir = /tmp
transaction_isolation = READ-COMMITTED
user = mysql
wait_timeout = 600

[mysqldump]
max_allowed_packet = 16M
quick
quote-names

[isamchk]
!include /etc/mysql/mariadb.cnf
!includedir /etc/mysql/conf.d/
key_buffer = 16M

gas85 · October 29, 2019, 10:39am

Check your preview directory size and be happy
I wrote about preview a bit here: Previews / data size - #2 by gas85 also there is a linked information how to decrease preview folder (and may be DB) size.

Did you moved you data directory even once after installation? I have had this issue when DB was twice bigger than it should: Files amount after moving of data directory is wrong (much bigger)

nxcl · October 29, 2019, 1:32pm

I adapted my config.php to yours but it didn’t help.

No, I set the data directory to /var/nc_data during the installation. At that time this problem didn’t exist either.

gas85 · October 29, 2019, 1:50pm

Config only will not helps you. You have to “reset” your preview cache also as described here

github.com/nextcloud/previewgenerator

How to config? How to reclaim space?

opened 07:07AM - 04 Jan 18 UTC

tavinus

Hi! #### Say I apply these settings ```sh sudo -u www-data php /var/www/nex…tcloud/occ config:app:set --value="64 128" previewgenerator squareSizes sudo -u www-data php /var/www/nextcloud/occ config:app:set --value="64 128" previewgenerator widthSizes sudo -u www-data php /var/www/nextcloud/occ config:app:set --value="64 128" previewgenerator heightSizes ``` #### And my `config.php` has ```php 'preview_max_x' => 2048, 'preview_max_y' => 2048, 'preview_max_scale_factor' => 2, ``` OR ```php 'preview_max_x' => null, 'preview_max_y' => null, 'preview_max_scale_factor' => 2, ``` #### And then I run `generate-all` Would the generator set the `'_max'` file to 128 pixels only and then just use that size as maximum even when I click to open an image? Or is just for pre-generating and nothing else and then the bigger image will be generated when clicked? I am but confused because I got some mixed results here, but I am guessing the browser cache was doing its thing. What would be the recommended settings for a big install with lots of images? We want to be able to browse it fast and have the best full-screen experience, but we would like to avoid having 30+Gb (still processing) of preview images data (which also goes to the backup, btw). And then there is also the question if it is safe to remove the preview files and/or folders to reclaim that space, which I already asked on core. https://github.com/nextcloud/server/issues/5263#issuecomment-355202378 I would probably prefer to make some simple bash scripting to remove just some bigger sized PNGs so I don't have to reprocess everything. That is easy if there is no problem with the database and other stuff. In any case, because I was testing different configs as well, I can see many different preview images above 1024px for some files, which is probably the reason for such a big footprint. I would prefer to have just the smaller sizes pre-generated (for menus and icons) and let the big stuff be done as needed. Can this be done with the config above? #### Edit Also, what does `pre-generate` really do? It runs in 1s and does not seem to create any files here. `generate-all` scans everything (for days) and sucks the processor and creates the pngs, but `pre-generate` does not seem to do anything.

nxcl · October 30, 2019, 8:18am

Now I’ve done it as described.
But unfortunately still no improvement.
I’m really helpless now and I’m running out of ideas.
After all, the download works, but the long loading time until it starts is really corrosive.

A new installation would be really fatal.

nxcl · October 31, 2019, 10:42am

<IfModule mod_headers.c>
  <IfModule mod_setenvif.c>
    <IfModule mod_fcgid.c>
       SetEnvIfNoCase ^Authorization$ "(.+)" XAUTHORIZATION=$1
       RequestHeader set XAuthorization %{XAUTHORIZATION}e env=XAUTHORIZATION
    </IfModule>
    <IfModule mod_proxy_fcgi.c>
       SetEnvIfNoCase Authorization "(.+)" HTTP_AUTHORIZATION=$1
    </IfModule>
  </IfModule>

  <IfModule mod_env.c>
    # Add security and privacy related headers
    Header always set Referrer-Policy "no-referrer"
    Header always set X-Content-Type-Options "nosniff"
    Header always set X-Download-Options "noopen"
    Header always set X-Frame-Options "SAMEORIGIN"
    Header always set X-Permitted-Cross-Domain-Policies "none"
    Header always set X-Robots-Tag "none"
    Header always set X-XSS-Protection "1; mode=block"
    SetEnv modHeadersAvailable true
  </IfModule>

  # Add cache control for static resources
  <FilesMatch "\.(css|js|svg|gif)$">
    Header set Cache-Control "max-age=15778463"
  </FilesMatch>

  # Let browsers cache WOFF files for a week
  <FilesMatch "\.woff2?$">
    Header set Cache-Control "max-age=604800"
  </FilesMatch>
</IfModule>
<IfModule mod_php7.c>
  php_value mbstring.func_overload 0
  php_value default_charset 'UTF-8'
  php_value output_buffering 0
  <IfModule mod_env.c>
    SetEnv htaccessWorking true
  </IfModule>
</IfModule>
<IfModule mod_rewrite.c>
  RewriteEngine on
  RewriteCond %{HTTP_USER_AGENT} DavClnt
  RewriteRule ^$ /remote.php/webdav/ [L,R=302]
  RewriteRule .* - [env=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
  RewriteRule ^\.well-known/host-meta /public.php?service=host-meta [QSA,L]
  RewriteRule ^\.well-known/host-meta\.json /public.php?service=host-meta-json [QSA,L]
  RewriteRule ^\.well-known/webfinger /public.php?service=webfinger [QSA,L]
  RewriteRule ^\.well-known/nodeinfo /public.php?service=nodeinfo [QSA,L]
  RewriteRule ^\.well-known/carddav /remote.php/dav/ [R=301,L]
  RewriteRule ^\.well-known/caldav /remote.php/dav/ [R=301,L]
  RewriteRule ^remote/(.*) remote.php [QSA,L]
  RewriteRule ^(?:build|tests|config|lib|3rdparty|templates)/.* - [R=404,L]
  RewriteCond %{REQUEST_URI} !^/\.well-known/(acme-challenge|pki-validation)/.*
  RewriteRule ^(?:\.|autotest|occ|issue|indie|db_|console).* - [R=404,L]
</IfModule>
<IfModule mod_mime.c>
  AddType image/svg+xml svg svgz
  AddEncoding gzip svgz
</IfModule>
<IfModule mod_dir.c>
  DirectoryIndex index.php index.html
</IfModule>
AddDefaultCharset utf-8
Options -Indexes
<IfModule pagespeed_module>
  ModPagespeed Off
</IfModule>
#### DO NOT CHANGE ANYTHING ABOVE THIS LINE ####

ErrorDocument 403 //
ErrorDocument 404 //
<IfModule mod_rewrite.c>
  Options -MultiViews
  RewriteRule ^core/js/oc.js$ index.php [PT,E=PATH_INFO:$1]
  RewriteRule ^core/preview.png$ index.php [PT,E=PATH_INFO:$1]
  RewriteCond %{REQUEST_FILENAME} !\.(css|js|svg|gif|png|html|ttf|woff2?|ico|jpg|jpeg|map)$
  RewriteCond %{REQUEST_FILENAME} !core/img/favicon.ico$
  RewriteCond %{REQUEST_FILENAME} !core/img/manifest.json$
  RewriteCond %{REQUEST_FILENAME} !/remote.php
  RewriteCond %{REQUEST_FILENAME} !/public.php
  RewriteCond %{REQUEST_FILENAME} !/cron.php
  RewriteCond %{REQUEST_FILENAME} !/core/ajax/update.php
  RewriteCond %{REQUEST_FILENAME} !/status.php
  RewriteCond %{REQUEST_FILENAME} !/ocs/v1.php
  RewriteCond %{REQUEST_FILENAME} !/ocs/v2.php
  RewriteCond %{REQUEST_FILENAME} !/robots.txt
  RewriteCond %{REQUEST_FILENAME} !/updater/
  RewriteCond %{REQUEST_FILENAME} !/ocs-provider/
  RewriteCond %{REQUEST_FILENAME} !/ocm-provider/
  RewriteCond %{REQUEST_URI} !^/\.well-known/(acme-challenge|pki-validation)/.*
  RewriteRule . index.php [PT,E=PATH_INFO:$1]
  RewriteBase /
  <IfModule mod_env.c>
    SetEnv front_controller_active true
    <IfModule mod_dir.c>
      DirectorySlash off
    </IfModule>
  </IfModule>
</IfModule>

May there is a problem?

<IfModule mod_rewrite.c>
  RewriteEngine on
  RewriteCond %{HTTP_USER_AGENT} DavClnt
  RewriteRule ^$ /remote.php/webdav/ [L,R=302]
  RewriteRule .* - [env=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
  RewriteRule ^\.well-known/host-meta /public.php?service=host-meta [QSA,L]
  RewriteRule ^\.well-known/host-meta\.json /public.php?service=host-meta-json [QSA,L]
  RewriteRule ^\.well-known/webfinger /public.php?service=webfinger [QSA,L]
  RewriteRule ^\.well-known/nodeinfo /public.php?service=nodeinfo [QSA,L]
  RewriteRule ^\.well-known/carddav /remote.php/dav/ [R=301,L]
  RewriteRule ^\.well-known/caldav /remote.php/dav/ [R=301,L]
  RewriteRule ^remote/(.*) remote.php [QSA,L]
  RewriteRule ^(?:build|tests|config|lib|3rdparty|templates)/.* - [R=404,L]
  RewriteCond %{REQUEST_URI} !^/\.well-known/(acme-challenge|pki-validation)/.*
  RewriteRule ^(?:\.|autotest|occ|issue|indie|db_|console).* - [R=404,L]
</IfModule>

TP75 · November 4, 2019, 9:32pm

Please have a look at:

Hope this helps.

nxcl · November 11, 2019, 1:54pm

Nope

Anyone else have an idea?
That really sucks.

TP75 · November 11, 2019, 3:12pm

@nxcl – Please understand I am very sorry to hear of your mishap and hassle in continuity. Please find my virtual gesture of sympathy included.

Unfortunately, many cooks spoil the broth.

IMHO one could have chosen to stay at NC16 to reduce complexity and to avoid the usual beta tester issues of an early so-called stable NC 17 release. However, happens all the time and almost every fresh major release, unfortunately.

I am very sorry to not being able to provide some advice more specific to your issue and cannot help the repair, unfortunately.

However, some more general advice and again mainly aimed at the public domain of this user forum.

One could consider to remove the php-fpm and use the libapache2-mod-php7.3 instead. This would be quite appropriate to most home user NC scenarios and one’s data would stay safe with the more conservative approach, I presume.

php7.3-fpm — server-side, HTML-embedded scripting language (FPM-CGI binary)

This package provides the Fast Process Manager interpreter that runs as a daemon and receives Fast/CGI requests. Note that MOST Apache users probably want the libapache2-mod-php7.3 package.
— Debian -- Details of package php7.3-fpm in buster —

Furthermore, one may consult the below article for background information.

Why NextCloudPi uses Apache and not Nginx

Naturally, one may not use a RasPi embedded system. However, one may learn about Apache2 vs. NGINX in a more reasonable way and referring to factual triage.

Good luck and all the best.

Happy hacking.

nxcl · November 11, 2019, 3:20pm

I will try this!

Hmm, at the moment some employees are testing and see this:

CPU 398. ??

Now I’m getting 504 GW Timeout …

TP75 · November 11, 2019, 3:35pm

@nxcl – Sorry, this complete thread is too long and the IMHO somewhat contradictory steps far too confusing for my pour old brain.

Just guessing in the dark:

Did you truly remove the php-fpm from the install?
Did you test the NC cache setup and are you using the preferred setup?
Memcaching crashes NC17 - #2
Memcaching crashes NC17 - #3 – cc @j-ed
Did you consider to study the NC 17 in completeness and more thoroughly?

Please understand I do not blame you and just lending a helping hand basically. IMHO from learning of the many in-official or vague sources of some advice above you could be in some serious trouble, unfortunately.

This is the home user forum and although you may ask freely and virtually anything, it may depend what you get answered for free. Any enterprise should ref. to Nextcloud GmbH as there shall be professional support available:

Customers and Partners (mainly) closed portal — https://portal.nextcloud.com/
An Enterprise Subscription from Nextcloud is available with email and phone support.

NOTE: Please be aware I am not affiliated with Nextcloud GmbH and my views are the free views as both a volunteer and a EU citizen and appear in a private capacity solely.

Good luck.

nxcl · November 12, 2019, 7:13am

I think, i did
sudo apt-get purge 'php*'

How do i test this correctly?

do I have to give www-data permission here?

I installed Redis like here:

TP75 · November 12, 2019, 9:07am

@nxcl – You are in trouble and you may have lost your data or may lose it inadvertently. Honestly, this way of procedure was not exactly brilliant.

Warning: You may lose all of your data (inluding the data of presumed customers) if you have no backup.

Please make a full backup of both the database and the nextcloud/data file tree onto an external volume. Afterwards detach this volume from your computer and put this backup volume in a secure and safe place.

IMHO your team seems to be completely in the dark. If you are a business and if you truly have employees please get somebody professional to assist you at your earliest convenience.

Unfortunately, you did not although your statement is correct. Apparently, you did not only ‘purge’ the php-fpm module but instead you may have removed any and all PHP modules and thus the complete PHP software package including all configuration files from your system.

Please recall the phrase ‘remove’ but not ‘purge’ was used by me above.
Please be aware that the wildcard operator php* addresses any and almost all PHP modules on your system.
Please consult the CLI manual for the apt command by help of the CLI i.e. just make a man apt and look for the entries of ‘remove’ and ‘purge’ please.
Please consider the grave error you may have made and you may have destroyed your PHP configuration completely and utterly lost any chance of safe return inadvertently.

Please think first, act second only and always have the protection of your data in mind.

BTW a kind gesture like problem solved (i.e. ) or a little ACK to one or more of my comments (i.e. click on the heart icon ) would show you are satisfied. This could be a kind gesture and would motivate me like authors of other advice to continue in lending a hand freely…

Please immediately start studying the basics and in your own best interest consult the documentation:

NC 16 Admin Docs » Installation and server configuration
NC 16 Admin Docs » Upgrade manually
Official Ubuntu Documentation
Ubuntu Server Guide
Debian System Administration
The Programming Historian - Introduction to the Bash Command Line
Debian Installation manual
The Debian Administrator’s Handbook - Unfortunately, looks as if stuck with Jessie but may be still a valuable source of advice for any system admin aware of the deviations to a more current stable install or to a different flavour of Linux also.

Take your time to study and make some trials in a safe environment. Consider to install a test system, if you have a spare machine at hand and you can afford the time.

Stick to the more known Un*x flavours. Naturally, you can choose Ubuntu, CentOS, FreeBSD, ArchLinux and probably NextCloudPi or other flavours besides Debian Linux. Also available are some other OS …

As I am in a good mood today and lending a hand please find an excerpt from the a.m. APT manpage:

Removing a package removes all packaged data, but leaves usually small (modified) user configuration files behind, in case the remove was an accident. Just issuing an installation request for the accidentally removed package will restore its function as before in that case. On the other hand you can get rid of these leftovers by calling purge even on already removed packages. Note that this does not affect any data or configuration stored in your home directory.
apt(8) — apt — Debian stretch — Debian Manpages

Happy hacking.

nxcl · November 12, 2019, 12:39pm

We are still testing Nextcloud with pilot users only.
We also use the cloud exclusively for temporary file exchange with external users. Accordingly, we don’t make a real backup of the data, which is also communicated to the employees. We delete all uploaded files automatically after 30 days anyway. A loss of data is therefore relatively unimportant.

I’m new to the world of Linux.
I already noticed my mistake myself. Well, too late.

TP75 · November 12, 2019, 12:44pm

Good to know and I rest my case. Your approach is a good one and I may have misunderstood some of your a.m. phrasings, apparently.

Furthermore, one could consult the Nextcloud Community Guidelines

Be pragmatic

Nextcloud is a pragmatic community. We value tangible results over having the last word in a discussion. We defend our core values like freedom and respectful collaboration, but we don’t let arguments about minor issues get in the way of achieving more important results. We are open to suggestions and welcome solutions regardless of their origin. When in doubt support a solution which helps getting things done over one which has theoretical merits, but isn’t being worked on. Use the tools and methods which help getting the job done. Let decisions be taken by those who do the work.

Better safe than ugly.

Good luck.

nxcl · November 19, 2019, 9:41am

Unfortunately, the problem is still there.

I have executed the fatal command “sudo apt-get purge ‘php*’” before, when I was doing something else because of PHP.
I’m not really sure if this caused my problem - but I know that I didn’t have my above problem at the beginning.

So I will reinstall my Nextcloud completely.

However, I have set up about 50 users locally, which I don’t like to rebuild. Better, I want to reinstall the Nextcloud over the weekend so that the users don’t notice anything about it. So they should be able to log back in on Monday with the same user. Also their documents should be available.

How can I do this?
I once looked for something - but unfortunately I don’t find a really good, up-to-date manual.
Do you know any instructions, at best self-tested?

Currently I have on my screen:

Database backup - HOW?
/var/nc_data is on separate drive
/var/www/html/nextcloud/config/config.php
/etc/apache2/sites-available/ -everything-
/etc/php/7.3/apache2/php.ini
/etc/fail2ban/jail.d/nextcloud.local