Chunked upload /w pfsense haproxy as SSL offloader

hi,

since i think someone in the chain is not behaving right, i already already reported this is a bug (chunked upload /w pfsense haproxy as SSL offloader · Issue #26843 · nextcloud/server · GitHub) but i was referred here.

i have a dockerized nextcloud (21.0.1) running behind a pfsense haproxy.

like this:

client -- https --> pfsense haproxy --- http ["x-forwarded-proto"] --->[docker port exposed] nginx --- fpm ---> nextcloud

locally (i.e. connecting directly to pfsense) everything looks fine.

however, clients outside the local network sometimes are just a little bit slower (i think this is what makes the difference). then, what happens is this:

Error | no app in context | 
Sabre\DAV\Exception\BadRequest: Expected  filesize of 1724803 bytes but read (from Nextcloud client) and wrote  (to Nextcloud storage) 40960 bytes. Could either be a network problem on  the sending side or a problem writing to the storage on the server  side.
/var/www/html/apps/dav/lib/Connector/Sabre/Directory.php - line 156: 
OCA\DAV\Connector\Sabre\File->put(null)
/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php - line 1098: 
OCA\DAV\Connector\Sabre\Directory->createFile(" --- filename --- ", null)
/var/www/html/3rdparty/sabre/dav/lib/DAV/CorePlugin.php - line 504: 
Sabre\DAV\Server->createFile(" --- filename --- ", null, null)
/var/www/html/3rdparty/sabre/event/lib/WildcardEmitterTrait.php - line 89: 
Sabre\DAV\CorePlugin->httpPut(Sabre\HTTP\Request {}, Sabre\HTTP\Response {})
/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php - line 472: 
Sabre\DAV\Server->emit("method:PUT", [ Sabre\HTTP ... }])
/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php - line 253: 
Sabre\DAV\Server->invokeMethod(Sabre\HTTP\Request {}, Sabre\HTTP\Response {})
/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php - line 321: Sabre\DAV\Server->start()
/var/www/html/apps/dav/appinfo/v1/webdav.php - line 84: 
Sabre\DAV\Server->exec()/var/www/html/remote.php - line 167: require_once("/var/www/ht ... p")
-- | -- | --

i see no errors in the haproxy or nginx logs. when i do upload anything bigger than ~64k, i see the upload PUT just hanging there, until it eventually times out.

currently, the only workaround that enables me to upload again is:

a.) php occ config:app:set files max_chunk_size --value 32768
b.) disable any sort of connection keepalive in haproxy (“option forceclose”)

i can then see several (“numbered”) PUT requests of about 32k each. i have tried

  • various buffer-related configs in haproxy
  • turning gzip compression off in nginx
  • playing with other settings on the haproxy side

so far, nothing has worked. from the looks of it, something seems to be going wrong between the haproxy and nginx that messes up the headers OR breaks the connection keepalive.

any ideas?