When a file changes frequently the NC client hangs

smokingwheels · June 12, 2024, 1:19pm

Support intro

Sorry to hear you’re facing problems

help.nextcloud.com is for home/non-enterprise users. If you’re running a business, paid support can be accessed via portal.nextcloud.com where we can ensure your business keeps running smoothly.

In order to help you as quickly as possible, before clicking Create Topic please provide as much of the below as you can. Feel free to use a pastebin service for logs, otherwise either indent short log examples with four spaces:

example

Or for longer, use three backticks above and below the code snippet:

longer
example
here

Some or all of the below information will be requested if it isn’t supplied; for fastest response please provide as much as you can

Nextcloud version: Nextcloud Hub 7 (28.0.6)
Operating system and version : Ubuntu 22.04 Desktop Both
Server 14 year old laptop 1 TB HDD 32 GB
Client 14 year old laptop 1 TB SSD 16 GB Linux 3.13.0
14 june Item 10 and 11
Client 14 year old laptop 1 TB SSD 8 GB Windows 7
Apache or nginx version : snap
PHP version : 8.2.18

The issue you are facing:
When a file changes frequently the NC client hangs.
Possibly when the file changes more than once it the time it takes to update to the server.

Is this the first time you’ve seen this error? : Y

Steps to replicate it:

Create File in clients working folder.
Create copy of clients file but adding some random data to the file.
Keep changing random contents so client and server can’t keep up

Errors various on client.

Checksum error
Connection timeout
File has changed on resume sync.
Client log fails to update on client gui.

13 june after appending to files on client side.

Failed to find valid token/etag combination for file.
Is locked, existing lock on file: exclusive
Conflict: Server version, local copy renamed.
Triggered resolve conflict button, option to keep both failed with error No such file or directory.
Client reported local file as zero bytes.
Using the dd command on server HDD to create a 1 GB file causes client to give up but then it resumes sync again after error.

The dd command generates high I/O load, which can cause delays in the system’s response times. This does lead to timeouts or errors in the Nextcloud client.

Error	index	
NotPermittedException
Could not create folder "/appdata_oc5itj6yexl8/p

14 june
Ran test of continually appending to a file overnight on regular interval.

Had 4 resolve conflicts on the same file but the server had trunkated the file (allhosts2). Solved by resolving conflict in client and uploading again.

/snap/nextcloud/42563/htdocs/apps/dav/lib/Connector/Sabre/File.php","Line":505,"message":"Could not open file","exception":{},"CustomMessage":"Could not open file"}}

Windows client 3.13.0 slows network bandwidth/transfer when watching sync in progress.
Github issue 6804
https://youtu.be/wW8-WkeqXQA
Github issue 4918
Syncing of Large files fails.
https://youtu.be/t1mY3lr8OPs
Github Issue 6792
Video is 2:12:43

Server has HDD 5400 rpm.
Tested server with DD command result 24.3 MB/s

dd if=/dev/zero of=/tmp/test.img bs=1G count=1 oflag=dsync
dd if=/dev/zero of=/tmp/test.img bs=1G count=10 oflag=dsync
dd if=/dev/zero of=/tmp/test.img bs=1G count=100 oflag=dsync

kworker/u16:2+flush-8:0 at 100 %
Client Network transfer rate is half what the HDD is doing (IOTOP)
Server has to work **twice as hard** as network bandwidth. (No Cache eg smaller files)

Client reports error but server still processing.
Client cant upload reason Exclusive lock.
because server is still processing uploads.
Client trys to sync again and client starts uploading.
Network traffic length too short for size of file. (20 GB). 

2nd attempt from client fails because you try to have 2 files on the server with same name.

My logs 12 june 2024
cloudparty.evils.in/index.php/s/june12_2024

My Logs 13 june 2024
http://gts.undo.it/index.php/s/wPDrxzFjXnQH3MX

My logs 14 june 2024
http://gts.undo.it/index.php/s/YAYHKaFjeHGsXw4

I made a video 18 mins if anyone is interested here 18 mins

I made a slide show of my screen shots over the last 3 days.
I have really enjoyed learning about the Nextcloud system.
My english is very hard to understand even I have trouble with some of my posts here.
Slide show 15 june 2024

My QB64 program used to test client and server.
Note: There is a bug using the sleep command so you have to use for next loops for timing.
Bug fixed used shell command and for next loops.

Rem A tester for YaCy PiHole Nextcloud
Rem limited
Rem June 2024
Randomize Timer
DefInt A-Z
Timer On
On Timer(1) GoSub health

digloop:
Rem ESC to exit
If InKey$ = Chr$(27) Then End
a = (25 * Rnd) + 97
b = (25 * Rnd) + 97
c = (25 * Rnd) + 97
d = (25 * Rnd) + 97
e = (25 * Rnd) + 97
f = (25 * Rnd) + 97
g = (25 * Rnd) + 97
h = (25 * Rnd) + 97



domain$ = Chr$(a%) + Chr$(b%) + Chr$(c%) + Chr$(d%) + Chr$(e%) + Chr$(f%) + Chr$(g%) + Chr$(h%)
index& = index& + 1
Hz = Hz + 1

Open "/mnt/cloudparty/limited/allhosts" For Input As #1
Open "/mnt/cloudparty/limited/allhosts2" For Output As #2
'Open "/mnt/cloudparty/limited/allhosts3" For Output As #3
'Open "/mnt/cloudparty/limited/allhosts4" For Output As #4
'Open "/mnt/cloudparty/limited/allhosts5" For Output As #5
'Open "/mnt/cloudparty/limited/allhosts6" For Output As #6
'Open "/mnt/cloudparty/limited/allhosts7" For Output As #7
'Open "/mnt/cloudparty/limited/allhosts2" For Append As #2
'Open "/mnt/cloudparty/limited/allhosts3" For Append As #3
'Open "/mnt/cloudparty/limited/allhosts4" For Append As #4
'Open "/mnt/cloudparty/limited/allhosts5" For Append As #5
'Open "/mnt/cloudparty/limited/allhosts6" For Append As #6
'Open "/mnt/cloudparty/limited/allhosts7" For Append As #7




Do While Not EOF(1)
    Input #1, a$
    Print #2, a$
    Print #2, domain$
    '   Print #3, a$
    '  Print #3, domain$
    ' Print #4, a$
    'Print #4, domain$
    '  Print #5, a$
    ' Print #5, domain$
    'Print #6, a$
    ' Print #6, domain$
    ' Print #7, a$
    '  Print #7, domain$


Loop
Close #1: Close #2: Close #3: Close #4: Close #5: Close #6: Close #7
'Sleep 1000
Rem 15 sec one loop
For i& = 1 To 2100000000: Next
If InKey$ = Chr$(27) Then End
For i& = 1 To 2100000000: Next
If InKey$ = Chr$(27) Then End
Print "Going to sleep"
If InKey$ = Chr$(27) Then End
Shell "sleep 30"
If InKey$ = Chr$(27) Then End
Shell "sleep 30"
If InKey$ = Chr$(27) Then End
Shell "sleep 30"
If InKey$ = Chr$(27) Then End
Shell "sleep 30"
If InKey$ = Chr$(27) Then End
Shell "sleep 30"
If InKey$ = Chr$(27) Then End
Shell "sleep 30"
If InKey$ = Chr$(27) Then End
Shell "sleep 30"
If InKey$ = Chr$(27) Then End





Print "I'm back      "
'For i& = 1 To 2100000000: Next
'For i& = 1 To 2100000000: Next
'For i& = 1 To 2100000000: Next
'For i& = 1 To 2100000000: Next
'For i& = 1 To 2100000000: Next
'For i& = 1 To 2100000000: Next
'For i& = 1 To 2100000000: Next
'For i& = 1 To 2100000000: Next
'For i& = 1 To 2100000000: Next
'For i& = 1 To 2100000000: Next
'For i& = 1 To 2100000000: Next
'For i& = 1 To 2100000000: Next
Rem 50 sec all

If InKey$ = Chr$(27) Then End



Close #1: Close #2

'Sleep 1

Rem timed on my computer rough Hz with no sleep states.
Rem Hz 100
'For i& = 1 To 1770000: Next

Rem 51 Hz
'For i& = 1 To 4000000: Next

Rem 30 Hz
'For i& = 1 To 7500000: Next

Rem 15 Hz
'For i& = 1 To 15000000: Next

'Next


GoTo digloop

health:
Locate 1, 1
Print Hz, index&, timer$
Hz = 0
Return

Hope that helps I welcome any comments!