Files become zero bytes

tflidd · January 13, 2017, 7:43am

thanks. ref: Files getting set to 0 in data folder · Issue #3056 · nextcloud/server · GitHub

Sanook · January 13, 2017, 11:28pm

Owncloud users also reporting the “zero file size bug”:

feisar · January 17, 2017, 10:17pm

Running Rsnapshot on the Nextcloud data directory is a good idea, peace of mind that you can revert any file back to a previous state.

I also, now, run this script every night, if any zero byte file exist in an array of given filesystems it emails a list of them out.

#!/bin/bash

filesystems[0]="/mnt/local/nextcloud/data/user1/files"
filesystems[1]="/mnt/local/nextcloud/data/user2/files"

emAdy="email@example.com"
emSub="Warning: Nextcloud Contains 0 Byte Files"

for fs in "${filesystems[@]}"; do

        while read -d '' -r; do
    		arr+=( "$REPLY" )
		done < <(find $fs -size 0 -print0)

done

if [[ ! -z ${arr[@]} ]]; then

	echo ${arr[@]} | mail -s "$emSub" "$emAdy"

fi

tflidd · January 28, 2017, 11:34am

There is a small fix of code that helps to debug this problem:

Stay tuned on the github issue!

Sanook · May 9, 2018, 5:27am

I have installed a fresh version of NC 13 but the problem still persists.

Thanks to the script of @feisar i could find it out.

So this is a warning to all Nextcloud users: don’t trust the desktop client!

Edit:

Today i had it again! One file of 65 GB of data had 0 bytes filesize after syncing it with the desktop client.

For those who want to verify that issue:

Install a Nextcloud instance on localhost in a virtual machine
Sync a huge amount of data in one run with the desktop sync client, lets say 100 GB
after it run the script of @feisar
if no errors then 100 GB upload is not enough
delete everything and retry with more data then 100 GB
if the srcript found zero byte files, open web browser, locate the file and watch the file size: does not display zero bytes (!) but can’t be opened of course. This could cause preview-plugin or fulltextsearch-plugin to run forever

With that error, Nextcloud can’t be recommended for business, sorry guys. @tflidd
Only for private use to sync and share small amount of data with your peers.

JasonBayton · May 11, 2018, 8:03pm

The Nextcloud desktop client, right? Because I haven’t seen this affect my array and I don’t use the client.

Sotan · May 11, 2018, 9:29pm

It seems like the upload procedure from the Client can destroy the Files on the Server.

Wonder if another Client, for example a Combination of webdav mount and rsync, would make less Work than finding the Problem.

JasonBayton · May 11, 2018, 9:54pm

If it helps, I personally use a mix of Android client, webDAV permanently mounted, and SSHFS. The latter only because I interface my data via the external storage app to avoid having problems accessing the data locally via filesystem.

Sanook · May 11, 2018, 10:28pm

Yes.

Well, i have again a brand new installation of NC 13 and the desktop client 2.3.3 build 1 on Windows 10. So the bug occured two times in the last days.

I’m just filling up the NC server with data and before it goes live i check if some zero files were created. This is a must i believe. Otherwise maybe i can’t restore the zero-file anymore.

But then you don’t have a copy of your files on your workstation.

Sotan · May 12, 2018, 6:04am

Doing rsync on the webdav could solute this.

After Reading of the Zero Byte Problem, i also found this at my Server. I really Need a sync solution so either i find an alternative of the nextcloud sync or I find an alternative to the whole nextcloud.

For me,An alternative to nextcloud sync could be using webdav. And downloading it on some PCs AS an Backup, one-directional. This also saves Space and sync time on the clients.

But I am very afraid - the 0 Byte Problem is a very very big Problem that exists in both paid and free owncloud and free nextcloud. But noone found a solution and not really much people even seem to Know about.

This is Something that should let me drop the usage of nextcloud.

JasonBayton · May 12, 2018, 2:37pm

That has literally never been an issue. Nor have I ever needed the Dropbox/box/GDrive clients either though so YMMV

There are other sync clients that can be used IIRC. I’ll have a dig around.

tflidd · May 12, 2018, 5:36pm

Ok, it still is an open topic, I reopened the topic on github again. I have never seen data loss myself, but I started to look for empty files on my own setup. Some are just locking-files or temp-files a lot in the trash bin. There are however a couple of actual files that are 0-size (also on the synced client). i will see in the next couple of days, if I can track that down somehow.

@JasonBayton: You haven’t seen this problem, so this could be a good indicator that this is actually the client?

JasonBayton · May 12, 2018, 5:50pm

I’d say so, yes.

Sotan · May 12, 2018, 6:38pm

Do you Know Any alternative to the Client? Would a rsynced webdav be a good Idea?

Sotan · May 12, 2018, 6:39pm

And maybe, could all Users Post the Last Change Date of the Zero Byte Files? So for example If they all are 2017/2018, You will quickly See in what Version of the Client the Problem occured First.

MikeLupe · May 13, 2018, 8:57am

@tflidd

I did some further research and updated as well the github issue https://github.com/nextcloud/server/issues/3056#issuecomment-274770918

"Update after some research:

I must correct that I only really lost two files to 0 but was able to recover them from a backup.

These two files may be from an older sync executed with an old windows NC client around end of 2016 / beginning of 2017. The other 0-ed files were either really empty (text files not really used) or were in one of these directories, which didn’t really bother:

Some files:
./data/appdata_50793ab339aa5/preview/16024/64-64-crop.png
./data/rainloop-storage/data/default/cache/c5/0c/c50c82f5bf4415357b97f23ab6deb16c6cd4d695

And some folders:
./data/user/files_trashbin/files/…
./data/appdata_ochidghtcad7/avatar/users/generated
./data/updater-*****/…

So this may really be related to a problem we had with a much older windows client. Sorry for the alarmism from my side."

Sotan · May 13, 2018, 9:15am

I am using the latest nextcloud Server Forum Ubuntu 18.04 and the latest nextcloud Client for Ubuntu 18.04. both installed two weeks ago.

The Zero File bug occured anyway. So I dont think that relies to an old client.

MikeLupe · May 13, 2018, 9:17am

So your affected files are from the past two weeks? Because for me besides the two older files it seems ok. NC Clients: Win7 / Deb 9.x

Sotan · May 13, 2018, 9:19am

I never Used nextcloud before. I migrated from Google two weeks ago after Testing nextcloud vor two weeks. After that two weeks, i reinstalled the whole VM with all actual versions.

But I only use the Ubuntu sync Client. Maybe the Error was corrected in the Windows Version, but Not in the Ubuntu Version

MikeLupe · May 13, 2018, 9:22am

Hmmm… ok.

So what does this one results for you in a bash on the server (-v to exclude updater dir):

find ./yournextcloud/data/ -size 0 -type f | grep -v updater | wc -l

and general
find ./yournextcloud/data/ -size 0 -type f | grep -v updater