Not all files show under a specific folder

ℹ️ Support
, ,
1

Having an issue where about 8 files are not showing up in a specific folder on Nextcloud. They are physically there when I look at the actual folder. The files are on a CIFS/SMB share we have connected using external storage support. Basically files are on the host Windows 10 box. Nextcloud is running on a VMware image from techandme on this Windows 10 box.

I’ve tried to rename the file still don’t show. Moved it to a different folder still don’t show. Made sure all permissions were reset inside windows for that folder still doesn’t show. I don’t get why it’s not showing these 8 specific files in Nextcloud browser or iOS app. Can’t see much in the log that points me to why It’s not showing up. Any help would be appreciated on what’s causing this.

Nextcloud version 13.0.3
Operating system and version Ubuntu 16.04 (VM techandme)
Apache or nginx version VM Techandme
PHP version VM Techandme

Is this the first time you’ve seen this error? Yes

Steps to replicate it:

  1. Go to folder. Refresh, files are missing
  2. Go to Nextcloud iOS app, go to same folder, files missing.
2

The problem here is that Nextcloud has cached the list of files/folders in the directory, to reduce the latency when users ask what’s available. It fetches the full file when one is requested, but that navigation is cached.

This caching assumes nothing else is going to change the folder, unless you tell it differently. The admin guide’s SMB/CIFS page has instructions for this. It’ll request for the SMB/CIFS server to notify it when files/folders change, then it will update its cache in batches. If there’s too long between those batches (so that users are noticing the delay), you can look at “Decrease sync delay” to have those run more often without too much overhead.

3

Thanks for the clarification.

I have each of the shares set with the option of " Check for changes - Once every direct access" So I was assuming it should be doing it each time.

Now to further test this, I created another folder, moved 5 of the files (which I cant see) into that folder. In Nextcloud looked at that new folder, still no files in there. Created a random text file, that shows instantly. So Im leaning on something to do with these files, I dont get why its these specific files, they are all the same extension. Ive also made sure all permissions at the file level the same as I asked Windows to overwrite all child permissions (which I did at the folder level). So not sure if its permissions or something else.

Dixit

4

Just to close the loop on this mostly, I got it finally to appear (all except one file for some reason) by issuing an “occ files:scan --all” command. Now I see all but one file. May try to run it again, but not sure why still one file missing. Even at one point the other day trashed the entire install, installed Nextcloud from scratch again via techandme image and still same issue, same files were missing.

5

Ok something definitely not right, I decided to run the command again, and somehow instead of getting that one file back, we just lost 7 files, so back to square one again somehow. So seems like this cache for SMB or something around it is not staying correctly updated. I noticed another folder with even larger amount of files missing that one has over 450files on actual disk, but only showing about 200 of them. Same thing happened on that folder.

6

Still cant seem to get a handle on why certain files dont show up in folders yet they are physically there. I dont understand why when running the command to scan all files causes it sometimes to pull in the files (yet some are still missing) and then other times to cause massive amount of files missing.

I have the share set to re-scan on access of the share under the actual external file share mount settings.

Any additional troubleshooting help would be appreciated on whats causing this. Im on the latest 13.0.4 and still having the issue

7

Finally getting some more information in the log. Now Im getting this for the one file thats missing when I attempt to refresh this directory:

Exception while scanning file “Recent/test.mkv”: An exception occurred while executing ‘INSERT INTO oc_filecache (mimepart,mimetype,mtime,size,etag,storage_mtime,permissions,parent,checksum,path_hash,path,name,storage) SELECT ?,?,?,?,?,?,?,?,?,?,?,?,? FROM oc_filecache WHERE storage = ? AND path_hash = ? HAVING COUNT(*) = 0’ with params [“5”, “19”, 4039369200, 9388699060, “5b38143a90b15”, 4039369200, 27, 387217, “”, “17fd63d39459b2e7d24f4e6117b594e0”, “Recent/test.mkv”, “test.mkv”, 13, 13, “17fd63d39459b2e7d24f4e6117b594e0”]: SQLSTATE[22003]: Numeric value out of range: 1264 Out of range value for column ‘mtime’ at row 1

So looked at this file and somehow it had a date modified of 12/31/2097. Since I noticed it was complaining about out of range value for ‘mtime’. So I used a tool to change this to the same as date created and still didnt get the file to show, even after refreshing the directory, but the error is now gone, so we making some progress. So seems like the 12/31/2097 date breaks this.

There is also something else causing it attributes wise that doesnt show up in the logs. So I managed to FINALLY fix this one fix. I basically just used MKVToolnix to simply rewrite this file, didnt change anything inside it, just asked it to remux it and deleted this original test.mkv and added the new one and it finally showed.

8

Any idea how that happened. Could be worth reporting this on the bugtracker: Issues · nextcloud/server · GitHub
Such a case should somehow be handled correctly without manual intervention on the database.

Are you already using bigint on this column?

9

Dont believe im using bigint, didnt modify anthing on the DB side as its a stock version of the techandme VMware image for nextcloud.

How it happened, not sure, could be maybe an incorrect time somehow on another machine where we created the file and brought it over to this one.

10

There is a occ-command to convert your tables to bigint. After you have done this, it should not happen again. If it does, please report it to the bug tracker.

11

Sorry for necromancing the older thread. - But i have the same issue.

Some of the files added by various users have wrong timestamps (like year 2097). Those files are simply not displayed then within the webapp.
Its pretty painful to always correct those manually for each file, with thirdparty tools.
And plenty of users are calling us every now and then, because they cannot see some files.

Is there any reasonable way to fix Nextclouds problem with those timestamps?
How about this occ command to change the tables to bigint?
Is that safe to do in a productive system?
Does this fix the problem for good?

Thx in advance,
ND.

12

I’m a new user of Nextcloud. It seems that I have the same issue. I added my network shares (from unRaid) as external storage. However, some folders seem to be 0 B and doesn’t contain any files, even they have GB’s of data in them. Permissions should be okay. Also modify dates seem to be okay. I can’t find any logic why most of my files are okay but some (usually all the files within one folder) are not showing at all.