Purging OC_STORAGES and OC_FILECACHE

Nextcloud version (eg, 18.0.2): 15.07
Operating system and version (eg, Ubuntu 20.04): 16.04? 16.04.6 LTS
Apache or nginx version (eg, Apache 2.4.25): 2.4.18
PHP version (eg, 7.1): ??

The issue you are facing:

I am running into the issue of OC_FILECACHE and OC_STORAGES records not being deleted when you remove an External Storage from the system through the UI.

Because we had some very large shares (which we are not using anymore), OC_FILECACHE has grown to be excessively large and has caused disk space problems.

My question is – can I just delete the records for the deprecated External storages directly in MySQL? This seems like a couple of trivial delete statements (one for each table, records corresponding to the same storage ID). Or will this corrupt the system somehow?

Please advise.

Thanks. John

1 Like

I followed number of post here and on github, unfortunately I have no references by hand. There is a known bug/misconception in nextcloud which turns into issue

  • storages, most often related to external storage used by multiple users is indexed multiple times
  • storages removed from NC config remain in DB (check oc_storages DB table)
  • records in oc_filecache table remain there (at least until corresponding storage is removed/or forever)

there was an issue on github discussing this more in depth and it looks there is an improvement now filecache records removed by occ files:scan --all/files:cleanup

at least for me running NC 19.0.8 following procedure helped:

  • enable external_files app (was removed earlier due to performance issues)
  • remove all configured storages
  • some external storages are still listed in oc_storages table
  • review this records
    select * from oc_storages where id like 'smb:%'
  • review this records
    delete from oc_storages where id like 'smb:%'
  • run occ files:cleanup

as a result it removed 160k stale records from file cache.

  • running ALTER TABLE oc_filecache FORCE;
    returned free space as described here

hopefully this helps!

So I’m following the migration guide which at some point speaks of replacing the storage entry for the file location. Makes sense, however looking at my table, I see a bunch of confusing entries. Just seconds before dumping the database, I thought i’d add a dedicated admin user, just because (my own user was the admin user up until now, and I probably want to not do this in the future).

Anyhow, looking at the table, I see:

 home::user1                                                   |          1 |         1 |             
 home::user2                                                   |          2 |         1 |             
 home::user3                                                   |          3 |         1 |             
 local::/opt/some/local/path/                                  |          4 |         1 |             
 home::user4                                                   |          5 |         1 |             
 home::user5                                                   |          6 |         1 |             
 local::/var/www/owncloud.domain.nl/ htdocs/data/              |          7 |         1 |             
 google::someoldgooglemount                                    |          8 |         1 |             
 shared::/photos (2)                                           |          9 |         1 |             
 shared::/documents (2)                                        |         10 |         1 |             
 shared::/blabla.odt                                           |         11 |         1 |             
 home::testuser                                                |         12 |         1 |             
 shared::/IMG_20160424_162231.jpg                              |         13 |         1 |             
 home::user6                                                   |         14 |         1 |             
 home::user7                                                   |         15 |         1 |             
 shared::/Huwelijk                                             |         16 |         1 |             
 local::/var/www/owncloud.domain.nl/cache/user1/               |         17 |         1 |             
 local::/var/www/owncloud.domain.nl/cache/user2/               |         18 |         1 |             
 local::/var/www/owncloud.domain.nl/cache/user3/               |         19 |         1 |             
 local::/var/www/owncloud.domain.nl/cache/user4/               |         20 |         1 |             
 local::/var/www/owncloud.domain.nl/cache/user5/               |         21 |         1 |             
 local::/var/www/owncloud.domain.nl/cache/user6/               |         22 |         1 |             
 local::/var/www/owncloud.domain.nl/cache/user7/               |         23 |         1 |             
 local::/var/www/owncloud.domain.nl/cache/testuser/            |         24 |         1 |             
 shared::/gebouw_brief.odt                                     |         25 |         1 |             
 local::/var/www/nextcloud.domain.nl/htdocs/data/              |         26 |         1 |             
 local::/var/www/nextcloud.domain.nl/cache/user1/              |         27 |         1 |             
 local::/var/www/nextcloud.domain.nl/htdocs/cache/user2/       |         28 |         1 |             
 local::/var/www/nextcloud.domain.nl/htdocs/cache/user3/       |         29 |         1 |             
 local::/var/www/nextcloud.domain.nl/htdocs/cache/user4/       |         30 |         1 |             
 b6b3c73b0f407870fb9b1325804bf076                              |         31 |         1 |             
 2cb9001c9d47ee0c335caafdeec00a41                              |         32 |         1 |             
 fe65301173af99ddcc2df3748f547f18                              |         33 |         1 |             
 local::/var/www/nextcloud.domain.nl/htdocs/cache/user1/       |         34 |         1 |             
 fec9905a238118f16700b92a88cc9ee2                              |         35 |         1 |             
 local::/var/www/nextcloud.domain.nl/htdocs/cache/user2/       |         36 |         1 |             
 84ffae455c8dbfabac6e3226c38780f3                              |         37 |         1 |             
 local::/var/www/nextcloud.domain.nl/htdocs/cache/user3/       |         38 |         1 |             
 4ee49a93bf5b74c2a198a2e0bf8822a0                              |         39 |         1 |             
 local::/var/www/nextcloud.domain.nl/htdocs/cache/user4/       |         40 |         1 |             
 065bae8c933042eb1a47c06885fa1baa                              |         41 |         1 |             
 local::/var/www/nextcloud.domain.nl/htdocs/cache/testuser/    |         42 |         1 |             
 1ea41684bef21c3646937296583a5d62                              |         43 |         1 |             
 local::/var/www/nextcloud.domain.nl/htdocs/cache/user5/       |         44 |         1 |             
 a175c5257a0ddd50b5aafe878559c12b                              |         45 |         1 |             
 home::user5                                                   |         46 |         1 |             
 home::admin                                                   |         47 |         1 |             
 local::/var/www/nextcloud.domain.nl/htdocs/cache/admin/       |         48 |         1 |             
 23d4809d24c83c2114861cb735264369         

The first issue I have, the directory ‘cache’ doesn’t exist in the new server. Would nextcloud create it itself? I do also see data/user1/cache on the data volume.

Anyway, the cache is easy to resolve, and caches should be recreated. The UID created just after the caches worry me a little bit, what about that?

Further more, its clear I migrated this database from the owncloud to the nextcloud days eons ago. Can I just delete these records? Renaming the variables as per migration guide, would result in duplicates that are not allowed. Are the numeric_id’s referenced anywhere at all? or are they just serials. the table type seems to suggest so. But removing records may have an impact on this.

Before the import, obviously when cleaning up this table, the sequence must be re-adjusted as well; much aware of that :stuck_out_tongue:

Finally, a little off-topic; but while searching for oc_storages in the dump file, I noticed a whole lot of weird duplicates. E.g. oc_oc_storages_randomstring_numeric_id_seq which seems a little bit like a left over from a data migration script? E.g. copy a table, upgrade table, forget to remove copy?

Anyway, sorry for necroling this old thread, but it seems to be still quite accurate, and maybe I just go unlucky with a really old database :stuck_out_tongue: