Object Name and Folder Structure When Using S3 Storage (IDrive e2) as Primary Storage

[fsyamala]

Nextcloud version : Nextcloud Hub 8 (29.0.4) via Snap
Operating system and version : Ubuntu 24.04
Apache or nginx version : 2.4
PHP version : 8.2

When I connect the IDrive e2 bucket as External Storage in Nextcloud, then the object name, folder name, and structure stored in the bucket are the same as the file name, folder name, and structure in Nextcloud or my computer. But if I connect the IDrive e2 bucket as Primary Storage in Nextcloud, by implementing it as the source below (config.php):
https://docs.nextcloud.com/server/latest/admin_manual/configuration_files/primary_storage.html

'objectstore' => [
     'class' => '\\OC\\Files\\ObjectStore\\S3',
     'arguments' => [
          'bucket' => 'xxxx',
          'hostname' => 'xxxx',
          'key' => 'xxxx',
          'secret' => 'xxxx',
          'port' => 443,
          'use_ssl' => true,
          'use_path_style' => true,
     ],
],

then the object name changes to sequential numbers that do not reflect or are different from the file name or folder name in Nextcloud or my computer. This is messy, and I will have difficulty if I have to download it directly from the IDrive e2 website.

Why does this happen? And is there a way I can do to make the object name regular, when I use IDrive e2 as Primary Storage, just like when I use IDrive e2 as External Storage.

I’ve asked this question to IDrive support either. If I get any useful answers or insights, I will forward them here as well.

Thank you.

[fsyamala]

Using S3 as External Storage. File name and structure being normal. Metadata all good.

Using S3 as Primary Storage. All files name become random object name. Metadata all gone.

[ernolf]

@fsyamala

if I connect the IDrive e2 bucket as Primary Storage in Nextcloud
/…/
then the object name changes to sequential numbers that do not reflect or are different from the file name or folder name in Nextcloud or my computer.

Object storage can be used either as primary storage or as external storage. When it is used as primary storage, the object store only store the file content by using a unique identifier. No metadata are stored in the object store: no files names, no directory structures, etc. Nextcloud assumes exclusive access over the bucket being used.

This has benefits and downsides: performance is better when you use an object store as primary storage, but it restricts being able to access the files from outside of Nextcloud.

(From: Object Store as Primary Storage)

hth.

---

Much and good luck,
ernolf

[fsyamala]

Thanks, ernolf.
I’ve read that article.
But do you think we should have an option to do otherwise (keep metadata while using S3 for Primary Storage)? Because it will help beginner users like me. It will help when I have to backup/download all my files while my computer and my web are not accessible or broken.

[jtr]

@fsyamala

Why does this happen? And is there a way I can do to make the object name regular, when I use IDrive e2 as Primary Storage, just like when I use IDrive e2 as External Storage.

Yes. It’s called External Storage. Basically pick one or the other. The trade-offs are described in the docs: https://docs.nextcloud.com/server/latest/admin_manual/configuration_files/primary_storage.html#performance-implications