Configuring S3 Object Storage as Primary Storage: Error

Hi,

I want to use an existing, pre-created and empty AWS S3 Bucket to be integrated as primary storage.

My config.php before finishing initial installation (installation wizard) looks like this:

<?php
$CONFIG = array (
  'htaccess.RewriteBase' => '/',
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'apps_paths' => 
  array (
    0 => 
    array (
      'path' => '/var/www/html/apps',
      'url' => '/apps',
      'writable' => false,
    ),
    1 => 
    array (
      'path' => '/var/www/html/custom_apps',
      'url' => '/custom_apps',
      'writable' => true,
    ),
  ),
  'instanceid' => 'ocaeodq8o7rp',
  'objectstore' => 
  array (
    'class' => '\\OC\\Files\\ObjectStore\\S3',
    'arguments' => 
    array (
      'bucket' => 'nyendo-nextcloud',
      'autocreate' => false,
      'key' => 'XXX',
      'secret' => 'XXX',
      'use_ssl' => true,
    ),
  ),
);

After trying to finish the setup, I get an internal server error. The server logs tell the following:

"message":{"Exception": ... Creation of bucket \"nyendo-nextcloud\" failed. Error executing \"CreateBucket\" on \"https://nyendo-nextcloud.s3.eu-west-1.amazonaws.com/\"; AWS HTTP error: Client error: PUT https:/ /nyendo-nextcloud.s3.eu-west-1.amazonaws.com/ resulted in a 409 Conflict response: ... <Error><Code>BucketAlreadyOwnedByYou</Code><Message> ... BucketAlreadyOwnedByYou (client): Your previous request to create the named bucket succeeded and you already own it. ...

I assumed that the config field 'autocreate' => false, should prevent nextcloud from creating a bucket. Correct?

Still Nextcloud fails to finish the setup because it apparently fails to create the S3 bucket which it is supposed to just use. Did I do something wrong with the configuration?

No ideas? I’m just wondering if this can be considered as a bug …
(Antworten gerne auch auf deutsch)

Did you get any further with this?

I get exactly the same, have tried all kinds of suggestions. Initially I was trying on shared hosting but have since spun it up on docker both locally and on Digital Ocean. Every time the exact same result.

EDIT: Turns out there was an issue with my ACCESS KEYs. Not sure what it was exactly but after reissuing them and updating my config it worked!

Unfortunately, I haven’t tested it further since I’ve setup the application. I deleted my own-created bucket and let Nextcloud create it with the same name. I’m confident that I used the correct access key + secret at that time.

I’m happy to hear that others don’t have that issue. Which version are you using?

I’m using the latest version, have had good consistent luck with it since the above. I’ve found using the nextcloud docker image is a good approach as you can set the S3 details up and then the set-up process creates the config.php, so less chance of getting the file itself wrong.

Probably worth noting in the below example I’ve been using Wasabi for S3 rather than Amazon… this is what I’ve been using both locally and on my digital ocean droplet (should it help anyone)

version: '2.3'

services:
  db:
    image: mariadb
    restart: always
    volumes:
      - db:/var/lib/mysql/data
    environment:
      - MYSQL_DATABASE=nextcloud
      - MYSQL_USER=nextcloud
      - MYSQL_PASSWORD=nextcloud

  app:
    image: nextcloud
    restart: always
    ports:
      - 80:80
    volumes:
      - nextcloud:/var/www/html
    environment:
      - MYSQL_HOST=db
      - MYSQL_DATABASE=nextcloud
      - MYSQL_USER=nextcloud
      - MYSQL_PASSWORD=nextcloud
      - OBJECTSTORE_S3_HOST=s3.eu-central-1.wasabisys.com
      - OBJECTSTORE_S3_BUCKET=nextcloud-test
      - OBJECTSTORE_S3_SECRET=XXXXXXXXXX
      - OBJECTSTORE_S3_KEY=XXXXXXXXXX
      - OBJECTSTORE_S3_REGION=eu-central-1
      - OBJECTSTORE_S3_SSL=true
    depends_on:
      - db

  cron:
    image: nextcloud
    restart: always
    volumes:
      - nextcloud:/var/www/html
    entrypoint: /cron.sh
    depends_on:
      - db


volumes:
  db:
  nextcloud:

Just bare in mind if you use in production, you really need to make sure your nextcloud config.php and MySQL database are backed up, as if you lose either of these, your data isn’t very nice to recover.

When it saves in S3 as primary storage, all the meta data is stored in MySQL (filename, etc.) and then it names the files stored in S3 by database ID (it seems) rather than filename.type… so although the files are all there for you. It would be hideous to recover as you can’t tell what is what.

1 Like

Just looking at your error report again, I notice it has this in there:

Following this:

I wonder if it was just that the S3 service hadn’t finished deleting the bucket? Bucket names must be unique so i suppose it must take some time to remove them from Amazon’s ‘registry’ so to speak. Did you have any files in there? As that certainly would have slowed the progress.

Incidentally I returned to this page as I had a similar issue the other day, where i had foolishly just called the bucket ‘nextcloud-test’. But of course, as bucket names need to be unique across the S3 system, someone else was already using that one. So i guess something else to look out for there.