Configuring S3 Object Storage as Primary Storage: Error

šŸš§ Installation
, ,
1

Hi,

I want to use an existing, pre-created and empty AWS S3 Bucket to be integrated as primary storage.

My config.php before finishing initial installation (installation wizard) looks like this:

<?php
$CONFIG = array (
  'htaccess.RewriteBase' => '/',
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'apps_paths' => 
  array (
    0 => 
    array (
      'path' => '/var/www/html/apps',
      'url' => '/apps',
      'writable' => false,
    ),
    1 => 
    array (
      'path' => '/var/www/html/custom_apps',
      'url' => '/custom_apps',
      'writable' => true,
    ),
  ),
  'instanceid' => 'ocaeodq8o7rp',
  'objectstore' => 
  array (
    'class' => '\\OC\\Files\\ObjectStore\\S3',
    'arguments' => 
    array (
      'bucket' => 'nyendo-nextcloud',
      'autocreate' => false,
      'key' => 'XXX',
      'secret' => 'XXX',
      'use_ssl' => true,
    ),
  ),
);

After trying to finish the setup, I get an internal server error. The server logs tell the following:

"message":{"Exception": ... Creation of bucket \"nyendo-nextcloud\" failed. Error executing \"CreateBucket\" on \"https://nyendo-nextcloud.s3.eu-west-1.amazonaws.com/\"; AWS HTTP error: Client error: PUT https:/ /nyendo-nextcloud.s3.eu-west-1.amazonaws.com/ resulted in a 409 Conflict response: ... <Error><Code>BucketAlreadyOwnedByYou</Code><Message> ... BucketAlreadyOwnedByYou (client): Your previous request to create the named bucket succeeded and you already own it. ...

I assumed that the config field 'autocreate' => false, should prevent nextcloud from creating a bucket. Correct?

Still Nextcloud fails to finish the setup because it apparently fails to create the S3 bucket which it is supposed to just use. Did I do something wrong with the configuration?

2

No ideas? Iā€™m just wondering if this can be considered as a bug ā€¦
(Antworten gerne auch auf deutsch)

3

Did you get any further with this?

I get exactly the same, have tried all kinds of suggestions. Initially I was trying on shared hosting but have since spun it up on docker both locally and on Digital Ocean. Every time the exact same result.

EDIT: Turns out there was an issue with my ACCESS KEYs. Not sure what it was exactly but after reissuing them and updating my config it worked!

4

Unfortunately, I havenā€™t tested it further since Iā€™ve setup the application. I deleted my own-created bucket and let Nextcloud create it with the same name. Iā€™m confident that I used the correct access key + secret at that time.

Iā€™m happy to hear that others donā€™t have that issue. Which version are you using?

5

Iā€™m using the latest version, have had good consistent luck with it since the above. Iā€™ve found using the nextcloud docker image is a good approach as you can set the S3 details up and then the set-up process creates the config.php, so less chance of getting the file itself wrong.

Probably worth noting in the below example Iā€™ve been using Wasabi for S3 rather than Amazonā€¦ this is what Iā€™ve been using both locally and on my digital ocean droplet (should it help anyone)

version: '2.3'

services:
  db:
    image: mariadb
    restart: always
    volumes:
      - db:/var/lib/mysql/data
    environment:
      - MYSQL_DATABASE=nextcloud
      - MYSQL_USER=nextcloud
      - MYSQL_PASSWORD=nextcloud

  app:
    image: nextcloud
    restart: always
    ports:
      - 80:80
    volumes:
      - nextcloud:/var/www/html
    environment:
      - MYSQL_HOST=db
      - MYSQL_DATABASE=nextcloud
      - MYSQL_USER=nextcloud
      - MYSQL_PASSWORD=nextcloud
      - OBJECTSTORE_S3_HOST=s3.eu-central-1.wasabisys.com
      - OBJECTSTORE_S3_BUCKET=nextcloud-test
      - OBJECTSTORE_S3_SECRET=XXXXXXXXXX
      - OBJECTSTORE_S3_KEY=XXXXXXXXXX
      - OBJECTSTORE_S3_REGION=eu-central-1
      - OBJECTSTORE_S3_SSL=true
    depends_on:
      - db

  cron:
    image: nextcloud
    restart: always
    volumes:
      - nextcloud:/var/www/html
    entrypoint: /cron.sh
    depends_on:
      - db


volumes:
  db:
  nextcloud:

Just bare in mind if you use in production, you really need to make sure your nextcloud config.php and MySQL database are backed up, as if you lose either of these, your data isnā€™t very nice to recover.

When it saves in S3 as primary storage, all the meta data is stored in MySQL (filename, etc.) and then it names the files stored in S3 by database ID (it seems) rather than filename.typeā€¦ so although the files are all there for you. It would be hideous to recover as you canā€™t tell what is what.

1 Like
6

Just looking at your error report again, I notice it has this in there:

Following this:

I wonder if it was just that the S3 service hadnā€™t finished deleting the bucket? Bucket names must be unique so i suppose it must take some time to remove them from Amazonā€™s ā€˜registryā€™ so to speak. Did you have any files in there? As that certainly would have slowed the progress.

Incidentally I returned to this page as I had a similar issue the other day, where i had foolishly just called the bucket ā€˜nextcloud-testā€™. But of course, as bucket names need to be unique across the S3 system, someone else was already using that one. So i guess something else to look out for there.

7

Just in case:

I had the same problem. My setup:

  • AWS S3 Bucket (already created)
  • IAM User with S3 Read/Write Access restricted to the bucket
  • => Failed

Apparently the IAM User also needs ā€œListā€ access to check whether the bucket already exists. Strictly this shouldnā€™t be necessary right?

The problem has been fixed by giving the user List/Read/Write access

8

I was able to reproduce and solve the error.

This occurred because nextcloud sets by default region = eu-west-1.
But I created my S3 bucket in eu-central-1, now when nextcloud wants to connect the bucket in this region is not found and it tries to create a new bucket with that name.

The error occurs because a bucket with this name already exists in another region.

Solution: Set correct region in configuration.