Object storage S3 as primary storage - issue

xavierl · December 15, 2019, 6:58pm

[/details]

Nextcloud version _: 17.0.2
Operating system and version _: Ubuntu 18.04
Apache or nginx version
PHP version : 7.3
db: pgsql 11

Issue :

i want to use s3 type object storage as primary storage

i can connect from nextcloud to the bucket as external storage.

but when i configure as primary storage
nextcloud does not work anymore

here is the configuration i have included in the config file

'objectstore' => array(
        'class' => '\\OC\\Files\\ObjectStore\\S3',
        'arguments' => array(
                'bucket' => 'my-nextcloud-storage-bucket',
                'autocreate' => true,
                'key'    => 'ACCESS-KEY',
                'secret' => 'SECRET-KEY',
                'hostname' => 's3.fr-par.scw.cloud',
                'port' => 443,
                'use_ssl' => true,
                'region' => 'fr-par',
        ),
  ),

is there something wrong in  the configuration that prevent nextcloud to work properly ?

Reiner_Nippes · December 15, 2019, 7:33pm

You must configure S3 as primary storage before you create any user.
If your create any user before you change to S3 the user won’t have a home directory anymore more.

xavierl · December 15, 2019, 7:40pm

thank you for the information.
actually there is only the admin user

i have used this tutorial : https://www.scaleway.com/en/docs/install-and-configure-nextcloud-object-storage/

Reiner_Nippes · December 15, 2019, 10:41pm

so your admin user has no home directory now.

just create a new admin user with occ on the cli. and delete the old one.

xavierl · December 16, 2019, 9:59am

i have rewrite the config file again
it is working now.

i should have done something wrong the first time

thank you

Nextcloud_rocks · March 2, 2020, 10:28pm

Hi @xavierl! I’m sort of newbie with Nextcloud admin… been using other file storage providers for some time, but now’s the time for me to set up my own! I really don’t know how to setup the config… Could you please show me how your config.php is correctly setup?
I’d really appreciate it!!

Nextcloud_rocks · March 2, 2020, 10:37pm

@Reiner_Nippes could you help me with this issue?
Anyone else?

Reiner_Nippes · March 3, 2020, 7:26am

did you follow the manual?

https://docs.nextcloud.com/server/18/admin_manual/configuration_files/primary_storage.html

and did you put the setup of object storage into the config.php before you first initialize nextcloud?

Nextcloud_rocks · March 3, 2020, 12:56pm

No, thank you for sharing the link!! Could you please give me an example of an accurately setup config? I really just don’t know where in the config to enter the objectstore line.
(I would love if you’d share yours! ) You could share a link of it here and DM me with it or post it publicly! Thank you very much!!

Reiner_Nippes · March 3, 2020, 1:16pm

i don’t use s3 as storage backend.

if you want to use s3 as backend - as far as i know - you have to start with an “empty” installation. there is no migration of any data for existing user.

you may have a look at the template from my playbook.

github.com

ReinerNippes/nextcloud/blob/nextcloud-reloaded/roles/nextcloud/templates/objectstore_s3_config.php.j2

<?php
$CONFIG = array (
  'objectstore' => array(
          'class' => '\\OC\\Files\\ObjectStore\\S3',
          'arguments' => array(
                  'bucket' => '{{ objectstore_s3_bucket_name | default('nextcloud-'+nextcloud_fqdn) }}',
                  'autocreate' => true,
                  'key'    => '{{ objectstore_s3_key }}',
                  'secret' => '{{ objectstore_s3_secret }}',
                  'hostname' => '{{ objectstore_s3_hostname | default('s3.amazonaws.com') }}',
                  'port' => '{{ objectstore_s3_port | default('') }}',
                  'use_ssl' => {{ objectstore_s3_use_ssl | default('true') }},
                  'region' => '{{ objectstore_s3_region | default('us-east-1') }}',
                  // required for some non Amazon S3 implementations
                  'use_path_style' => {{ objectstore_s3_use_path_style | default('true') }}
          ),
  ),
);

you simply replace the vaiables {{ }} (e.g. {{ objectstore_s3_key }}) with your values and put it in nextcloud/config/config.php after you untar nextcloud and before you first login into nextcloud.

or you run the playbook on a new installed ubuntu machine. with correct values in the inventory you’ll get a working config.

github.com

ReinerNippes/nextcloud/blob/31bbb25dfb49fe969bce769bf4a5beae5556abb0/inventory#L71




# webserver type nginx or apache
nextcloud_webserver_type    = nginx


# php version
php_version                 = '7.3'


# optional php packages see: roles/php/var/main.yml
php_install_optional_packages = false


# Use S3 Bucket as primary storage
objectstore_s3_key            = ''
objectstore_s3_secret         = ''
# objectstore_s3_bucket_name    = ''
# objectstore_s3_hostname       = 's3.amazonaws.com'
# objectstore_s3_port           = '443'
# objectstore_s3_use_ssl        = 'true'
# objectstore_s3_region         = 'us-east-1'
# objectstore_s3_use_path_style = 'true'


# Install turn server for Nextcloud Talk

Nextcloud_rocks · March 4, 2020, 11:45am

Thank you very much. I really appreciate this!!!

Could this possibly work on the one click Nextcloud app from Vultr, or should I rather start with a LAMP stack and install that way? Thank you again!

Reiner_Nippes · March 5, 2020, 3:15pm

if you mean that i could integrated s3 as primary storage in the vultr on click nextcloud. no. because vultr is using a kind af of script like my playbook. but i have nor access to it.

if you mean if i could think about a way to use the vultr one click nextcloud machine. yes. if you have shell access you insert the s3 storage php config into the existing config. after that you have to create a new admin user with the occ command. and delete the old one. might work. not tested.

Nextcloud_rocks · March 7, 2020, 4:48pm

Thank you very much!! You have been extremely helpful.
I’ll have a shot at it today!

I’m using your playbook… Brilliantly designed. Thank you!!

Nextcloud_rocks · March 25, 2020, 12:08am

HI! This may be a stupid question…
I just used your playbook… It’s really powerfully built, thanks for all the work.
I’m wondering how to login as root for the first time? Where do I find the password and username?
I’m trying to use OCC, but it returns
Could not open input file: /var/www/nextcloud/occ

Please give me advice!
Thank you!

Reiner_Nippes · March 25, 2020, 1:01pm

which one? there are three (+ one new branch).

normally the last message of the playbook looks like:

ok: [localhost] => {
    "msg": [
        "Your Nextcloud at https://nextcloud.example.tld is ready.",
        "Login with user: admin and password: fTkLgvPYdmjfalP8XgMsEg7plnoPsTvp ",
        "Other secrets you'll find in the directory /opt/nextcloud/secrets "
    ]
}

the path /opt/nextcloud/secrets is definded in a variable in group_vars/all.yml named nextcloud_credentials_store or credential_store.

e.g.

github.com

ReinerNippes/nextcloud_on_docker/blob/e61b42653f832c522203ff46102fcdbb32be131e/group_vars/all.yml#L4


      
          ---
          # Some common variables used in more then one roles
           
          nextcloud_credential_store:    "{{ nextcloud_base_dir }}/secrets"
          nextcloud_data_dir:            "{{ nextcloud_base_dir }}/data"
          nextcloud_www_dir:             "{{ nextcloud_base_dir }}/www"
          nextcloud_config_dir:          "{{ nextcloud_base_dir }}/config"
          nextcloud_database_dump_dir:   "{{ nextcloud_base_dir }}/database-dump"
          
          
          traefik_config_dir:            "{{ nextcloud_base_dir }}/traefik"
          
          portainer_data_dir:            "portainer-data-vol"

the name of the default user of nextcloud is defined in the inventory:

https://github.com/ReinerNippes/nextcloud/blob/753f060a710afe0bfbe71f43966af6d0507fea0d/inventory#L32

Nextcloud_rocks · March 25, 2020, 3:32pm

I don’t know which one I was using! But I just opened a new server and ran the Docker one.

I could log in perfectly! Thank you.

However, I can’t see the normal admin settings! When I log in, I can’t see the settings tab like normal. What is the reason for this / how can I make the button appear?
Thank you!

Reiner_Nippes · March 25, 2020, 5:34pm

could you make a screenshot?

Nextcloud_rocks · March 25, 2020, 5:53pm

Certainly!

Notice that there’s no avatar in the upper right corner like there normally is.

EDIT: I was wrong… There’s actually an avatar. However, it’s hidden! I had to hover the arrow over that spot. I’m sorry…

I can’t tell if my bucket was sucessfully integrated, how can I check it?
Here is another screenshot (from the settings I did eventually successfully access…) It looks like my s3 bucket is not connected. How do I check?

Reiner_Nippes · March 25, 2020, 6:07pm

first you can login and you see the default files. so everything should be ok. if i remember well from my testing, if the s3 backend is not created you can’t login.

go to your s3 bucket and check if there are objects. go to /opt/nextcloud/data - if you didn’t change my default - and check if it’s empty. at least there should be not admin folder.

the avatar is also empty for me during first login. if you create a user or relogin you should get an avatar.

Nextcloud_rocks · March 25, 2020, 8:32pm

Ok, thank you so much!! I think everything’s ok. I really appreciate your help! It is as you said.