Distributed file system (SeeweedFS?)

Krischan · April 23, 2020, 12:49am

The recent post about OpenIO ( OpenIO, the ideal storage technology to protect data on a Nextcloud platform, enabling collaboration at scale ) got me thinking about what solutions exist to scale a Nextcloud instance beyond what a single server can easily provide. Besides OpenIO some other distributed file storage options usually come up: MinIO & Cepth.

However on a first glance all these seem quite a bit more advanced and for datacenters (and also partially Open Core) thus I am not sure if they are the right choice for someone wishing to scale up their home server a bit.

But today I came across SeeweedFS, which seems like a really interesting solution:

SeaweedFS is a simple and highly scalable distributed file system, to store and serve billions of files fast! SeaweedFS implements an object store with O(1) disk seek, transparent cloud integration, and an optional Filer supporting POSIX, S3 API, AES256 encryption, Erasure Coding for warm storage, FUSE mount, Hadoop compatible, WebDAV.

Anyone tried it with Nextcloud already? It seems to be possible to use both block-storage or a S3 like object storage. And you could even link via Webdav (no auth yet though).

I also like that it focuses on small files and quick access. The cloud backup options also seem very interesting for a home setup, if you don’t mind using those.

Any other comments or further suggestions?

wscott · January 20, 2024, 10:37am

This post is related:

gist.github.com

https://gist.github.com/egg82/d260c195bda2c9ed9b4835fe0fc0e0a3

seaweedfs_owncloud.md

## A quick write-up on SeaweedFS + OwnCloud
This guide aims to take a look at a distributed, scalable SeaweedFS as a backend storage for an OwnCloud server. Why not NextCloud? Because NextCloud's S3 connector is outdated and not maintained, and won't work with SeaweedFS. Both OwnCloud and NextCloud have the same support and the same plugins. The only concern anyone has is some weird split that happened between the two years ago. It's fine.

This was a giant pain in the ass, but eventually I got it working. This is how.

### Filesystem/Hardware
All servers start from fresh, clean installs of [Ubuntu 18.04.3](https://ubuntu.com/download/server)

This guide will use three seperate boxes (one master/OwnCloud and two slaves).
<br/>I'm assuming there's spare, unformatted disks attached to the slaves. If not, ignore or change the section on mounting to fit your needs.

This file has been truncated. show original

Considering this thread and that gist are from the same time I wonder if they are very related.

system · September 23, 2024, 1:36pm

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.