Presently, S3 storage is offered as a backend for auxiliary storage.
Some users have requested support for S3 as a primary storage system. To date, no plans have been given to adopt such support.
Nevertheless, the request is germane.
S3 storage offers various advantages over storage as files on the local file system, in terms of scalability and reliability. As use cases are evolving, such advantages are becoming more manifest.
S3 is based on data and access models that support site replication and migration much more naturally and robustly than file systems. Of particular relevance is the flexibility to separate application server deployments from the backend file storage.
For example, through such a model, migration to new a application hosting solution is possible without disruption of file storage or transfer or file content.
As upward scalability of deployments is one consideration, a further development is the downward scalability of S3 storage solutions, many light enough now to operate on standalone appliances such as ones that have historically been targeted for Nextcloud.
One serious limitation is that at present, the S3 storage system relies on the local relational database as a single point of failure for representing the file tree, including file names and their mappings to payload objects in the S3 bucket.
Thus, the resilience of S3 storage for file payloads is offset by any vulnerability in the application and especially database deployment. Database backup is currently essential for disaster recovery, but still extremely clumsy as a solution for extracting file trees or file contents directly from S3 storage.
However, such vulnerabilities may be largely mitigated by certain enhancements that require no redesign of the overall system. An S3 bucket may easily store data objects that include at least recent if not fully current representations of the file tree data. By replicating such a representation, as in a serialized form, completely or incrementally, at periodic intervals, in the S3 backend, the application easily may preserve all data required for reconstruction of the entire tree of stored files. If the object data is structured transparently, then recovery by standalone utilities is feasible, in case administrators need to extract file contents without reconstituting a full deployment of the application. An object referenced by a single prescribed key may serve as a manifest for the contents of the entire bucket.
The solution opens opportunities for further enhancements, such as complete replication of all site data into an S3 store for purposes of disaster recovery, as well as seamless site migration.
Discussions of general and long-term improvements submitted to the issue tracker are often closed. Hopefully such a discussion will be well received in the support forum.
