I have an idea for an extension/app, which could help offload server bandwidth to a CDN which consists of multiple VPS servers, serving not static Nextcloud files, but actual files stored in Nextcloud.
I have a Nextcloud instance at home, but my home internet connection only has a 5Mbps uplink, and accessing anything remotely is a pain. For privacy and data custody reasons, I don’t want to store all my data on someone else’s server, i.e. VPS. This could also work with people with faster connections with a lot of people accessing large files at once.
I just want to know whether this idea has been tried before, and if not, whether there are any reasons it hasn’t.
Possible Implementation
- Master server (home/slow) mirrors all files to a CDN server in the background, slave servers (CDN) syncs amongst themselves.
- Master server prioritizes foreground downloads and pauses mirroring when files are being accessed over WAN.
- Slave servers can use disk or file encryption, to prevent VPS provider from potentially accessing the files.
- When you’re accessing from your “home”, i.e. the same IP as the server, the server does not rewrite the URL (or rewrites it to the local ip), so you can access the file over LAN.
- When you’re accessing remotely:
- Master server first checks if CDN server has the latest copy of the file, if not, treat as cache miss and download (slowly) from master server directly.
- If cache hit, send an authentication token to the CDN server, if it matches your browser, the file is downloaded from the CDN server.
- Possible support for deleting rarely accessed files if CDN storage server is full.
- Possible multiple server support with load balancing and GeoIP based url rewrites.