Initial backup fails: /nextcloud_aio_volumes/nextcloud_aio_nextcloud_data/images/107/vm-107-disk-0.qcow2: file changed while we backed it up

The Basics

  • Nextcloud Server version (e.g., 29.x.x):
    • AIO 12.3.0
  • Operating system and version (e.g., Ubuntu 24.04):
    • Debian GNU/Linux 12 (bookworm)
  • Web server and version (e.g, Apache 2.4.25):
    • Apache/2.4.66
  • Reverse proxy and version _(e.g. nginx 1.27.2)
    • Caddy v2.10.2
  • PHP version (e.g, 8.3):
    • 8.3.28
  • Is this the first time you’ve seen this error? (Yes / No):
    • Initial setup of backup, tried 3x
  • When did this problem seem to first start?
    • First (and subsequent) time
  • Installation method (e.g. AlO, NCP, Bare Metal/Archive, etc.)
    • AIO
  • Are you using CloudfIare, mod_security, or similar? (Yes / No)
    • Cloudflare

Summary of the issue you are facing:

Trying to set up backup via AIO. Using remote borg server over ssh. Backup seems to run successfully but when it completes the log shows “Deleting the failed backup archive…”. The only error I see in the log is:

2025-12-20T11:04:58.238865000Z /nextcloud_aio_volumes/nextcloud_aio_nextcloud_data/images/107/vm-107-disk-0.qcow2: file changed while we backed it up

Steps to replicate it (hint: details matter!):

  1. Nextcloud AIO running on Proxmox 9.1.2.

  2. Borg 1.4.0 running on Debian GNU/Linux 13 (trixie)

  3. Set the remote borg repo to ssh://root@borgserver.XXX/mnt/nextcloud-backup

  4. Create backup

  5. Runs for 6 hours or so and reports backup failed

Log entries

Nextcloud

Please provide the log entries from your Nextcloud log that are generated during the time of problem (via the Copy raw option from Administration settings->Logging screen or from your nextcloud.log located in your data directory). Feel free to use a pastebin/gist service if necessary.

2025-12-20T11:04:48.726618000Z 621.39 GB O 539.16 GB C 476.41 GB D 293157 N nextcloud_aio_volumes/nextcloud_aio_nextcloud_data/images/107/vm-107-disk-0.qcow2
2025-12-20T11:04:49.016087000Z 621.41 GB O 539.16 GB C 476.41 GB D 293157 N nextcloud_aio_volumes/nextcloud_aio_nextcloud_data/images/107/vm-107-disk-0.qcow2
2025-12-20T11:04:49.217368000Z 621.43 GB O 539.16 GB C 476.41 GB D 293157 N nextcloud_aio_volumes/nextcloud_aio_nextcloud_data/images/107/vm-107-disk-0.qcow2
2025-12-20T11:04:49.508269000Z 621.46 GB O 539.16 GB C 476.41 GB D 293157 N nextcloud_aio_volumes/nextcloud_aio_nextcloud_data/images/107/vm-107-disk-0.qcow2
2025-12-20T11:04:58.235512000Z Saving files cache
hunks cache
ache config
zing cache transaction: Reading config
zing cache transaction: Reading chunks
zing cache transaction: Reading files
B O 539.16 GB C 476.41 GB D 293158 N nextcloud_aio_volumes/nextcloud_aio_nextcloud_data/images/107/vm-107-disk-0.qcow2
2025-12-20T11:04:58.238865000Z /nextcloud_aio_volumes/nextcloud_aio_nextcloud_data/images/107/vm-107-disk-0.qcow2: file changed while we backed it up

... backup continues with remaining files, then completes with ...

2025-12-20T11:06:05.097735000Z Repository: ssh://root@borgserver.XXX/mnt/nextcloud-backup
2025-12-20T11:06:05.098103000Z Archive name: 20251220_045805-nextcloud-aio
2025-12-20T11:06:05.098204000Z Archive fingerprint: xxx
2025-12-20T11:06:05.098286000Z Time (start): Sat, 2025-12-20 04:58:06
2025-12-20T11:06:05.098390000Z Time (end):   Sat, 2025-12-20 11:05:54
2025-12-20T11:06:05.098466000Z Duration: 6 hours 7 minutes 48.10 seconds
2025-12-20T11:06:05.098540000Z Number of files: 332365
2025-12-20T11:06:05.098624000Z Utilization of max. archive size: 0%
2025-12-20T11:06:05.098696000Z ------------------------------------------------------------------------------
2025-12-20T11:06:05.099053000Z                        Original size      Compressed size    Deduplicated size
2025-12-20T11:06:05.099146000Z This archive:              623.90 GB            539.85 GB            476.81 GB
2025-12-20T11:06:05.099232000Z All archives:              623.90 GB            539.85 GB            476.84 GB
2025-12-20T11:06:05.099315000Z 
2025-12-20T11:06:05.099382000Z                        Unique chunks         Total chunks
2025-12-20T11:06:05.099452000Z Chunk index:                  305665               475752
2025-12-20T11:06:05.099536000Z ------------------------------------------------------------------------------
2025-12-20T11:06:05.267666000Z Deleting the failed backup archive...
2025-12-20T11:27:24.465473000Z ------------------------------------------------------------------------------
2025-12-20T11:27:24.466949000Z                        Original size      Compressed size    Deduplicated size
2025-12-20T11:27:24.467038000Z Deleted data:             -624.08 GB           -539.88 GB           -476.84 GB
2025-12-20T11:27:24.467118000Z All archives:                    0 B                  0 B                  0 B
2025-12-20T11:27:24.467206000Z 
2025-12-20T11:27:24.467287000Z                        Unique chunks         Total chunks
2025-12-20T11:27:24.467339000Z Chunk index:                       0                    0
2025-12-20T11:27:24.467393000Z ------------------------------------------------------------------------------
2025-12-20T11:27:24.632744000Z Backup failed!
2025-12-20T11:27:24.633112000Z You might want to check the backup integrity via the AIO interface.
2025-12-20T11:27:24.633189000Z Deleting borg.config file so that you can choose a different location for the backup.

Configuration

Nextcloud

The output of occ config:list system or similar is best, but, if not possible, the contents of your config.php file from /path/to/nextcloud is fine (make sure to remove any identifiable information!):

{
    "system": {
        "one-click-instance": true,
        "one-click-instance.user-limit": 100,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "check_data_directory_permissions": false,
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "password": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "overwritehost": "nextcloud.XXX",
        "overwriteprotocol": "https",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "localhost",
            "nextcloud.XXX"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "pgsql",
        "version": "32.0.3.2",
        "overwrite.cli.url": "https:\/\/nextcloud.XXX\/",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "maintenance": false,
        "updatechecker": false,
        "loglevel": 2,
        "log_type": "file",
        "logfile": "\/var\/www\/html\/data\/nextcloud.log",
        "log_rotate_size": 10485760,
        "log.condition": {
            "apps": [
                "admin_audit"
            ]
        },
        "preview_max_x": 2048,
        "preview_max_y": 2048,
        "jpeg_quality": 60,
        "enabledPreviewProviders": {
            "1": "OC\\Preview\\Image",
            "2": "OC\\Preview\\MarkDown",
            "3": "OC\\Preview\\MP3",
            "4": "OC\\Preview\\TXT",
            "5": "OC\\Preview\\OpenDocument",
            "6": "OC\\Preview\\Movie",
            "7": "OC\\Preview\\Krita"
        },
        "enable_previews": true,
        "upgrade.disable-web": true,
        "mail_smtpmode": "smtp",
        "trashbin_retention_obligation": "auto, 30",
        "versions_retention_obligation": "auto, 30",
        "activity_expire_days": 30,
        "simpleSignUpLink.shown": false,
        "share_folder": "\/Shared",
        "one-click-instance.link": "https:\/\/nextcloud.com\/all-in-one\/",
        "upgrade.cli-upgrade-link": "https:\/\/github.com\/nextcloud\/all-in-one\/discussions\/2726",
        "updatedirectory": "\/nc-updater",
        "maintenance_window_start": 100,
        "allow_local_remote_servers": true,
        "davstorage.request_timeout": 3600,
        "documentation_url.server_logs": "https:\/\/github.com\/nextcloud\/all-in-one\/discussions\/5425",
        "htaccess.RewriteBase": "\/",
        "dbpersistent": false,
        "auth.bruteforce.protection.enabled": true,
        "ratelimit.protection.enabled": true,
        "files_external_allow_create_new_local": true,
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "app_install_overwrite": [
            "maps"
        ],
        "onlyoffice": {
            "jwt_secret": "***REMOVED SENSITIVE VALUE***",
            "jwt_header": "AuthorizationJwt"
        },
        "DOMAIN": "nextcloud.XXX"
    },
    "apps": {
        "activity": {
            "enabled": "yes",
            "installed_version": "5.0.0-dev.0",
            "types": "filesystem"
        },
        "admin_audit": {
            "logfile": "\/var\/www\/html\/data\/audit.log"
        },
        "app_api": {
            "enabled": "no",
            "installed_version": "32.0.0",
            "types": ""
        },
        "backgroundjob": {
            "lastjob": "274954"
        },
        "bruteforcesettings": {
            "enabled": "yes",
            "installed_version": "5.0.0-dev.0",
            "types": ""
        },
        "calendar": {
            "installed_version": "6.1.2",
            "types": "",
            "enabled": "yes"
        },
        "circles": {
            "enabled": "yes",
            "installed_version": "32.0.0",
            "loopback_tmp_path": "\/",
            "loopback_tmp_scheme": "https",
            "maintenance_update": "{\"maximum\":3,\"3\":1766002205,\"2\":1766002806,\"1\":1766002806,\"4\":1667440801,\"5\":1667012400}",
            "migration_22": "1",
            "migration_22_1": "1",
            "types": "filesystem,dav",
            "maintenance_run": "1766241650",
            "migration_run": "0"
        },
        "cloud_federation_api": {
            "enabled": "yes",
            "installed_version": "1.16.0",
            "types": "filesystem"
        },
        "comments": {
            "enabled": "yes",
            "installed_version": "1.22.0",
            "types": "logging"
        },
        "contacts": {
            "installed_version": "8.1.2",
            "types": "dav",
            "enabled": "yes"
        },
        "contactsinteraction": {
            "enabled": "yes",
            "installed_version": "1.13.1",
            "types": "dav"
        },
        "core": {
            "backgroundjobs_mode": "cron",
            "emailTestSuccessful": "1",
            "files_metadata_installed": "1",
            "installedat": "1643649273.4845",
            "lastupdateResult": "{\"version\":\"32.0.3.2\",\"versionstring\":\"Nextcloud 32.0.3\",\"url\":\"https:\\\/\\\/download.nextcloud.com\\\/server\\\/releases\\\/nextcloud-32.0.3.zip\",\"web\":\"https:\\\/\\\/docs.nextcloud.com\\\/server\\\/32\\\/admin_manual\\\/maintenance\\\/upgrade.html\",\"changes\":\"https:\\\/\\\/updates.nextcloud.com\\\/changelog_server\\\/?version=32.0.3\",\"autoupdater\":\"1\",\"eol\":\"0\"}",
            "lastupdatedat": 1765950171,
            "metadataGenerationDone": true,
            "moveavatarsdone": "yes",
            "previewsCleanedUp": "1",
            "public_files": "files_sharing\/public.php",
            "public_webdav": "dav\/appinfo\/v1\/publicwebdav.php",
            "theming.variables": "b92d206521717ac032f8aa58d3c7ff2f",
            "vendor": "nextcloud",
            "lastcron": 1766243450,
            "files_metadata": {
                "photos-original_date_time": {
                    "value": null,
                    "type": "int",
                    "etag": "",
                    "indexed": true,
                    "editPermission": 0
                },
                "photos-exif": {
                    "value": null,
                    "type": "array",
                    "etag": "",
                    "indexed": false,
                    "editPermission": 0
                },
                "photos-ifd0": {
                    "value": null,
                    "type": "array",
                    "etag": "",
                    "indexed": false,
                    "editPermission": 0
                },
                "photos-size": {
                    "value": null,
                    "type": "array",
                    "etag": "",
                    "indexed": false,
                    "editPermission": 0
                },
                "photos-gps": {
                    "value": null,
                    "type": "array",
                    "etag": "",
                    "indexed": false,
                    "editPermission": 0
                },
                "photos-place": {
                    "value": null,
                    "type": "string",
                    "etag": "",
                    "indexed": true,
                    "editPermission": 0
                },
                "files-live-photo": {
                    "value": null,
                    "type": "string",
                    "etag": "",
                    "indexed": false,
                    "editPermission": 2
                },
                "blurhash": {
                    "value": null,
                    "type": "string",
                    "etag": "4da8fa36618ec3d3092363f4b5660f89",
                    "indexed": false,
                    "editPermission": 0
                }
            },
            "oc.integritycheck.checker": []
        },
        "dashboard": {
            "enabled": "yes",
            "installed_version": "7.12.0",
            "types": ""
        },
        "dav": {
            "buildCalendarReminderIndex": "yes",
            "buildCalendarSearchIndex": "yes",
            "builtSocialSearchIndex": "yes",
            "chunks_migrated": "1",
            "enabled": "yes",
            "installed_version": "1.34.2",
            "regeneratedBirthdayCalendarsForYearFix": "yes",
            "types": "filesystem"
        },
        "deck": {
            "installed_version": "1.16.2",
            "types": "dav",
            "enabled": "no"
        },
        "federatedfilesharing": {
            "enabled": "yes",
            "installed_version": "1.22.0",
            "types": ""
        },
        "federation": {
            "enabled": "yes",
            "installed_version": "1.22.0",
            "types": "authentication"
        },
        "files": {
            "enabled": "yes",
            "installed_version": "2.4.0",
            "mimetype_version": "32.0.1.2",
            "types": "filesystem"
        },
        "files_antivirus": {
            "av_path": "\/usr\/bin\/clamscan",
            "installed_version": "6.1.0",
            "types": "filesystem,dav",
            "enabled": "yes",
            "av_mode": "daemon",
            "av_port": "3310",
            "av_host": "nextcloud-aio-clamav",
            "av_stream_max_length": "2147483648",
            "av_max_file_size": "-1",
            "av_infected_action": "only_log"
        },
        "files_downloadlimit": {
            "enabled": "yes",
            "installed_version": "5.0.0-dev.0",
            "types": ""
        },
        "files_external": {
            "enabled": "yes",
            "installed_version": "1.24.0",
            "types": "filesystem"
        },
        "files_pdfviewer": {
            "enabled": "yes",
            "installed_version": "5.0.0-dev.0",
            "types": ""
        },
        "files_reminders": {
            "enabled": "yes",
            "installed_version": "1.5.0",
            "types": ""
        },
        "files_rightclick": {
            "enabled": "no",
            "installed_version": "1.6.0",
            "types": ""
        },
        "files_sharing": {
            "enabled": "yes",
            "installed_version": "1.24.1",
            "types": "filesystem"
        },
        "files_trashbin": {
            "enabled": "yes",
            "installed_version": "1.22.0",
            "types": "filesystem,dav",
            "background_job_expire_trash_offset": 0
        },
        "files_versions": {
            "enabled": "yes",
            "installed_version": "1.25.0",
            "types": "filesystem,dav"
        },
        "firstrunwizard": {
            "enabled": "yes",
            "installed_version": "5.0.0-dev.0",
            "types": ""
        },
        "groupfolders": {
            "installed_version": "20.1.6",
            "types": "filesystem,dav",
            "enabled": "yes",
            "cron_last_folder_index": 1
        },
        "logreader": {
            "enabled": "yes",
            "installed_version": "5.0.0-dev.0",
            "shownLevels": "[2,3,4]",
            "types": "logging"
        },
        "lookup_server_connector": {
            "enabled": "yes",
            "installed_version": "1.20.0",
            "types": "authentication"
        },
        "mail": {
            "installed_version": "5.6.5",
            "types": "",
            "enabled": "yes"
        },
        "maps": {
            "installed_version": "1.6.0",
            "types": "filesystem",
            "enabled": "no"
        },
        "nextcloud-aio": {
            "installed_version": "0.8.0",
            "types": "",
            "enabled": "yes"
        },
        "nextcloud_announcements": {
            "enabled": "yes",
            "installed_version": "4.0.0-dev.0",
            "pub_date": "Thu, 24 Oct 2019 00:00:00 +0200",
            "types": "logging"
        },
        "notes": {
            "installed_version": "4.12.4",
            "types": "",
            "enabled": "yes"
        },
        "notifications": {
            "enabled": "yes",
            "installed_version": "5.0.0-dev.0",
            "types": "logging"
        },
        "notify_push": {
            "installed_version": "1.2.1",
            "types": "filesystem",
            "base_endpoint": "https:\/\/nextcloud.XXX\/push",
            "enabled": "yes"
        },
        "oauth2": {
            "enabled": "yes",
            "installed_version": "1.20.0",
            "types": "authentication"
        },
        "onlyoffice": {
            "verify_peer_off": "false",
            "DocumentServerInternalUrl": "",
            "jwt_header": "AuthorizationJwt",
            "settings_error": "",
            "StorageUrl": "",
            "installed_version": "9.11.0",
            "types": "prevent_group_restriction",
            "enabled": "yes",
            "jwt_secret": "***REMOVED SENSITIVE VALUE***",
            "DocumentServerUrl": "https:\/\/nextcloud.XXX\/onlyoffice",
            "demo": "{\"available\":true,\"enabled\":false,\"start\":{\"date\":\"2025-12-19 04:49:00.273352\",\"timezone_type\":3,\"timezone\":\"UTC\"}}"
        },
        "password_policy": {
            "enabled": "yes",
            "installed_version": "4.0.0-dev.0",
            "types": "authentication"
        },
        "phonetrack": {
            "installed_version": "0.9.1",
            "types": "",
            "enabled": "yes"
        },
        "photos": {
            "enabled": "yes",
            "installed_version": "5.0.0-dev.1",
            "lastPlaceMappedUser": "ncadmin",
            "lastPlaceMappingDone": "true",
            "types": "dav,authentication"
        },
        "privacy": {
            "enabled": "yes",
            "installed_version": "4.0.0-dev.0",
            "types": ""
        },
        "profile": {
            "enabled": "yes",
            "installed_version": "1.1.0",
            "types": ""
        },
        "provisioning_api": {
            "enabled": "yes",
            "installed_version": "1.22.0",
            "types": "prevent_group_restriction"
        },
        "recommendations": {
            "enabled": "yes",
            "installed_version": "5.0.0-dev.0",
            "types": ""
        },
        "related_resources": {
            "enabled": "yes",
            "installed_version": "3.0.0-dev.0",
            "types": ""
        },
        "richdocuments": {
            "enabled": "no",
            "installed_version": "7.1.1",
            "types": "prevent_group_restriction"
        },
        "serverinfo": {
            "enabled": "yes",
            "installed_version": "4.0.0-dev.0",
            "types": "",
            "size_appdata_storage": 4158592,
            "cached_count_storages": 10,
            "cached_count_appdata_files": 1163,
            "cached_count_filecache": 442492
        },
        "settings": {
            "enabled": "yes",
            "installed_version": "1.15.1",
            "types": ""
        },
        "sharebymail": {
            "enabled": "yes",
            "installed_version": "1.22.0",
            "types": "filesystem"
        },
        "spreed": {
            "enabled": "no",
            "has_reference_id": "yes",
            "installed_version": "22.0.7",
            "project_access_invalidated": "1",
            "signaling_token_privkey_es256": "***REMOVED SENSITIVE VALUE***",
            "signaling_token_pubkey_es256": "***REMOVED SENSITIVE VALUE***",
            "types": "dav,prevent_group_restriction",
            "stun_servers": "***REMOVED SENSITIVE VALUE***"
        },
        "support": {
            "SwitchUpdaterServerHasRun": "yes",
            "enabled": "yes",
            "installed_version": "4.0.0-dev.0",
            "types": "session"
        },
        "survey_client": {
            "enabled": "yes",
            "installed_version": "4.0.0-dev.0",
            "types": ""
        },
        "systemtags": {
            "enabled": "yes",
            "installed_version": "1.22.0",
            "types": "logging"
        },
        "tasks": {
            "installed_version": "0.17.1",
            "types": "",
            "enabled": "yes"
        },
        "text": {
            "enabled": "yes",
            "installed_version": "6.0.1",
            "types": "dav"
        },
        "theming": {
            "enabled": "yes",
            "installed_version": "2.7.0",
            "types": "logging"
        },
        "twofactor_backupcodes": {
            "enabled": "yes",
            "installed_version": "1.21.0",
            "types": ""
        },
        "twofactor_totp": {
            "enabled": "yes",
            "installed_version": "14.0.0",
            "types": ""
        },
        "updatenotification": {
            "calendar": "6.1.1",
            "contacts": "8.1.1",
            "core": "32.0.3.2",
            "deck": "1.16.2",
            "enabled": "yes",
            "groupfolders": "20.1.6",
            "installed_version": "1.22.0",
            "mail": "5.6.5",
            "maps": "1.5.0",
            "notes": "4.12.4",
            "phonetrack": "0.9.1",
            "related_resources": "1.0.4",
            "spreed": "21.0.4",
            "tasks": "0.17.1",
            "twofactor_totp": "6.4.1",
            "types": "",
            "update_check_errors": 0
        },
        "user_status": {
            "enabled": "yes",
            "installed_version": "1.12.0",
            "types": ""
        },
        "viewer": {
            "enabled": "yes",
            "installed_version": "5.0.0-dev.0",
            "types": ""
        },
        "weather_status": {
            "enabled": "yes",
            "installed_version": "1.12.0",
            "types": ""
        },
        "webhook_listeners": {
            "enabled": "yes",
            "installed_version": "1.3.0",
            "types": "filesystem"
        },
        "workflowengine": {
            "enabled": "yes",
            "installed_version": "2.14.0",
            "types": "filesystem"
        }
    }
}

Apps

The output of occ app:list (if possible).

Enabled:
  - activity: 5.0.0-dev.0
  - bruteforcesettings: 5.0.0-dev.0
  - calendar: 6.1.2
  - circles: 32.0.0
  - cloud_federation_api: 1.16.0
  - comments: 1.22.0
  - contacts: 8.1.2
  - contactsinteraction: 1.13.1
  - dashboard: 7.12.0
  - dav: 1.34.2
  - federatedfilesharing: 1.22.0
  - federation: 1.22.0
  - files: 2.4.0
  - files_antivirus: 6.1.0
  - files_downloadlimit: 5.0.0-dev.0
  - files_external: 1.24.0
  - files_pdfviewer: 5.0.0-dev.0
  - files_reminders: 1.5.0
  - files_sharing: 1.24.1
  - files_trashbin: 1.22.0
  - files_versions: 1.25.0
  - firstrunwizard: 5.0.0-dev.0
  - groupfolders: 20.1.6
  - logreader: 5.0.0-dev.0
  - lookup_server_connector: 1.20.0
  - mail: 5.6.5
  - nextcloud-aio: 0.8.0

Original post:

I just realized I had a few leftover mount points I was using to copy things around during setup. I’ve cleaned that up and am trying the backup again. :crossed_fingers:

Update 1:

Nope. It’s still failing in the same place.

How is the AIO backup finding its way to 107/vm-107-disk-0.qcow2? Of course that’s going to change - it’s the image that’s running the backup!

Resolution:

Oh wow… I somehow managed to create my Borg Server ON my Nextcloud data drive! :clown_face:

I think I can take it from here.

(Mods, feel free to delete this thread if you see fit.)