Major issue with potential data loss

Support intro

Sorry to hear youā€™re facing problems. :slightly_frowning_face:

The community help forum (help.nextcloud.com) is for home and non-enterprise users. Support is provided by other community members on a best effort / ā€œas availableā€ basis. All of those responding are volunteering their time to help you.

If youā€™re using Nextcloud in a business/critical setting, paid and SLA-based support services can be accessed via portal.nextcloud.com where Nextcloud engineers can help ensure your business keeps running smoothly.

Getting help

In order to help you as efficiently (and quickly!) as possible, please fill in as much of the below requested information as you can.

Before clicking submit: Please check if your query is already addressed via the following resources:

(Utilizing these existing resources is typically faster. It also helps reduce the load on our generous volunteers while elevating the signal to noise ratio of the forums otherwise arising from the same queries being posted repeatedly).

Some or all of the below information will be requested if it isnā€™t supplied; for fastest response please provide as much as you can. :heart:

The Basics

  • Nextcloud Server version (e.g., 29.x.x):
    • Nextcloud Hub 9 (30.0.4)
  • Operating system and version (e.g., Ubuntu 24.04):
    • Client: Pop!_OS 22.04 LTS
    • Server: Ubuntu 22.04.4 LTS
  • Web server and version (e.g, Apache 2.4.25):
    • Apache through AIO
  • Reverse proxy and version _(e.g. nginx 1.27.2)
    • replace me
  • PHP version (e.g, 8.3):
    • replace me
  • Is this the first time youā€™ve seen this error? (Yes / No):
    • Yes
  • When did this problem seem to first start?
    • 18 December 2024 @1426UTC
  • Installation method (e.g. AlO, NCP, Bare Metal/Archive, etc.)
    • AIO
  • Are you using CloudfIare, mod_security, or similar? (Yes / No)
    • No

Summary of the issue you are facing:

Possible data loss. When I now try to access the Personal Board (Deck) I get a screen saying I need to create a List (I had four lists before the error started) and a warning similar to this: ``

There are errors in the log shown below.

This started when I was entering a new card on my Personal board in my Backlog List, and tried to add a due date (which I typically do not use). The autosave feature kept kicking me out of the date field while trying to enter month, year and date, so I switched to the date picker. Something went haywire, then the card showed invalid date. I couldnā€™t change it, so I created a new card, and just used the date picker. But then I couldnā€™t delete the ā€œinvalidā€ card, and on screen refresh I got the notice described above.

Steps to replicate it (hint: details matter!):

These are the steps that led to the issue. Iā€™ve not tried replicating because my first concern is to recover my data (I do have a backup I can restore if necessary):

  1. Selected Personal Board
  2. In Backlog list, clicked icon to ā€œCreate Cardā€
  3. Entered name of card, then accepted
  4. Opened details of card
  5. Clicked + symbol to add due date
  6. Started typing date 03/15 etc. As autosave keeps kicking me out of the field Iā€™m typing (a different issuee) I switched to the date selector.
  7. Tried to select correct due date and time
  8. Saved card, but found it showed data as invalid. Edited a couple of times but still it showed an invalid data.
  9. Created new card and used date picker to selected due date. Saved card.
  10. Tried to delete card showing the invalid date. I couldnā€™t delete the offending card.
  11. Refreshed screen, and got the screen showing above in the screen cap.

Log entries

Nextcloud

Please provide the log entries from your Nextcloud log that are generated during the time of problem (via the Copy raw option from Administration settings->Logging screen or from your nextcloud.log located in your data directory). Feel free to use a pastebin/gist service if necessary.

The Paste Bin contains the first entry in the log. This basic error is repeated dozens of times, showing for both Deck and Core applications.

https://pastebin.com/NnvNrmuc

docker logs nextcloud-aio-database

2024-12-18 14:35:49.692 UTC [16298] ERROR:  duplicate key value violates unique constraint "oc_text_documents_pkey"
2024-12-18 14:35:49.692 UTC [16298] DETAIL:  Key (id)=(10077) already exists.
2024-12-18 14:35:49.692 UTC [16298] STATEMENT:  INSERT INTO "oc_text_documents" ("id", "last_saved_version_time", "last_saved_version_etag", "base_version_etag") VALUES($1, $2, $3, $4)

Web Browser

If the problem is related to the Web interface, open your browser inspector Console and Network tabs while refreshing (reloading) and reproducing the problem. Provide any relevant output/errors here that appear.

PASTE

Web server / Reverse Proxy

The output of your Apache/nginx/system log in /var/log/____:

PASTE HERE

Configuration

Nextcloud

The output of occ config:list system or similar is best, but, if not possible, the contents of your config.php file from /path/to/nextcloud is fine (make sure to remove any identifiable information!):

{
    "system": {
        "one-click-instance": true,
        "one-click-instance.user-limit": 100,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "appsallowlist": false,
        "check_data_directory_permissions": false,
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "password": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "overwritehost": "***REMOVED",
        "overwriteprotocol": "https",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "localhost",
            "***REMOVED***"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "pgsql",
        "version": "30.0.4.1",
        "overwrite.cli.url": "https:\/\/***REMOVED***\/",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "maintenance": false,
        "loglevel": "2",
        "log_type": "file",
        "logfile": "\/var\/www\/html\/data\/nextcloud.log",
        "log_rotate_size": "10485760",
        "log.condition": {
            "apps": [
                "admin_audit"
            ]
        },
        "preview_max_x": "2048",
        "preview_max_y": "2048",
        "jpeg_quality": "60",
        "enabledPreviewProviders": {
            "1": "OC\\Preview\\Image",
            "2": "OC\\Preview\\MarkDown",
            "3": "OC\\Preview\\MP3",
            "4": "OC\\Preview\\TXT",
            "5": "OC\\Preview\\OpenDocument",
            "6": "OC\\Preview\\Movie",
            "7": "OC\\Preview\\Krita",
            "0": "OC\\Preview\\Imaginary",
            "23": "OC\\Preview\\ImaginaryPDF"
        },
        "enable_previews": true,
        "upgrade.disable-web": true,
        "mail_smtpmode": "smtp",
        "trashbin_retention_obligation": "auto, 30",
        "versions_retention_obligation": "auto, 30",
        "activity_expire_days": "30",
        "simpleSignUpLink.shown": false,
        "share_folder": "\/Shared",
        "one-click-instance.link": "https:\/\/nextcloud.com\/all-in-one\/",
        "upgrade.cli-upgrade-link": "https:\/\/github.com\/nextcloud\/all-in-one\/discussions\/2726",
        "updatedirectory": "\/nc-updater",
        "davstorage.request_timeout": 3600,
        "htaccess.RewriteBase": "\/",
        "dbpersistent": false,
        "files_external_allow_create_new_local": false,
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "allow_local_remote_servers": true,
        "preview_imaginary_url": "***REMOVED SENSITIVE VALUE***",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_sendmailmode": "smtp",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauth": 1,
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "587",
        "default_phone_region": "US",
        "maintenance_window_start": 100,
        "preview_imaginary_key": "***REMOVED SENSITIVE VALUE***",
        "auth.bruteforce.protection.enabled": true,
        "ratelimit.protection.enabled": true,
        "updater.release.channel": "stable",
        "documentation_url.server_logs": "https:\/\/github.com\/nextcloud\/all-in-one\/discussions\/5425",
        "DOMAIN": "***REMOVED***"
    }
}

Apps

The output of occ app:list (if possible).

Enabled:
  - activity: 3.0.0
  - admin_audit: 1.20.0
  - app_api: 4.0.3
  - bruteforcesettings: 3.0.0
  - calendar: 5.0.7
  - circles: 30.0.0
  - cloud_federation_api: 1.13.0
  - comments: 1.20.1
  - contacts: 6.1.1
  - contactsinteraction: 1.11.0
  - dashboard: 7.10.0
  - dav: 1.31.1
  - deck: 1.14.2
  - federatedfilesharing: 1.20.0
  - federation: 1.20.0
  - files: 2.2.0
  - files_downloadlimit: 3.0.0
  - files_fulltextsearch: 30.0.0
  - files_pdfviewer: 3.0.0
  - files_reminders: 1.3.0
  - files_sharing: 1.22.0
  - files_trashbin: 1.20.1
  - files_versions: 1.23.0
  - firstrunwizard: 3.0.0
  - fulltextsearch: 30.0.0
  - fulltextsearch_elasticsearch: 30.0.0
  - logreader: 3.0.0
  - lookup_server_connector: 1.18.0
  - maps: 1.5.0
  - nextcloud-aio: 0.7.0
  - nextcloud_announcements: 2.0.0
  - notes: 4.11.0
  - notifications: 3.0.0
  - notify_push: 0.7.0
  - oauth2: 1.18.1
  - password_policy: 2.0.0
  - photos: 3.0.2
  - privacy: 2.0.0
  - provisioning_api: 1.20.0
  - recommendations: 3.0.0
  - related_resources: 1.5.0
  - richdocuments: 8.5.3
  - serverinfo: 2.0.0
  - settings: 1.13.0
  - sharebymail: 1.20.0
  - support: 2.0.0
  - survey_client: 2.0.0
  - systemtags: 1.20.0
  - tasks: 0.16.1
  - text: 4.1.0
  - theming: 2.5.0
  - twofactor_backupcodes: 1.19.0
  - twofactor_totp: 12.0.0-dev
  - user_status: 1.10.0
  - viewer: 3.0.0
  - weather_status: 1.10.0
  - webhook_listeners: 1.1.0-dev
  - workflowengine: 2.12.0
Disabled:
  - encryption: 2.18.0
  - files_external: 1.22.0
  - files_rightclick: 0.15.1 (installed 1.6.0)
  - suspicious_login: 8.0.0
  - twofactor_nextcloud_notification: 4.0.0
  - user_ldap: 1.21.0

Tips for increasing the likelihood of a response

  • Use the preformatted text formatting option in the editor for all log entries and configuration output.
  • If screenshots are useful, feel free to include them.
    • If possible, also include key error output in text form so it can be searched for.
  • Try to edit log output only minimally (if at all) so that it can be ran through analyzers / formatters by those trying to help you.

The error states a lot of information actually, would you be familiar to databases. Entries (rows) in database tables have their own identifier, an ID, similar to names for people. Sometimes, errors occur, as in your case, when an identical combination is found. Regard it as the unique identification of a person being the combination of initials. In your case, the error is provided that the system tries to write an entry for your deck, however it finds that there is already such an entry.
What I would do, given that I have experience with databases, is trying to erase the current row in the database for the same ID, so the process can finish writing the old entry.
However, in my experience most plugins have their own data registered, so I would not be concerned about the files on the system. If you would, please make a backup of the data directory, please use recursiveness (I had one of my first backups only populated with the few files in the main directory and a list of empty directories.)

I do have some familiarity with databases, and will try to find the offending entry to delete it. Youā€™re thinking this will ā€œrecoverā€ the rest of the entries in Deck?

I realize the lack of data shown on the screen for that board might simply be because the data canā€™t be loaded. If this is the case, Iā€™d suggest the error message isnā€™t helpful.

Assuming I am able to restore things to prior to the crash, then the question will become what caused the duplicate key, and is that related to the date picker issue that started all of this?

In looking at the database dump, it is clear I do not have enough knowledge of how NextCloud data is structured or sql databases.

If you have a suggested query to find the duplicate ID, Iā€™d be happy to try it.

Thanks.