Calcardbackup: bash script to backup Nextcloud calendars and addressbooks as *.ics/*.vcf files

I had the honour to help Bernie_O with some translation of documentation for the newest release of this backup script which I find really useful.
But I am writing this to point out to the fact that calcardbackup does not need cleartext passwords (nor usernames) anymore, since it fetches all data directly from the database now. So there’s no security issue anymore to use the script, which at least to me is a better feeling to have backup and security both.

Thanks a lot Bernie_O for your constant work on this script!

1 Like

You Sir are my absolute hero! The script worked like a charm and everything was perfect. Thank you for your work!

2 Likes

@Bernie_O
You linked me to this thread.
I’m currently trying to extract the data from a backup set and it seems your script is pulling data only from the running instance.
I decompressed the backup to the “/var/tmp/workcloud” and changed the settings like this.

This ist the output of your script, but when checking the contents of the ics, I have the content of the currently running nextcloud 15.0.4 database. Did I miss something?

root@bananapim2:/var/tmp/calcardbackup/calcardbackup# ./calcardbackup /var/tmp/workcloud

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+ Sat Feb 16 12:08:12 CET 2019 → START calcardbackup ver. 0.8.6 (13.02.2019), AGPL-3.0
+ Checking dependencies and preparing…
+ no usersfile given: will backup all available items.
+ Using URL: http://localhost
+ no valid status.php found at http://localhost.
+ Nextcloud 14.0.4 detected.
+ Database of chosen Nextcloud installation is MySQL/MariaDB.
+ Looking for calendars in your Nextcloud:
+ saving calendar ncpadmin-Personal.ics (from database)…success!
+ saving calendar user1-Personal.ics (from database)…success!
+ saving calendar user2-Personal.ics (from database)…success!
+ Looking for calendarsubscriptions in your Nextcloud:
+ No calendarsubscriptions found.
+ Looking for addressbooks in your Nextcloud:
+ saving addressbook ncpadmin-Contacts.vcf (from database)…empty file. Addressbook ‘Contacts’ of user ‘ncpadmin’ is empty.
+ saving addressbook system_system-system.vcf (from database)…success!
+ saving addressbook user1-Contacts.vcf (from database)…empty file. Addressbook ‘Contacts’ of user ‘Karin’ is empty.
+ saving addressbook user2-Contacts.vcf (from database)…success!
+ Compressing backup as *.tar.gz file. Be patient - this may take a while.
+ Backup successfully compressed!
+ Find your backup here: /var/tmp/calcardbackup/calcardbackup/backups/calcardbackup-2019-02-16.tar.gz
+ Sat Feb 16 12:08:14 CET 2019 → END calcardbackup
+
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Your config.php from /var/tmp/workcloud is pointing to the database of your new installation.
Do you have a backup of the database your old installation was using? Or did you only backup the nextcloud folder (including the data folder)?

Can you tell me where you can see:
“Your config.php from /var/tmp/workcloud is pointing to the database of your new installation.”
I have a file called nextcloud-sqlbkp_20190210.bak that was unpacked from the backup, but I don’t know how to use that.
I guess it’s the sql dump.

Here are some of the last lines (human readable):

-- Dumping data for table `oc_whats_new`
--

LOCK TABLES `oc_whats_new` WRITE;
/*!40000 ALTER TABLE `oc_whats_new` DISABLE KEYS */;
/*!40000 ALTER TABLE `oc_whats_new` ENABLE KEYS */;
UNLOCK TABLES;
/*!40103 SET TIME_ZONE=@OLD_TIME_ZONE */;

/*!40101 SET SQL_MODE=@OLD_SQL_MODE */;
/*!40014 SET FOREIGN_KEY_CHECKS=@OLD_FOREIGN_KEY_CHECKS */;
/*!40014 SET UNIQUE_CHECKS=@OLD_UNIQUE_CHECKS */;
/*!40101 SET CHARACTER_SET_CLIENT=@OLD_CHARACTER_SET_CLIENT */;
/*!40101 SET CHARACTER_SET_RESULTS=@OLD_CHARACTER_SET_RESULTS */;
/*!40101 SET COLLATION_CONNECTION=@OLD_COLLATION_CONNECTION */;
/*!40111 SET SQL_NOTES=@OLD_SQL_NOTES */;

-- Dump completed on 2019-02-10 10:04:49

I can‘t see that, but you said, that the script is pulling data from the running instance and my script is using the credentials from config.php, so obviously your config.php is pointing to the database of your new installation.

Good you have a database dump. It should be not a big problem to restore the calendars. Replace [DBPASSWORD] with the according value in config.php from your old installation and make sure that there is no space between -p and [DBPASSWORD]:

  1. create a new database (e.g. called „workcloud“) with:
    EDIT: make sure there is no database called „workcloud“ in your database - or choose a different name in the following commands.
mysql -u ncadmin -p[DBPASSWORD] -e "CREATE DATABASE workcloud CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;"
  1. import your MySQL-dump (issue this command from the folder where your MySQL-dump resides):
mysql -u ncadmin -p[DBPASSWORD] workcloud < nextcloud-sqlbkp_20190210.bak
  1. run calcardbackup like you did before
  2. have a look the .ics files calcardbackup created
    • if the calendars aren‘t the ones from your old installation, write again in this thread
    • if you got the calendars from your old installation you can delete the database again with:
mysql -u ncadmin -p[DBPASSWORD] -e "DROP DATABASE workcloud;"

In any case let me know how it goes :slight_smile:

Setting up the new database gives me error.
Copied the passphrase from the old config.php (it’s the same like the one from the active instance, by the way)

root@bananapim2:/var/tmp/workcloud/config# mysql -u ncadmin -pblablablalbalikethis -e "CREATE DATABASE workcloud CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;"
ERROR 1044 (42000) at line 1: Access denied for user 'ncadmin'@'localhost' to database 'workcloud'

Hm… Sorry - I am not very familiar with the code of nextcloudpi.
You need to issue those commands then as the mysql root user.
You can try to issue the commands with sudo and leaving out option -u and -p so that the commands start like this (you probably will be asked for a password - enter the one you use when logging in via ssh):

sudo mysql -e "CREATE DATABASE workcloud [...] "

If this doesn’t work - @nachoparker has to help here for how gain access to mysql as the mysql root-user to create a temporary table for this purpose.

I issued both commands without -u and -p, as root user.
The calcardbackup is still exporting the nextcloud contents only.
This is the current list of databases:

   MariaDB [(none)]> show databases;
    +--------------------+
    | Database           |
    +--------------------+
    | MyMusic60          |
    | MyVideos107        |
    | information_schema |
    | mysql              |
    | nextcloud          |
    | performance_schema |
    | workcloud          |
    +--------------------+
    7 rows in set (0.00 sec)

You need to edit /var/tmp/workcloud/config/config.php to point to the database created from the dump:

"dbname"  => "workcloud",

and grant the according privileges to the database user ncadmin with those two commands:

mysql -e "GRANT ALL PRIVILEGES on workcloud.* to 'ncadmin'@'localhost';"
mysql -e "FLUSH privileges;"

Then run calcardbackup again:

cd /var/tmp/calcardbackup/calcardbackup
./calcardbackup /var/tmp/workcloud

EDIT: in your case, you need to run the mysql commands as root user

:man_facepalming: of course…I’m an idiot!
Will come back later with more results - hopefully.

Edit:
Worked as expected - flawless.
Thanks for your help and your nice tool! :handshake:

1 Like

Glad I could help - don’t forget to delete the temporary database again
:love_you_gesture:

After installing calcardbackup and configuring calcardbackup.conf I get following messages with an error message included:

START calcardbackup ver. 0.9.3 (24.06.2019), AGPL-3.0

  • Checking dependencies and preparing…
  • Using configuration file /usr/local/bin/calcardbackup/calcardbackup.conf, ignoring all other command line options.
  • no usersfile given:
    • will fetch all available items from database
  • Using URL: https://nextcloud
  • Nextcloud 16.0.1 detected.
  • Database of chosen Nextcloud installation is MySQL/MariaDB.
  • Looking for calendars in your Nextcloud:
    ./calcardbackup: Zeile 991: mysql: Befehl nicht gefunden

The created archive in the backup folder is empty.

Any help is appreciated.

You need to install the MySQL client.
Which operating system and database (MySQL or MariaDB?) are you using?

EDIT: the next release of calcardbackup will check, wether the command line client for the according database is installed.

I am using Ubuntu and MySQL. After I installed the MySQL client from repo I get now this error:

Looking for calendars in your Nextcloud:
ERROR 2002 (HY000): Can’t connect to local MySQL server through socket ‘/var/run/mysqld/mysqld.sock’ (2)

The folder/file ‘/var/run/mysqld/mysqld.sock’ does not exist on my system.

Maybe important: I am using XAMPP/LAMPP for running my nextcloud instance.

After copying the my.cnf from /opt/lampp/etc/ to /etc/ (my.cnf including correct socket path) everything works.

Many thanks for this backup solution.

1 Like

It may be worth to take a look at vdirsyncer. It syncs calendar/card data from a server (e.g. nextcloud) to local files/folders.
And it can perform 2-way sync, so you can use other tools on your local files and the changes will be synched back.

vdirsyncer and calcardbackup have quite different use cases:

  • vdirsyncer syncronizes calendars/addressbooks of a single user. A single username/password combination must be given in a configuration file.

  • calcardbackup is meant as a backup solution for every single calendar and addressbook found in the database of Nextcloud/ownCloud without the need to give any usernames or passwords.

I just move servers and had to reinstall nextcloud and I wanted to say a big THANK YOU for saving me hours of trying to recover my user’s calendar!

1 Like

Thanks for this nice script! Excellent work!

Just the “–include-shares” option doesn’t work as expected, I still get this message:

Skipping calendar 'xxxxxxxxxxxxxx_shared_by_yyyyyy': already saved as 'yyyyy.ics"