Calcardbackup: bash script to backup Nextcloud calendars and addressbooks as *.ics/*.vcf files

📑 How to
, ,
1

I like to backup my Nextcloud calendars and addressbooks as *.ics/*.vcf files, because it is very easy to restore: just import the backed up *.ics/*.vcf file to the server or a client and (accidentally) deleted data is back online.

calcardbackup exports all available calendars/addressbooks from a Nextcloud instance as .ics/.vcf files and compresses them to a single file with the current date in the filename.

Your Nextcloud instance is broken? No problem: as long as you have access to the database, calcardbackup is able to export all calendars and addressbooks that are in there.

calcardbackup is compatible with all currently released Nextcloud versions (and ownCloud >= 5.0) and has been tested under Debian, Ubuntu, FreeBSD, SunOS and Darwin.
Though shell access to the server is needed, the included calcardbackup-wrapper.php script might be able to run calcardbackup and Nextclouds occ command in shared hosting environments without shell access (depending on your webhosters configuration).

The usage is simple: in most cases the path to the Nextcloud directory is sufficient. It is not necessary to run the script with root privileges - it only needs read access to Nextclouds configuration file config.php. At no point any data leaves the server.

https://github.com/BernieO/calcardbackup
(don’t use the repo at GitHub anymore as it is heavily outdated and contains lots of unfixed bugs!)

EDIT (17.01.2020): the repository moved to Codeberg:
:point_right: https://codeberg.org/BernieO/calcardbackup :point_left:

EDIT (06.05.2021): calcardbackup v2.0.0 might be able to run on your server even if you don’t have shell access to the server. Check out the section ‘Can I use this without shell access to the server?’ in the README of the repo.

Best wishes,
Bernie_O

Please note, that this text has been adapted several times to match the newest release of calcardbackup.

11 Likes
2

Thanks for sharing your script with us. Some remarks:

  • why do you need the server-path and the folder on the server? For me it makes more sense to make a backup from an external host (like https://uriesk.wordpress.com/2015/02/13/backup-your-caldav-calendar-with-cadaver/) or just run a backup server-side script for each user (directly use the database, don’t ask the user’s password).
  • Having a file with all users and clear-text passwords on the server is not good
  • Tables can have a prefix different from “oc_”
3

The folder is needed for reading values of config.php - the URL is only needed, if overwrite.cli.uri is not present in config.php.
Cadaver sounds interesting. I’ll have a look at that. But wouldn’t you need usernames and passwords then as well?
I also thought about directly reading the values from the database. Haven’t read enough into ics-standard though and don’t want to read values, put them somehow together eventually resulting in a non valid ics file. And with only backing up the database tables (with e.g. mysqldump) restoring wouldn’t be as easy as it is with ics/vcf files.

I agree, this is not optimal. The script is mainly meant to be for users who host their own Nextcloud Installation at home with not so many users (e.g. family) and not so many people accessing the server (being able to read that file).

Thanks, didn’t notice that because there is no option to choose a table prefix string when installing Nextcloud. So it might not affect so many people. Anyway: fixed it for the next release.

4

I just released version 0.2.0 of the script, which comes with a new option to include shared addressbooks/calendars to the backup. This may be used to provide only one username + password in the credentials-file to whom all addressbooks/calendars (to be backed up) are shared. A just for this purpose created new user would make sense. If the file gets somehow in wrong hands, only the credentials of this single user will be compromised.

Drawback of this approach is: added addressbooks/calendars will not be backed up, unless they are shared with that user. That’s the price for improved security…

Have a look at the section “About the option -i / --include-shares” in the README of the repo for a detailed “How to”. There you can also find some usage examples: https://github.com/BernieO/calcardbackup (EDIT 28.05.2021: don’t use the repo at GitHub anymore as it is heavily outdated and contains lots of unfixed bugs!!)

EDIT (17.01.2020): the repository moved to Codeberg:
:point_right: https://codeberg.org/BernieO/calcardbackup

Cheers, Bernie_O

5

I had the honour to help Bernie_O with some translation of documentation for the newest release of this backup script which I find really useful.
But I am writing this to point out to the fact that calcardbackup does not need cleartext passwords (nor usernames) anymore, since it fetches all data directly from the database now. So there’s no security issue anymore to use the script, which at least to me is a better feeling to have backup and security both.

Thanks a lot Bernie_O for your constant work on this script!

1 Like
6

You Sir are my absolute hero! The script worked like a charm and everything was perfect. Thank you for your work!

2 Likes
7

@Bernie_O
You linked me to this thread.
I’m currently trying to extract the data from a backup set and it seems your script is pulling data only from the running instance.
I decompressed the backup to the “/var/tmp/workcloud” and changed the settings like this.

This ist the output of your script, but when checking the contents of the ics, I have the content of the currently running nextcloud 15.0.4 database. Did I miss something?

root@bananapim2:/var/tmp/calcardbackup/calcardbackup# ./calcardbackup /var/tmp/workcloud

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+ Sat Feb 16 12:08:12 CET 2019 → START calcardbackup ver. 0.8.6 (13.02.2019), AGPL-3.0
+ Checking dependencies and preparing…
+ no usersfile given: will backup all available items.
+ Using URL: http://localhost
+ no valid status.php found at http://localhost.
+ Nextcloud 14.0.4 detected.
+ Database of chosen Nextcloud installation is MySQL/MariaDB.
+ Looking for calendars in your Nextcloud:
+ saving calendar ncpadmin-Personal.ics (from database)…success!
+ saving calendar user1-Personal.ics (from database)…success!
+ saving calendar user2-Personal.ics (from database)…success!
+ Looking for calendarsubscriptions in your Nextcloud:
+ No calendarsubscriptions found.
+ Looking for addressbooks in your Nextcloud:
+ saving addressbook ncpadmin-Contacts.vcf (from database)…empty file. Addressbook ‘Contacts’ of user ‘ncpadmin’ is empty.
+ saving addressbook system_system-system.vcf (from database)…success!
+ saving addressbook user1-Contacts.vcf (from database)…empty file. Addressbook ‘Contacts’ of user ‘Karin’ is empty.
+ saving addressbook user2-Contacts.vcf (from database)…success!
+ Compressing backup as *.tar.gz file. Be patient - this may take a while.
+ Backup successfully compressed!
+ Find your backup here: /var/tmp/calcardbackup/calcardbackup/backups/calcardbackup-2019-02-16.tar.gz
+ Sat Feb 16 12:08:14 CET 2019 → END calcardbackup
+
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

8

Your config.php from /var/tmp/workcloud is pointing to the database of your new installation.
Do you have a backup of the database your old installation was using? Or did you only backup the nextcloud folder (including the data folder)?

9

Can you tell me where you can see:
“Your config.php from /var/tmp/workcloud is pointing to the database of your new installation.”
I have a file called nextcloud-sqlbkp_20190210.bak that was unpacked from the backup, but I don’t know how to use that.
I guess it’s the sql dump.

Here are some of the last lines (human readable):

-- Dumping data for table `oc_whats_new`
--

LOCK TABLES `oc_whats_new` WRITE;
/*!40000 ALTER TABLE `oc_whats_new` DISABLE KEYS */;
/*!40000 ALTER TABLE `oc_whats_new` ENABLE KEYS */;
UNLOCK TABLES;
/*!40103 SET TIME_ZONE=@OLD_TIME_ZONE */;

/*!40101 SET SQL_MODE=@OLD_SQL_MODE */;
/*!40014 SET FOREIGN_KEY_CHECKS=@OLD_FOREIGN_KEY_CHECKS */;
/*!40014 SET UNIQUE_CHECKS=@OLD_UNIQUE_CHECKS */;
/*!40101 SET CHARACTER_SET_CLIENT=@OLD_CHARACTER_SET_CLIENT */;
/*!40101 SET CHARACTER_SET_RESULTS=@OLD_CHARACTER_SET_RESULTS */;
/*!40101 SET COLLATION_CONNECTION=@OLD_COLLATION_CONNECTION */;
/*!40111 SET SQL_NOTES=@OLD_SQL_NOTES */;

-- Dump completed on 2019-02-10 10:04:49
10

I can‘t see that, but you said, that the script is pulling data from the running instance and my script is using the credentials from config.php, so obviously your config.php is pointing to the database of your new installation.

Good you have a database dump. It should be not a big problem to restore the calendars. Replace [DBPASSWORD] with the according value in config.php from your old installation and make sure that there is no space between -p and [DBPASSWORD]:

  1. create a new database (e.g. called „workcloud“) with:
    EDIT: make sure there is no database called „workcloud“ in your database - or choose a different name in the following commands.
mysql -u ncadmin -p[DBPASSWORD] -e "CREATE DATABASE workcloud CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;"
  1. import your MySQL-dump (issue this command from the folder where your MySQL-dump resides):
mysql -u ncadmin -p[DBPASSWORD] workcloud < nextcloud-sqlbkp_20190210.bak
  1. run calcardbackup like you did before
  2. have a look the .ics files calcardbackup created
    • if the calendars aren‘t the ones from your old installation, write again in this thread
    • if you got the calendars from your old installation you can delete the database again with:
mysql -u ncadmin -p[DBPASSWORD] -e "DROP DATABASE workcloud;"

In any case let me know how it goes :slight_smile:

11

Setting up the new database gives me error.
Copied the passphrase from the old config.php (it’s the same like the one from the active instance, by the way)

root@bananapim2:/var/tmp/workcloud/config# mysql -u ncadmin -pblablablalbalikethis -e "CREATE DATABASE workcloud CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;"
ERROR 1044 (42000) at line 1: Access denied for user 'ncadmin'@'localhost' to database 'workcloud'
12

Hm… Sorry - I am not very familiar with the code of nextcloudpi.
You need to issue those commands then as the mysql root user.
You can try to issue the commands with sudo and leaving out option -u and -p so that the commands start like this (you probably will be asked for a password - enter the one you use when logging in via ssh):

sudo mysql -e "CREATE DATABASE workcloud [...] "

If this doesn’t work - @nachoparker has to help here for how gain access to mysql as the mysql root-user to create a temporary table for this purpose.

13

I issued both commands without -u and -p, as root user.
The calcardbackup is still exporting the nextcloud contents only.
This is the current list of databases:

   MariaDB [(none)]> show databases;
    +--------------------+
    | Database           |
    +--------------------+
    | MyMusic60          |
    | MyVideos107        |
    | information_schema |
    | mysql              |
    | nextcloud          |
    | performance_schema |
    | workcloud          |
    +--------------------+
    7 rows in set (0.00 sec)
14

You need to edit /var/tmp/workcloud/config/config.php to point to the database created from the dump:

"dbname"  => "workcloud",

and grant the according privileges to the database user ncadmin with those two commands:

mysql -e "GRANT ALL PRIVILEGES on workcloud.* to 'ncadmin'@'localhost';"
mysql -e "FLUSH privileges;"

Then run calcardbackup again:

cd /var/tmp/calcardbackup/calcardbackup
./calcardbackup /var/tmp/workcloud

EDIT: in your case, you need to run the mysql commands as root user

15

:man_facepalming: of course…I’m an idiot!
Will come back later with more results - hopefully.

Edit:
Worked as expected - flawless.
Thanks for your help and your nice tool! :handshake:

1 Like
16

Glad I could help - don’t forget to delete the temporary database again
:love_you_gesture:

17

After installing calcardbackup and configuring calcardbackup.conf I get following messages with an error message included:

START calcardbackup ver. 0.9.3 (24.06.2019), AGPL-3.0

  • Checking dependencies and preparing…
  • Using configuration file /usr/local/bin/calcardbackup/calcardbackup.conf, ignoring all other command line options.
  • no usersfile given:
    • will fetch all available items from database
  • Using URL: https://nextcloud
  • Nextcloud 16.0.1 detected.
  • Database of chosen Nextcloud installation is MySQL/MariaDB.
  • Looking for calendars in your Nextcloud:
    ./calcardbackup: Zeile 991: mysql: Befehl nicht gefunden

The created archive in the backup folder is empty.

Any help is appreciated.

18

You need to install the MySQL client.
Which operating system and database (MySQL or MariaDB?) are you using?

EDIT: the next release of calcardbackup will check, wether the command line client for the according database is installed.

19

I am using Ubuntu and MySQL. After I installed the MySQL client from repo I get now this error:

Looking for calendars in your Nextcloud:
ERROR 2002 (HY000): Can’t connect to local MySQL server through socket ‘/var/run/mysqld/mysqld.sock’ (2)

The folder/file ‘/var/run/mysqld/mysqld.sock’ does not exist on my system.

Maybe important: I am using XAMPP/LAMPP for running my nextcloud instance.

20

After copying the my.cnf from /opt/lampp/etc/ to /etc/ (my.cnf including correct socket path) everything works.

Many thanks for this backup solution.

1 Like