Cool Cloud Solution thanks


#1
Hi Team,
Before I share some findings about vcf handling in nextcloud, let me praise the team for the great work first. Last week I decided to setup my locally hosted cloud. Nextcloud was my second attempt after having tested Seafile server. My family obviously loves nextcloud as they have flooded the server already and I am getting in trouble now with my backups ; -).
What was important to me and why did I choose nextcloud?
-       Self hosted. No provider involved. Check
-       Easy to install. Low hardware requirements. Check
-       Easy to use and reasonable apps for the smartphone. Check
-       Easy access to user data "under the hood". Check. Thanks to /var/www/nextcloud/data..... I can get to all the files easily. I can even load great bunches of data and do an "occ files:scan" afterwards. Great (it tooks some research to find you have to better use "... /usr/bin/php7.0 occ files:scan" rather than "php occ files:scan")
-       And many many other advantages I do see with using nextcloud.
Now, I tried to sync my contacts. This caused some headaches. What you can find in the internet is syncing from nextcloud to A, B or C. But how do you initially get your contacts (from your smartphone) into the cloud? Syncing via davdroid or whatever apps there are out there? No luck. Importing a vcf directly? No luck. I now know why. It is (as also the developers pointed out) due to different versions. My Smartphone has 2.1 and nextcloud has 3.0 vcf version data.
When you try to load the data you see the circling donut and that's it. No error. Nothing. Ok, I did a search/replace of 2.1 to 3.0 within the files and the import finished but produced some crappy data. No problem, the data cleanup will take several hours but anyhow, that's still fine as long as I can get to the data later. Can I? Where is the data? What, if I want to get to my contacts later and export them? I couldn't find a solution and therefore did a deeper dive into the system. Installing phpmyadmin and looking around did the trick. Aah, it is stored as a blob. So, here we go. I am a programming hobbyist and took the challenge and spent some time on coding. Forgive me for not using try, catch, building a class and the many other code weaknesses but I hope people using nextcloud can benefit from my code snippet and can also get to their contacts later. If the feedback is good enough I can probably do some tweaking to the code and leverage it to the next level. Here we go. I am using Python3.
--------------------------------------------------------------------------------
# Install the missing packages using your package management system (apt-get) 
# or via pip install
# The script can be run by copy+paste into a jupyter notebook
import MySQLdb
import vobject
import io, os
import pandas as pd

conn = MySQLdb.connect(host= "192.168.100.100", # put in the IP of the nextcloud server
                       user= "Standard", # User must have at least the rights for SQL-SELECT 
                       passwd= "Standard", # Type in the password for this user
                       db= "nextcloud") # the database name for your nextcloud

x= conn.cursor()
# "carddata" is the BlOB column, "oc_cards" is the table name in the database 
# (see above: db="nextcloud", for a default installation)
x.execute("SELECT carddata FROM oc_cards")
blob= x.fetchone()
list_of_dicts= []


while blob is not None:
    # Convert the binary (BLOB) to utf8 text and separate the lines
    lines=blob[0].decode('utf8').split('\r\n')
    
    # Depending on what you want to do, here are some code ideas:
    # You can just print out what you have
    for line in lines:
        if 'PHOTO' not in line and ':' in line[:15]:
            print(line)
    print('-' * 30)
            # Alternatively you can make row a vcard object directly.
            # And use it for further coding.
            # vcard= vobject.readOne(row[0].decode('utf8'))
            # vcard.prettyPrint()

    
    # Read the key:value pair into a dictionary
    blob_dict = {}
    for line in lines:
        if 'PHOTO' not in line and ':' in line[:15]:
            pair = line.split(':')
            if len(pair) == 2:
                extra = {pair[0]: pair[1]}
                # append the key:value pair to the dictionary
                blob_dict.update(extra)
    # Append all blob lines to a list of dicts (used at the end of the script)
    list_of_dicts.append(blob_dict)

    # You can output single vcf files, like this:
    # Take the full name (FN) of the contact(blob_dict) and create a vcf file from it
    with open(blob_dict.get("FN") + '.vcf', 'wt') as f:
        for line in lines:
            if 'PHOTO' not in line and ':' in line[:15]:
                f.write(line + '\n')

    blob= x.fetchone()
    i = i + 1

# see above: We created key:value pairs and put them together in a list_of_dicts.
# Having all that we can now write a csv file containing all contacts.
df = pd.DataFrame(list_of_dicts)
# We do not need the nextcloud specific data UID and REV
df = df.drop(['UID', 'REV'], axis= 1)
df.to_csv('Contacts_v30.csv', sep='|', encoding='utf8', index=False)

# Please recognise the separator for a later spreadsheet import. 
# It is set to the pipe symbol "|", not comma

x.close()
conn.close()

Hope this helps somebody. Thanks.
HZ

#2

Hello @newlandlord

thank you for your warm words.

Please file an topic in suitable category to get assistance.

Cheers