Developing an app, with python module - is it ok?

Good day.
My friend and i decided to develop an app for finding duplicate media(image,photos).

I need it for my NC instance(have huge amount of duplicate photos/videos with different resolutions, sizes and formats). And we want to publish app on GitHub and in App NC store. We think many will like it to use it.
Frontend and backend will be written as recommended in guides, but we need a part that will be written in python.
And that part need access for our app DB tables(we need create 3 tables for this app). What is right way to do that?(using database)
App needs to supports minimum of 3 types of db(sqlite, mysql/maria and postgre) - so python module will read NC config, get type of db and connect to it.

Is it a right way?

I found only recommended ways to work with DB for php in guides. I can rewrite what I wrote in python in php, but it will be slower and I try to optimise code as much as possible.
Also a third way when php will load data from tables to pass that data to python looks like very ugly for me.

1 Like

Hi,

you’ll mostly have to explore this yourself. The Python dependency will mean your app will only run in some environments where the right Python with the right modules is installed.

You can use proc_open and similar to invoke a Python script from php, just as an idea.

Cheers

I don’t know much systems(and i know many linux distros, some FreeBSD clones and macOS) where is a problem to install python3.
And during app install we can do it automatically(or throw web interface), and install all module needs(there are only a few amount we need) in venv.

Thanks for your attention and reply. :slight_smile:

There is a python library for ownCloud (and worked/works with Nextcloud) that uses the Nextcloud webdav interface: GitHub - owncloud/pyocclient: ownCloud client library for Python
Perhaps some functions have to be implemented if needed (e.g. tagging). This way, you could use the webdav-interface and the NC-APIs and you don’t need to deal with the database.

You cannot directly access a shared sqlite with python, since sqlite doesn’t use a server. The others, yes. Those will be fine.

However to be honest, I think you’d be better off leaving the database access in nextcloud/php, and passing data between them.

I’m not trying to talk you out of using python, but what is the reason you need it? Curious.

Thanks, forgot about that sqlite “feature”.

Leaving access to files/db from php and pass all data from php to python seems a bit overhead.
Then it will become an app, I wish not to use myself. And then someone can write app in full php and it will be better.

If you interested why we choose to use python this is a brief number of reasons:

  • Better support for different hashing methods(whash, dhash)
  • Future support of CNN networks, most of such libraries written in Python
  • Admin will be able to restrict python script, for max cpu load/memory consumption or even host it on another machine if needed, php will commutate with it with REST API.
  • Python script will be able to work in two modes:
    a) As service, processing user tasks, with/without unloading hashing data(8/32/128 bytes for each unique image(depends what option selected), and if it is video then multiply value by 2-3) That will be useful for instances like mine, with currently 10k+ photos.
    b) As on demand start to perfom task and quit
    That will lead to maximum of speed and customisation. I dont know how to write in php first variant, I am not so good at php to write the best code for this situation.

Oh, ok, so this isn’t actually a plugin for nextcloud, its a completely separate system that shares the database. Gotcha.

We are close to alpha release, need some GUI improvements and adjustments, doc writing for admins and finish video processing part - maybe in two-three weeks we will finish it, as alpha version and publish on Github.
Based on the results of, it will be a nextcloud plugin, not a separate system.
OS where we are testing it: clear install of Centos, Debian and Ubuntu and WSL(without pip and otherm only python that ships with).
Mysql,Maria,Postgre support.(in future maybe we add Oracle if needed)
Requirements will be: Python >= 3.6 and it is all.
All other modules will be auto installed locally(if user prefer this method) or user can follow guide and install all it manually.
Trying to make it maximum user friendly and easy to use.

P.S: a question about nextcloud-core - do we need to check if file is locked? we properly handle situation when file is invalid/damaged - and we prefer to skip those files. Currently we don’t do that check for lock - do we need it? Files are open only for read - can it cause some unknown issues?

We finished first version of it. If someone will be interested, here is out repo:
https://github.com/andrey18106/mediadc

1 Like

Thanks for the app. and more importantly, opening the doors of Python to Next Cloud.

An off topic question though,
I want to synch my self hosted Kanboard with nextcloud tasks.
Any idea how can I do that?
PS. Not a professional developer is talking here.

I haven’t any idea for that, i am not familiar with nextcloud app named tasks. Maybe, if you will provide some more info about this…
I just don’t understand for what reason you need python to integrate Kanboard? =)
Did you tried this app? [https://apps.nextcloud.com/apps/external]

Currently we are working on nextcloud python framework, and will rewrite our MediaDC app after that.
Framework will allow:

  1. Call from php python modules with arguments and return value in json.
  2. Automatically install required python modules by app if they have packages on pip, if not then gcc compilers must be installed on system(that requires sudo access) and they will build from sources.
  3. Provide API for Python for NC FS access(list, read, write, create, delete) (in alpha state already, almost finished) (not a webdav)
  4. Provide API for Python for NC DB access(not done/tested yet, in process)
  5. Framework will install Python 3.9.9, but there will be ability to specify own interpreter for admins(currently we are working on this, in a few days will finish if there will be no more bugs during tests)
  6. There will be option in feature(not in a first version) to run python module for app on separate machine. Steps 3/4 are able to work through unix sock or network socket. Read File access will be slower in this case.
  7. Very nice configuration page, and no need in sudo access to install something, ability to do silent auto-install install in most cases.
  8. It will run on Debian/Centos family and currently we fighting with musli linuxes to support from box Alpine family without glibc. Arm 64bit CPU’s will be supported from box, too.
    And for some reason no one installs nextcloud on Mac, but it works on Mac too…

We can’t say yet how fast it will be, php has too many restrictions for threadings, but what we have tested it looks promising.