Search images power by AI with python

đź’» Development
1

Hello everyone,

I’m seeking help to get started with a simple idea.

Use python face_recognition to search people on picture by name.

Introduction

Python face_recognition is an interface to easily search person in picture by given only one picture of a given face. Nextcloud is well you know what it is, so I will skeep introduction.

Idea

I Want to bring together this two great application together. Allowing people to search in pictures by simply spelling name of the person there are looking for.

Developement

I know, nextcloud server run with PHP and face_recognition with python, but on server side it will be easy to run both PHP and python. However I have no clue to where I should start, and many questions. I know, I can read nextcloud’s docs, but it will take me time to figure out where I should start when I can ask for some clues.

Questions

  1. What do you think of this idea?
  2. How can I modify search in nextcloud to use python app?
  3. Can somone tell me, as pro dev, what step should I do?

Background

I know well python, I have some insight on PHP. I run nextcloud on a FreeBSD server.

Thank you in advance for your precious help.

2

Hey,

interesting idea. But I think you need to view the architecture a little different in my opinion.

  1. real time search is not a good idea
  2. you need background/batch processing and storing the metadata in the database
  3. the search can not be modified - you “register” your app to the search and feed it with feedback

So if you want to proceed, I could imagine the following on a very high level

  • create you own app which delivers the py script to be executed from php
  • dependencies/prerequisites to have py installed on the server
  • provide a scanner logic which loops all images; does the AI; stores the metadata per img into the DB
  • call this scanner from the UI of your app via an ajax=>php
  • provide an UI to tag known names
  • register an occ and/or cronjob command to enable background processing
  • register a search provider to feed your results into the standard search list
  • register file hooks to update the index in the background for newly added files

what do you think?
Depending on your php skills, this will be week-filling-task :slight_smile:

cheers,
Rello

3

Hey @Hatoris ,
I like your idea. This would be a nice feature. :+1:

4

Hi,

There is an app which can process external scripts:
https://apps.nextcloud.com/apps/workflow_script

Not sure if this could be a first step for face recognition when there is an existing python script already.

5

I already replied in this thread about other face recognition and face grouping application we are working on, so see that for more details. I just want to add content relevant to this thread over here.

@Hatoris - we are working on something similar, so see how it fits your case. Maybe you want to join us, maybe learn from our mistakes, maybe you just want to have fun building stuff on your own… YMMV. Matias (original author of this app) also initially started with Python and I thought it was bad idea (by adding more dependencies, complicating workflows, making onboard harder), so somehow:slight_smile: I managed to pursuade him to adopt native approach. So, we worked hard on PDLib (PHP binding for DLib), to get to that point. We also implemented full queuing logic where images are processed. Take a look at our thoughts along the way in functional spec, maybe it will give you some ideas! If you feel like this is interesting to you, or have more ideas on top of our work, feel free to join us on Github (I mean, I also joined to Matias’ repo myself:D) - you are welcome. Otherwise, just go for your implementation, who knows how awesome it turns to and you will certainly learned a lot along the way! Cheers!