FullNextSearch is now Beta

A new version of FullNextSearch and its components have been release on the appstore. The app is switch from Alpha to Beta. As a reminder, FullNextSearch is replacing Nextant.

A complete guide on the implementation of this Full Text Search engine into your Nextcloud is also available on the wiki:

If you’re already using Nextant, now is a good time to switch to FullNextSearch

If you’re upgrading from FullNextSearch ALPHA, you will need to

  • check the setup of ReadonlyREST (check the wiki) and more the configuration from elasticsearch.yml to readonlyrest.yml

  • reset the current index:

./occ fullnextsearch:reset
./occ fullnextsearch:index

Hi, I have used the nextant app for the last year and it was really good, I tried the new app with the latest elasticsearch but I noticed a couple of problems:
elasticsearch always used 1G of memory (the mini server only has 2) while solr releses it when not used, but this is probably a configuration problem in elasticsearch’s rpm
Since I had a solr instance I used it to index my emails using dovecot fts-solr that also works very well, unfortunately dovecot’s fts-elasticsearch plugin is unmaintained and doesn’t compile.
Anyway, I was able to setup fullnextsearch but the initial indexing always failed, on the same file I think:
[root@erm67 html]# sudo -u apache ./occ fullnextsearch:index
indexing.
USER: ermanno

  • RAM: 18.558967590332

  • RAM: 22.121726989746

  • RAM: 22.91674041748

  • RAM: 23.664680480957

    • RAM: 24.007484436035
    • RAM: 24.847015380859
    • RAM: 25.668411254883
    • RAM: 25.916816711426
    • RAM: 25.933441162109
    • RAM: 26.759033203125
    • RAM: 28.999183654785
    • RAM: 31.035209655762
    • RAM: 31.18936920166
    • RAM: 31.652488708496
    • RAM: 33.37100982666
    • RAM: 33.690185546875
    • RAM: 33.954521179199
    • RAM: 34.290542602539
    • RAM: 34.571563720703
    • RAM: 35.062767028809
      . Indexing: 2017-7-31_3840.pdf

    [Elasticsearch\Common\Exceptions\BadRequest400Exception]
    {“error”:{“root_cause”:[{“type”:“illegal_argument_exception”,“reason”:“pipeline with id [attachment] does not exist”}],“type”:“illegal_argument_exception”,“reas
    on”:“pipeline with id [attachment] does not exist”},“status”:400}

fullnextsearch:index [–output [OUTPUT]]

Have you installed the ingest-attachment plugins ?

You should reset the index first and try again:

./occ fullnextsearch:reset
./occ fullnextsearch:index

Ahh ok I did not install the plugin, I will try again, but first I have to find a way to make elasticsearch require less memory, there is also a mock-solr plugin that should work with the dovecot solr plugin.
It’s Fedora 27 Server aarch64 btw.

I installed the plugin, reset the index and started indexing again, now I get a different error:

. Indexing: 6274262B -Murelle HE -IT.pdf
result: {"_index":“erm67_index”,"_type":“files”,"_id":“68719”,"_version":1,“result”:“created”,"_shards":{“total”:2,“successful”:1,“failed”:0},"_seq_no":0,"_primary_term":1}

[OCA\FullNextSearch\Exceptions\DatabaseException]
An exception occurred while executing ‘INSERT INTO oc_fullnextsearch_indexes (owner_id, provider_id, document_id, status, indexed) VALUES(?, ?, ?, ?
, ?)’ with params [“ermanno”, “files”, 64995, 2, 1514637251]:
SQLSTATE[HY000]: General error: 1364 Field ‘err’ doesn’t have a default value

fullnextsearch:index [–output [OUTPUT]]

@erm67: Will be fixed in next update

Hello,
is there any cause why Nextant/Solr will be dropped with the new fullnextsearch?

Can I use my currently installed nextant version with the upcoming nextcloud 13?

best regards,
christian

Mod: ping @Cult

Yes, you can still use Nextant/Solr. You might need to edit the info.xml to have it work on nc13.

Is it planned to leave beta status for nc13 release?

Please don´t get me wrong. Your work on nc and fulltext searching is great. At the moment I just miss the options I know from nextant and the documentation for fullnextsearch is not yet very detailed. I was just a bit confused that nextant got replaced with the new app in beta status.

I will check the new beta and give feedback asap.

Best regards,
Christian

0.4 will be release within the next week, which include features that are not in Nextant: pagination and search options.

Next step will be the CLI output/interface during the index, and importing features from Nextant (mainly filters on file type during index).

Out of curiosity, what features from Nextant v1 is really missing in the v2 ?

I first thought the auto suggest feature of nextant is missing, but I´m not sure if it´s needed. It depends on the query and index mappings. In case of a fuzzy search suggestions won´t be needed.

Indeed one thing is great with v2. No more servlet needed with configuring a path-based proxypass setting in apache/nginx. Routing queries through the apps api is less server maintenance and configuration pain.

Correct me if I’m wrong, but I don’t see whether we can use Solr with FullNextSearch.

Not a big problem to me, I’m only playing with Nextant and just installed the Solr server, so it wouldn’t be very difficult if I changed to Elastisearch, but I’m curious.

Support for Solr will come later :roll_eyes:

I supposed so, but wasn’t sure. Thanks for your effort!

those instructions are not sooooo complete after all lots of missing stuff. for example i installed and successfully indexed however the search form at next cloud is not full text search like nextant used to be. any ideas?