Fulltextsearch: No response in NC UI

Seems to be related to 53776

What works:
curl -XGET 'localhost:9200/nextcloud/_search?q=schmidt&pretty'

What not:
image
(no response from the fulltextsearch app)

What I tried:

  • deactivate and remove all fulltextsearch apps from NC
  • drop both fulltextsearch tables from database (via cPanel / phpMyAdmin)
  • systemctl stop elasticsearch (ssh root@…)
  • delete /var/lib/elasticsearch/nodes/0
  • systemctl start elasticsearch
  • download and activate all fulltextsearch apps
  • configure fulltextsearch
  • ssh to your nextcloud (not root), occ fulltextsearch:stop, :reset, :test (ok); :index

(credits to @RussianNeuroMancer from the forum here)

There are 350 GB of meaningful data (no large photo collections). It used to work until the NC 17 upgrade.

Nextcloud 17.0.2, PHP 7.2.23, Full Text Search 1.3.6, Full Text Search Elastic Platform 1.4.0, Full Text Search Files 1.3.6, Full Text Search Files Tesseract 1.3.1, Elasticsearch 6.8.5-1

Problem still persists even after upgrade to

Nextcloud 18.0.3
Full text search 1.4.1
Full text search - Elasticsearch Platform 1.5.1
Full text search - Files 1.4.2
Full text search - Files - Tesseract OCR 1.4.1

Any help will be greatly appreciated.

please run:

./occ fulltextsearch:test
./occ fulltextsearch:check

and post the results

php occ fulltextsearch:test

.Testing your current setup:  
Creating mocked content provider. ok  
Testing mocked provider: get indexable documents. (2 items) ok  
Loading search platform. (Elasticsearch) ok  
Testing search platform. ok  
Locking process ok  
Removing test. ok  
Pausing 3 seconds 1 2 3 ok  
Initializing index mapping. ok  
Indexing generated documents. ok  
Pausing 3 seconds 1 2 3 ok  
Retreiving content from a big index (license). (size: 32386) ok  
Comparing document with source. ok  
Searching basic keywords:  
 - 'test' (result: 1, expected: ["simple"]) ok  
 - 'document is a simple test' (result: 2, expected: ["simple","license"]) ok  
 - '"document is a test"' (result: 0, expected: []) ok  
 - '"document is a simple test"' (result: 1, expected: ["simple"]) ok  
 - 'document is a simple -test' (result: 1, expected: ["license"]) ok  
 - 'document is a simple +test' (result: 1, expected: ["simple"]) ok  
 - '-document is a simple test' (result: 0, expected: []) ok  
Updating documents access. ok  
Pausing 3 seconds 1 2 3 ok  
Searching with group access rights:  
 - 'license' - [] -  (result: 0, expected: []) ok  
 - 'license' - ["group_1"] -  (result: 1, expected: ["license"]) ok  
 - 'license' - ["group_1","group_2"] -  (result: 1, expected: ["license"]) ok  
 - 'license' - ["group_3","group_2"] -  (result: 1, expected: ["license"]) ok  
 - 'license' - ["group_3"] -  (result: 0, expected: []) ok  
Searching with share rights:  
 - 'license' - notuser -  (result: 0, expected: []) ok  
 - 'license' - user2 -  (result: 1, expected: ["license"]) ok  
 - 'license' - user3 -  (result: 1, expected: ["license"]) ok  
Removing test. ok  
Unlocking process ok  

php occ fulltextsearch:check
Full text search 1.4.1

- Search Platform:
Elasticsearch 1.5.1
{
    "elastic_host": [
        "http://localhost:9200"
    ],
    "elastic_index": "nextcloud",
    "fields_limit": "10000",
    "es_ver_below66": "0",
    "analyzer_tokenizer": "standard"
} 
 
- Content Providers:
Files 1.4.2
{
    "files_local": "1",
    "files_external": "1",
    "files_group_folders": "1",
    "files_encrypted": "0",
    "files_federated": "0",
    "files_size": "45",
    "files_pdf": "1",
    "files_office": "1",
    "files_image": "0",
    "files_audio": "0",
    "files_fulltextsearch_tesseract": {
        "version": "1.4.1",
        "enabled": "1",
        "psm": "2",
        "lang": "eng,fra,deu",
        "pdf": "0",
        "pdf_limit": "0"
    }
}

As I wrote above, a curl request from the console works, e.g.:

curl -XGET 'localhost:9200/nextcloud/_search?q=schmidt&pretty'

The error could be fixed in the meantime. The problem was that somehow the initial indexing process for the files_fulltextsearch didn’t write into the appconfig table. Usually if the process is finished, a dataset like this is written:

appid configkey configvalue
fulltextsearch provider_indexed {“files”:“1”}

Because this entry was missing the files-configvalue the corresponding files_fulltextsearch-provider was skipped when the fulltextsearch was trying to create all indexed and available providers to satisfy the search-request.

The fix in this case was very stupid: just add the "files":"1"value to the mentioned JSON. I created a pull-request on github to write to the logfile in that case (see https://github.com/nextcloud/fulltextsearch/pull/578). This sould improve troubleshooting in case someone else is facing this issue.

2 Likes

The entry was indeed missing from my DB. I then added it and restarted the server. Unfortunately, this did not solve the problem for me. The search results still remain empty. Do I have to create the index again?

You should not update the provider_indexed entry yourself. Have you run:

./occ fulltextsearch:check
./occ fulltextsearch:test
./occ fulltextsearch:index

?

Yes. The full-text search ran for weeks without any problems. Then at some point we had problems with our connected “archive”. The archive is on an externally connected NAS while the Nextcloud itself is on a full SSD storage.

The contents of the NAS were also fully indexed, which I think was a mistake due to the speed of the NAS.

I then tried everything:

Rebuilding the index via
./occ fulltextsearch:reset
./occ fulltextsearch:index

Then I completely removed elasticsearch and all plugins, reinstalled them and rebuilt the index.

Then I found the thread here, which unfortunately didn’t help me either.

Full text search 22.0.1

- Search Platform:
Elasticsearch 22.0.1 (Selected)
{
    "elastic_host": [
        "http://localhost:9200"
    ],
    "elastic_index": "local_index",
    "fields_limit": "10000",
    "es_ver_below66": "0",
    "analyzer_tokenizer": "standard"
}

- Content Providers:
Deck 1.5.5
[]
Files 22.0.1
{
    "files_local": "1",
    "files_external": "2",
    "files_group_folders": "1",
    "files_encrypted": "0",
    "files_federated": "0",
    "files_size": "25",
    "files_pdf": "1",
    "files_office": "1",
    "files_image": "0",
    "files_audio": "0",
    "files_fulltextsearch_tesseract": {
        "version": "22.0.0",
        "enabled": "1",
        "psm": "4",
        "lang": "eng,deu",
        "pdf": "1",
        "pdf_limit": "0"
    }
}

.Testing your current setup:
Creating mocked content provider. ok
Testing mocked provider: get indexable documents. (2 items) ok
Loading search platform. (Elasticsearch) ok
Testing search platform. ok
Locking process ok
Removing test. ok
Pausing 3 seconds 1 2 3 ok
Initializing index mapping. ok
Indexing generated documents. ok
Pausing 3 seconds 1 2 3 ok
Retreiving content from a big index (license). (size: 32386) ok
Comparing document with source. ok
Searching basic keywords:
 - 'test' (result: 1, expected: ["simple"]) ok
 - 'document is a simple test' (result: 2, expected: ["simple","license"]) ok
 - '"document is a test"' (result: 0, expected: []) ok
 - '"document is a simple test"' (result: 1, expected: ["simple"]) ok
 - 'document is a simple -test' (result: 1, expected: ["license"]) ok
 - 'document is a simple +test' (result: 1, expected: ["simple"]) ok
 - '-document is a simple test' (result: 0, expected: []) ok
 - 'document is a simple +test +testing' (result: 1, expected: ["simple"]) ok
 - 'document is a simple +test -testing' (result: 0, expected: []) ok
 - 'document is a +simple -test -testing' (result: 0, expected: []) ok
 - '+document is a simple -test -testing' (result: 1, expected: ["license"]) ok
 - 'document is a +simple -license +testing' (result: 1, expected: ["simple"]) ok
Updating documents access. ok
Pausing 3 seconds 1 2 3 ok
Searching with group access rights:
 - 'license' - [] -  (result: 0, expected: []) ok
 - 'license' - ["group_1"] -  (result: 1, expected: ["license"]) ok
 - 'license' - ["group_1","group_2"] -  (result: 1, expected: ["license"]) ok
 - 'license' - ["group_3","group_2"] -  (result: 1, expected: ["license"]) ok
 - 'license' - ["group_3"] -  (result: 0, expected: []) ok
Searching with share rights:
 - 'license' - notuser -  (result: 0, expected: []) ok
 - 'license' - user2 -  (result: 1, expected: ["license"]) ok
 - 'license' - user3 -  (result: 1, expected: ["license"]) ok
Removing test. ok
Unlocking process ok

The search results still remain empty

Which JSON file, exactly, did you add this to?
I just setup a new NC server and installed Elasticsearch. Also, just like the OP, all my tests and checks work fine, however, I don’t get any results in the web UI. Again, like the OP, it works when running from the CLI.

did anyone find a solution to this issue?