Bug: Fulltextsearch_elasticsearch 31.0.1 needs Elasticsearch 9

Support intro

Sorry to hear you’re facing problems. :slightly_frowning_face:

The community help forum (help.nextcloud.com) is for home and non-enterprise users. Support is provided by other community members on a best effort / “as available” basis. All of those responding are volunteering their time to help you.

If you’re using Nextcloud in a business/critical setting, paid and SLA-based support services can be accessed via portal.nextcloud.com where Nextcloud engineers can help ensure your business keeps running smoothly.

Getting help

In order to help you as efficiently (and quickly!) as possible, please fill in as much of the below requested information as you can.

Before clicking submit: Please check if your query is already addressed via the following resources:

(Utilizing these existing resources is typically faster. It also helps reduce the load on our generous volunteers while elevating the signal to noise ratio of the forums otherwise arising from the same queries being posted repeatedly).

Some or all of the below information will be requested if it isn’t supplied; for fastest response please provide as much as you can. :heart:

The Basics

  • Nextcloud Server version (e.g., 29.x.x):
    • 31.0.12
  • Operating system and version (e.g., Ubuntu 24.04):
    • Ubuntu 24
  • Web server and version (e.g, Apache 2.4.25):
    • Apache 2
  • Reverse proxy and version _(e.g. nginx 1.27.2)
    • No
  • PHP version (e.g, 8.3):
    • 8.3
  • Is this the first time you’ve seen this error? (Yes / No)
  • Yes
  • When did this problem seem to first start?
    • During live deployment
  • Installation method (e.g. AlO, NCP, Bare Metal/Archive, etc.)
    • VM
  • Are you using CloudfIare, mod_security, or similar? (Yes / No)
    • No

Summary of the issue you are facing:

Full text search working nicely in a sandpit with a disposable Nextcloud and ElasticSearch 8 server. Built real ElasticSearch 8 server and deployed full text search on a dedicated test Nextcloud and also worked fine.

Started deployment of full text search on my live Nextcloud and after debugging typos the occ:fulltextsearch:test kept failing. Error in the nextcloud log:

[fulltextsearch_elasticsearch] Warning: 400 Bad Request: {“error”:{“root_cause”:[{“type”:“media_type_header_exception”,“reason”:“Invalid media-type value on headers [Content-Type, Accept]”}],“type”:“media_type_header_exception”,“reason”:“Invalid media-type value on headers [Content-Type, Accept]”,“caused_by”:{“type”:“status_exception”,“reason”:“Accept version must be either version 8 or 7, but found 9. Accept=application/vnd.elasticsearch+json; compatible-with=9”}},“status”:400}

Fulltext search had updated in the app store from 31.0.0 to 31.0.1 an hour before I did the live deployment. Test and Live share the same Elasticsearch so that’s common to both.

According to the doco on https://github.com/nextcloud/fulltextsearch_elasticsearchAs of version 26.0.0 this app is only compatible with Elasticsearch 8

I presume this is an oops and not a design decision. Mandating a reinstall of ElasticSearch is not something I would expect as an undocumented feature of a point release.

At the very least this should be a config option if it’s not possible to look up the version of the elastic search server.

Downgrading the app to 31.0.0 (not Nextcloud!) gets me out of this hole for now, but of course has implications for future Nextcloud patches. And I was starting to consider NC32 over the Christmas break too…

Steps to replicate it (hint: details matter!):

  1. Install fulltextsearch_elastic 30.0.1 search from app store

  2. Configure against ElasticSearch 8.19.8 that’s only week old

  3. occ fulltextsearch:test will fail

  4. Manually downgrade fulltextsearch_elasticsearch to 30.0.0 to match test system

  5. occ fulltextsearch:test now works

Log entries

Nextcloud

Please provide the log entries from your Nextcloud log that are generated during the time of problem (via the Copy raw option from Administration settings->Logging screen or from your nextcloud.log located in your data directory). Feel free to use a pastebin/gist service if necessary.

[fulltextsearch_elasticsearch] Warning: 400 Bad Request: {“error”:{“root_cause”:[{“type”:“media_type_header_exception”,“reason”:“Invalid media-type value on headers [Content-Type, Accept]”}],“type”:“media_type_header_exception”,“reason”:“Invalid media-type value on headers [Content-Type, Accept]”,“caused_by”:{“type”:“status_exception”,“reason”:“Accept version must be either version 8 or 7, but found 9. Accept=application/vnd.elasticsearch+json; compatible-with=9”}},“status”:400}

Web Browser

If the problem is related to the Web interface, open your browser inspector Console and Network tabs while refreshing (reloading) and reproducing the problem. Provide any relevant output/errors here that appear.

NA

Web server / Reverse Proxy

The output of your Apache/nginx/system log in /var/log/____:

NA

Configuration

Nextcloud

The output of occ config:list system or similar is best, but, if not possible, the contents of your config.php file from /path/to/nextcloud is fine (make sure to remove any identifiable information!):

NA

Apps

The output of occ app:list (if possible).

Tips for increasing the likelihood of a response

  • Use the preformatted text formatting option in the editor for all log entries and configuration output.
  • If screenshots are useful, feel free to include them.
    • If possible, also include key error output in text form so it can be searched for.
  • Try to edit log output only minimally (if at all) so that it can be ran through analyzers / formatters by those trying to help you.

I do see the change was made and also backported. I’m not sure what the reasoning was. Can you create an issue in the GitHub repo? It won’t necessarily get seen by the right folks here on the community forum (at least on a timely basis).

Same here, but:

You can simply upgrade elasticsearch to version 9.

The latest elasticsearch version 8 point release seamlessly upgrades to the latest version 9 point release, this even applies to the versions mentionned at readonlyrest.com, which I use to secure it a little.

All you have to do is to change the 8.x in your repository configuration into 9.x, then apt-get update and then you can upgrade elasticsearch.

Elasticsearch version 9 even works with fulltextsearch_elasticsearch app version 31.0.0.

It just seems to be a matter of documentation as the sentence on GitHub doesn’t seem to be reviewed after app version 26.

So, basically, you’re done within five minutes - they just didn’t announce this “minor unimportant change” that breaks part of your system. :wink:

But hey, having a job and developping an app in your spare time probably won’t make you a documentation nazi.

1 Like

Be careful about this. If you are using elasticsearch since 7.x, you’ll have to do some upgrade steps (index versions are probably outdated). Easiest way is to use kibana upgrade assistant.
Be sure to make a backup first before upgrading ;).

Hello I have this problem with Nextcloud AIO

Nextcloud AIO v12.3.0
Nextcloud Hub 25 Autumn (32.0.3)

The installation is new (now with 8 GB RAM) but the restored Nextcloud borg backup was from a Synology without SECCOMP and Full text search. Hence, there has never been a full text search. And I was keen to get it.

I wonder why AIO delivers “8.19.8” - if the compatibility requests version 9.
What is wrong: compatibility header or version of ‘nextcloud-aio-fulltextsearch‘?
Or something else I’m overlooking?
Should I change the API_COMPATIBILITY_HEADER?

me@rpitrix5-8G:~ $ sudo docker exec -it nextcloud-aio-fulltextsearch curl -s -X GET “http://localhost:9200” | grep number
“number” : “8.19.8”,

me@rpitrix5-8G:~ $ sudo docker exec -it nextcloud-aio-nextcloud grep -r “compatible-with=9” /var/www/html/custom_apps/fulltextsearch_elasticsearch
/var/www/html/custom_apps/fulltextsearch_elasticsearch/lib/Vendor/Elastic/Elasticsearch/Client.php: const API_COMPATIBILITY_HEADER = ‘%s/vnd.elasticsearch+%s; compatible-with=9’;

Hello,
I could solve the issue for my AIO by setting ‘compatible-with=9’ to ‘compatible-with=8’.

The correction of the required version could be done with this command:
sudo docker exec -it nextcloud-aio-nextcloud sed -i ‘s/compatible-with=9/compatible-with=8/g’ /var/www/html/custom_apps/fulltextsearch_elasticsearch/lib/Vendor/Elastic/Elasticsearch/Client.php

After that I checked if the 9 was replaced by 8 correctly:
sudo docker exec -it nextcloud-aio-nextcloud grep -r “compatible-with=” /var/www/html/custom_apps/fulltextsearch_elasticsearch
/var/www/html/custom_apps/fulltextsearch_elasticsearch/lib/Vendor/Elastic/Elasticsearch/Client.php: const API_COMPATIBILITY_HEADER = ‘%s/vnd.elasticsearch+%s; compatible-with=8’;

Then the point ‘Testing search platform.’ was ok:
me:~$ sudo docker exec --user www-data -it nextcloud-aio-nextcloud php occ fulltextsearch:test

.Testing your current setup:
Creating mocked content provider. ok
Testing mocked provider: get indexable documents. (2 items) ok
Loading search platform. (Elasticsearch) ok
Testing search platform. ok
Locking process ok
Removing test. ok
Pausing 3 seconds 1 2 3 ok
Initializing index mapping. ok
Indexing generated documents. ok
Pausing 3 seconds 1 2 3 ok
Retreiving content from a big index (license). (size: 32386) ok
Comparing document with source. ok
Searching basic keywords:

  • ‘test’ (result: 1, expected: [“simple”]) ok
  • ‘document is a simple test’ (result: 2, expected: [“simple”,“license”]) ok
  • …..

and the indexing could be started.

1 Like

Thanks for this fix. I was doing setup for the first time and could not figure out what why fulltextsearch was not passing the tests. Although it’s a bit of a hacky solution, I’m hoping this gets hotfixed.

“Thanks for this fix. I was doing setup” Same here did the trick before switch to ES 9 on Ubuntu 24 vm apache…Happy new year :wink: