Using nc16 fulltextsearch with elasticsearch external node

🚧 Installation
1

Hi nextcloudies,

i’ am try to use fulltextsearch /elasticsearch with a external fulltextsearch node. Curl tells me
that everything is fine …

{
“name” : “”,
“cluster_name” : “elasticsearch”,
“cluster_uuid” : “LSpyel10TIaamdTbhDYmEg”,
“version” : {
“number” : “7.0.1”,
“build_flavor” : “default”,
“build_type” : “deb”,
“build_hash” : “e4efcb5”,
“build_date” : “2019-04-29T12:56:03.145736Z”,
“build_snapshot” : false,
“lucene_version” : “8.0.0”,
“minimum_wire_compatibility_version” : “6.7.0”,
“minimum_index_compatibility_version” : “6.0.0-beta1”
},

and php occ fulltextsearch:test tells me:

.Testing your current setup:
Creating mocked content provider. ok
Testing mocked provider: get indexable documents. (2 items) ok
Loading search platform. (Elasticsearch) ok
Testing search platform. ok
Locking process ok
Removing test. ok
Pausing 3 seconds 1 2 3 ok

Initializing index mapping. fail

In Connection.php line 615:

{“error”:{“root_cause”:[{“type”:“mapper_parsing_exception”,“reason”:“Root mapping definition has unsupported parameters: [standard : {dynamic=true, properties={owner={type=keyword}, groups={type=keyword}, circles={type=keyw
ord}, metatags={type=keyword}, source={type=keyword}, title={copy_to=combined, analyzer=keyword, term_vector=yes, type=text}, content={copy_to=combined, analyzer=analyzer, term_vector=yes, type=text}, users={type=keyword}, t
ags={type=keyword}, provider={type=keyword}, subtags={type=keyword}, links={type=keyword}, combined={analyzer=analyzer, term_vector=yes, type=text}, hash={type=keyword}}}]”}],“type”:“mapper_parsing_exception”,“reason”:“Faile
d to parse mapping [_doc]: Root mapping definition has unsupported parameters: [standard : {dynamic=true, properties={owner={type=keyword}, groups={type=keyword}, circles={type=keyword}, metatags={type=keyword}, source={typ
e=keyword}, title={copy_to=combined, analyzer=keyword, term_vector=yes, type=text}, content={copy_to=combined, analyzer=analyzer, term_vector=yes, type=text}, users={type=keyword}, tags={type=keyword}, provider={type=keyword
}, subtags={type=keyword}, links={type=keyword}, combined={analyzer=analyzer, term_vector=yes, type=text}, hash={type=keyword}}}]”,“caused_by”:{“type”:“mapper_parsing_exception”,“reason”:“Root mapping definition has unsuppor
ted parameters: [standard : {dynamic=true, properties={owner={type=keyword}, groups={type=keyword}, circles={type=keyword}, metatags={type=keyword}, source={type=keyword}, title={copy_to=combined, analyzer=keyword, term_vec
tor=yes, type=text}, content={copy_to=combined, analyzer=analyzer, term_vector=yes, type=text}, users={type=keyword}, tags={type=keyword}, provider={type=keyword}, subtags={type=keyword}, links={type=keyword}, combined={anal
yzer=analyzer, term_vector=yes, type=text}, hash={type=keyword}}}]”}},“status”:400}

If i try to start indexing, this appears:

Options: []
Memory:
┌─ Indexing ────
│ Action:
│ Provider: Account:
│ Document:
│ Info:
│ Title:
│ Content size:
│ Chunk: /
│ Progress: /
└──
┌─ Results ────
│ Result: 0/0
│ Index:
│ Status:
│ Message:
│
│
└──
┌─ Errors ────
│ Error: 0/0
│ Index:
│ Exception:
│ Message:
│
│
└──

x:first result ## c/v:prec/next result ## b:last result

f:first error ## h/j:prec/next error ## d:delete error ## l:last error

q:quit ## p:pause

In Connection.php line 615:
(the same as above …)

Whats happened there? Is it possible to use a external elasticsearch node?

Thanks in advance.

2

Yeah, it works AND the mistake is in front of the keyboard …

Causes:

  1. Install elasticsearch 6.7 (NOT 7.x)
  2. add / modify network.host: to /etc/elasticsearch/elasticsearch.yml
3

And which entries did you add/modify on elasticsearch.yml??

4

network.host must be the (from your nextcloud instance reachable) ip v4 address.

5

That’s it? Because the elasticsearch service won’t start if I delete the “#”.
I have to add
“node.master: true
node.data: true
transport.host: localhost
transport.tcp.port: 9300”
to bring the service up running.

How do you tell NC to use this elasticsearch instance? I have an automated “sudo -u www-data php /var/www/nextcloud/occ fulltextsearch:index” running and it doesn’t use this external instance.
Beforehand I used to use an internal instance, but out of the sudden it stopped working.

it tells me now " Exception: Elasticsearch\Common\Exceptions\NoNodesAvailableException
│ Message: No alive nodes found in your cluster"
but curl works

6

Its simply to configure and start. Please tell a little more about your config like Linux Distro, Snap, Docker, native install, elasticsearch version and so on.

I am using a elasticsearch 6.8 on my host and a complete docker based config for nextcloud and lool.

7

Sure.
I have a Debian 9.9 LXC-Container running Nextcloud (the Turnkey-Application) within Proxmox 5.4.
My elasticsearch (v 7.2) is now running within a Ubuntu 19.04 unprivileged LXC-Container.

cheers

8

see above: elasticsearch 6.x and NOT 7.x

9

I tried it now with elasticsearch 6.7. But if I enable the network.host with 192.168.100.40 (the IP of the elasticsearch-CT) the elasticsearch service won’t start. I still need to add these lines:

to even get the service running.

If I run occ fulltextsearch:index at the NC-instance (192.168.100.41) it starts and is indexing, but with the message:
“Exception: Elasticsearch\Common\Exceptions\NoNodesAvailableException
Message: No alive nodes found in your cluster”
And there is no traffic between the ES (192.168.100.40) and NC-instance (192.168.100.41).

10

Update:
it connects to the elasticsearch, because in the log is written:
“[o.e.c.m.MetaDataCreateIndexService] [vakS0xp] [mycloudindex2] creating index, cause [api], templates [], shards [5]/[1], mappings [standard]”
And I just changed the index to “mycloudindex2”. BUT the occ fulltextsearch:index still tells me “no node alive”.

11

This config can not work.

And i see there are some misunderstoods about the concepts, ip’s and so on. First of all be sure your esearch and your nc docker instances can communicate between ip’s and ports. To do this, install as root relavated packages and tools like ping or nmap.

Then you should start your esearch container with a custom volume (for example /etc/elasticsearch) to persist config changes.

After this you should be extended the log information for your esearch and your nextcloud container. If the ip of the nextcloud container does not exist in the elasticsearch.yml the service can not be connected by the nextcloud instance.

You have extended possibilities to test your config with the occ command “fulltextsearch:test”
If this test is not ok you never can build a proper index.