Add ingest-attachment plugin to elasticsearch

tykayn · May 27, 2020, 9:52am

using nextcloud snap 18.0.4, i have no idea how i can install the plugin ingest-attachment. it might be needed to have the search indexing inside my office and pdf files.

installing the elastic search app and file full text search app was not enough to have it working.
i have to setup separately elasticsearch on my ubuntu and set the server to be localhost:9200.
now Elastic seem to be working, when i run the snap occ command i get this:

$ sudo snap run nextcloud.occ fulltextsearch:test

.Testing your current setup:
Creating mocked content provider. ok
Testing mocked provider: get indexable documents. (2 items) ok
Loading search platform. (Elasticsearch) ok
Testing search platform. ok
Locking process ok
Removing test. ok
Pausing 3 seconds 1 2 3 ok
Initializing index mapping. fail
Error detected, unlocking process ok
In IndexService.php line 253:

please add ingest-attachment plugin to elasticsearch

fulltextsearch:test [–output [OUTPUT]] [-j|–json] [-d|–platform_delay PLATFORM_DELAY]

also here is the check command :
$ sudo snap run nextcloud.occ fulltextsearch:check

Full text search 1.4.1

Search Platform:
Elasticsearch 1.5.1
{
“elastic_host”: [
“://localhost:9200”
],
“elastic_index”: “ncloud”,
“fields_limit”: “10000”,
“es_ver_below66”: “0”,
“analyzer_tokenizer”: “standard”
}

Content Providers:
Deck 1.0.1

Files 1.4.2
{
“files_local”: “1”,
“files_external”: “1”,
“files_group_folders”: “1”,
“files_encrypted”: “0”,
“files_federated”: “0”,
“files_size”: “20”,
“files_pdf”: “0”,
“files_office”: “0”,
“files_image”: “0”,
“files_audio”: “0”
}

Django43 · May 27, 2020, 7:59pm

Please try /usr/share/elasticsearch/bin/elasticsearch-plugin install ingest-attachment

see also https://decatec.de/home-server/volltextsuche-in-nextcloud-mit-ocr/

Have a nice day

Django43

tykayn · May 28, 2020, 9:40am

i did install the plugin.
but nextcloud still fails to see it when i check.

$ sudo snap run nextcloud.occ fulltextsearch:test

.Testing your current setup:
Creating mocked content provider. ok
Testing mocked provider: get indexable documents. (2 items) ok
Loading search platform. (Elasticsearch) ok
Testing search platform. ok
Locking process ok
Removing test. ok
Pausing 3 seconds 1 2 3 ok
Initializing index mapping. fail
Error detected, unlocking process ok
In IndexService.php line 253:

please add ingest-attachment plugin to elasticsearch

fulltextsearch:test [–output [OUTPUT]] [-j|–json] [-d|–platform_delay PLATFORM_DELAY]

gargancian · May 30, 2020, 4:54am

Looks like Elasticsearch is not supported in Nextcloud Snap - read this one: https://github.com/nextcloud/nextcloud-snap/issues/708

tykayn · May 31, 2020, 8:52am

i arrived to make the plugin work somehow.
however there is this issue.

thanks @gargancian i will report that to github issues

$ sudo snap run nextcloud.occ fulltextsearch:test

[sudo] password for tykayn:

.Testing your current setup:
Creating mocked content provider. ok
Testing mocked provider: get indexable documents. (2 items) ok
Loading search platform. (Elasticsearch) ok
Testing search platform. ok
Locking process ok
Removing test. ok
Pausing 3 seconds 1 2 3 ok
Initializing index mapping. ok
Indexing generated documents. ok
Pausing 3 seconds 1 2 3 ok
Retreiving content from a big index (license). (size: 32386) ok
Comparing document with source. ok
Searching basic keywords:

‘test’ An unhandled exception has been thrown:
TypeError: Return value of OCA\FullTextSearch\Model\SearchRequest::getProviders() must be of the type array, null returned in /var/snap/nextcloud/21026/nextcloud/extra-apps/fulltextsearch/lib/Model/SearchRequest.php:114
Stack trace:
#0 /var/snap/nextcloud/21026/nextcloud/extra-apps/fulltextsearch/lib/Model/SearchRequest.php(700): OCA\FullTextSearch\Model\SearchRequest->getProviders()
#1 [internal function]: OCA\FullTextSearch\Model\SearchRequest->jsonSerialize()
#2 /var/snap/nextcloud/21026/nextcloud/extra-apps/fulltextsearch_elasticsearch/lib/Service/SearchService.php(103): json_encode(Object(OCA\FullTextSearch\Model\SearchRequest))
#3 /var/snap/nextcloud/21026/nextcloud/extra-apps/fulltextsearch_elasticsearch/lib/Platform/ElasticSearchPlatform.php(336): OCA\FullTextSearch_ElasticSearch\Service\SearchService->searchRequest(Object(Elasticsearch\Client), Object(OCA\FullTextSearch\Model\SearchResult), Object(OC\FullTextSearch\Model\DocumentAccess))
#4 /var/snap/nextcloud/21026/nextcloud/extra-apps/fulltextsearch/lib/Command/Test.php(584): OCA\FullTextSearch_ElasticSearch\Platform\ElasticSearchPlatform->searchRequest(Object(OCA\FullTextSearch\Model\SearchResult), Object(OC\FullTextSearch\Model\DocumentAccess))
#5 /var/snap/nextcloud/21026/nextcloud/extra-apps/fulltextsearch/lib/Command/Test.php(436): OCA\FullTextSearch\Command\Test->search(Object(Symfony\Component\Console\Output\ConsoleOutput), Object(OCA\FullTextSearch_ElasticSearch\Platform\ElasticSearchPlatform), Object(OCA\FullTextSearch\Provider\TestProvider), Object(OC\FullTextSearch\Model\DocumentAccess), ‘test’, Array)
#6 /var/snap/nextcloud/21026/nextcloud/extra-apps/fulltextsearch/lib/Command/Test.php(171): OCA\FullTextSearch\Command\Test->testSearchSimple(Object(Symfony\Component\Console\Output\ConsoleOutput), Object(OCA\FullTextSearch_ElasticSearch\Platform\ElasticSearchPlatform), Object(OCA\FullTextSearch\Provider\TestProvider))
#7 /snap/nextcloud/21026/htdocs/3rdparty/symfony/console/Command/Command.php(255): OCA\FullTextSearch\Command\Test->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#8 /snap/nextcloud/21026/htdocs/core/Command/Base.php(168): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#9 /snap/nextcloud/21026/htdocs/3rdparty/symfony/console/Application.php(915): OC\Core\Command\Base->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#10 /snap/nextcloud/21026/htdocs/3rdparty/symfony/console/Application.php(272): Symfony\Component\Console\Application->doRunCommand(Object(OCA\FullTextSearch\Command\Test), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#11 /snap/nextcloud/21026/htdocs/3rdparty/symfony/console/Application.php(148): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#12 /snap/nextcloud/21026/htdocs/lib/private/Console/Application.php(214): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#13 /snap/nextcloud/21026/htdocs/console.php(99): OC\Console\Application->run()
#14 /snap/nextcloud/21026/htdocs/occ(11): require_once(‘/snap/nextcloud…’)

pohuing · February 25, 2021, 1:03am

As a note to anyone coming by where the app still won’t connect to ES. After installing ES plugins you have to restart ElasticSearch before the ingest-attachment plugin gets loaded.