Fulltextsearch v0.7 is Out!

🍱 Features & apps fulltextsearch
1

And you can search more content, with (almost) one click !

Selection_012
Selection_0121489Ă—540 26.6 KB

The main idea of the Fulltextsearch app is to provide a complete library and API for other apps to index their content. Fulltextsearch displays a search box to those apps, but also provide a page that centralize the search to all indexed content.

Today, we released 4 apps:

  • Fulltextsearch
    The core app

  • Fulltextsearch - Elasticsearch (search platform)
    The gateway for the Elasticsearch servlet

  • Fulltextsearch - Files (content provider)
    Index the content of your files, works with local, external files and group folders

  • Fulltextsearch - Bookmarks (content provider)
    Index the content of the web pages added to the bookmarks app

Important note

It is strongly adviced to disable/uninstall any old version of the app before proceding to the upgrade

After the upgrade, a reset and a full index is necessary:

 ./occ fulltextsearch:reset
 ./occ fulltextsearch:index

Changelog

Fulltextsearch

  • A navigation page to search within all content from the providers.
  • Fixing the display of the search results in case of multiple page.
  • The content (index and search) can be splited in Parts for a pinpointed search.
  • Rework on the exchange between platform and content providers.

Fulltextsearch - Elasticsearch

  • Improvement in the index removing

Fulltextsearch - Files

  • Added the option to limit the search to the current directory.
  • Added the option to filter by file extension.
  • Improved index and search within local files, external filesystem and group folders.
  • Changing the available sources (in the settings) does not require a full re-index anymore.
  • Bugfix: compat with strange username in shared name index.

General

  • Smaller improvement/bugfixes
5 Likes
2

I can’t start the worker service?

fulltextsearch-service
fulltextsearch-service982Ă—181 24 KB

3

the Live feature is only available on NC14 as it was locking file on previous version. However, I don’t know anything related to that Worker

4

I’m just starting it as a service, but it’s disabled for now until NC 14 comes out.

I have another problem with Fulltexsearch 0.7: After resetting the index i tried to re-index it, but it seems that is not possible anymore.

I found a big file with ~780 MB after the crash: /var/lib/elasticsearch/java_pid7705.hprof

And this is the output of /var/lib/elasitcsearch/elasticsearch.log:

Maybe it needs more memory? I have installed 8 GB RAM.

[2018-05-17T07:58:02,243][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [] fatal error in thread [elasticsearch[gbqajo8][bulk][T#1]], exiting
java.lang.OutOfMemoryError: Java heap space
        at java.util.Arrays.copyOfRange(Arrays.java:3664) ~[?:1.8.0_171]
        at java.lang.String.<init>(String.java:207) ~[?:1.8.0_171]
        at java.lang.StringBuilder.toString(StringBuilder.java:407) ~[?:1.8.0_171]
        at com.fasterxml.jackson.core.util.TextBuffer.contentsAsString(TextBuffer.java:356) ~[jackson-core-2.8.10.jar:2.8.10]
        at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishAndReturnString(UTF8StreamJsonParser.java:2470) ~[jackson-core-2.8.10.jar:2.8.10]
        at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.getText(UTF8StreamJsonParser.java:315) ~[jackson-core-2.8.10.jar:2.8.10]
        at org.elasticsearch.common.xcontent.json.JsonXContentParser.text(JsonXContentParser.java:83) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.common.xcontent.support.AbstractXContentParser.readValue(AbstractXContentParser.java:385) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.common.xcontent.support.AbstractXContentParser.readMap(AbstractXContentParser.java:333) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.common.xcontent.support.AbstractXContentParser.readMap(AbstractXContentParser.java:296) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.common.xcontent.support.AbstractXContentParser.map(AbstractXContentParser.java:251) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.common.xcontent.XContentHelper.convertToMap(XContentHelper.java:135) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.common.xcontent.XContentHelper.convertToMap(XContentHelper.java:108) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.action.index.IndexRequest.sourceAsMap(IndexRequest.java:294) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.ingest.PipelineExecutionService.innerExecute(PipelineExecutionService.java:167) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.ingest.PipelineExecutionService.access$000(PipelineExecutionService.java:42) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.ingest.PipelineExecutionService$2.doRun(PipelineExecutionService.java:94) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:672) ~[elasticsearch-6.2.4.jar:6.2.4]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-6.2.4.jar:6.2.4]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_171]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_171]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]
5

How much heap space did you configure?
I am using elasticsearch docker image with environment variable "ES_JAVA_OPTS=-Xms1g -Xmx1g" giving it 1GB of heap space. You might need more depending on the size of your data.

6

I don’t have docker therefore i had to edit jvm.options

#-Xms1g
#-Xmx1g
-Xms4g
-Xmx4g

Now it’s working.

Thank you!