fulltextsearch with elasticsearch test fail: Search platform (Elasticsearch) down

Support intro

Sorry to hear you’re facing problems. :slightly_frowning_face:

The community help forum (help.nextcloud.com) is for home and non-enterprise users. Support is provided by other community members on a best effort / “as available” basis. All of those responding are volunteering their time to help you.

If you’re using Nextcloud in a business/critical setting, paid and SLA-based support services can be accessed via portal.nextcloud.com where Nextcloud engineers can help ensure your business keeps running smoothly.

Getting help

In order to help you as efficiently (and quickly!) as possible, please fill in as much of the below requested information as you can.

Before clicking submit: Please check if your query is already addressed via the following resources:

(Utilizing these existing resources is typically faster. It also helps reduce the load on our generous volunteers while elevating the signal to noise ratio of the forums otherwise arising from the same queries being posted repeatedly).

Some or all of the below information will be requested if it isn’t supplied; for fastest response please provide as much as you can. :heart:

The Basics

  • Nextcloud Server version (e.g., 29.x.x):
  • Operating system and version (e.g., Ubuntu 24.04):
    • Debian13with Docker version 28.5.1, build e180ab8
  • Web server and version (e.g, Apache 2.4.25):
    • nextcloud:latest (apache image)
  • Reverse proxy and version _(e.g. nginx 1.27.2)
    • traefik:latest
  • PHP version (e.g, 8.3):
    • PHP 8.3.27
  • Is this the first time you’ve seen this error? (Yes / No):
    • Yes
  • When did this problem seem to first start?
    • After installing Full text search, Full text search - files, and Full text search - elasticsearch
  • Installation method (e.g. AlO, NCP, Bare Metal/Archive, etc.)
    • Docker stack in swarm mode
  • Are you using CloudfIare, mod_security, or similar? (Yes / No)
    • Usinng traefik crowdsec bouncer plugin

Summary of the issue you are facing:

Within nextcloud container, it can communicate with elasticsearch container.

www-data@e0e85328d571:~/html$ curl -v http://elasticsearch:9200
* Uses proxy env variable NO_PROXY == 'localhost,127.0.0.1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16'
* Host elasticsearch:9200 was resolved.
* IPv6: (none)
* IPv4: 10.0.9.78
*   Trying 10.0.9.78:9200...
* Connected to elasticsearch (10.0.9.78) port 9200
* using HTTP/1.x
> GET / HTTP/1.1
> Host: elasticsearch:9200
> User-Agent: curl/8.14.1
> Accept: */*
> 
* Request completely sent off
< HTTP/1.1 200 OK
< X-elastic-product: Elasticsearch
< content-type: application/json
< content-length: 539
< 
{
  "name" : "ea0d65729450",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "YYA7eCQ_T1a6bwmVn162GQ",
  "version" : {
    "number" : "8.6.1",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "180c9830da956993e59e2cd70eb32b5e383ea42c",
    "build_date" : "2023-01-24T21:35:11.506992272Z",
    "build_snapshot" : false,
    "lucene_version" : "9.4.2",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}
* Connection #0 to host elasticsearch left intact

But when I ran the test command, it would be failed.

www-data@e0e85328d571:~/html$ php occ fulltextsearch:test -vvv
 
.Testing your current setup:  
Creating mocked content provider. ok  
Testing mocked provider: get indexable documents. (2 items) ok  
Loading search platform. (Elasticsearch) ok  
Testing search platform. fail 
In Test.php line 238:
                                          
  [Exception]                             
  Search platform (Elasticsearch) down ?  
                                          

Exception trace:
  at /var/www/html/apps/fulltextsearch/lib/Command/Test.php:238
 OCA\FullTextSearch\Command\Test->testLoadingPlatform() at /var/www/html/apps/fulltextsearch/lib/Command/Test.php:92
 OCA\FullTextSearch\Command\Test->execute() at /var/www/html/3rdparty/symfony/console/Command/Command.php:326
 Symfony\Component\Console\Command\Command->run() at /var/www/html/core/Command/Base.php:218
 OC\Core\Command\Base->run() at /var/www/html/3rdparty/symfony/console/Application.php:1078
 Symfony\Component\Console\Application->doRunCommand() at /var/www/html/3rdparty/symfony/console/Application.php:324
 Symfony\Component\Console\Application->doRun() at /var/www/html/3rdparty/symfony/console/Application.php:175
 Symfony\Component\Console\Application->run() at /var/www/html/lib/private/Console/Application.php:187
 OC\Console\Application->run() at /var/www/html/console.php:90
 require_once() at /var/www/html/occ:33

fulltextsearch:test [--output [OUTPUT]] [-j|--json] [-d|--platform_delay PLATFORM_DELAY]

And I aslo setup tcpdump in elasticsearch container, and it could capture TCP traffic and generate dump file when I ran curl -v http://elasticsearch:9200 in nextcloud container. But tcpdump didn’t trigger to capture the traffic and log the dump file when I ran php occ fulltextsearch:test -vvv in nextcloud container.

Steps to replicate it (hint: details matter!):

  1. Deploy the latest nextcloud docker image

  2. Deploy elasticsearch official image with version 8.1.x, or 8.6.1, or 7.1.x

  3. Install apps of Full text search, Full text search - files, Full text search - elasticsearch

  4. Configure the full text search in the admin web ui

Log entries

Nextcloud

Checked the log, there is no log entry for full text search app.

Configuration

Nextcloud

The output of occ config:list system or similar is best, but, if not possible, the contents of your config.php file from /path/to/nextcloud is fine (make sure to remove any identifiable information!):

www-data@e0e85328d571:~/html$ php occ config:list system
{
    "system": {
        "default_phone_region": "CN",
        "maintenance_window_start": 3,
        "htaccess.RewriteBase": "\/",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "loglevel": 0,
        "appstore_curl_timeout": 600,
        "appstoreenabled": true,
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "password": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "upgrade.disable-web": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "nextcloud.mydomain:port"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "32.0.1.2",
        "overwrite.cli.url": "http:\/\/nextcloud.mydomain:port",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "overwritehost": "nextcloud.mydomain:port",
        "overwriteprotocol": "https",
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "maintenance": false,
        "mail_smtpmode": "smtp",
        "mail_sendmailmode": "smtp",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "465",
        "mail_smtpauth": true,
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpsecure": "ssl",
        "mail_smtptimeout": 30,
        "app_install_overwrite": []
    }
}

www-data@e0e85328d571:~/html$ php occ fulltextsearch:configure
{
    "app_navigation": true,
    "search_platform": "OCA\\FullTextSearch_Elasticsearch\\Platform\\ElasticSearchPlatform",
    "collection_internal": "local",
    "cron_err_reset": 1763363717,
    "tick_ttl": 1800,
    "collection_indexing_list": 50,
    "collection_links": []
}

www-data@e0e85328d571:~/html$ php occ fulltextsearch_elasticsearch:configure
{
    "fields_limit": 10000,
    "elastic_host": "http:\/\/elasticsearch:9200",
    "elastic_index": "my_index",
    "elastic_logger_enabled": true,
    "analyzer_tokenizer": "standard",
    "allow_self_signed_cert": false
}

Docker stack file

version: '3.8'

services:
  nextcloud-db:
    image: mariadb:lts
    command: --transaction-isolation=READ-COMMITTED --log-bin=ROW --skip-innodb-read-only-compressed --max_allowed_packet=2048M --wait_timeout=28800 --interactive_timeout=28800
    volumes:
      - /mnt/nextcloud-db-data:/var/lib/mysql
    environment:
      - MYSQL_DATABASE=nextcloud
      - MYSQL_USER=nextcloud
      - MYSQL_ROOT_PASSWORD=mypassword 
      - MYSQL_PASSWORD_FILE=/run/secrets/nc_db_user_pass
    secrets:
      - nc_db_user_pass
    networks:
      - nextcloud
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.hostname == worker02
      restart_policy:
        condition: on-failure

  nextcloud-redis:
    image: redis:alpine
    networks:
      - nextcloud
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 5s
      retries: 5
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.role == worker
      restart_policy:
        condition: on-failure

  nextcloud-app:
    image: nextcloud:latest
    depends_on:
      - nextcloud-db
      - nextcloud-redis
    environment:
      - REDIS_HOST=nextcloud-redis
      - MYSQL_HOST=nextcloud-db
      
      - MYSQL_USER=nextcloud
      - MYSQL_DATABASE=nextcloud
      - MYSQL_PASSWORD_FILE=/run/secrets/nc_db_user_pass
      - NEXTCLOUD_TRUSTED_DOMAINS=nextcloud.mydomain:port
      - TRUSTED_PROXIES=10.0.2.0/24 
      - OVERWRITEPROTOCOL=https
      - OVERWRITEHOST=nextcloud.mydomain:port
      
      - MAIL_MAILER=smtp 
      - MAIL_SMTPHOST=smtp.smtpserver.com 
      - MAIL_SMTPPORT=465 
      - MAIL_SMTPSECURE=ssl 
      
      - MAIL_SMTPAUTHTYPE=LOGIN 
      - MAIL_SMTPNAME=emailaddress 
      - MAIL_SMTPTIMEOUT=30
      - SMTP_PASSWORD_FILE=/run/secrets/nc_mail_pass  
      - MAIL_FROM_ADDRESS=emailaddress
      - MAIL_FROM_NAME=Nextcloud

      - PHP_MAX_EXECUTION_TIME=600
      - PHP_MAX_INPUT_TIME=600
      


      - HTTP_PROXY=http://192.168.1.222:7800

      - HTTPS_PROXY=http://192.168.1.222:7800
    
      - NO_PROXY=localhost,127.0.0.1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16


    secrets:
      - nc_db_user_pass
      - nc_mail_pass      
    volumes:
      - /mnt/swarm-data/nextcloud/html:/var/www/html
    networks:
      - traefik-public
      - nextcloud
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.hostname == worker03
      restart_policy:
        condition: on-failure
      labels:
        - "traefik.enable=true"
        - "traefik.docker.network=traefik-public"

        - "traefik.http.routers.nextcloud-secure.entrypoints=https"
        - "traefik.http.routers.nextcloud-secure.rule=Host(`nextcloud.mydomain`)"

        - "traefik.http.routers.nextcloud-secure.middlewares=nc-rep@file,nc-header@file,my-crowdsec@file"  
        - "traefik.http.routers.nextcloud-secure.tls=true"
        - "traefik.http.routers.nextcloud-secure.tls.certresolver=le"
        - "traefik.http.routers.nextcloud-secure.service=nextcloud-svc"

        - "traefik.http.services.nextcloud-svc.loadbalancer.server.port=80"
        - "traefik.http.services.nextcloud-svc.loadbalancer.passHostHeader=true"


  nextcloud-cron:
    image: nextcloud:fpm 
    depends_on:
      - nextcloud-db
      - nextcloud-redis
      - nextcloud-app

    environment:
      - REDIS_HOST=nextcloud-redis
      - MYSQL_HOST=nextcloud-db
      - MYSQL_USER=nextcloud
      - MYSQL_DATABASE=nextcloud
      - MYSQL_PASSWORD_FILE=/run/secrets/nc_db_user_pass
    secrets:
      - nc_db_user_pass
    volumes:
      - /mnt/swarm-data/nextcloud/html:/var/www/html
    networks:
      - nextcloud
    deploy:
      mode: replicated
      replicas: 1
      placement:
        constraints:
          - node.role == worker 
      restart_policy:
        condition: on-failure
    command: /bin/sh -c "while true; do su -s /bin/bash www-data -c 'php -f /var/www/html/cron.php' ; sleep 300; done"


  elasticsearch:
    image: elasticsearch:8.6.1 
    environment:
      - discovery.type=single-node 
      - xpack.security.enabled=false
      - xpack.security.transport.ssl.enabled=false

      - network.host=0.0.0.0
      - http.host=0.0.0.0
      
      - ES_JAVA_OPTS=-Xms2g -Xmx2g

    ulimits:
      memlock:
        soft: -1
        hard: -1

    volumes:
      - /mnt/swarm-data/nextcloud_es:/usr/share/elasticsearch/data 
    networks:
      - nextcloud
    deploy:
      replicas: 1
      restart_policy:
        condition: on-failure
      resources:
        limits:
          memory: 4096M 
        reservations:
          memory: 2048M 
      placement:
        constraints:
          - node.hostname == worker03

networks:
  traefik-public:
    external: true
  
  nextcloud:
    driver: overlay
    attachable: false

secrets:
  nc_db_user_pass:
    external: true
  nc_mail_pass:
    external: true

Apps

The output of occ app:list (if possible).

www-data@e0e85328d571:~/html$ php occ app:list
Enabled:
  - activity: 5.0.0-dev.0
  - bruteforcesettings: 5.0.0-dev.0
  - circles: 32.0.0
  - cloud_federation_api: 1.16.0
  - comments: 1.22.0
  - contactsinteraction: 1.13.1
  - dashboard: 7.12.0
  - dav: 1.34.2
  - federatedfilesharing: 1.22.0
  - federation: 1.22.0
  - files: 2.4.0
  - files_downloadlimit: 5.0.0-dev.0
  - files_fulltextsearch: 32.0.2
  - files_pdfviewer: 5.0.0-dev.0
  - files_reminders: 1.5.0
  - files_sharing: 1.24.0
  - files_trashbin: 1.22.0
  - files_versions: 1.25.0
  - firstrunwizard: 5.0.0-dev.0
  - fulltextsearch: 32.0.0
  - fulltextsearch_elasticsearch: 32.0.0
  - logreader: 5.0.0-dev.0
  - lookup_server_connector: 1.20.0
  - nextcloud_announcements: 4.0.0-dev.0
  - notifications: 5.0.0-dev.0
  - oauth2: 1.20.0
  - password_policy: 4.0.0-dev.0
  - photos: 5.0.0-dev.1
  - privacy: 4.0.0-dev.0
  - profile: 1.1.0
  - provisioning_api: 1.22.0
  - recommendations: 5.0.0-dev.0
  - related_resources: 3.0.0-dev.0
  - serverinfo: 4.0.0-dev.0
  - settings: 1.15.1
  - sharebymail: 1.22.0
  - support: 4.0.0-dev.0
  - survey_client: 4.0.0-dev.0
  - systemtags: 1.22.0
  - text: 6.0.1
  - theming: 2.7.0
  - twofactor_backupcodes: 1.21.0
  - updatenotification: 1.22.0
  - user_oidc: 8.1.0
  - user_status: 1.12.0
  - viewer: 5.0.0-dev.0
  - weather_status: 1.12.0
  - webhook_listeners: 1.3.0
  - workflowengine: 2.14.0
Disabled:
  - admin_audit: 1.22.0
  - app_api: 32.0.0 (installed 32.0.0)
  - encryption: 2.20.0
  - files_external: 1.24.0
  - suspicious_login: 10.0.0-dev.0
  - twofactor_nextcloud_notification: 6.0.0-dev.0
  - twofactor_totp: 14.0.0
  - user_ldap: 1.23.0

Are you certain?

Given the loglevel of 0 you should be seeing lines in your nextcloud log when the connection attempt to http://elasticsearch:9200 is made by occ fulltextsearch:test -vvv like the following:

  • “Retry” messages
  • “Exceeded maximum number of retries”
  • Response (retry ...): <code>

Given your curl test returned a 200 I’d expect it to succeed, but if it’s not succeeding the <code> there should at least big a different way that might give some hints.

Yes. I can confirm that there is no log entry for test command, even for curl command. And I can confirm there would be no connection attempt to http://elasticsearch:9200 since the tcpdump running in elasticsearch container doesn’t capture any tcp traffic at all. So weird…

Confirmed it’s proxy setting issue. Checkout here fulltextsearch with elasticsearch test fail: Search platform (Elasticsearch) down · Issue #433 · nextcloud/fulltextsearch_elasticsearch · GitHub