„X-Robots-Tag“-HTTP-Header not configured with „noindex, nofollow“ since NC 26.0.0

Would you mind explaining exactly how to solve this with apache2 please?

For apache2 there is a .htaccess file with the “solution” so there is nothing to be done:

… unless you disabled .htaccess by server configuration with

  AllowOverride None

ernolf

Thanks, the only AllowOverride statement I have is in my site configuration file like this:

  <Directory /var/www/nextcloud/>
    Require all granted
    AllowOverride All
    Options FollowSymLinks MultiViews

    <IfModule mod_dav.c>
      Dav off
    </IfModule>
  </Directory>

But I still see the warning in Nextcloud. I checked the .htaccess file in /var/www/nextcloud and it contains those directives. Any suggestions?

The problem when using old topics, is that you did not provide any information about your system, your setup and your hosting environment.

Are you on a shared hoster?

It only works if the apache2 module mod_env is loaded.

If you have got full access to your webserver, you could find out if mod_env is loaded with this call

~$ sudo apachectl -M 2>/dev/null | grep env_module

it should echo this:

 env_module (shared)

Much and good luck,
ernolf

Thanks, yes the output of

sudo apachectl -M 2>/dev/null | grep env_module

is

env_module (shared)

In my case I am running nextcloud on my own server with a bare install (no containers). I’ve been running it this way since about NC 13, but the headers warning seems to be since upgrading to NC 29.

My full apache site config is below:

<IfModule mod_ssl.c>
<VirtualHost *:443>

  # enable HTTP/2
  Protocols h2 http/1.1

	# The ServerName directive sets the request scheme, hostname and port that
	# the server uses to identify itself. This is used when creating
	# redirection URLs. In the context of virtual hosts, the ServerName
	# specifies what hostname must appear in the request's Host: header to
	# match this virtual host. For the default virtual host (this file) this
	# value is not decisive as it is used as a last resort host regardless.
	# However, you must set it for any further virtual host explicitly.
	ServerName <redacted>
        <IfModule mod_headers.c>
                Header always set Strict-Transport-Security "max-age=15552000; includeSubDomains"
        </IfModule>

	ServerAdmin sysadmin@<redacted>
	DocumentRoot /var/www/nextcloud


	<Directory /var/www/nextcloud/>
		Require all granted
		AllowOverride All
		Options FollowSymLinks MultiViews

		<IfModule mod_dav.c>
			Dav off
		</IfModule>
	</Directory>

	# For most configuration files from conf-available/, which are
	# enabled or disabled at a global level, it is possible to
	# include a line for only one particular virtual host. For example the
	# following line enables the CGI configuration for this host only
	# after it has been globally disabled with "a2disconf".
	#Include conf-available/serve-cgi-bin.conf

	<IfModule mod_rewrite.c>
		RewriteEngine on

    # to prevent direct logins
    #RewriteCond %{QUERY_STRING} direct
    #RewriteRule ^login$ - [R=404]

    # well-known
		RewriteRule ^/\.well-known/host-meta /public.php?service=host-meta [QSA,L]
		RewriteRule ^/\.well-known/host-meta\.json /public.php?service=host-meta-json [QSA,L]
		RewriteRule ^/\.well-known/carddav /remote.php/dav/ [R=301,L]
		RewriteRule ^/\.well-known/caldav /remote.php/dav/ [R=301,L]
		RewriteRule ^\.well-known/nodeinfo /index.php/.well-known/nodeinfo [R=301,L]
		RewriteRule ^\.well-known/webfinger /index.php/.well-known/webfinger [R=301,L]
	</IfModule>

	# Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
	# error, crit, alert, emerg.
	# It is also possible to configure the loglevel for particular
	# modules, e.g.
	#LogLevel info ssl:warn

	ErrorLog ${APACHE_LOG_DIR}/error.log
	CustomLog ${APACHE_LOG_DIR}/access.log combined


	ProxyPass /push/ws ws://127.0.0.1:7867/ws
	ProxyPass /push/ http://127.0.0.1:7867/
	ProxyPassReverse /push/ http://127.0.0.1:7867/

	# For most configuration files from conf-available/, which are
	# enabled or disabled at a global level, it is possible to
	# include a line for only one particular virtual host. For example the
	# following line enables the CGI configuration for this host only
	# after it has been globally disabled with "a2disconf".
	#Include conf-available/serve-cgi-bin.conf


	Include /etc/letsencrypt/options-ssl-apache.conf
	SSLCertificateFile /etc/letsencrypt/live/<redacted>/fullchain.pem
	SSLCertificateKeyFile /etc/letsencrypt/live/<redacted>/privkey.pem
</VirtualHost>
</IfModule>

Hi,

I’m in exactly same case :

  • Server Ubuntu 20.04 (updated)
  • PHP 8.2.20
  • Apache 2.4.41

http headers give this log :

  • HTTP/1.1 301 Moved Permanently
  • Date: Mon, 17 Jun 2024 08:25:26 GMT
  • Server: Apache/2.4.41 (Ubuntu)
  • Location: https://myserver.com/nextcloud/
  • Content-Length: 312
  • Connection: close
  • Content-Type: text/html; charset=iso-8859-1
  • HTTP/1.1 200 OK
  • Date: Mon, 17 Jun 2024 08:25:27 GMT
  • Server: Apache/2.4.41 (Ubuntu)
  • Last-Modified: Sun, 16 Jun 2024 18:13:00 GMT
  • ETag: “9c-61b05cc8b06c2”
  • Accept-Ranges: bytes
  • Content-Length: 156
  • Vary: Accept-Encoding
  • X-Frame-Options: sameorigin
  • Connection: close
  • Content-Type: text/html

Apache Config :

<VirtualHost *:80>
        ServerName myserver.com
        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html
        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

        <Directory /var/www/html>
                Options -Indexes
        </Directory>
        RewriteEngine on
        RewriteCond %{SERVER_NAME} =myserver.com
        RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
        <Directory /var/www/html/nextcloud/>
                Require all granted
                AllowOverride All
                Options FollowSymLinks MultiViews
                <IfModule mod_dav.c>
                        Dav off
                </IfModule>
        </Directory>
</VirtualHost>

Loaded modules :

core_module (static)
so_module (static)
watchdog_module (static)
http_module (static)
log_config_module (static)
logio_module (static)
version_module (static)
unixd_module (static)
access_compat_module (shared)
alias_module (shared)
auth_basic_module (shared)
authn_core_module (shared)
authn_file_module (shared)
authz_core_module (shared)
authz_host_module (shared)
authz_user_module (shared)
autoindex_module (shared)
deflate_module (shared)
dir_module (shared)
env_module (shared)
filter_module (shared)
headers_module (shared)
mime_module (shared)
mpm_prefork_module (shared)
negotiation_module (shared)
php_module (shared)
reqtimeout_module (shared)
rewrite_module (shared)
setenvif_module (shared)
socache_shmcb_module (shared)
ssl_module (shared)
status_module (shared)

.htaccess in /var/www/html/nextcloud :

<IfModule mod_headers.c>
  <IfModule mod_setenvif.c>
    <IfModule mod_fcgid.c>
       SetEnvIfNoCase ^Authorization$ "(.+)" XAUTHORIZATION=$1
       RequestHeader set XAuthorization %{XAUTHORIZATION}e env=XAUTHORIZATION
    </IfModule>
    <IfModule mod_proxy_fcgi.c>
       SetEnvIfNoCase Authorization "(.+)" HTTP_AUTHORIZATION=$1
    </IfModule>
    <IfModule mod_lsapi.c>
      SetEnvIfNoCase ^Authorization$ "(.+)" XAUTHORIZATION=$1
      RequestHeader set XAuthorization %{XAUTHORIZATION}e env=XAUTHORIZATION
    </IfModule>
  </IfModule>

  <IfModule mod_env.c>
    # Add security and privacy related headers

    # Avoid doubled headers by unsetting headers in "onsuccess" table,
    # then add headers to "always" table: https://github.com/nextcloud/server/pull/19002
    Header onsuccess unset Referrer-Policy
    Header always set Referrer-Policy "no-referrer"

    Header onsuccess unset X-Content-Type-Options
    Header always set X-Content-Type-Options "nosniff"

    Header onsuccess unset X-Frame-Options
    Header always set X-Frame-Options "SAMEORIGIN"

    Header onsuccess unset X-Permitted-Cross-Domain-Policies
    Header always set X-Permitted-Cross-Domain-Policies "none"

    Header onsuccess unset X-Robots-Tag
    Header always set X-Robots-Tag "noindex, nofollow"

    Header onsuccess unset X-XSS-Protection
    Header always set X-XSS-Protection "1; mode=block"

    SetEnv modHeadersAvailable true
  </IfModule>

  # Add cache control for static resources
  <FilesMatch "\.(css|js|mjs|svg|gif|png|jpg|ico|wasm|tflite)$">
    <If "%{QUERY_STRING} =~ /(^|&)v=/">
      Header set Cache-Control "max-age=15778463, immutable"
    </If>
    <Else>
      Header set Cache-Control "max-age=15778463"
    </Else>
  </FilesMatch>

  # Let browsers cache WOFF files for a week
  <FilesMatch "\.woff2?$">
    Header set Cache-Control "max-age=604800"
  </FilesMatch>
</IfModule>

<IfModule mod_php.c>
  php_value mbstring.func_overload 0
  php_value default_charset 'UTF-8'
  php_value output_buffering 0
  <IfModule mod_env.c>
    SetEnv htaccessWorking true
  </IfModule>
</IfModule>

<IfModule mod_mime.c>
  AddType image/svg+xml svg svgz
  AddType application/wasm wasm
  AddEncoding gzip svgz
  # Serve ESM javascript files (.mjs) with correct mime type
  AddType text/javascript js mjs
</IfModule>

<IfModule mod_dir.c>
  DirectoryIndex index.php index.html
</IfModule>

<IfModule pagespeed_module>
  ModPagespeed Off
</IfModule>

<IfModule mod_rewrite.c>
  RewriteEngine on
  RewriteCond %{HTTP_USER_AGENT} DavClnt
  RewriteRule ^$ /remote.php/webdav/ [L,R=302]
  RewriteRule .* - [env=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
  RewriteRule ^\.well-known/carddav /nextcloud/remote.php/dav/ [R=301,L]
  RewriteRule ^\.well-known/caldav /nextcloud/remote.php/dav/ [R=301,L]
  RewriteRule ^\.well-known/webfinger /nextcloud/index.php/.well-known/webfinger [R=301,L]
  RewriteRule ^\.well-known/nodeinfo /nextcloud/index.php/.well-known/nodeinfo [R=301,L]
  RewriteRule ^remote/(.*) remote.php [QSA,L]
  RewriteRule ^(?:build|tests|config|lib|3rdparty|templates)/.* - [R=404,L]
  RewriteRule ^\.well-known/(?!acme-challenge|pki-validation) /index.php [QSA,L]
  RewriteRule ^ocm-provider/?$ index.php [QSA,L]
  RewriteRule ^(?:\.(?!well-known)|autotest|occ|issue|indie|db_|console).* - [R=404,L]
</IfModule>

https://docs.nextcloud.com/server/latest/admin_manual/issues/general_troubleshooting.html#troubleshooting-webdav
<IfModule mod_setenvif.c>
  SetEnvIf Transfer-Encoding "chunked" proxy-sendcl=1
</IfModule>

AddDefaultCharset utf-8
Options -Indexes
#### DO NOT CHANGE ANYTHING ABOVE THIS LINE ####

ErrorDocument 403 /nextcloud/index.php/error/403
ErrorDocument 404 /nextcloud/index.php/error/404

And I’m still having this issue : Some headers are not set correctly on your instance …

Any help would be really appreciated

Ok, so I just found where the issue come from for my own case.
I did not realized there was a separate file for SSL.
Once I did realized this I addzd this in the SSL conf file and now headers are well handled :

        <Directory /var/www/html/nextcloud/>
                Require all granted
                AllowOverride All
                Options FollowSymLinks MultiViews
                <IfModule mod_dav.c>
                        Dav off
                </IfModule>
        </Directory>

Hi guys,

I found the problem : the code running test is base on doing GET on /heartbeat. But apparently, this route has a problem : fix(settings): define a 'heartbeat' route, so SecurityHeaders can handle redirected root by smokris · Pull Request #45228 · nextcloud/server · GitHub

Fix is scheduled so : wait or manually set URL in apps/settings/lib/SetupChecks/SecurityHeaders.php

Replace

	public function run(): SetupResult {
		$urls = [
			['get', $this->urlGenerator->linkToRoute('heartbeat'), [200]],
		];

by

	public function run(): SetupResult {
		$urls = [
			['get', '/heartbeat' , [200]],
		];
1 Like

Yep. The latest update of NC fixed this issue. I was going nuts trying to figure out if my instance is not setup correctly as I am using HAProxy on pfsense.

Now it’s all good. :smile:

1 Like

Yep the fix works :wink:

This topic was automatically closed 8 days after the last reply. New replies are no longer allowed.