HTTP/2 on "Apache/2.4.27 (Debian 9)"

Hi together,
I’m struggling since a while on enabling HTTP/2.
I have read HTTP/2, yes or no?.

I have enabled http2 with a2enmod http2

So my /etc/apache2/conf-available/http2.conf and I have run a2enconf http2

IfModule http2_module>
  Protocols h2 h2c http/1.1

  H2Push          on
  H2PushPriority  *                       after
  H2PushPriority  text/css                before
  H2PushPriority  image/jpeg              after   32
  H2PushPriority  image/png               after   32
  H2PushPriority  application/javascript  interleaved

  SSLProtocol all -SSLv2 -SSLv3
  SSLHonorCipherOrder on
  SSLCipherSuite 'EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256 EECDH+aRSA+RC4 EECDH EDH+aRSA !RC4 !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS'
</IfModule>

And the /etc/apache2/sites-enabled/001-domain.com-le-ssl.conf

<IfModule mod_ssl.c>
        <VirtualHost _default_:443>
                ServerAdmin arne92@gmail.com
                ServerName domain.com
#               Header always add Strict-Transport-Security "max-age=15768000; includeSubDomains; preload"
                DocumentRoot /var/www/nextcloud

                Protocols h2 http/1.1
                H2Direct on


                ErrorLog ${APACHE_LOG_DIR}/001_error.log
                CustomLog ${APACHE_LOG_DIR}/001_access.log combined

                SSLEngine on

                SSLCertificateFile /etc/letsencrypt/live/domain.com-0001/fullchain.pem
                SSLCertificateKeyFile /etc/letsencrypt/live/domain.com-0001/privkey.pem

                <FilesMatch "\.(cgi|shtml|phtml|php)$">
                                SSLOptions +StdEnvVars
                </FilesMatch>
                <Directory /var/www/nextcloud>
                        Options Indexes FollowSymLinks MultiViews
                        AllowOverride All
                        Require all granted
                        SetEnv MOD_X_SENDFILE_ENABLED 1
                        LimitRequestBody 0
                        XSendFile On
                        XSendFilePath /owncloud_data
                        <IfModule mod_dav.c>
                                Dav off
                        </IfModule>
                        Satisfy Any
                        SSLRenegBufferSize 10486000
                </Directory>


                BrowserMatch "MSIE [2-6]" nokeepalive ssl-unclean-shutdown downgrade-1.0 force-response-1.0
                # MSIE 7 and newer should be able to use keepalive
                BrowserMatch "MSIE [17-9]" \
                                ssl-unclean-shutdown




                # Intermediate configuration, tweak to your needs
                SSLCipherSuite ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!3DES:!MD5:!PSK
                SSLProtocol All -SSLv2 -SSLv3

                SSLHonorCipherOrder          on
                SSLOptions          +StrictRequire


                ProxyPass /http-bind/ http://localhost:5280/http-bind/
                ProxyPassReverse /http-bind/ http://localhost:5280/http-bind/

                Redirect permanent /sharelatex https://sharelatex.domain.com/

                <Location /webrtc>
                        ProxyPass http://127.0.0.1:8080/webrtc
                        ProxyPassReverse /webrtc
                </Location>

                <Location /webrtc/ws>
                        ProxyPass ws://127.0.0.1:8080/webrtc/ws
                </Location>

                ProxyVia On
                ProxyPreserveHost On
                RequestHeader set X-Forwarded-Proto 'https' env=HTTPS
                php_flag output_buffering off

#               This is also to prevent high memory usage
                php_flag always_populate_raw_post_data off

#               This is almost a given, but magic quotes is *still* on on some
#               linux distributions
                php_flag magic_quotes_gpc off

                # SabreDAV is not compatible with mbstring function overloading
                php_flag mbstring.func_overload off

                # Solr
                ProxyPass /solr/ http://localhost:8983/solr/
                ProxyPassReverse /solr/ http://localhost:8983/solr/

                <Location /solr>
                        AuthType Basic
                        AuthName "solr"
                        AuthUserFile /etc/apache2/htpasswd-solr
                        Require valid-user
                </Location>


        </VirtualHost>
</IfModule>

So I think http/2 should be enabled and working but the browser console and Curl is saying:

* Rebuilt URL to: https://domain.com/
*   Trying 192.168.0.57...
* TCP_NODELAY set
* Connected to domain.com (192.168.0.57) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
} [5 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [113 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [2722 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [589 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [70 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
{ [1 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: CN=domain.com
*  start date: Aug  8 21:03:00 2017 GMT
*  expire date: Nov  6 21:03:00 2017 GMT
*  subjectAltName: host "domain.com" matched cert's "domain.com"
*  issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
*  SSL certificate verify ok.
} [5 bytes data]
> GET / HTTP/1.1
> Host: domain.com
> User-Agent: curl/7.54.1
> Accept: */*
> 
{ [5 bytes data]
< HTTP/1.1 302 Found
< Date: Mon, 21 Aug 2017 07:27:16 GMT
< Server: Apache/2.4.27 (Debian)
< Upgrade: h2
< Connection: Upgrade
< Set-Cookie: oc93z1rhfmbp=0eh06ob303o5jr2ejdo488hpp6; path=/; HttpOnly
< Expires: Thu, 19 Nov 1981 08:52:00 GMT
< Cache-Control: no-store, no-cache, must-revalidate
< Pragma: no-cache
< Set-Cookie: oc_sessionPassphrase=IJdJYT28vxdIbOcgDh7hEvt9eARx6Sikek8zQWR9iaRLrOGoFsgTpDx2Tr44bfVXK%2F4dQTeTKv38x10lvV1gSzSi8znsGRHsR2T3B%2BX3ElOrLzr0O5WOLqKp33UwkxTE; path=/; secure; HttpOnly
< Content-Security-Policy: default-src 'self'; script-src 'self' 'unsafe-eval' 'nonce-L3NldndFaWhVaVhVbTNWeFc1aFRNL0J2RTlqR2hhRjJUNmh1YkpMTlFPYz06bHBDZStRcjJHMENPMERvWkNjaGljcmNxSjZpeXlvb2hMc29jQ3ZXQktOOD0='; style-src 'self' 'unsafe-inline'; frame-src *; img-src * data: blob:; font-src 'self' data:; media-src *; connect-src *; object-src 'none'; base-uri 'self';
< X-Frame-Options: SAMEORIGIN
< Set-Cookie: __Host-nc_sameSiteCookielax=true; path=/; httponly;secure; expires=Fri, 31-Dec-2100 23:59:59 GMT; SameSite=lax
< Set-Cookie: __Host-nc_sameSiteCookiestrict=true; path=/; httponly;secure; expires=Fri, 31-Dec-2100 23:59:59 GMT; SameSite=strict
< Location: https://domain.com/login
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< X-Robots-Tag: none
< X-Download-Options: noopen
< X-Permitted-Cross-Domain-Policies: none
< Content-Length: 0
< Content-Type: text/html; charset=UTF-8
< 
* Connection #0 to host domain.com left intact

So if someone has an idea what could be going wrong, or where I could get more debug information please let me know.

Hello,

Is-it working with this directive ? ProtocolsHonorOrder On (activated by default in apache)
and putting the protocols list in the other way like this :

Protocols http/1.1 h2c h2

https://httpd.apache.org/docs/trunk/mod/core.html#protocolshonororder

With ProtocolsHonorOrder set to on (default), the client ordering does not matter and only the ordering in the server settings influences the outcome of the protocol negotiation.

I guess this would lead to http/1.1 in any situation, always found it the other way round and is working fine for me.

@tacruc Could you try it by just use the /etc/apache2/conf-available/http2.conf like this:

<IfModule http2_module>
        Protocols h2 h2c http/1.1
        H2Direct on
</IfModule>

and leave out every http/2 related setting in vhost/every ssl related setting in http2.conf? At the moment it looks like you double many of them.

It should (does fine for me) work, if you use the default (before http/2) ssl vhost with SSLProtocol SSLCipherSuite etc inside, so that it will work totally independent from http/2 or not. And the http2.conf on the other hand really only enables the protocol order and in case these H2… settings, so that you can simply plug&play http/2 as desired.

On the other hand in your log I see for browser

* ALPN, offering h2
* ALPN, offering http/1.1
...
> GET / HTTP/1.1

and curl:

< HTTP/1.1 302 Found
...
< Upgrade: h2
< Connection: Upgrade

So the server well offers both.

  • Did you check, if your browser/version really supports http2?
  • And about curl for me it looks like everything is working as expected: http/1.1 found but redirected and upgraded to h2, or do I interpret the log wrong?

Guys, this is interesting. Do you think you could make a PR to change to HTTP2 as default config in the VM?

Would be no big deal, but e.g. spreed video calls app does not yet support http/2 and there might be other web services that have issues with it.

1 Like

Nice! That’s an easy fix, just warn ppl it won’t work on HTTP2, but leave it in until they fixed it.

Anyway, would love if you could help me with a PR. <3

Yes, I will do that. Most elegant way for my impression is to put http2.conf into /etc/apache2/mods-available/ like this:

<IfModule http2_module>
        Protocols h2 h2c http/1.1
        H2Direct on
</IfModule>

Just tested it and works well :slight_smile:. That way it will be automatically loaded (symlinked to mods-enabled) together with the module (http2.load) by a2enmod http2 and unloaded as well. “Protocols h2 h2c http/1.1” leads to the preferred use of http2 on any connection (h2 for https, h2c (clear) for http) and falls back to http/1.1, if unsupported.

H2Direct is by default just active for h2c, because some modules seem to have compatibility issues. On the other hand it has some performance advantage and I never had any issues with it.

More about that: mod_http2 - Apache HTTP Server Version 2.4
HTTP/2 guide - Apache HTTP Server Version 2.4


The maybe simpler solution would be to just add Protocols h2 http/1.1 into the ssl vhost, optionally within <IfModule http2_module>. As far as I know, no browser supports http2 for non ssl connections so far, so h2c has currently no effect, if I understood right. But I like to have thinks as modular as possible with a clean plug&play behaviour.

What you think?

I think option 1 sounds great! Apply it system wide and make it possible to disable easily without changing individual vhost parameters.

I back you 100%. Looking forward to the PR so that we can discuss further. :slight_smile:

1 Like

Hi guys, please note that the Upgrade: h2 response header you’re seeing from Apache is a bug in older versions of of the web server. You should unset the response header to avoid compatibility issues with clients who don’t understand the unexpected Upgrade header. (The Upgrade header is a HTTP request and not a response header!)

1 Like

Thanks for the info. Is there an easy fast check, if the bug is fixed on ones server/nextcloud?

Because link says it will be fixed with 2.4.21 and TO still faces it with 2.4.27.

Is there an easy fast check, if the bug is fixed on ones server/nextcloud?

Not really. The header should never be sent in a response. You can just always unset it with the Header unset Upgrade directive. If the header isn’t set, nothing happens. If it is set, it will be removed.

Hi,
Thanks, for your Help, but the ProtocolsHonorOrder doesn’t make any difference. Protocolls in both orders.

I cleaned the http2.conf and the virtual host configuring ssl only in the vhost and the http2 only in the http2.conf, but it the output of the Browser (Firefox 52.3.0 and 54.0.1) both telling to use HTTP1.1. But help.nextcloud.com it say’s http2.0, so I would say the browser supports HTTP2.0

From this log I would agree that the server seem’s to offer both, but the output of help.nextcloud.com looks more like it is direct using HTTP2.0.

* ALPN, server accepted to use h2

vs on my server:

* ALPN, server accepted to use http/1.1

The Full Log of curl on curl -vso /dev/null --http2 https://help.nextcloud.com

* Rebuilt URL to: https://help.nextcloud.com/
*   Trying 88.198.160.135...
* TCP_NODELAY set
* Connected to help.nextcloud.com (88.198.160.135) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
} [5 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [103 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [2734 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [621 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [102 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
{ [1 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=help.nextcloud.com
*  start date: Jul  6 23:01:00 2017 GMT
*  expire date: Oct  4 23:01:00 2017 GMT
*  subjectAltName: host "help.nextcloud.com" matched cert's "help.nextcloud.com"
*  issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
} [5 bytes data]
* Using Stream ID: 1 (easy handle 0xacd6a940c0)
} [5 bytes data]
> GET / HTTP/2
> Host: help.nextcloud.com
> User-Agent: curl/7.54.1
> Accept: */*
> 
{ [5 bytes data]
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
} [5 bytes data]
< HTTP/2 200 
< server: nginx
< date: Mon, 28 Aug 2017 13:39:42 GMT
< content-type: text/html; charset=utf-8
< vary: Accept-Encoding
< x-frame-options: SAMEORIGIN
< x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
< x-discourse-route: list/latest
< cache-control: no-store, must-revalidate, no-cache, private
< x-request-id: 28f42b75-0933-458c-b9e1-66df40884560
< x-runtime: 0.170438
< x-discourse-trackview: 1
< referrer-policy: no-referrer-when-downgrade
< strict-transport-security: max-age=31536000
< 
{ [7790 bytes data]
* Connection #0 to host help.nextcloud.com left intact

thanks for the info.
So where would you sugest to set it. In the HTTP2.conf or in each vhost section?

Yeah, could be related to the Upgrade header then. I would try to solve it by setting Header unset Upgrade inside http2.conf then, as the error only occurs in connection with it.

I am still struggling to check if the header is also derived with my server. My browser console doesn’t show this log, even in verbose mode. I am just able too see “h2” as negotiated protocol for it, using opera://net-internals/#http2 / chrome://net-internals/#http2 . But actually this should prove the successful http/2 use.

I unset the header in http2.conf but still I get the

< Server: Apache/2.4.27 (Debian)
< Connection: Upgrade

with curl -vso and still the the

* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
...
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use http/1.1

So I think for the momement I am giving up and stick to http1.1
Thanks to @MichaIng @da2x and @zfzfzf33 for your Help

Just found curl --http2 https://my.domain.org -v now :slight_smile::

* Rebuilt URL to: https://my.domain.org/
*   Trying 127.0.1.1...
* TCP_NODELAY set
* Connected to micha.gnoedi.org (127.0.1.1) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=my.domain.org
*  start date: Jul 18 08:32:00 2017 GMT
*  expire date: Oct 16 08:32:00 2017 GMT
*  subjectAltName: host "micha.gnoedi.org" matched cert's "micha.gnoedi.org"
*  issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x4a6e38)
> GET / HTTP/1.1
> Host: my.domain.org
> User-Agent: curl/7.52.1
> Accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 200
< date: Mon, 28 Aug 2017 20:59:53 GMT
< server: Apache
< strict-transport-security: max-age=15552000; includeSubDomains; preload
< last-modified: Wed, 21 Jun 2017 10:32:53 GMT
< etag: "29cd-55275e408f75f"
< accept-ranges: bytes
< content-length: 10701
< vary: Accept-Encoding
< content-type: text/html
...

So it looks similar until on your client http/1.1 is accepted, where it is h2 on mine, linke it is for help.nextcloud.com.

Just using curl --http2 my.domain.org -v leads to:

* Rebuilt URL to: my.domain.org/
*   Trying 127.0.1.1...
* TCP_NODELAY set
* Connected to my.domain.org (127.0.1.1) port 80 (#0)
> GET / HTTP/1.1
> Host: my.domain.org
> User-Agent: curl/7.52.1
> Accept: */*
> Connection: Upgrade, HTTP2-Settings
> Upgrade: h2c
> HTTP2-Settings: AAMAAABkAARAAAAA
>
< HTTP/1.1 301 Moved Permanently
< Date: Mon, 28 Aug 2017 21:06:42 GMT
< Server: Apache
< Location: https://my.domain.org/
< Content-Length: 233
< Content-Type: text/html; charset=iso-8859-1
<
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="https://my.domain.org/">here</a>.</p>
</body></html>
* Curl_http_done: called premature == 0
* Connection #0 to host my.domain.org left intact

So the Upgrade header appears for 80/http connections (where HTTP/2 is not supported by browsers) and which will be redirected to 443/https, where HTTP/2 is supported again. Also read somewhere about HTTP/1.1 => HTTP/2 protocol “upgrade”, so seems to be not necessarily a bug here. Also 3rd party server tests (https://http2.pro/, HTTP/2 Test - Verify HTTP/2 Support | KeyCDN Tools) give me positive HTTP/2 test result.

Hmm yeah, so far no idea what could make the server just HTTP/1.1 in your case, where actually both protocols are offered and the order h2 (> h2c) > http/1.1 is set.
Maybe have a close look at all your apache config files (sites-enabled, conf-enabled, .htaccess in case). Also follow all IncludeOptional directives within your apache2.conf to not forget any config file. Perhaps there are some config collisions, “Protocols” already set somewhere else or what.