Big performance differences: Debian (Bullseye) vs Alpine-based Redis images (Docker)

🏷️ General
, , , ,
1

I recently benchmarked something I’d been curious about for awhile. I thought this community might find some of my findings interesting.

Particularly considering the popularity of Alpine based Docker containers and the importance of Redis to Nextcloud.

I run most of my deployments in Docker, and Alpine-based containers are extremely common. They’re often the default in example Docker instructions and example Compose files.

I wanted to assess what, if any, performance impact arose from using Alpine (which is musl libc based) versus Debian (which is glibc based). There are other trade-off and advantages to both, but my interest here was performance. If you don’t know what I’m even referring to that’s okay - you can still benefit.

Redis comes with a nice little benchmarking tool that makes quick and dirty testing easy. It’s modestly called redis-benchmark. You can run the same tests I did using the general default test based like so:

time redis-benchmark -q

The differences were far more dramatic than I expected.

Here are two representative samplings from of my test runs. I simply used the standard available Docker images available for Redis. Redis gives the user the choice of which variant container OS to run - Alpine or Bullseye (aka: Debian) - by way of appending the appropriate tag to your image name in your Docker Compose file or in your CLI syntax. This is fairly typical for most available commonly used off-the-shelf Docker images. All tests were done on the same hardware back to back around a half dozen times each.

A representative pair of results are below.

After testing and concluding which variant I preferred to run, I promptly transitioned (particularly given it’s as easy as just changing which image is referenced in my Compose files). Can you guess which I chose? (Hint: The big picture number is the real wall clock time it took to run each benchmark - so the lower that figure is the better. I’ve added asterisks below to call that figure out).

Redis v7.0.10 Alpine Docker Image
TCP (Localhost)

PING_INLINE: 95969.28 requests per second, p50=0.263 msec
PING_MBULK: 94607.38 requests per second, p50=0.263 msec
SET: 96153.85 requests per second, p50=0.263 msec
GET: 96246.39 requests per second, p50=0.263 msec
INCR: 97370.98 requests per second, p50=0.255 msec
LPUSH: 97276.27 requests per second, p50=0.255 msec
RPUSH: 97560.98 requests per second, p50=0.255 msec
LPOP: 96525.09 requests per second, p50=0.263 msec
RPOP: 96246.39 requests per second, p50=0.263 msec
SADD: 96432.02 requests per second, p50=0.263 msec
HSET: 96618.36 requests per second, p50=0.263 msec
SPOP: 96805.42 requests per second, p50=0.263 msec
ZADD: 95877.28 requests per second, p50=0.263 msec
ZPOPMIN: 97560.98 requests per second, p50=0.255 msec
LPUSH (needed to benchmark LRANGE): 97943.19 requests per second, p50=0.255 msec
LRANGE_100 (first 100 elements): 31496.06 requests per second, p50=0.791 msec
LRANGE_300 (first 300 elements): 12377.77 requests per second, p50=2.007 msec
LRANGE_500 (first 500 elements): 8471.70 requests per second, p50=2.935 msec
LRANGE_600 (first 600 elements): 7355.10 requests per second, p50=3.407 msec
MSET (10 keys): 94250.71 requests per second, p50=0.263 msec

**real    0m 53.29s**
user    0m 28.02s
sys     0m 24.99s

Redis v7.0.10 Bullseye (Debian) Docker Image
TCP (localhost)

PING_INLINE: 99700.90 requests per second, p50=0.255 msec
PING_MBULK: 99502.48 requests per second, p50=0.255 msec
SET: 99403.58 requests per second, p50=0.255 msec
GET: 99800.40 requests per second, p50=0.255 msec
INCR: 99601.60 requests per second, p50=0.255 msec
LPUSH: 99108.03 requests per second, p50=0.255 msec
RPUSH: 98135.42 requests per second, p50=0.255 msec
LPOP: 98716.68 requests per second, p50=0.255 msec
RPOP: 99108.03 requests per second, p50=0.255 msec
SADD: 98522.17 requests per second, p50=0.255 msec
HSET: 98135.42 requests per second, p50=0.255 msec
SPOP: 97087.38 requests per second, p50=0.255 msec
ZADD: 98716.68 requests per second, p50=0.255 msec
ZPOPMIN: 99800.40 requests per second, p50=0.255 msec
LPUSH (needed to benchmark LRANGE): 99601.60 requests per second, p50=0.255 msec
LRANGE_100 (first 100 elements): 59988.00 requests per second, p50=0.415 msec
LRANGE_300 (first 300 elements): 26553.37 requests per second, p50=0.911 msec
LRANGE_500 (first 500 elements): 18723.09 requests per second, p50=1.303 msec
LRANGE_600 (first 600 elements): 16520.73 requests per second, p50=1.495 msec
MSET (10 keys): 97656.24 requests per second, p50=0.255 msec

**real    0m33.045s**
user    0m17.631s
sys     0m15.112s

Caveat: As with all generalized benchmarking, the real performance benefit depends on the specific use case. Digging into that will have to wait for another day.

3 Likes
2

That’s a surprising difference. I have read that there are similar performance improvements reported for other applications when comparing Alpine with other distro containers. It’s always described as Alpine being aimed at minimizing storage size vs light weight operations.

I wonder if the Nextcloud container and DB containers see similar performance differences?

1 Like
3

I did some reading and found this interesting discussion regarding why there is such a difference

Yes. It is known that some workloads, mostly involving malloc, and heavy
amounts of C string operations benchmark poorly verses glibc.

This is largely because the security hardening features of musl and
Alpine are not zero cost, but also because musl does not contain
micro-architecture specific optimizations, meaning that on glibc you
might get strlen/strcpy type functions that are hand-tuned for the exact
CPU you are using.

In practice though, performance is adequate for most workloads.

https://lists.alpinelinux.org/~alpine/users/<6df8863e77b970b466dbfc9a3a5c2bcec3199f48.camel%40aquilenet.fr>

1 Like