From what you described here, I think that you shot yourself in the foot on that one… The client IP is extracted from the IP packet received by the HTTP engine. From what you described, you put your HAProxy in TCP mode. That is, HAProxy terminates the TCP socket and starts a new one, without being able to put a note in the HTTP Header about it. It can not because it did not decrypt the SSL and did not handled the HTTP request.
So for that reason, the HTTP engine will receive a packet that is from HAProxy’s IP address. Because of that, this IP is what looks like the client.
To avoid that, you must work at either a higher level (HTTP) or a lower level (IP). Should you terminate the SSL on HAProxy, there it can see what is the “external” IP address and save it in a header like X-Forwarded-For. That way, when the HTTP engine will process the request, it can extract that header instead of the IP header and use it as a client IP address.
The other option would be to work at IP level. Instead of terminating the TCP session, just do NAT and port forwarding. That will replace only the destination part of the packet, not the source. That way, when the HTTP engine will see the IP packet, the original source will still be there.
But by playing at TCP level, you end up with the worst of each mode: IP is masked in the process, the request is not filtered at application layer and the HTTP engine ends up blinded without gaining any protection.
If you are about not adding protection in front of HTTP, just do NAT - Port forwarding and you will save your IP. Should you wish to add extra protection by filtering the requests before the HTTP engine, you need to terminate SSL and decrypt in front of the HTTP engine.