To answer my own question: Yes that was the problem. 
I had to set the proxy read timeout for nginx to a higher value according to this post 504 Gateway timeout for large file uploads - #7 by GR4 and now everything works!
Thank you @jtr for spending all the time helping me troubleshoot this! 