Too Many Authentication Failures

Nextcloud version (eg, 10.0.2): 11.0
Operating system and version (eg, Ubuntu 16.04): 16.04 server
Apache or nginx version (eg, Apache 2.4.25):
PHP version (eg, 5.6): 7.0

The issue you are facing:
I am trying to use FTP so I can upgrade my nextcloud to 11.01. When I try to connect I get the message about Too Many Authentication Failures.

I do have login with SSH Keys enabled. I have turned on the password login back on in the sshd_config file. Thats when I got the Too Many Authentication Failures instead of the Connection refused So I am guessing that was needed.

However I do not know how to resolve the Too Many Authentication failures message. Its been awhile so I`m guessing it doesnt reset on its own and I have no idea what to do to get this to work again.

Thanks for the help.