What steps should I take to make sure I am the only one with access to my Linux VPS?

I recently set up a NextCloud VPS. At first I was a bit disturbed that the hosting company emailed me the root password in plain text, and I changed the root password right away. But later, in response to a ticket I opened, someone at the hosting company installed another app on my VPS. I’m not sure how they had access. I don’t think my hosting company has malicious intentions, but I’d like to make sure that my NextCloud instance is not accessible by anyone but myself. How should I do that?

I’m a web programmer, but not a linux/unix expert. I plan to set up SSH keys and disable password access, but I am not sure if that’s enough to ensure that I’m the only one who can log in, see files, and run commands on the VPS. Do I need to do anything else to control access?

A few other notes:

  • My host is webo.hosting, which I chose because they are Europe-based and offered managed daily backups, but I’m wondering if I should switch to something more mainstream like DigitalOcean.
  • I also have questions about encryption and attaching external storage, but that’s probably a slightly different topic.

TL;DR

What steps should I take to make sure I am the only one with access to my Linux VPS?