How to Log In to a VPS Over SSH and Secure It (Beginner Guide)

Your provider emails an IP address, a root user and a password. You type ssh root@the-ip, and the terminal asks about a fingerprint you do not understand. A few minutes later the server is open, but anyone who knows the address can try guessing the root password. Beginners usually stop at one of three points: the fingerprint prompt, the Permission denied (publickey) message, or the most expensive one, switching off password login and getting locked out of their own server.

This guide takes you from the first login to a server that only accepts keys, in six ordered steps. I checked every command and option against the Ubuntu Server documentation and the OpenSSH and ufw manual pages, and every example uses placeholders: the IP 203.0.113.10 and a user named deploy. The examples are written for Ubuntu or Debian. This site's own deploy uses the same pattern: GitHub Actions logs in as a normal user with a key from a secret, with no password and no address or user name written in the repository.

What you need

• A running Linux VPS (this guide uses Ubuntu or Debian), with the IP address, initial user and password from your provider.
• A terminal on your laptop: Terminal on macOS or Linux, or PowerShell or WSL on Windows, with an ssh client installed.
• Access to the web console in your provider's panel. It is the emergency door if SSH fails, so find the button now, before you need it.
• Two terminal windows you can open side by side. You will use them for one important trick in Step 4.

No VPS yet? Read the guide to choosing a cheap VPS in Indonesia first, then come back once the server is running.

Step 1: Log in to the VPS for the first time

The first login uses the user and password from your provider. Type the command below, with your own IP address. If the SSH port is not 22, add -p followed by the port number.

# Dari laptop. Pakai user dan IP dari email provider (di sini: placeholder).
ssh root@203.0.113.10

# Pertama kali, kira-kira muncul ini. Fingerprint-nya milik server kamu:
# The authenticity of host '203.0.113.10 (203.0.113.10)' can't be established.
# ED25519 key fingerprint is SHA256:<fingerprint-server-kamu>.
# Are you sure you want to continue connecting (yes/no/[fingerprint])?
# Ketik: yes   (ssh lalu menyimpannya di ~/.ssh/known_hosts)

The fingerprint prompt is not an error. It is ssh asking whether this is the server you meant. Once you answer yes, the fingerprint is stored in ~/.ssh/known_hosts, and if it ever changes, ssh warns you and refuses to continue. If you want to be thorough, compare it with what the provider console shows, or run ssh-keygen -l -f /etc/ssh/ssh_host_ed25519_key.pub there.

Step 2: Create a normal user with sudo rights

Logging in as root is like holding the keys to every door at once: one typo can break anything. Create a normal user for daily work and give it sudo rights, so dangerous commands have to be typed deliberately. Run this on the server, still as root.

# Di server, masih sebagai root
adduser deploy                # isi password + data singkat; ini password untuk sudo
usermod -aG sudo deploy       # beri hak sudo (Ubuntu dan Debian)

# Cek dari sesi root: pindah ke user baru lalu coba sudo
su - deploy
sudo whoami                   # harus mencetak: root
exit

Do not continue until sudo whoami prints root. If you get a message saying the deploy user is not in the sudoers file, repeat usermod from the root session. Switching off root login in Step 4 without a working sudo user is one of the fastest routes to being locked out.

Step 3: Create an SSH key and install it on the server

An SSH key is a pair of files: a private key that stays on your laptop and a public key installed on the server. The Ubuntu documentation recommends ssh-keygen -t ed25519. The ssh-keygen manual says the private key file should not be readable by anyone but you, and that the passphrase may be empty but is better filled in so a stolen key is not instantly usable. Run all of this on your laptop.

# Di LAPTOP kamu (bukan di server)
ssh-keygen -t ed25519
# Enter file in which to save the key: tekan Enter (jadi ~/.ssh/id_ed25519)
# Enter passphrase: isi passphrase; kosong juga boleh, tapi lebih lemah

# Pasang public key ke user deploy di server
ssh-copy-id -i ~/.ssh/id_ed25519.pub deploy@203.0.113.10

# Tes: harus masuk TANPA diminta password server
ssh deploy@203.0.113.10

ssh-copy-id logs in to the server and appends the public key to ~/.ssh/authorized_keys of the target user, creating the file and folder if they do not exist. If that command is not available on your laptop, do it by hand with the commands below. This is also how you install a second key later.

# Kalau ssh-copy-id tidak ada di laptop kamu: tempel manual, di server sebagai user deploy
mkdir -p ~/.ssh
chmod 700 ~/.ssh
nano ~/.ssh/authorized_keys   # tempel SATU baris isi id_ed25519.pub di sini
chmod 600 ~/.ssh/authorized_keys

The two modes above are not decoration. The sshd manual recommends that ~/.ssh and authorized_keys are not accessible to other users, and sshd refuses the file if others can write to it while StrictModes is on, which is the default. So if a key will not work, do not run chmod 777: it makes things worse.

Step 4: Switch off password login and root login

Now that the key works, close the password route. While the server accepts passwords, bots on the internet can keep guessing. With PasswordAuthentication no, guessing passwords gets nowhere because the server no longer accepts any. PermitRootLogin defaults to prohibit-password, which means root can still log in with a key. Setting it to no forces everyone in as a normal user.

# Wrong: ubah, restart, lalu menutup satu-satunya sesi.
# Satu salah ketik dan pintu tertutup dari luar.
sudo nano /etc/ssh/sshd_config
sudo systemctl restart ssh.service
exit

# Right: cadangkan, ubah lewat file drop-in, cek sintaks, restart,
# lalu tes dari terminal KEDUA sementara terminal pertama tetap terbuka.
sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.original
sudo nano /etc/ssh/sshd_config.d/00-hardening.conf
sudo sshd -t                      # tidak mencetak apa-apa = konfigurasi valid
sudo systemctl restart ssh.service

The Ubuntu documentation explains that the main sshd_config starts with an Include line for the sshd_config.d folder, and OpenSSH uses the first value it finds for each setting, so files in that folder override the main file. That makes one small file in that folder the safest approach. It holds only three lines. KbdInteractiveAuthentication defaults to yes, so it is switched off too, leaving the key as the only way in.

# /etc/ssh/sshd_config.d/00-hardening.conf
PasswordAuthentication no
KbdInteractiveAuthentication no
PermitRootLogin no

The sshd -t command only checks that the configuration is valid, and Ubuntu recommends it before a restart. After the restart, sshd -T prints the effective configuration. If another file sets the same thing earlier, this is where you see it.

# Lihat nilai yang BENAR-BENAR dipakai sshd, bukan yang kamu kira
sudo sshd -T | grep -E '^(passwordauthentication|kbdinteractiveauthentication|permitrootlogin) '
# passwordauthentication no
# kbdinteractiveauthentication no
# permitrootlogin no

Step 5: Add the ufw firewall, allowing port 22 first

ufw is Ubuntu's built-in firewall tool. The ufw manual page says it is disabled by default with a default incoming policy of deny, and that enabling it may drop existing connections, such as ssh. The order therefore matters: allow SSH before enable, not after.

# Wrong: aktifkan dulu, izinkan nanti. Sesi SSH-mu bisa putus di tengah jalan.
sudo ufw enable
sudo ufw allow 22/tcp

# Right: izinkan SSH DULU, baru aktifkan
sudo ufw allow 22/tcp
sudo ufw allow 80/tcp          # hanya kalau server ini melayani web
sudo ufw allow 443/tcp         # hanya kalau server ini melayani web
sudo ufw enable
# Command may disrupt existing ssh connections. Proceed with operation (y|n)? y
# Firewall is active and enabled on system startup

sudo ufw status verbose        # baris pertama: Status: active

Open only the ports you actually use. A server with no website needs only port 22. If your SSH runs on another port, allow that port, not 22.

Step 6: Test from a second session, and what to do if you are locked out

Open a second terminal without closing the first, and prove three things: the deploy user gets in with the key, root is refused, and passwords are refused.

# Terminal KEDUA, saat terminal pertama masih terbuka
ssh deploy@203.0.113.10                                   # harus masuk dengan key

ssh root@203.0.113.10                                     # harus ditolak
# root@203.0.113.10: Permission denied (publickey).

ssh -o PubkeyAuthentication=no deploy@203.0.113.10        # password harus ditolak juga
# deploy@203.0.113.10: Permission denied (publickey).

If all three match, only then close the old session. If one fails, the first session is still open to fix the configuration, with no rescue from outside needed.

# Dari console / rescue provider, sebagai root: kembalikan SSH ke keadaan awal
rm /etc/ssh/sshd_config.d/00-hardening.conf
sshd -t
systemctl restart ssh.service

# Kalau penyebabnya firewall
ufw disable

What if the first session is already closed and you are locked out? Open the console or rescue mode from your provider's panel, log in there and undo the settings above. The name and location of the console menu differ between providers, so check your provider's documentation. The sshd_config.original backup from Step 4 is a copy of the starting state if you need to compare.

Common errors and how to fix them

Almost every message below shows up on a beginner's first attempt. Add -v to the ssh command to see which step failed.

Final checklist

• A normal user with sudo rights exists, and sudo whoami prints root.
• Key login works from your laptop without asking for the server password.
• PasswordAuthentication no and PermitRootLogin no show in the sshd -T output.
• ssh as root and password login are both refused from a second terminal.
• ufw is active with port 22 allowed, plus only the other ports you actually need.
• The private key has a passphrase, and a copy is stored somewhere safe other than your laptop.

A secured server can still break for other reasons. The next step I suggest is a 3-2-1 backup strategy for your VPS, so one typo never means lost data.

Learn it in depth

This guide is deliberately practical. If you want to understand why SSH checks file permissions, how known_hosts and authorized_keys work, and how to run a safe SSH lab in Docker on your own computer, take the free lesson.

SSH for beginners: log in to a Linux server with a key (lesson 2, DevOps from zero)

The rule to carry home: never change how you log in while you hold only one session. Add the new route first, prove it works from a second session, then close the old route.