Connect to a VM via SSH

You’ve just created a Linux VM in Azure, the portal shows it as Running, and now you need to get inside. But there’s no friendly desktop button — just a public IP address and a message that says “Connect”. This is the moment every cloud beginner hits: you need a secure shell into your machine, and if you don’t know how to use SSH, you’re stuck at the gate. In this lesson, you’ll learn exactly how to connect to a VM via SSH from your local machine, using the built-in tools that come with every operating system.

The problem this lesson solves

When you deploy a virtual machine in Azure, you rarely get a graphical console by default. For Linux VMs, the standard way to interact is SSH — Secure Shell. Without SSH, you can’t run commands, install software, edit config files, or monitor logs. And if you used Azure’s own SSH keys during creation, you might have downloaded a .pem file — but do you know how to use it? Many beginners copy-paste the public IP into their browser, see a blank page, and give up. This lesson removes that confusion: you’ll learn the four components needed to establish an SSH connection and walk through the exact commands to get from your terminal to a working shell on your Azure VM.

Core concept / mental model

Think of SSH as a secure tunnel between your laptop and your VM. On one end, you have your local machine — the client. On the other, the Azure Linux VM — the server. The tunnel is encrypted, so even if someone snoops on the network, they only see gibberish.

At a high level, an SSH connection needs four things:

1. Address — the public IP or hostname of your VM (e.g., 20.213.123.45 or myvm.eastus.cloudapp.azure.com).
2. Port — by default, 22. Azure allows you to change it, but we’ll stick with the default.
3. Username — the admin username you chose when creating the VM (e.g., azureuser).
4. Authentication — either a password or an SSH private key.

You can think of the private key as a digital ID card. You hold one copy; Azure holds the matching public lock. When you attempt to connect, Azure checks that your key opens the lock. If it does, you’re in.

The whole exchange happens in about half a second. You type a command, see a prompt asking to confirm the host’s fingerprint, and then you’re presented with a $ or # prompt — the shell. From that moment, every command you type runs inside your VM, not on your laptop.

More formally, SSH uses a client-server model. The ssh command is your client; the sshd service running on the VM is the server. They communicate over TCP port 22 using cryptographic handshakes (often RSA or Ed25519 keys).

How it works step by step

Here’s the logical flow that happens between your terminal and your Azure VM:

1. You invoke the SSH client with a command like ssh azureuser@20.213.123.45.
2. DNS resolution (if using a hostname) translates the name to an IP address.
3. TCP handshake — your client and the VM agree on a connection over port 22.
4. Server authentication — the VM sends its host key; your client stores it for future sessions (this is the “yes/no” prompt you see).
5. User authentication — you provide your private key (or password). The server checks it against the public key stored in ~/.ssh/authorized_keys of your user.
6. Session establishment — once authenticated, the server starts a shell for you.
7. Interaction — you run commands; they execute on the VM and return output to your terminal.

If any step fails — wrong IP, key not found, port closed — you get an error message, and we’ll address those in the Troubleshooting section.

Hands-on walkthrough

Let’s actually connect to a running Azure VM. I’ll assume you have a Linux VM named myVM with public IP 20.213.123.45 and a username azureuser. If you used a password, you’ll be prompted for it; if you used an SSH key, you’ll need the private key file.

Step 1: Open your terminal

• On macOS/Linux, open Terminal.
• On Windows, open PowerShell or Windows Terminal. Windows 10+ includes an OpenSSH client built-in, so you don’t need extra software.

Step 2: Find your VM’s public IP

In the Azure portal, go to Virtual machines → your VM → Overview. Note the Public IP address field. If you have a DNS label, you can use that instead.

Step 3: Connect using SSH

Option A — key-based authentication (recommended)

If you downloaded a private key during VM creation, it might be named myVM_key.pem. Change its permissions and connect:

# Make sure the key is not world-readable
chmod 400 ~/Downloads/myVM_key.pem

# Connect using the key
ssh -i ~/Downloads/myVM_key.pem azureuser@20.213.123.45

Option B — password authentication

If you chose a password, simply type:

ssh azureuser@20.213.123.45

You’ll be prompted for the password. Note: your password will not echo as you type.

Step 4: Confirm the host fingerprint

The first time you connect, SSH will ask you to verify the host’s fingerprint. This is a security feature to prevent man-in-the-middle attacks. Type yes and press Enter.

Step 5: You’re in!

You should see a welcome message and a command prompt like:

azureuser@myVM:~$

Now run a command to verify you’re on the remote machine:

hostname

Expected output (your VM name):

myVM

You can now install packages, edit files, or run any command as if you were sitting at the VM’s console.

Alternative: Use the Azure CLI to get the IP

If you prefer the command line, you can get the IP with az vm show:

az vm show -g MyResourceGroup -n myVM -d --query publicIps -o tsv

This prints just the IP, which you can pipe directly into ssh if you’re feeling fancy:

ssh azureuser@$(az vm show -g MyResourceGroup -n myVM -d --query publicIps -o tsv)

Compare options / when to choose what

There’s more than one way to reach your VM. Here’s a quick comparison so you know which approach fits your workflow:

Key insight: For daily work, use your local SSH client. For secure, managed access without a public IP, consider Azure Bastion. If you lose connectivity, the serial console is your rescue tool.

Troubleshooting & edge cases

You will hit errors. Here are the most common ones and how to fix them.

1. “Permission denied (publickey)”

Worse: You’re using a key but the server refuses it.

Fix: Check that you’re using the correct -i path, and that the key is the same one you uploaded to Azure. Also verify the key is not world-readable (must be 400 or 600).

2. “Connection timed out”

Cause: Port 22 is blocked by a Network Security Group (NSG) rule, or your VM has no public IP.

Fix: In the Azure portal, go to your VM’s Networking blade. Ensure there’s an Inbound port rule allowing SSH (22) from your IP or from the Internet. Also verify the VM has a public IP associated.

3. “Host key verification failed”

Cause: The VM’s host key changed (e.g., you recreated the VM with the same IP).

Fix: Remove the old key from ~/.ssh/known_hosts. Run:

ssh-keygen -R 20.213.123.45

Then connect again.

4. “Bad permissions” on key file

On Linux/macOS, SSH refuses keys with open permissions. Fix:

chmod 400 ~/Downloads/myVM_key.pem

5. Password won’t paste

Some terminals don’t paste into SSH password prompts. Use right-click pasting (Windows) or Ctrl+Shift+V (most terminals).

6. Can’t connect from Windows PowerShell

PowerShell versions before 7.4 might not have the OpenSSH client enabled. Install it via Settings → Optional features, or use Windows Terminal with the latest PowerShell.

What you learned & what's next

You now know the core idea behind connecting to a VM via SSH: a secure client-server tunnel that lets you run commands on your Azure Linux VM from any local terminal. You practiced with both key-based and password-based authentication, and you learned how to troubleshoot common failures like timeouts and permission issues.

You applied this in a hands-on walkthrough — you ran your first remote command (hostname) and saw the output. That means you’ve achieved the two learning objectives: explaining the concept and completing a practical exercise.

What’s next: In the next lesson, you’ll learn how to manage your VM’s state — starting, stopping, and restarting it — and how to resize it without downtime. With SSH in your toolkit, you’ll be ready to install and configure software directly on your cloud infrastructure.