Connect to Your VM via SSH

So you've provisioned an Azure Linux VM — maybe through the portal, maybe with az vm create — and now you're staring at a public IP address wondering what to do next. The painful truth is that a VM without a secure way to reach its shell is just a monthly bill, and the fastest path to that shell is SSH. In this lesson, you'll learn how to connect to your VM via SSH — from generating keys on your local machine to running your first remote command — and you'll come away with the confidence to manage any Azure Linux VM from your terminal, whether you're on macOS, Windows, or Linux.

The problem this lesson solves

You've deployed a VM, but now what? Without SSH, you're locked out of the command line — you can't install packages, configure services, or debug application failures. The portal's "Run command" feature is clunky and limited; it's not a real terminal. And the most common mistake beginners make is trying to use a password instead of key-based authentication, which is both less secure and surprisingly harder to set up on Azure.

Pro tip: Azure disables password authentication by default for Linux VMs created via the CLI. If you try to SSH with a password and get Permission denied (publickey), that's why.

Core concept / mental model

Think of SSH as a secure tunnel between your laptop and your Azure VM. The tunnel is authenticated using a key pair: a private key that stays on your machine (like your house key) and a public key that you place on the VM (like a lock that only your key can open). When you attempt to connect, the server checks that your private key matches the public key it holds — no passwords sent over the wire.

Azure makes this easy by letting you inject the public key during VM creation, or you can add it later. Once SSH is working, you have a full remote shell — you can run sudo apt update, edit config files with vim, or tail application logs, all as if you were sitting at the machine's keyboard.

How it works step by step

1. Generate a key pair on your local machine using ssh-keygen — this creates id_rsa (private) and id_rsa.pub (public) in ~/.ssh/.
2. Provide the public key to Azure — either at VM creation time with --ssh-key-values or by later appending it to ~/.ssh/authorized_keys on the VM.
3. Connect with ssh azureuser@<public-ip> — the SSH client and server negotiate encryption, verify your private key, and open the tunnel.
4. Run commands — once connected, every keystroke is sent securely to the remote shell.

The first time you connect, you'll see a host authenticity prompt — type yes to accept the VM's fingerprint. This is your protection against man-in-the-middle attacks, so don't ignore it.

Hands-on walkthrough

Step 1: Generate your SSH key pair

Open a terminal (or PowerShell with OpenSSH client) on your local machine and run:

ssh-keygen -t rsa -b 4096 -C "you@example.com" -f ~/.ssh/azure_vm_key

Expected output (paraphrased):

Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/you/.ssh/azure_vm_key
Your public key has been saved in /home/you/.ssh/azure_vm_key.pub

Pro tip: Add a passphrase to encrypt your private key at rest. You'll type it each time you connect, but it prevents misuse if your laptop is lost.

Step 2: Create a VM with your public key

Use the az CLI to create the VM, passing the public key content directly:

az vm create \
  --resource-group MyResourceGroup \
  --name MyVM \
  --image Ubuntu2204 \
  --admin-username azureuser \
  --ssh-key-values ~/.ssh/azure_vm_key.pub

Expected output: a JSON blob with publicIpAddress — note it down.

Step 3: Connect to your VM via SSH

Now the moment you've been waiting for — connect:

ssh -i ~/.ssh/azure_vm_key azureuser@<public-ip>

If it's your first time, you'll see:

The authenticity of host '40.118.90.45 (40.118.90.45)' can't be established.
ED25519 key fingerprint is SHA256:...
Are you sure you want to continue connecting? Type 'yes' and hit Enter.

Type yes, and you're in! You should see a prompt like azureuser@MyVM:~$. Try a quick command:

uname -a

Output shows the kernel info — proof you're connected to the VM.

Step 4: Add the public key to an existing VM

If you already created a VM without SSH keys, use az vm user update to inject your key:

az vm user update \
  --resource-group MyResourceGroup \
  --name MyVM \
  --username azureuser \
  --ssh-key-value ~/.ssh/azure_vm_key.pub

Then connect as above.

Compare options / when to choose what

For most scenarios, key-based SSH is the way to go. Azure Bastion shines when you need to restrict public exposure entirely. az ssh is handy for on-the-go troubleshooting from a web browser.

Troubleshooting & edge cases

Pro tip: If you're on Windows, use the built-in OpenSSH client (available in PowerShell). If you encounter issues, ensure the ssh-agent service is running and your key is added with ssh-add.

What you learned & what's next

You now own the core skill of remote administration: connect to your VM via SSH. You can generate key pairs, provision a VM with your public key, tackle common connectivity errors, and decide when a password, Bastion, or az ssh makes more sense. This unlocks the next lesson in the Azure Tutorial track, where you'll use this SSH connection to configure your VM for real workloads — perhaps installing a web server or connecting to a managed database.

Keep your private key safe, practice connecting a few times, and remember: every cloud engineer's journey starts with a single ssh command.