Securing DevOps Pipeline with a Private Azure Network: A Step-by-Step Guide

When working with Azure Kubernetes Service (AKS) and private networks, configuring the pipeline to work seamlessly with private resources is crucial.

I recently encountered a challenge while setting up an Azure DevOps pipeline with a private Azure network. Here's how I overcame several blockers, including setting up a NAT gateway, installing necessary tools on my VM (like Docker, Helm, kubectl, Git secrets), and configuring a self-hosted agent in a secure environment.

Let’s dive into the steps I took! 🔒🎯

🚧 The Challenge: A Private Network with Azure DevOps

My current project involves Azure Kubernetes Service (AKS), but I needed to make everything work in a private, secure environment. The VM hosting the Azure DevOps agent was placed in a private subnet, and the pipeline needed to interact with Azure resources and run Docker builds, Kubernetes commands, and security scans, all while maintaining a strict security posture.

Here’s the outline of the journey I took to get everything up and running:

• Setting up a NAT Gateway for outbound access from the VM to the internet.

• Installing necessary tools like Docker, kubectl, Helm, Git secrets, and others.

• Troubleshooting the pipeline, which included Docker-related issues, missing tools, and configuration problems.

  
  

🛠 Step 1: Setting Up the NAT Gateway to Access Azure DevOps

Why the NAT Gateway?

Since my VM was in a private subnet without a public IP, it couldn’t reach Azure DevOps or external resources. To fix this, I configured a NAT Gateway to allow the VM to access the internet while keeping it isolated within the private network.

How I Did It:

• Create a NAT Gateway in Azure: First, I created the NAT Gateway in the same VNET as the VM.

• Assign the NAT Gateway to the Subnet: The next step was to attach the NAT Gateway to the subnet that my VM was using. This ensured the VM could access the internet but still remained private.

After setting up the NAT Gateway, the VM now had access to external resources like Azure DevOps, allowing the agent to properly communicate with the Azure DevOps servers.

🐳 Step 2: Installing Docker and Other Required Tools on the VM

With the NAT Gateway in place, I proceeded to install the tools that the Azure DevOps agent and pipeline would rely on. This included Docker, kubectl, Helm, and Git secrets.

Installing Docker:

Docker was required to build and run images in the pipeline. Here's how I installed it:

Installing kubectl:

kubectl was required to interact with the Kubernetes cluster from the VM.

Installing Helm:

Helm is essential for managing Kubernetes applications via Helm charts. Here’s how I installed it:

Installing Git Secrets:

Git secrets help identify and prevent sensitive data from being pushed into repositories. Here’s how I installed it:

Step 3: Troubleshooting Pipeline Issues

I ran into a few hiccups along the way, including missing tools (like kubectl and docker) and permission issues for the Azure DevOps agent. Here’s how I resolved them:

• Missing kubectl or Docker permissions: After setting up the agent, I found that the agent didn’t have sufficient permissions to interact with the Docker daemon. I solved this by adding the user to the Docker group:

• Missing npm for Git Secrets: as npm was missing, I simply installed Node.js and npm first:

• Helm failed due to missing unzip: Installing unzip resolved the issue:

🎉 Conclusion: Successful Pipeline Setup

After configuring the VM with the required tools, setting up the NAT gateway for internet access, and troubleshooting the agent, the pipeline is now fully operational! Here's a summary of the steps I took:

• Set up a NAT Gateway for outbound internet access from the private subnet.

• Installed necessary tools like Docker, kubectl, Helm, and Git secrets on the VM.

• Configured and connected the Azure DevOps self-hosted agent to the project.

• Looked into common issues like permissions and missing tools in the pipeline.

This setup allows me to have a private Azure DevOps environment, securely building and deploying Kubernetes applications, all while ensuring that sensitive resources remain behind the firewall.

If you’ve faced similar issues or have any questions, feel free to drop a comment or connect with me. Let’s keep learning and growing! 🚀