How to Automate Infra setup in Azure Cloud using Terraform and Azure DevOps | Automate App Service or Web App creation in Azure cloud using Terraform and Azure DevOps and store Terraform state remotely

Automating infrastructure setup in Azure Cloud using Terraform and Azure DevOps is a powerful approach to managing your infrastructure as code (IaC). This allows you to define and manage your Azure resources programmatically, version control your infrastructure configurations, and automate the deployment process. 

We will be creating a Java based WebApp(app service) in Azure cloud using Terraform and automate the process using Azure Devops Pipelines.

Watch Steps in YouTube channel:

Prerequistes:

• Azure subscription
• Azure DevOps organization and project
• Terraform Build & Release Tasks extension in Azure DevOps
• Terraform files created and stored in SCM for creating resources in Azure cloud

Implementation Steps:

• Create a resource group first in Azure cloud
• Create storage account, container for Terraform to store state information in Azure cloud remotely.
• Create pipeline and add Terraform tasks
• Execute pipeline to deploy resources(App service plan and WebApp) in Azure cloud
• Verify resources are created in Azure cloud
• Confirm if the terraform.tfstate file is updated

Create Resource Group in Azure cloud

Login to Azure portal, Create a new RG, or you can skip this step if you already have existing group.

Create Storage Account in Azure cloud

Create a new resource, type storage account

Click on create

select RG, enter unique account name, select standard as performance, select locally redundant storage

Click on create

Create a container inside the storage account

Select the storage account you just created, click on containers under Data storage

enter a name for the container

Create a pipeline in Azure DevOps

Login to Azure Devops, select Pipelines, select use the classic editor to create a pipeline

Configure Pipeline with tasks

Add Terraform installer task to install Terraform on build agent

Add Terraform task 

Configure the task

enter storage account, container, key information as below:

Add terraform tasks to plan, apply

Add Terraform task for plan

change the command to plan from drop down and also select azure subscription from drop down

Add Terraform task for apply

change the command to apply from drop down and also select azure subscription from drop down

Now verify to make sure if we have correct values in each/every task. Now click on Save + Queue.

Click on Save and Run

This confirms that pipeline have successfully created resources in Azure cloud. You can login to Azure portal to see the resources - app service plan and web app.

You can also verify terraform state info which has resources entry for all the resources created

Click on Containers, mytfstatecontainer

Click on terraform.tfstate

Click on Edit to view the content of terraform state file

Clean up resources created in Azure using the pipeline - destroy command

change the command to destroy instead of apply from the drop down

You can either add a new task for destroy or modify to destroy from apply in the existing task.

Now save the pipeline and run the pipeline. check the output of destroy task

Check terraform.tfstate file after destroy.

How to connect to Azure Virtual Machine from your local machine? | Connect to Azure VM from your local machine | Connect to Azure VM from your local machine

Let's learn how to connect to a Virtual machine running in Azure cloud from your local machine. Your local machine can be a Windows laptop or MacBook laptop.

Pre-requisites:

1. Keys(for e.g., yourkey.pem) already downloaded in your local machine, preferably in downloads folder.
2. Azure VM is up and running
3. SSH client - for Windows laptop, you need to install an SSH client such as Git bash or putty. You can download Git from this URL - https://git-scm.com/downloads. For Apple laptop you need to download iTerm from here.

Steps to connect to your Azure VM instance:

1. Go to Azure console --> https://portal.azure.com

2. Click on Virtual machines, click on VM

3. Click on connect

Choose Native SSH

Enter SSH key name in step # 3 in the screen

4. copy the values from step # 4

 

Copy the url from SSH which looks like below:

For e.g.

ssh -i myAzVMKey.pem azureuser@104.43.214.12

Watch steps in YouTube channel:

Windows Laptop instructions
5. Go to your local machine, Open Git Bash in Windows

make sure you are in downloads directory where your keys got downloaded. Type the below commands:

type below commands: 

pwd

this should tell you which directory you are and then navigate to downloads dir.

cd  ~/downloads 

Now copy the value from Example in the above screen

ssh -i myAzVMKey.pem azureuser@104.43.214.12

and then type enter, say yes and enter

now you should be in Azure cloud, screen should show something like this, It means you are successfully connected to VM instance running on Azure cloud. 

Mac Book Laptop or iMac Instructions

Open iTerm window, type the below command to go to downloads directory.

cd downloads

For few Mac laptops, it may add .txt in the end of pem file. in that case you need to remove .txt in the end

ssh -i myAzVMKey.pem azureuser@104.43.214.12

pwd
and then execute below command to make sure the keys have only read permissions.

chmod 400 *.pem

6. Paste the url from example highlighted above in step # 4.

ssh -i myAzVMKey.pem azureuser@104.43.214.12

7. type yes when it is asking to connect.

8. now you should be in Azure cloud, screen should show something like this, It means you are successfully connected to VM instance running on Azure cloud. 

Automate Azure cloud infrastructure setup using Ansible and Azure DevOps pipeline | How to integrate Ansible with Azure DevOps | Integrating Ansible with Azure DevOps Pipelines |

Ansible is an open-source, configuration management tool that automates cloud provisioning, configuration management, and application deployments. Using Ansible you can provision virtual machines, containers, network, and complete cloud infrastructures. 

Automate Azure cloud infrastructure setup using Ansible and Azure pipeline

Integrate Ansible with Azure Cloud

Integrating Ansible with Microsoft Azure allows you to automate and manage your Azure infrastructure using Ansible playbooks and modules. Ansible provides a collection of Azure-specific modules that enable you to provision and configure resources in Azure.

To configure Azure credentials, you need the following information:

• Your Azure subscription ID and tenant ID
• The service principal application ID and secret

Pre-requisites:

• Azure account subscription, click here if you don't have one.
• Service principal to create any resources in Azure cloud using Azure cloud shell or Azure CLI

Create Azure Service Principal

Run the following commands to create an Azure Service Principal:

az ad sp create-for-rbac --name <service-principal-name> \ 

--role Contributor \ 

--scopes /subscriptions/<subscription_id>

Save the above output in a file as you will not be able retrieve later.

Create an Ansible playbook

Create a simple playbook to create resource group in Azure. Make sure you modify the name of the resource group and location below.

---

- hosts: localhost

  connection: local

  tasks:

    - name: Creating resource group

      azure_rm_resourcegroup:

        name: "my-rg12"

        location: "eastus"

Create Azure YAML build pipeline:

Login to Azure Devops --> https://dev.azure.com

Select project dashboard.

Go to Pipelines -> New pipeline --> Click on Azure Repos Git or any SCM where you have playbooks stored. Select repo, click on Starter pipeline.

Add below four pipeline variables with value received from service principal creation.

AZURE_SUBSCRIPTION_ID

AZURE_CLIENT_ID

AZURE_SECRET

AZURE_TENANT

Add below tasks:

• Install Ansible on build agent
• Install Ansible rm module on build agent
• Execute Ansible playbook for creating resource group in Azure cloud.

trigger:

- main

pr: none # Disable PR triggers, can be adjusted as needed

pool:

vmImage: 'ubuntu-latest'

steps:

- script: |

# Install Ansible

pip3 install "ansible==2.9.17"

displayName: 'Install Ansible'

- script: |

# Install Ansible rm module

pip3 install ansible[azure]

displayName: 'Install Ansible rm module'

- script: |

# Run Ansible playbook to create resource group

ansible-playbook create-rg.yml

displayName: 'Run Ansible Playbook'

env:

AZURE_SUBSCRIPTION_ID: $(AZURE_SUBSCRIPTION_ID)

AZURE_CLIENT_ID: $(AZURE_CLIENT_ID)

AZURE_SECRET: $(AZURE_SECRET)

AZURE_TENANT: $(AZURE_TENANT)

Save the pipeline and run it.

Now Login to Azure cloud to see if the resource group have been created.

Watch Steps in YouTube channel:

Install Ansible on Red Hat Linux | How to setup Ansible on Red Hat Linux VM | Ansible install on Azure Linux Virtual Machine | Ansible Azure Integration

How to setup Ansible on Red Hat Linux VM and Integrate with Azure Cloud?

Ansible is #1 configuration management tool. It can also be used for infrastructure provisioning as well. or You can use Ansible in combination of Terraform which can take care of infra automation and Ansible can do configuration management. We will be setting up Ansible on Red Hat VM in Azure cloud And create some resources in Azure Cloud by using Ansible playbooks.

 

Ansible Architecture:

 

The best way to install Ansible in Linux is to use PIP, a package manager for Python.

Pre-requisites:

• Create new Red Hat VM in Azure Cloud for setting up Ansible, just open port 22 in firewall rule.
• Install Azure CLI on Red Hat Linus VM
• Service principal to create any resources in Azure cloud using Azure CLI

How to setup Ansible on Red Hat Linux VM

Watch Steps in YouTube channel:

Change host name to AnsibleMgmtNode
sudo hostnamectl set-hostname AnsibleMgmtNode

Update Repository
sudo yum update -y

Install Python-pip3

sudo yum install python3-pip -y

Upgrade pip3 sudo pip3 install --upgrade pip

# Install Ansible pip3 install "ansible==2.9.17"

check Ansible version

ansible --version

# Install Ansible azure_rm module for interacting with Azure.

pip3 install ansible[azure]

Authenticate with Azure

To configure Azure credentials, you need the following information:

• Your Azure subscription ID and tenant ID
• The service principal application ID and secret

Create an Azure Service Principal

Login to Azure first

az login

Enter Microsoft credentials

Run the following commands to create an Azure Service Principal:

az ad sp create-for-rbac --name <service-principal-name> \ 

--role Contributor \ 

--scopes /subscriptions/<subscription_id>

Save the above output in a file as you will not be able retrieve later.
Configure the Ansible credentials using one of the following techniques:

• Option 1: Create an Ansible credentials file

• Option 2: Define Ansible environment variables

Option 1: Create Ansible credentials file

In this section, you create a local credentials file to provide credentials to Ansible. For security reasons, credential files should only be used in development environments.

mkdir ~/.azure 

vi ~/.azure/credentials

Insert the following lines into the file. Replace the placeholders with the service principal values.

[default] subscription_id=<subscription_id> client_id=<service_principal_app_id> secret=<service_principal_password> tenant=<service_principal_tenant_id>

Option 2: Define Ansible environment variables

On the host virtual machine, export the service principal values to configure your Ansible credentials.

export AZURE_SUBSCRIPTION_ID=<subscription_id> export AZURE_CLIENT_ID=<service_principal_app_id> export AZURE_SECRET=<service_principal_password> export AZURE_TENANT=<service_principal_tenant_id>

Test Ansible installation

You now have a virtual machine with Ansible installed and configured!

This section shows how to create a test resource group within your new Ansible configuration. If you don't need to do that, you can skip this section.

• Option 1: Use an ad-hoc ansible command
• Option 2: Write and run an Ansible playbook

Option 1: Use an ad-hoc ansible command

Run the following ad-hoc Ansible command to create a resource group:

ansible localhost -m azure_rm_resourcegroup -a "name=my-rg123 location=eastus"

Option 2: Write and run an Ansible playbook

Create a simple playbook to create resource group in Azure.

sudo vi create-rg.yml

---

- hosts: localhost

  connection: local

  tasks:

    - name: Creating resource group

      azure_rm_resourcegroup:

        name: "myResourceGroup"

        location: "eastus"

Execute the playbook using ansible-playbook command.

ansible-playbook create-rg.yml

Now Login to Azure cloud to see if the resource group have been created.

Clean up Resources

Save the following code as delete-rg.yml

sudo vi delete-rg.yml

--- - hosts: localhost tasks: - name: Deleting resource group - "{{ name }}" azure_rm_resourcegroup: name: "{{ name }}" state: absent register: rg - debug: var: rg

ansible-playbook delete-rg.yml --extra-vars "name=myResourceGroup"

check in Azure cloud to see if the resource group have been deleted.