How to Setup Self Hosted Linux Agent in Azure DevOps | How to configure Self Hosted Agent for Azure Pipelines | Create Build Agent in Azure Cloud

Let us learn how to create and configure a Self-Hosted Agent in Azure DevOps (ADO).

What is an Agent?

An agent is computing infrastructure with installed agent software that runs one job at a time.

To build your code or deploy your software using Azure Pipelines, you need at least one agent. As you add more code and people, you'll eventually need more.

When your pipeline runs, the system begins one or more jobs. 

In Azure pipelines, there are two types of build agents:

1. Microsoft-hosted agents - This is a service totally managed by Microsoft and it's cleared on every execution of the pipeline (on each pipeline execution, you have a fresh new environment).
2. Self-hosted agents - This is a service that you can to set up and manage by yourself. This can be a custom virtual machine on Azure or a custom on-premise machine inside your infrastructure. In a self-hosted agent, you can install all the software you need for your builds, and this is persisted on every pipeline execution. A self-hosted agent can be on Windows, Linux, macOS, or in a Docker container.

Pre-requisites:

• Microsoft account setup
• Azure account and subscription setup
• Create a VM(Ubuntu 18.0.4) in Azure Cloud
• Create Personal Access Token in Azure DevOps

Watch Steps in YouTube channel:

How to configure Self-hosted build agent?

1. Go to Azure DevOps dashboard - https://dev.azure.com/

2. Select your project dashboard

3. Go to your project settings

4. Click on Agent pools

Create a new Agent pool name

Enter name as Ubuntu18-VM-Pool or any name

Make sure you select Grant access permission to all pipelines

click on Ubuntu18-VM-Pool, Agents, New agent

Click on Linux

Note down the steps to configure Linux build agent.

Login to your Azure VM now.

Step #1 - Create the Agent

mkdir myagent && cd myagent

Step #2 - Download the agent

wget https://vstsagentpackage.azureedge.net/agent/2.214.1/vsts-agent-linux-x64-2.214.1.tar.gz

Step #3 - Configure the Agent

tar zxvf vsts-agent-linux-x64-2.214.1.tar.gz

List the files in the directory after extracting.

ls -al

Step #4:

Run the below command:

./config.sh

Accept the Team Explorer Everywhere license agreement now?

Type Y and enter

Step #5:

Enter server URL >
https://dev.azure.com/{yourorganization}

Step #6:

Enter authentication type (press enter for PAT) > PAT

Step #7:

Enter personal access token, generated from this step

Step #8:

Enter Agent pool

Give some name

Step #9:

Enter Agent name --> myBuildAgent_1

Step #10:

Enter work folder > enter

that's it agent is successfully configured.

Configure the Agent to run as a Service

sudo ./svc.sh install &

Execute now to run as a service

./runsvc.sh &

Check the status of build Agent

Click on Ubuntu-18-VM pool name

Click on Agents

This confirms that Build agent is successfully configured in Azure DevOps and is available to run builds.

Steps for removing Agent from the agent pool

Remove the service first

sudo ./svc.sh uninstall

./config.sh remove

To Perform Java related builds on this Agent, make sure you install Java and Maven on this VM.

Install Java 11

sudo apt-get install default-jdk -y

Maven Installation
Maven is a popular build tool used for building Java applications. Please click here to learn more about Maven. You can install Maven by executing below command:

sudo apt update && sudo apt install maven -y

Check if Maven got installed

mvn --version

How to run Ansible playbook from Jenkins pipeline job | Automate EC2 provisioning in AWS using Jenkins and Ansible Playbook | Create new EC2 instance in AWS cloud using Ansible Playbook and Jenkins Pipeline

We will learn how to create new EC2 instance using Ansible playbook and automate using Jenkins Pipeline. 

Watch Steps in YouTube Channel:

Pre-requisites:

• Ansible is installed and Boto is also installed on Jenkins instance
• Ansible plug-in is installed in Jenkins. 
• Make sure you create an IAM role with AmazonEC2FullAccess policy and attach the role to Jenkins EC2 instance.
• Playbook for creating new EC2 instance needs to be created but you can refer my GitHub Repo

Steps:

Create Ansible playbook for provisioning EC2 instance

(Sample playbook is available in my GitHub Repo, you can use that as a reference)

Create Jenkins Pipeline 

pipeline {

    agent any

    stages {

        

        stage ("checkout") {

            steps {

                        checkout([$class: 'GitSCM', branches: [[name: '*/master']], extensions: [],                                                     userRemoteConfigs: [[url: 'https://github.com/akannan1087/myAnsibleInfraRepo']]])         

            }

        }

        stage('execute') {

            steps {

                //to suppress warnings when you execute playbook    

                sh "pip install --upgrade requests==2.20.1"

                // execute ansible playbook

                ansiblePlaybook playbook: 'create-EC2.yml'

            }

        }

    }

}

Execute Pipeline

Pipeline Console output

Ansible playbook for Tomcat Installation on Ubuntu 18.0.4/20.0.4 | Ansible Tomcat Playbook on Ubuntu 18.0.4/20.0.4

Ansible Playbook for installing Tomcat 9 on Ubuntu 18.0.4

sudo vi installTomcat.yml
---
- hosts: My_Group
  tasks:

    - name: Task # 1 Update APT package manager repositories cache

      become: true

      apt:

        update_cache: yes

    - name: Task # 2 - Install Tomcat using Ansible
      become: yes
      apt:
        name: "{{ packages }}"
        state: present
      vars:
        packages:
           - tomcat9

           - tomcat9-examples

           - tomcat9-docs

sudo ansible-playbook installTomcat.yml

This is the execution result of Ansible playbook.

Now access Tomcat on port 8080 in the target machine where you have installed it.

Online DevOps Coaching on AWS and Azure Cloud by Coach AK | Jan 2023 Schedule

Are you in IT? Tired of your work? Are you not able to make any good progress in your career? 

Are you not having a job? Looking for a break in IT? Are you interested in learning DevOps? 

 

Did you get laid off from your previous job due to Covid-19? 

 

If the answer is YES to all the above questions, You are in the right place to kick start your career in DevOps. DevOps is one of the top and hot IT skills right now. Currently almost all the employers are struggling to get right resources in their teams who can do the DevOps and automation work..You could be that person by attending this coaching program.

DevOps Coaching Classes schedules for Jan 2023:

Date	Time	Type	When?
Jan 7th	09:45 AM to 11:30 AM CST on Saturdays          & 10:30 AM to 12:30 pm CST on Sundays	Weekends	Sat/Sun
Jan 19th	6:00 to 8:00 PM CST	Weekdays	Tuesdays/Thursdays

DevOps Coaching Highlights:

- Comprehensive hands on knowledge on Git, Jenkins, Maven, SonarQube, Nexus, Terraform, Ansible, Puppet, Docker, AWS IAM, ECR, Docker registry. AWS and Azure cloud platforms.

- Coach is having about 22+ yrs of professional IT experience, 8+ Yrs in DevOps/Cloud/Automation.

- Many students already got placed in reputed companies from this coaching program successfully.

- Working as a Sr.DevOps Coach/Architect in a one of the top IT services companies in USA.

- Unique program...less theory, more hands on lab exercises

 
- Resume preparation will be done with candidates personally.

- One-to-one Interview coaching.

- Coaching is purely hands on with 101% job relevant.

- 100% Job assistance.

- Coached about 1300+ students successfully for past five years and many of my students got placed with many large enterprises in DFW, Charlotte, Houston, Austin, Chicago, Florida, Seattle, Bay area, Ohio, NJ and NY areas..

To join DevOps Coaching classes, contact coach below:

Contact no # : +1(469)733-5248
Email id: devops.coaching@gmail.com
Contact: Coach AK

If you live in India, please contact assistant coach Gunal to learn about the program:

Name - Gunal

Email id: gunal.j0907@gmail.com

Contact no: +91 87600 02237

How to Deploy Springboot App into AKS cluster using Jenkins Pipeline and Kubectl CLI Plug-in | Deploy Microservices into AKS cluster using Jenkins Pipeline

We are going to learn how to Automate build and deployment of Springboot Microservices App into Azure Kubernetes Cluster(AKS) using Jenkins pipeline. 

Sample springboot App Code:

I have created a sample Springboot App setup in GitHub. Click here to access code base in GitHub. 

Jenkins pipeline will:

- Automate maven build(jar) using Jenkins
- Automate Docker image creation
- Automate Docker image upload into Azure container registry
- Automate Deployments to Azure Kubernetes Cluster

Watch Steps in YouTube Channel:

Pre-requistes:

1. AKS cluster needs to be up running. You can create AKS cluster using any of one of the below options:

• Create AKS cluster in Azure portal directly
• Create AKS cluster using Azure CLI
• Create AKS cluster using Terraform

2. Jenkins instance is setup and running

3. Make sure to Install Docker, Docker pipeline and Kubectl CLI plug-ins are installed in Jenkins

4.  Install Docker in Jenkins and Jenkins have proper permission to perform Docker builds

5. Install Kubectl on Jenkins instance

6. ACR is also setup in Azure cloud. 

7. Make sure AKS has pull access from ACR

8. Dockerfile created along with the application source code for springboot App.

9. Modify K8S manifest file per acr, image name for AKS Deployment.

10. Install Azure CLI on your local machine. (We will be creating the AKS cluster from our local machine)

The Code for this video is here:

Make sure you fork my repo https://github.com/akannan1087/springboot-app

and make necessary changes in jenkins-aks-deploy-from-acr.yaml file after you fork into your account.

Step # 1 - Create Credentials to connect to ACR from Jenkins

Go to Azure Portal console, go to container registry

Settings--> Access keys

Get the username and password 

Go to Jenkins-> Manage Jenkins. Create credentials.

Enter ID as ACR and enter some text for description and Save

Step #2 - Create Credentials for connecting to AKS cluster using Kubeconfig

Go to Jenkins UI, click on Credentials -->

Click on Global credentials

Click on Add Credentials

use secret file from drop down.

you should see the nodes running in EKS cluster.

kubectl get nodes

Execute the below command to get kubeconfig info, copy the entire content of the file:

cat ~/.kube/config

Open your text editor or notepad, copy and paste the entire content and save in a file.

We will upload this file.

Enter ID as K8S and choose File and upload the file and save.

Step # 3 - Create a pipeline in Jenkins

Create a new pipeline job.

Step # 4 - Copy the pipeline code from below
Make sure you change values as per your settings highlighted in yellow below:

pipeline {

  tools {

        maven 'Maven3'

    }

    agent any

        environment {

        //once you create ACR in Azure cloud, use that here

        registryName = "myacrrepo3210"

        //- update your credentials ID after creating credentials for connecting to ACR

        registryCredential = 'ACR'

        dockerImage = ''

        registryUrl = 'myacrrepo3210.azurecr.io'

    }

    

    stages {

        stage('checkout') {

            steps {

                checkout([$class: 'GitSCM', branches: [[name: '*/main']], extensions: [], userRemoteConfigs: [[url: 'check_out_from_your_repo_after_forking_my_repo']]])

            }

        }

        

        stage ('Build') {

        steps {

            sh 'mvn clean install'           

        }

     }

     

    stage ('Build Docker image') {

        steps {

                script {

                    dockerImage = docker.build registryName

                }

            }

        }

        

    // Uploading Docker images into ACR

        stage('Upload Image to ACR') {

         steps{   

             script {

                docker.withRegistry( "http://${registryUrl}", registryCredential ) {

                dockerImage.push()

                }

            }

          }

        }

        

        stage ('K8S Deploy') {

          steps {

            script {

                withKubeConfig([credentialsId: 'K8S', serverUrl: '']) {

                sh ('kubectl apply -f  jenkins-aks-deploy-from-acr.yaml')

                }

            }

        }

     }

    }

}

Step # 5 - Build the pipeline

Step # 6 - Verify deployments to AKS

kubectl get pods

kubectl get services

Steps # 7 - Access Springboot App Deployed in AKS cluster

Once deployment is successful, go to browser and enter above load balancer URL mentioned above

You should see page like below:

Clean up the Cluster:

To avoid charges from Azure, you should clean up unneeded resources. When the cluster is no longer needed, use the az group delete command to remove the resource group, container service, and all related resources. 

az group delete --name myResourceGroup --yes --no-wait

How to Enable Web hooks in Azure Pipeline in Azure DevOps | Enable Web hooks in Azure Pipeline in Azure DevOps | Enable Automate Build in ADO

Webhooks allows developers to trigger jobs in CI server (such as Jenkins or Azure DevOps) for every code changes in SCM. In this article, we will learn how to trigger Azure Pipeline build jobs instantly for every code change in SCM.

Pre-requistes:

1. Azure Devops pipeline already configured. If you dont know how to create Azure pipeline, click on this link.

2. SCM repo have been setup, either in GitHub or Bitbucket or any SCM

Watch Steps in YouTube:

Steps to Enable Webhooks in Azure Build Pipeline

Go to Azure DevOps project dash board.

Go to Pipelines

Click on Pipelines

Click on Edit

Click on Triggers tab, Click Continuous Integration checkbox to enable Webhooks.

Click on Save the Job. You don't have to Queue the job.

Now go to your SCM and make a code change, you will see pipeline will trigger immediately.

No hosted parallelism has been purchased or granted in Azure Devops Pipeline | Azure DevOps Pipeline Error

  

Root cause and Fix:

Microsoft has temporarily disabled the free grant of parallel jobs for public projects and for certain private projects in new organizations. However, you can request this grant by submitting a request. Submit a ticket using below url to request increased parallelism in Azure DevOps. 

https://forms.office.com/pages/responsepage.aspx?id=v4j5cvGGr0GRqy180BHbR63mUWPlq7NEsFZhkyH8jChUMlM3QzdDMFZOMkVBWU5BWFM3SDI2QlRBSC4u