How to Deploy Springboot App into AKS cluster using Jenkins Pipeline and Kubectl CLI Plug-in | Deploy Microservices into AKS cluster using Jenkins Pipeline

We are going to learn how to Automate build and deployment of Springboot Microservices App into Azure Kubernetes Cluster(AKS) using Jenkins pipeline. 

Sample springboot App Code:

I have created a sample Springboot App setup in GitHub. Click here to access code base in GitHub. 

Jenkins pipeline will:

- Automate maven build(jar) using Jenkins
- Automate Docker image creation
- Automate Docker image upload into Azure container registry
- Automate Deployments to Azure Kubernetes Cluster

Watch Steps in YouTube Channel:

Pre-requistes:

1. AKS cluster needs to be up running. You can create AKS cluster using any of one of the below options:

• Create AKS cluster in Azure portal directly
• Create AKS cluster using Azure CLI
• Create AKS cluster using Terraform

2. Jenkins instance is setup and running

3. Make sure to Install Docker, Docker pipeline and Kubectl CLI plug-ins are installed in Jenkins

4.  Install Docker in Jenkins and Jenkins have proper permission to perform Docker builds

5. Install Kubectl on Jenkins instance

6. ACR is also setup in Azure cloud. 

7. Make sure AKS has pull access from ACR

8. Dockerfile created along with the application source code for springboot App.

9. Modify K8S manifest file per acr, image name for AKS Deployment.

10. Install Azure CLI on your local machine. (We will be creating the AKS cluster from our local machine)

The Code for this video is here:

Make sure you fork my repo https://github.com/akannan1087/springboot-app

and make necessary changes in jenkins-aks-deploy-from-acr.yaml file after you fork into your account.

Step # 1 - Create Credentials to connect to ACR from Jenkins

Go to Azure Portal console, go to container registry

Settings--> Access keys

Get the username and password 

Go to Jenkins-> Manage Jenkins. Create credentials.

Enter ID as ACR and enter some text for description and Save

Step #2 - Create Credentials for connecting to AKS cluster using Kubeconfig

Go to Jenkins UI, click on Credentials -->

Click on Global credentials

Click on Add Credentials

use secret file from drop down.

you should see the nodes running in EKS cluster.

kubectl get nodes

Execute the below command to get kubeconfig info, copy the entire content of the file:

cat ~/.kube/config

Open your text editor or notepad, copy and paste the entire content and save in a file.

We will upload this file.

Enter ID as K8S and choose File and upload the file and save.

Step # 3 - Create a pipeline in Jenkins

Create a new pipeline job.

Step # 4 - Copy the pipeline code from below
Make sure you change values as per your settings highlighted in yellow below:

pipeline {

  tools {

        maven 'Maven3'

    }

    agent any

        environment {

        //once you create ACR in Azure cloud, use that here

        registryName = "myacrrepo3210"

        //- update your credentials ID after creating credentials for connecting to ACR

        registryCredential = 'ACR'

        dockerImage = ''

        registryUrl = 'myacrrepo3210.azurecr.io'

    }

    

    stages {

        stage('checkout') {

            steps {

                checkout([$class: 'GitSCM', branches: [[name: '*/main']], extensions: [], userRemoteConfigs: [[url: 'check_out_from_your_repo_after_forking_my_repo']]])

            }

        }

        

        stage ('Build') {

        steps {

            sh 'mvn clean install'           

        }

     }

     

    stage ('Build Docker image') {

        steps {

                script {

                    dockerImage = docker.build registryName

                }

            }

        }

        

    // Uploading Docker images into ACR

        stage('Upload Image to ACR') {

         steps{   

             script {

                docker.withRegistry( "http://${registryUrl}", registryCredential ) {

                dockerImage.push()

                }

            }

          }

        }

        

        stage ('K8S Deploy') {

          steps {

            script {

                withKubeConfig([credentialsId: 'K8S', serverUrl: '']) {

                sh ('kubectl apply -f  jenkins-aks-deploy-from-acr.yaml')

                }

            }

        }

     }

    }

}

Step # 5 - Build the pipeline

Step # 6 - Verify deployments to AKS

kubectl get pods

kubectl get services

Steps # 7 - Access Springboot App Deployed in AKS cluster

Once deployment is successful, go to browser and enter above load balancer URL mentioned above

You should see page like below:

Clean up the Cluster:

To avoid charges from Azure, you should clean up unneeded resources. When the cluster is no longer needed, use the az group delete command to remove the resource group, container service, and all related resources. 

az group delete --name myResourceGroup --yes --no-wait

How to Enable Web hooks in Azure Pipeline in Azure DevOps | Enable Web hooks in Azure Pipeline in Azure DevOps | Enable Automate Build in ADO

Webhooks allows developers to trigger jobs in CI server (such as Jenkins or Azure DevOps) for every code changes in SCM. In this article, we will learn how to trigger Azure Pipeline build jobs instantly for every code change in SCM.

Pre-requistes:

1. Azure Devops pipeline already configured. If you dont know how to create Azure pipeline, click on this link.

2. SCM repo have been setup, either in GitHub or Bitbucket or any SCM

Watch Steps in YouTube:

Steps to Enable Webhooks in Azure Build Pipeline

Go to Azure DevOps project dash board.

Go to Pipelines

Click on Pipelines

Click on Edit

Click on Triggers tab, Click Continuous Integration checkbox to enable Webhooks.

Click on Save the Job. You don't have to Queue the job.

Now go to your SCM and make a code change, you will see pipeline will trigger immediately.

No hosted parallelism has been purchased or granted in Azure Devops Pipeline | Azure DevOps Pipeline Error

  

Root cause and Fix:

Microsoft has temporarily disabled the free grant of parallel jobs for public projects and for certain private projects in new organizations. However, you can request this grant by submitting a request. Submit a ticket using below url to request increased parallelism in Azure DevOps. 

https://forms.office.com/pages/responsepage.aspx?id=v4j5cvGGr0GRqy180BHbR63mUWPlq7NEsFZhkyH8jChUMlM3QzdDMFZOMkVBWU5BWFM3SDI2QlRBSC4u

CICD Process Flow Diagram | Implement CICD using Jenkins

 CICD Process Flow Diagram - Implement CICD using Jenkins

What is Continuous Integration?

Continuous integration is a DevOps software development practice where developers regularly merge their code changes into a central repository, after which automated builds and tests are run.

The key goals of continuous integration are to find and address bugs quicker, improve software quality, and reduce the time it takes to validate and release new software updates.

How does Continuous Integration Work?

Developers frequently commit to a shared repository using a version control system such as Git. Prior to each commit, developers may choose to run local unit tests on their code as an extra verification layer before integrating. A continuous integration service automatically builds and runs unit tests on the new code changes to immediately surface any errors.

Benefits of Continuous Integration

• Improve Developers productivity 
• Find bugs early in the software development stage
• Deliver products into market place sooner
• Improve the feedback loop

What is Continuous Delivery?

Continuous delivery is a software development practice where code changes are automatically prepared for a release to production. Continuous delivery is the next extension of continuous integration. The delivery phase is responsible for packaging an artifact together to be delivered to end-users. This phase runs automated building tools to generate this artifact.

Benefits of Continuous Delivery

• Automate the Software Release Process
• Improve Developer Productivity
• Find bugs early in the software development stage
• Deliver updates faster

How to Recover SonarQube Admin password | How to unlock SonarQube Admin password?

Let's say you have setup SonarQube using Docker or Docker Compose, you have forgotten the admin password for SonarQube. This article helps you to reset/recover the admin password. If you changed and then lost the admin password, you can reset it using the following steps.

Watch Steps in YouTube channel:

Pre-requisites:

• SonarQube is up and running using Docker image
• Database is either MSSQL or PostgreSQL

As we have configured SonarQube using Docker compose, We need to login to PostgreSQL running inside postgres docker container and execute update command to reset to default password.

Step 1: Login into PostgreSQL docker container

type below command to see the list of containers running in your EC2 instance.

sudo docker ps

Copy the container ID from above command. 

Now login into PostgresSQL docker container

docker exec -it <container_id> /bin/bash

Step 2:  Connect to PostgreSQL database by executing below command:

psql -p 5432 -d sonarqube -U sonar -h <container_id>

now enter the password for sonarqube database:

from my lab exercise, password for sonar user is admin123

Make sure it shows sonarqube which is your database schema inside PostgresSQL db.

Step 3: Execute the below query to change admin password to default password which is also admin

update users set crypted_password='100000$t2h8AtNs1AlCHuLobDjHQTn9XppwTIx88UjqUm4s8RsfTuXQHSd/fpFexAnewwPsO6jGFQUv/24DnO55hY6Xew==', salt='k9x9eN127/3e/hf38iNiKwVfaVk=', hash_method='PBKDF2', reset_password='true', user_local='true' where login='admin';

Step 4: Login to SonarQube UI and login as admin/admin

Login as admin/admin

Now it will immediately ask you to change the default admin password to something else:

That's it! That is how you recover SonarQube admin password.

References:

https://docs.sonarqube.org/latest/instance-administration/security/

How To Setup Jenkins using BootStrap Scripts in AWS EC2 Instance while Launching | How to Run commands on your EC2 instance at launch

 

What is bootstrap script in aws?

If you want to execute some commands during boot up(launch), you can execute it easily by loading script in user data section during EC2 launch. Bootstrap scripts run only once - when the instance is instantiated for the 1st time.

Please follow the below steps to create an EC2 instance.  We will be installing Java, Maven and Jenkins during boot up.

How to create EC2 instance in AWS console?

What is EC2 instance? 

It is virtual server provided by AWS. We will be using this EC2 to setup Jenkins. Please follow the below steps to create an EC2 instance.

Pre-requisites:

Script contains instructions for installing Jenkins

Steps:
1: Login to AWS console by clicking this link -->  https://aws.amazon.com/console/

click on All services, Click on Compute -->  Click on EC2

2. Click on Launch instance

3. Enter Name as Jenkins-EC2 and enter 1 as number of instance

4. Select Ubuntu and choose Ubuntu server 18.0.4 as AMI

5. Enter t2.small as instance type

6. You can choose existing Key

7. Under Network settings, Click Edit

Add port range as 8080 and select AnyWhere as Source Type, that should enter 0.0.0.0/0 as Source

8. Enter 10 GB as storage 

Steps to add bootstrap script during EC2 launch

Click on Advanced Details:

go to User Data section and Copy the script from this link.

9. Click on Launch Instance.

Click on View instances

Now you should be able to view instances in AWS console. 

once EC2 is provisioned, you can login and you will be able to see Java, Maven and Jenkins installed in EC2 instance after launch.

Check the Console Output Logs in EC2 instance

Login to EC2 instance, and type the below command:

tail -f /var/log/cloud-init-output.log

This will give the output of bootstrap execution

Verify if Java got installed.

java -version
mvn --version

Go to the browser and try to access Jenkins in the browser, Jenkins should be coming up.(make sure you open port 8080 in the firewall rules)

Enter Jenkins Admin password:

Execute below command to get Jenkins admin password

sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Enter the password and click on continue

Click on Suggested plug-ins and setup the admin user for Jenkins.

Watch the steps in YouTube channel:

Deploy Springboot Microservices App into Amazon EKS Cluster using Jenkins Pipeline and Kubectl CLI Plug-in | Containerize Springboot App and Deploy into EKS Cluster using Jenkins Pipeline

We will learn how to automate springboot microservices builds using Jenkins pipeline and Deploy into AWS EKS Cluster with help of Kubernetes CLI plug-in.

We will use Springboot Microservices based Java application. I have already created a repo with source code + Dockerfile. The repo also have Jenkinsfile for automating the following:

- Automating builds using Jenkins
- Automating Docker image creation
- Automating Docker image upload into AWS ECR
- Automating Docker Containers Deployments to Kubernetes Cluster

 

 

Watch steps in YouTube channel:

Same Code for this video is here:

Make sure you fork my repo https://github.com/akannan1087/springboot-app

Pre-requistes:
1. Amazon EKS Cluster is setup and running. Click here to learn how to create Amazon EKS cluster.

2. Create ECR repo in AWS

3. Jenkins Master is up and running

4. Docker installed on Jenkins instance 

5. Docker, Docker pipeline and Kubernetes CLI plug-ins are installed in Jenkins

6. Install kubectl on your instance

Step # 1 - Create Maven3 variable under Global tool configuration in Jenkins

Make sure you create Maven3 variable under Global tool configuration. 

Step #2 - Create Credentials for connecting to Kubernetes Cluster using kubeconfig

Click on Add Credentials, use Kubernetes configuration from drop down.

use secret file from drop down.

execute the below command to login as jenkins user.

sudo su - jenkins

you should see the nodes running in EKS cluster.

kubectl get nodes

Execute the below command to get kubeconfig info, copy the entire content of the file:

cat /var/lib/jenkins/.kube/config

Open your text editor or notepad, copy and paste the entire content and save in a file.

We will upload this file.

Enter ID as K8S and choose File and upload the file and save.

Enter ID as K8S and choose enter directly and paste the above file content and save.

Step # 3 - Create a pipeline in Jenkins

Create a new pipeline job.

Step # 4 - Copy the pipeline code from below
Make sure you change red highlighted values below as per your settings:
Your docker user id should be updated.
your registry credentials ID from Jenkins from step # 1 should be copied

pipeline {

   tools {

        maven 'Maven3'

    }

    agent any

    environment {

        registry = "account_id.dkr.ecr.us-east-2.amazonaws.com/my-docker-repo"

    }

   

    stages {

        stage('Cloning Git') {

            steps {

                checkout([$class: 'GitSCM', branches: [[name: '*/main']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: '', url: 'https://github.com/akannan1087/springboot-app']]])     

            }

        }

      stage ('Build') {

          steps {

            sh 'mvn clean install'           

            }

      }

    // Building Docker images

    stage('Building image') {

      steps{

        script {

          dockerImage = docker.build registry 

        }

      }

    }

   

    // Uploading Docker images into AWS ECR

    stage('Pushing to ECR') {

     steps{  

         script {

                sh 'aws ecr get-login-password --region us-east-2 | docker login --username AWS --password-stdin account_id.dkr.ecr.us-east-2.amazonaws.com'

                sh 'docker push account_id.dkr.ecr.us-east-2.amazonaws.com/my-docker-repo:latest'

         }

        }

      }

       stage('K8S Deploy') {

        steps{   

            script {

                withKubeConfig([credentialsId: 'K8S', serverUrl: '']) {

                sh ('kubectl apply -f  eks-deploy-k8s.yaml')

                }

            }

        }

       }

    }

}

Step # 5 - Build the pipeline

Once you create the pipeline and changes values per your configuration, click on Build now:

Step # 6 - Verify deployments to K8S

kubectl get pods

kubectl get deployments

kubectl get services

Steps # 7 - Access SpringBoot App in K8S cluster

Once build is successful, go to browser and enter master or worker node public ip address along with port number mentioned above

http://loadbalancer_ip_address

You should see page like below:

Note:

Make sure you fork my repo https://github.com/akannan1087/springboot-app

and make changes in eks-deploy-k8s.yaml to pull Docker image from your AWS ECR repo.