Showing posts with label Docker. Show all posts
Showing posts with label Docker. Show all posts

Saturday, April 6, 2024

GitHub Actions Pipeline to Create Docker Image and Push Docker Image into DockerHub | GitHub Actions Integration with DockerHub

 Please find steps for integrating DockerHub with GitHub Actions:


Implementation steps:

  • Create access token in DockerHub
  • Add access token, docker hub user name as secrets in GitHub Actions
  • Create GitHub Actions workflow yaml in your repo
  • Add tasks for Maven build, docker image creation, tagging and docker push 
  • Run the workflow/build in GitHub hosted runner(e.g. Ubuntu)
  • Verify docker image have been uploaded into DockerHub

Pre-requisites:

Steps below to create access token in DockerHub:

Click on new access token:
    Copy on Generate

 

Add Docker Hub user name and token as Secrets in GitHub Actions

Go to your GitHub Repo --> Settings --> 

Click on Secrets and Variables under Security in left nav 
Click new Repository Secret


Enter DOCKERHUB_USERNAME as secret name and 
Enter your docker hub user name


Enter DOCKERHUB_TOKEN as secret name and 
Enter your token as secret



Create GitHub Actions CICD workflow yaml:

Go to GitHub repo where your Java project is, create a new file:

.github/workflows/cicd.yml


The below file have four steps(tasks) 
    - Checkout
    - Install Java on runner
    - Build using Maven
    - Build docker image and tag it
    - Upload docker image into DockerHub

Copy below YAML code:

name: cicd-workflow to create docker image and upload into Dockerhub
on:
  push:
    branches: [ "master" ]
jobs:
  job1:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - name: Set up JDK 17
      uses: actions/setup-java@v2
      with:
        distribution: 'adopt'
        java-version: '17'
    - name: Build with Maven
      run: mvn clean install
    - name: Login to Docker Hub
      uses: docker/login-action@v3
      with:
        username: ${{ secrets.DOCKERHUB_USERNAME }}
        password: ${{ secrets.DOCKERHUB_TOKEN }}
    - name: Build and push Docker image
      env:
        IMAGE_TAG: ${{ github.sha }}
      run: |
        docker build -t ${{ secrets.DOCKERHUB_USERNAME }}/myspringbootapp:${IMAGE_TAG} .
        docker push ${{ secrets.DOCKERHUB_USERNAME }}/myspringbootapp:${IMAGE_TAG}


Commit the file.

As soon as you commit, build will run immediately in GitHub Actions. 
Now you can see the output of build in Actions tab.


Login to DockerHub to verify if docker image have been uploaded successfully.


Watch steps in YouTube channel:

Wednesday, April 5, 2023

SonarQube Server Cannot be reached Error Resolution | SonarQube Troubleshooting | Fix for max virtual memory areas vm.max_map_count [65530] is too low

 How to Resolve SonarQube Server cannot be reached Error?

Pre-requisites:

refer this page for pre-requisites to install SonarQube


Resolution:

Make sure EC2 instance or server where you are installing SonarQube has enough virtual memory and open file limits for  Linux OS.

How to change default value for vm.max_map_count temporarily
sudo sysctl vm.max_map_count=262144
sudo sysctl fs.file-max=65536
The above command will load the max_map_count values and open file limits till the next system restart.

How to update vm.max_map_count directly in sysctl.conf permanently
Login to instance where you will be installing SonarQube, perform the below command to configure virtual memory permanently for SonarQube to function:
sudo vi /etc/sysctl.conf

Add the following lines to the bottom of that file:

vm.max_map_count=262144
fs.file-max=65536

To make sure changes are getting into effect:
sudo sysctl -p

Make sure SonarQube is up and running by checking the logs
sudo docker-compose logs --follow


Once you see the message, that's it. SonarQube have been configured successfully. press control C and enter.

Watch Steps in YouTube channel: 

Monday, February 27, 2023

How to Create a Docker Image for Springboot App and Push Docker image into Amazon ECR from Azure DevOps Pipelines | Azure DevOps Pipelines to Build and Push a Docker image to AWS ECR

Amazon Elastic Container Registry (ECR) is a fully managed Docker container registry that makes it easy to store, share, and deploy container images. We will learn how to build docker image for a springboot microservices app using Azure DevOps(ADO) build pipeline and push docker image into AWS ECR.



What are we going to do in this lab?
1. Create a Repository in AWS ECR for storing docker images
2. Create an IAM user and AmazonEC2ContainerRegistryFullAccess policy.
3. Create access keys for IAM user in AWS console
4. Create service connection in AzureDevOps to connect to AWS using IAM user access & secret keys.
5. Create Azure DevOps Build pipeline with below tasks:
  • maven build for building JAR
  • build docker image
  • push docker image into ECR
6. Verify if docker image has been pushed to AWS ECR

Pre-requisites:

Watch Steps in YouTube channel:

Step 1 - Create a repo in ECR 

Go to AWS console, type ECR



Click on Create Repository

Enter name for your repo - all lower case and Click create repository


Create an IAM user

Go to AWS console --> IAM --> Add Users

Enter name for the user
Search for EC2 and choose AmazonEC2ContainerRegistryFullAccess
Click on Create User

Create security credentials

Click on user name ecr-user
Click on Security credentials



Create Access key

Create Service connection 

Go to Project settings --> Service Connections


Enter Access keys and Secret keys

Enter Connect name and select Grant access to all pipelines
Click on save

Create a classic Azure Build pipeline

Click on use the classic editor

Select GitHub and choose your spring-boot project and click continue



Choose a template for the pipeline, type docker and select docker container

Click on Apply

Select build Agent for the Pipeline
Choose Ubuntu latest as build agent


Add Pipeline variables
imageName as springboot-app
repoName as my-springboot-repo

Add Maven task for building springboot JAR file
Make sure Maven task is moved up..it should be a first task

enter maven goas as install


Customize build an image Task 

start customizing the task, choose the version as 2.0
Enter $(imageName) as  Container repository 
Select build as command from drop down
Tags as it have shown below
Remove push an image task



Add ECR push task


now configure the task

Select as shown in screenshot
Choose aws service connection from drop down
select region as per your settings.
select Image ID
enter as $(imageName):$(Build.BuildId)


Repo Name as $(repoName)
$(Build.BuildId) as tar repo tag


Save and Queue
Select ubuntu latest as build agent..do NOT select window agents.

Now make sure build is successful.


Verify if Docker image has been pushed into AWS ECR

Now login to AWS console --> Go to ECR--> select your repo. verify if image has been uploaded successfully.

Tuesday, January 31, 2023

How to Setup JFrog Artifactory using Docker-Compose | Install Artifactory using Docker Compose | Install Artifactory using Docker Compose on Ubuntu 18.0.4

How to setup Artifactory using Docker compose?

Artifactory is open source, binary repository manager. Artifactory is the single solution for managing all the artifacts, binaries, files and containers throughout your software supply chain. 

Some of the key features of Artifactory:
  • Supports 27 different package types including helm charts, docker images regardless of tech stack.
  • A single source of truth for all your binaries
  • Integration with all CICD tools
  • role based authorization with teams to manage artifacts 
  • you can create local, remote and virtual repositories
What is Docker Compose?
Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration. Since Docker Compose lets you configure related containers in a single YAML file, you get the same Infrastructure-as-Code abilities as Kubernetes. But they come in a simpler system that’s more suited to smaller applications that don’t need Kubernetes’ resiliency and scaling.
 
The purpose of docker-compose is to function as docker cli but to issue multiple commands much more quickly. To make use of docker-compose, you need to encode the commands you were running before into a docker-compose.yml file
 
Run docker-compose up and Compose starts and runs your entire app.

Watch the Steps in YouTube channel: 

Pre-requisites:
  • Ubuntu EC2 up and running with at least t2.medium (4GB RAM), 2GB will not work
  • Port 8081, 8082 is opened in security firewall rule
  • instance should have docker-compose installed

Change Host Name to Artifactory
sudo hostnamectl set-hostname Artifactory

Perform System update
sudo apt update

Install Docker-Compose
sudo apt install docker-compose -y

Create docker-compose.yml
this yml has all the configuration for installing Artifactory on Ubuntu EC2.
sudo vi docker-compose.yml 

(Copy the below code high-lighted in yellow color)
version: "3.3"
services:
  artifactory-service:
    image: docker.bintray.io/jfrog/artifactory-oss:7.49.6
    container_name: artifactory
    restart: always
    networks:
      - ci_net
    ports:
      - 8081:8081
      - 8082:8082
    volumes:
      - artifactory:/var/opt/jfrog/artifactory

volumes:
  artifactory:
networks:
  ci_net:

Save the file by entering :wq!

Now execute the compose file using Docker compose command to start Artifactory Container
sudo docker-compose up -d 

-d means detached mode

Make sure Artifactory is up and running
sudo docker-compose logs --follow


Once you see the message, that's it. Artifactory is been setup successfully. Now press Control C and enter to come out of the above screen.

Check Artifactory is up and running by typing below command:
curl localhost:8081
This confirms that Artifactory is up and running locally.

How to access Artifactory in the browser?

Now access Artifactory UI by going to browser and enter public dns name with port 8081
http://change to_artifactory_publicdns_name:8081

How to integrate Artifactory with Jenkins?

How to stop Artifactory container
sudo docker-compose down



Sunday, September 4, 2022

Deploy Springboot Microservices App into Amazon EKS Cluster using Jenkins Pipeline and Kubectl CLI Plug-in | Containerize Springboot App and Deploy into EKS Cluster using Jenkins Pipeline

We will learn how to automate springboot microservices builds using Jenkins pipeline and Deploy into AWS EKS Cluster with help of Kubernetes CLI plug-in.

We will use Springboot Microservices based Java application. I have already created a repo with source code + Dockerfile. The repo also have Jenkinsfile for automating the following:

- Automating builds using Jenkins
- Automating Docker image creation
- Automating Docker image upload into AWS ECR
- Automating Docker Containers Deployments to Kubernetes Cluster
 


 
Watch steps in YouTube channel:

Same Code for this video is here:

Pre-requisites:
1. Amazon EKS Cluster is setup and running. Click here to learn how to create Amazon EKS cluster.
5. Docker, Docker pipeline and Kubernetes CLI plug-ins are installed in Jenkins




6. Install kubectl on your instance

Step # 1 - Create Maven3 variable under Global tool configuration in Jenkins
Make sure you create Maven3 variable under Global tool configuration. 


Step #2 - Create Credentials for connecting to Kubernetes Cluster using kubeconfig
Click on Add Credentials, use Kubernetes configuration from drop down.

use secret file from drop down.


execute the below command to login as jenkins user.
sudo su - jenkins

you should see the nodes running in EKS cluster.

kubectl get nodes


Execute the below command to get kubeconfig info, copy the entire content of the file:
cat /var/lib/jenkins/.kube/config


Open your text editor or notepad, copy and paste the entire content and save in a file.
We will upload this file.

Enter ID as K8S and choose File and upload the file and save.


Enter ID as K8S and choose enter directly and paste the above file content and save.

Step # 3 - Create a pipeline in Jenkins
Create a new pipeline job.


Step # 4 - Copy the pipeline code from below
Make sure you change red highlighted values below as per your settings:
Your docker user id should be updated.
your registry credentials ID from Jenkins from step # 1 should be copied

pipeline {
   tools {
        maven 'Maven3'
    }
    agent any
    environment {
        registry = "account_id.dkr.ecr.us-east-2.amazonaws.com/my-docker-repo"
    }
   
    stages {
        stage('Cloning Git') {
            steps {
                checkout([$class: 'GitSCM', branches: [[name: '*/main']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: '', url: 'https://github.com/akannan1087/springboot-app']]])     
            }
        }
      stage ('Build') {
          steps {
            sh 'mvn clean install'           
            }
      }
    // Building Docker images
    stage('Building image') {
      steps{
        script {
          dockerImage = docker.build registry 
        }
      }
    }
   
    // Uploading Docker images into AWS ECR
    stage('Pushing to ECR') {
     steps{  
         script {
                sh 'aws ecr get-login-password --region us-east-2 | docker login --username AWS --password-stdin account_id.dkr.ecr.us-east-2.amazonaws.com'
                sh 'docker push account_id.dkr.ecr.us-east-2.amazonaws.com/my-docker-repo:latest'
         }
        }
      }

       stage('K8S Deploy') {
        steps{   
            script {
                withKubeConfig([credentialsId: 'K8S', serverUrl: '']) {
                sh ('kubectl apply -f  eks-deploy-k8s.yaml')
                }
            }
        }
       }
    }
}

Step # 5 - Build the pipeline
Once you create the pipeline and changes values per your configuration, click on Build now:


Step # 6 - Verify deployments to K8S

kubectl get pods



kubectl get deployments

kubectl get services



Steps # 7 - Access SpringBoot App in K8S cluster
Once build is successful, go to browser and enter master or worker node public ip address along with port number mentioned above
http://loadbalancer_ip_address

You should see page like below:



Note:

and make changes in eks-deploy-k8s.yaml to pull Docker image from your AWS ECR repo.