How to Implement CICD Pipeline using GitLab Yaml | GitLab CICD Tutorials | GitLab CICD Pipeline | Build Java WAR file using GitLab CICD YAML file

Here below is the code for creating GitLab CICD yaml file for Java Web App project to automate build and deployment. 

What is GitLab CICD?

GitLab CI/CD is a continuous integration and continuous deployment solution built into GitLab.

GitLab CI/CD

GitLab CI/CD is a feature of GitLab that automates:

• Building code
• Testing applications
• Scanning code
• Deploying applications

whenever developers push code into Git repositories.

What is .gitlab-ci.yml?

The .gitlab-ci.yml file is the heart of GitLab CI/CD pipelines.

It contains:

• Pipeline stages
• Jobs
• Scripts
• Variables
• Artifacts
• Deployment instructions

GitLab automatically reads this file whenever code changes are pushed into the repository. GitLab Runner uses a Docker container image to run the job. 

.gitlab-ci.yml for implementing CICD using GitLab

stages:

  - build

  - deploy

build_war:

  stage: build

  image: maven:3.8.6-eclipse-temurin-11

  script:

    - echo "Building WAR file using Maven"

    - mvn clean install -f MyWebApp/pom.xml

    - echo "Listing target directory"

    - ls -la MyWebApp/target

  artifacts:

    paths:

      - MyWebApp/target/*.war

    expire_in: 1 hour

deploy_to_tomcat:

  stage: deploy

  image: curlimages/curl:latest

  dependencies:

    - build_war

  script:

    - echo "Deploying WAR file to Tomcat running on AWS EC2"

    - |

      curl -v -u ${TOMCAT_USER}:${TOMCAT_PASSWORD} \

      -T MyWebApp/target/MyWebApp.war \

      "http://${TOMCAT_HOST}/manager/text/deploy?path=/MyWebApp&update=true"

GitHub Actions CICD Pipeline to Deploy Java WebApp into Azure App Service | Integration GitHub Actions with Azure App Service

 

Pre-requisites:

• Make sure Java web app is setup in GitHub
• Azure subscription to create web app

What are we going to do in this lab?

1. Create a Web App in Azure Cloud

2. Configure WebApp to Deploy using gitHub Actions

3. Create workflow yaml

4. Add steps/tasks in the yaml file

5. Run the workflow yaml

6. Check if Java Web App is deployed in Azure App Service

How to Create WebApp in Azure Portal?

1. Login portal.azure.com
2. Click on App services

3.Click on + Add or click on Create app service


Click on Web App. Choose your Azure subscription, usually Pay as you Go or Free trial subscription
Create a new resource group or you can use existing resource group)

Enter App service name(it should be unique)
Publish as Code
Run time stack as Java 17

Java Web Server stack --> Tomcat 10.0
Operating System as Linux
Region as Central US or where ever you are based at

Enter LinuxPlan name
Choose pricing plan

Now go to Deployment tab:

Enable basic authentication

and enable Continuous Deployment 

Click on GitHub account, Authorize.

Authorize AzureappService

now select organization, repo, branch

You can also click on preview file to get pipeline YAML code 

Click on Review and Create

Create Web App

Now make sure AzureAppService_PublishProfile secret is automatically created in GitHub repo you selected.

Create GitHub Actions CICD workflow yaml:

name: Build and deploy WAR app to Azure Web App
on:
  push:
    branches:
      - main
  workflow_dispatch:
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Set up Java version
        uses: actions/setup-java@v2
        with:
          java-version: '11'
          distribution: 'adopt'
      - name: Build with Maven
        run: mvn clean install -f MyWebApp/pom.xml
      - name: Upload artifact for deployment job
        uses: actions/upload-artifact@v3
        with:
          name: MyWebApp
          path: '${{ github.workspace }}'
  deploy:
    runs-on: ubuntu-latest
    needs: build
    environment:
      name: 'Production'
      url: ${{ steps.deploy-to-webapp.outputs.webapp-url }}
    steps:
      - name: Download artifact from build job
        uses: actions/download-artifact@v3
        with:
          name: MyWebApp
      - name: Deploy to Azure Web App
        id: deploy-to-webapp
        uses: azure/webapps-deploy@v2
        with:
          app-name: 'spingbootwebapp'
          slot-name: 'Production'
          publish-profile: ${{ secrets.AZUREAPPSERVICE_PUBLISHPROFILE_76B948D486E54ED7B06775D572207D40 }}
          package: '*.war'

Check the output after running the pipeline:

Verify if WebApp has been deployed into Azure App Service by browsing Web App url.

https://mysuperjavaapp.azurewebsites.net/MyWebApp/

Watch here all the steps in YouTube channel:

How to Create Quality Gate in SonarQube and integrate with GitHub Actions | SonarQube Integration with GitHub Actions | Automate Code Scan using SonarQube In GitHub Actions and Force build to Fail or Pass

 

Pre-requisites:

• Make sure SonarQube is up and running
• Make sure Java Project is setup in GitHub
• SonarQube is already integrated with GitHub Actions
• SONAR_HOST_URL and SONAR_TOKEN configured as secrets in GitHub Repo

How to Create Quality gate in SonarQube and integrate with GitHub Actions?

Make sure SonarQube is up and running and integrated with GitHub Actions. Please click here if you would like to setup SonarQube and integrate with GitHub Actions.

We will be executing below steps:

• Login to SonarQube
• Create Quality Gate in SonarQube
• Add conditions in Quality Gate
• Make quality gate as Default
• Create GitHub Actions CICD workflow yaml
• Add tasks for Maven build and Sonar Scan
• Add tasks for integrating Quality gate 
• pass/fail the builds in SonarQube

What is Quality gate?

In SonarQube a quality gate is a set of conditions that must be met in order for a project to be marked as passed.

Create Quality Gate

Login to SonarQube, Click on Quality gate, enter some name

Once you create the quality gate. Click on Add condition. 

Select new issues from the drop down and enter 2 

Select new bugs from the drop down and enter 1 as error

Setup a Default Gate

Create GitHub Actions CICD workflow yaml:

Go to GitHub repo where your Java project is, create a new file:

.github/workflows/cicd.yml

The below file have four steps(tasks) 

    - Checkout

    - Install Java on runner

    - Build using Maven

    - run Sonar Scan (this task need to have projectKey defined, otherwise build will fail)

    - run quality gate check

    - pass/fail the build

Copy the the whole yellow color marked content from below:

name: CI/CD workflow for Maven Build, Sonar Code scan and Quality gate check
on:
  push:
    branches:
      - main
  workflow_dispatch:
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - name: Checkout code
      uses: actions/checkout@v3
    - name: Set up JDK 11
      uses: actions/setup-java@v2
      with:
        distribution: 'adopt'
        java-version: '11'
    - name: Build with Maven
      run: mvn install -f MyWebApp/pom.xml
    - name: SonarQube Scan
      uses: sonarsource/sonarqube-scan-action@master
      with:
        projectBaseDir: .
        args: >
          -Dsonar.organization=my-org
          -Dsonar.projectKey=my-Java-web-app
      env:
        SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
        SONAR_HOST_URL: ${{ secrets.SONAR_HOST_URL }}
    # Check the Quality Gate status.
    - name: SonarQube Quality Gate check
      id: sonarqube-quality-gate-check
      uses: sonarsource/sonarqube-quality-gate-action@master
      # Force to fail step after specific time.
      timeout-minutes: 5
      env:
       SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
       SONAR_HOST_URL: ${{ secrets.SONAR_HOST_URL }} #OPTIONAL
    # Show the output from the Quality Gate.
    # The possible outputs of the `quality-gate-status` variable are `PASSED`, `WARN` or `FAILED`.
    - name: "Here is SonarQube Quality Gate Status value.."
      run: echo "The Quality Gate status is ${{ steps.sonarqube-quality-gate-check.outputs.quality-gate-status }}"

Commit the file.

As soon as you commit, build will run immediately in GitHub Actions. 

Now you can see the output of build in Actions tab.

Now login to SonarQube to see the Scan report

If your code have any defects, you can see some build fails.

SonarQube Quality gate failed:

Watch Steps in YouTube channel:

Notes:

You can also refer the documentation below from below websites.

https://docs.sonarsource.com/sonarqube/latest/devops-platform-integration/github-integration/

https://github.com/marketplace/actions/official-sonarqube-scan

https://github.com/marketplace/actions/sonarqube-quality-gate-check

Jenkins CI/CD Pipeline Optimization Best Practices | Optimizing Jenkins CI/CD pipelines

Optimizing Jenkins CI/CD pipelines is crucial for achieving faster, more efficient, and reliable software delivery. Here are some best practices and strategies for optimizing Jenkins pipelines:

1. Parallelization:

• Parallel Stages: Break down your pipeline into stages and parallelize independent stages to run concurrently. This can significantly reduce the overall pipeline execution time.
• 
  
• stages { stage('Build') { steps { script { parallel( unit_tests: { // Run unit tests }, integration_tests: { // Run integration tests } ) } } } // Other stages... }

2. Artifact Caching:

• Use Caches: Utilize Jenkins' built-in caching mechanisms to store and retrieve build artifacts between different pipeline runs. This reduces the time spent on redundant build steps.
• 
  
• pipeline { options { // Enable build caching buildDiscarder(logRotator(numToKeepStr: '5')) caches { gradle 'gradle-wrapper' } } // Pipeline stages... }

3. Agent Utilization:

• Node Pools: Distribute builds across multiple Jenkins agents or node pools to leverage available resources effectively. Adjust the number of executors on each agent based on workload.
• 
  
• pipeline { agent { label 'docker' } // Pipeline stages... }

4. Incremental Builds:

• Only Build Changes: Set up your pipeline to trigger builds only for changes in relevant branches. Use tools like Git SCM polling or webhooks to trigger builds on code changes.

5. Artifact Promotion:

• Promote Artifacts: Promote artifacts from one environment to another instead of rebuilding them. This helps in maintaining consistency across environments and reduces build times.

6. Pipeline DSL Optimization:

• Code Reusability: Use shared libraries and functions to avoid duplicating code across multiple pipeline scripts. This promotes code reusability and simplifies maintenance.

7. Conditional Execution:

• When Conditions: Use the when directive to conditionally execute stages based on certain criteria, such as branch names or environment variables.
• 
  
• stage('Deploy to Production') { when { expression { params.DEPLOY_TO_PROD == 'true' } } steps { // Deployment steps } }

8. Artifact Cleanup:

• Clean Workspace: Include a step to clean up the workspace at the end of each build to avoid accumulation of unnecessary artifacts and files.
• post { always { cleanWs() } }

9. Pipeline Visualization:

• Blue Ocean: Consider using the Blue Ocean plugin for Jenkins, which provides a more visually appealing and intuitive view of your pipeline.

10. Monitoring and Analytics:

• Collect Metrics: Implement monitoring and analytics to collect data on pipeline performance. Identify bottlenecks and areas for improvement.

11. Pipeline as Code:

• Declarative Syntax: Use the declarative syntax for Jenkins pipeline scripts whenever possible. It is more concise and easier to read.

12. Use Jenkins Shared Libraries:

• Library Usage: If you have common functionality across multiple pipelines, consider moving that logic into a shared library. This promotes code reuse and centralizes maintenance.

13. Artifact Signing and Verification:

• Security Checks: Integrate security checks into your pipeline, including artifact signing and verification steps, to ensure the integrity and authenticity of your artifacts.

14. Automated Testing:

• Automated Tests: Include automated tests for your pipeline scripts to catch issues early. Jenkins provides testing frameworks like Jenkins Pipeline Unit for this purpose.

15. Infrastructure as Code:

• Infrastructure Automation: Treat your Jenkins infrastructure as code. Use tools like Docker and Kubernetes for scalable and reproducible Jenkins environments.