Optimizing Jenkins CI/CD pipelines is crucial for achieving faster, more efficient, and reliable software delivery. Here are some best practices and strategies for optimizing Jenkins pipelines:
1. Parallelization:
Parallel Stages: Break down your pipeline into stages and parallelize independent stages to run concurrently. This can significantly reduce the overall pipeline execution time.
stages {
stage('Build') {
steps {
script {
parallel(
unit_tests: {
// Run unit tests
},
integration_tests: {
// Run integration tests
}
)
}
}
}
// Other stages...
}
2. Artifact Caching:
Use Caches: Utilize Jenkins' built-in caching mechanisms to store and retrieve build artifacts between different pipeline runs. This reduces the time spent on redundant build steps.
Node Pools: Distribute builds across multiple Jenkins agents or node pools to leverage available resources effectively. Adjust the number of executors on each agent based on workload.
Only Build Changes: Set up your pipeline to trigger builds only for changes in relevant branches. Use tools like Git SCM polling or webhooks to trigger builds on code changes.
5. Artifact Promotion:
Promote Artifacts: Promote artifacts from one environment to another instead of rebuilding them. This helps in maintaining consistency across environments and reduces build times.
6. Pipeline DSL Optimization:
Code Reusability: Use shared libraries and functions to avoid duplicating code across multiple pipeline scripts. This promotes code reusability and simplifies maintenance.
7. Conditional Execution:
When Conditions: Use the when directive to conditionally execute stages based on certain criteria, such as branch names or environment variables.
stage('Deploy to Production') {
when {
expression { params.DEPLOY_TO_PROD == 'true' }
}
steps {
// Deployment steps
}
}
8. Artifact Cleanup:
Clean Workspace: Include a step to clean up the workspace at the end of each build to avoid accumulation of unnecessary artifacts and files.
post {
always {
cleanWs()
}
}
9. Pipeline Visualization:
Blue Ocean: Consider using the Blue Ocean plugin for Jenkins, which provides a more visually appealing and intuitive view of your pipeline.
10. Monitoring and Analytics:
Collect Metrics: Implement monitoring and analytics to collect data on pipeline performance. Identify bottlenecks and areas for improvement.
11. Pipeline as Code:
Declarative Syntax: Use the declarative syntax for Jenkins pipeline scripts whenever possible. It is more concise and easier to read.
12. Use Jenkins Shared Libraries:
Library Usage: If you have common functionality across multiple pipelines, consider moving that logic into a shared library. This promotes code reuse and centralizes maintenance.
13. Artifact Signing and Verification:
Security Checks: Integrate security checks into your pipeline, including artifact signing and verification steps, to ensure the integrity and authenticity of your artifacts.
14. Automated Testing:
Automated Tests: Include automated tests for your pipeline scripts to catch issues early. Jenkins provides testing frameworks like Jenkins Pipeline Unit for this purpose.
15. Infrastructure as Code:
Infrastructure Automation: Treat your Jenkins infrastructure as code. Use tools like Docker and Kubernetes for scalable and reproducible Jenkins environments.
We will automate building a Java project configured in GitHub and we will use Maven to build, package in Azure YAML pipeline and upload WAR file into Artifactory.
select your SCM, in my case, Java Web App code is in GitHub.
Select the Java Repo, select Maven from the option. This will generate YAML file for us.
Add a task for uploading WAR file.
Click on Show assistant, search for generic and select JFrog Generic Artifacts
Choose Upload as command, choose service connection for Artifactory, enter *.war as pattern, enter repo name where artifact will be uploaded. Click on Add.
Now save the pipeline.. Run the pipeline
Azure DevOps YAML Pipeline Code
using the pipeline code we can automate build using Maven and upload WAR file into Artifactory
# Build your Java project using Maven and Upload WAR file to Artifactory
Let's say you have forgotten Jenkins admin password. How to reset or retrieve admin password? Here below are the steps for resetting Jenkins admin password:
1. Edit the Jenkins config.xml under /var/lib/jenkins to disable securityFlag by executing below command: sudo vi /var/lib/jenkins/config.xml
you need to disable security to Jenkins by below change 2. Find this <useSecurity>true</useSecurity> and change it to <useSecurity>false</useSecurity> 3. Restart jenkins by below command: sudo service jenkins restart 4. Access Jenkins now, click on Manage Jenkins, click on configure Global security
5. click on Enable security
6. Click on Jenkins Own user database option.
7. click on apply and save 8. Now click on Manage users, click on admin, click on configure
Artifactory is one of the popular binary repository managers. It is Java based open source tool, used for storing build artifacts and docker images.
Some of the key features of Artifactory:
Supports 27 different package types including helm charts, docker images regardless of tech stack.
A single source of truth for all your binaries
Integration with all CICD tools
role based authorization with teams to manage artifacts
you can create local, remote and virtual repositories
Artifactory can be integrated with many Continuous integration and Continuous delivery tools. Artifactory is mainly used by Ant, Maven and Gradle build tools. Let us see how to install Artifactory on Ubuntu 22.0.4 using Deb packages.
Pre-requisites:
VM needs to have at least 4GB RAM, for AWS choose at least medium instance type.
Default ports 8081 and 8082 needs to be opened. 8081 for Artifactory REST APIs. 8082 for everything else (UI, and all other product’s APIs).
Change Host Name to Artifactory sudo hostnamectl set-hostname Artifactory
Update Ubuntu OS sudo apt update
Add JFrog Artifactory APT repository
echo "deb https://releases.jfrog.io/artifactory/artifactory-debs xenial main" | sudo tee -a /etc/apt/sources.list.d/artifactory.list
Import repository GPG key by running the following commands