Monday, December 31, 2018

Can not run elasticsearch as root sonarqube Error - SonarQube Elasticsearch do not run as root

When you tried to configure SonarQube, you may experience the below error:

2018.10.15 17:32:13 ERROR es[][o.e.b.Bootstrap] Exception
java.lang.RuntimeException: can not run elasticsearch as root
        
 ~[elasticsearch-5.6.3.jar:5.6.3]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:195) 

 

SonarQube does not support being run as root user, so we need to create regular user like sonar and give privilege to that use.

Fix for the above issue:
create a user called sonar by executing below command:
sudo useradd sonar

sudo chown -R sonar:sonar sonar-install-folder/


Edit the sonar.sh start script and change the #RUN_AS_USER to be RUN_AS_USER=sonar

sudo vi /opt/sonarqube/bin/linux-x86-64/sonar.sh


Change 
RUN_AS_USER=sonar
now start the server 
sudo sh /opt/sonarqube/bin/linux-x86-64/sonar.sh start

This should fix that issue.

Thursday, December 27, 2018

Jenkins Scripted Pipeline - Create Jenkins Pipeline for Automating Builds, Code quality checks, Deployments to Tomcat - How to build, deploy WARs using Jenkins Pipeline - Build pipelines integrate with Bitbucket, Sonarqube, Slack, JaCoCo, Nexus, Tomcat

What are Pipelines in Jenkins?

- Pipelines are better than freestyle jobs, you can write a lot of complex tasks using pipelines when compared to Freestyle jobs.
- You can see how long each stage takes time to execute so you have more control compared to freestyle.
- Pipeline is groovy based script that have set of plug-ins integrated for automating the builds, deployment and test execution.
- Pipeline defines your entire build process, which typically includes stages for building an application, testing it and then delivering it. 
 - You can use snippet generator to generate pipeline code for the stages you don't know how to write groovy code.
- Pipelines are two types - Scripted pipeline and Declarative pipeline. Click here to know the difference between them.

Pre-requisites:
Install plug-ins
1. Install Pipeline stage view, Deploy to container, Slack, Jacoco, Nexus Artifact Uploader and SonarQube, pipeline stage view plug-ins (if already installed, you can skip it)

Steps to Create Scripted Pipeline in Jenkins

1. Login to Jenkins

2. Create a New item

3. Give name as MyfirstPipelineJob and choose pipeline

4. Click ok. Pipeline is created now

5. Under build triggers, click on poll SCM, schedule as

H/02 * * * *

6. Go to Pipeline definition section, click on Pipeline syntax link. under sample step drop down, choose checkout: Checkout from version control. enter bitbucket or GitHub Repository URL, and enter right credentials. Click here to learn to use PAT if you are using GitHub. scroll down, click on Generate Pipeline script. Copy the code.

7. Now copy the below pipeline code highlighted section into Pipeline section in the pipeline. Please copy stage by stage

8. Change Maven3, SonarQube variables and also Slack channel name as highlighted above in red as per your settings.

9. For Nexus Upload stage, You need to change the Nexus URL and credentials ID for Nexus (which you can grab from Credentials tab after login)

10. For Dev Deploy stage, you can copy credentials ID used for connecting to Tomcat.


Pipeline Code:

node {

    def mvnHome = tool 'Maven3'
    stage ("checkout")  {
       copy code here which you generated from step #6
    }

   stage ('build')  {
    sh "${mvnHome}/bin/mvn clean install -f MyWebApp/pom.xml"
    }

     stage ('Code Quality scan')  {
       withSonarQubeEnv('SonarQube') {
       sh "${mvnHome}/bin/mvn -f MyWebApp/pom.xml sonar:sonar"
        }
   }
  
   stage ('Code coverage')  {
       jacoco()
   }

   stage ('Nexus upload')  {
        nexusArtifactUploader(
        nexusVersion: 'nexus3',
        protocol: 'http',
        nexusUrl: 'nexus_url:8081',
        groupId: 'myGroupId',
        version: '1.0-SNAPSHOT',
        repository: 'maven-snapshots',
        credentialsId: '2c293828-9509-49b4-a6e7-77f3ceae7b39',
        artifacts: [
            [artifactId: 'MyWebApp',
             classifier: '',
             file: 'MyWebApp/target/MyWebApp.war',
             type: 'war']
        ]
     )
    }
   
   stage ('DEV Deploy')  {
      echo "deploying to DEV Env "
      deploy adapters: [tomcat9(credentialsId: '4c55fae1-a02d-4b82-ba34-d262176eeb46', path: '', url: 'http://your_tomcat_url:8080')], contextPath: null, war: '**/*.war'

    }

  stage ('Slack notification')  {
    slackSend(channel:'channel-name', message: "Job is successful, here is the info -  Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]' (${env.BUILD_URL})")
   }

   stage ('DEV Approve')  {
            echo "Taking approval from DEV Manager for QA Deployment"     
            timeout(time: 7, unit: 'DAYS') {
            input message: 'Do you approve QA Deployment?', submitter: 'admin'
            }
     }

stage ('QA Deploy')  {
     echo "deploying into QA Env " 
deploy adapters: [tomcat9(credentialsId: '4c55fae1-a02d-4b82-ba34-d262176eeb46', path: '', url: 'http://your_tomcat_url:8080')], contextPath: null, war: '**/*.war'

}

  stage ('QA notify')  {
    slackSend(channel:'channel-name', message: "QA Deployment was successful, here is the info -  Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]' (${env.BUILD_URL})")
   }

stage ('QA Approve')  {
    echo "Taking approval from QA manager"
    timeout(time: 7, unit: 'DAYS') {
        input message: 'Do you want to proceed to PROD Deploy?', submitter: 'admin,manager_userid'
  }
}

stage ('PROD Deploy')  {
     echo "deploying into PROD Env " 
deploy adapters: [tomcat9(credentialsId: '4c55fae1-a02d-4b82-ba34-d262176eeb46', path: '', url: 'http://your_tomcat_url:8080')], contextPath: null, war: '**/*.war'

}
}

11. Click Apply, Save
12. Now click on Build. It should execute all the stages and show pipeline view like this.




You can watch the Scripted pipeline video in my YouTube channel:

Wednesday, December 26, 2018

Pipeline code for uploading build artifacts to Nexus from Jenkins - How to upload build artifacts to Nexus from Jenkins

Here below is the pipeline code for uploading build artifacts to Nexus from Jenkins:

Prerequisite:
1) Install Nexus Artifact Uploader plug-in.


2) You also need to add Nexus credentials and use the credentials ID from Jenkins

Screen Shot 2018-12-12 at 6.53.40 PM.png


stage ('Nexus Artifact upload')
    {
        nexusArtifactUploader(
        nexusVersion: 'nexus3',
        protocol: 'http',
        nexusUrl: 'ec2-18-223-182-14.us-east-2.compute.amazonaws.com:8081',
        groupId: 'myGroupId',
        version: '1.0-SNAPSHOT',
        repository: 'maven-snapshots',
        credentialsId: '2c293828-9509-49b4-a6e7-77f3ceae7b39',
        artifacts: [
            [artifactId: 'MyWebApp',
             classifier: '',
             file: 'MyWebApp/target/MyWebApp.war',
             type: 'war']
        ]
     )
    }

Sunday, December 16, 2018

How to create S3 bucket in AWS using Terraform - Create S3 bucket in AWS using Terraform

Terraform is an infrastructure orchestration tool for creating web services in AWS automatically. You can use Terraform for provisioning S3 bucket in AWS.

sudo vi create_s3.tf

resource "aws_s3_bucket" "mybucket" {
  bucket = "my-tf-test-bucket"
  acl    = "public-read"
  website {
    index_document = "hello.html"
   routing_rules = <<EOF
[{
    "Condition": {
        "KeyPrefixEquals": "docs/"
    },
    "Redirect": {
        "ReplaceKeyPrefixWith": "documents/"
    }
}]
EOF
  }

  tags= {
    Name        = "My bucket"
    Environment = "Dev"
  }
}

Once you create the above file, execute terraform plan and then terraform apply to create S3 bucket in AWS.

Saturday, December 15, 2018

Ansible Playbook for provisioning a new EC2 instance in AWS - Create a new EC2 Using Ansible Playbook

Please find the Ansible Playbook for provisioning a new EC2 instance. Please follow the below steps in the machine where you installed Ansible.

Steps to create EC2 instance using Ansible:


1. Login to AWS console, click on username and go to My security credentials.
2. Continue on security credentials, click on access keys
3. Create a new access key if you dont have one. Make sure you download the keys.
4. Login to EC2 instance using Git bash or ITerm where you installed Ansible.

execute the below command

sudo vi ~/.boto

add below three lines in the above file, replace the ?? with access key and secret key values.
[Credentials]
aws_access_key_id = ??
aws_secret_access_key = ??





5. Edit Ansible hosts or inventory file
sudo vi /etc/ansible/hosts 
Add the below two lines in the end of the file:
[localhost]
local

6. cd ~
7. mkdir playbooks  
8. cd playbooks

Create Ansible playbook
9. sudo vi create_jenkins_ec2.yml 
(copy the below content in green color)
edit the create_jenkins_ec2.yml to make sure you update the key which is red marked below:
---
 - name:  provisioning EC2 Lab Exercises using Ansible
   hosts: localhost
   connection: local
   gather_facts: False
   tags: provisioning

   vars:
     keypair: MyEC2Key
     instance_type: t2.small
     image: ami-07c1207a9d40bc3bd
     wait: yes
     group: webserver
     count: 1
     region: us-east-2
     security_group: my-jenkins-security-grp
   
   tasks:

     - name: Create my security group
       local_action: 
         module: ec2_group
         name: "{{ security_group }}"
         description: Security Group for webserver Servers
         region: "{{ region }}"
         rules:
            - proto: tcp
              from_port: 22
              to_port: 22
              cidr_ip: 0.0.0.0/0
            - proto: tcp
              from_port: 8080
              to_port: 8080
              cidr_ip: 0.0.0.0/0
            - proto: tcp
              from_port: 80
              to_port: 80
              cidr_ip: 0.0.0.0/0
         rules_egress:
            - proto: all
              cidr_ip: 0.0.0.0/0
       register: basic_firewall
     - name: Launch the new EC2 Instance
       local_action:  ec2 
                      group={{ security_group }} 
                      instance_type={{ instance_type}} 
                      image={{ image }} 
                      wait=true 
                      region={{ region }} 
                      keypair={{ keypair }}
                      count={{count}}
       register: ec2
     - name: Add Tagging to EC2 instance
       local_action: ec2_tag resource={{ item.id }} region={{ region }} state=present
       with_items: "{{ ec2.instances }}"
       args:
         tags:
           Name: MyTargetEc2Instance




10. now execute the ansible playbook by
sudo ansible-playbook create_jenkins_ec2.yml



Fix the warnings by executing below command
pip install --upgrade requests==2.20.1

If everything is good, you should see the new instance created on AWS console. make sure you are able to connect to that instance.

That's it!! That is how you create a new EC2 instance using Ansible.