Monday, August 13, 2018

DevOps FAQs page - DevOps Wiki Page - DevOps Troubleshooting Page

Welcome to DevOps FAQs page. The objective of this page is to help you provides fixes for the issues you are facing during building pipelines.

Issue # 1 - Not able to connect to EC2 instance from my local machine and have No such file or directory error.
Root cause and fix:
1. Make sure you have downloaded the above keys under downloads directory.
2. Once you verified, you need to ssh from downloads directory on your local machine. make sure you type below command:

cd downloads (press enter)
and then copy ssh url copied from AWS console (and press enter)

Issue # 2 -  Unable to connect to Bitbucket in Jenkins

Root cause and fix:

There are two issues in the above picture.
1. you need to select add button and add your Bitbucket credentials and select it from drop down after adding it.
2. Do not add Jenkins credentials, it should be Bitbucket credentials.

Issues # 3 - There is no pom in this directory

Root cause and fix

Path of pom.xml was not specified in the build job. you need to click on advanced and add the path of pom.xml. in our case it will be MyWebApp/pom.xml

Issue - 4 Tomcat Deployment failed or not able to access WebApp after deployment

root cause and fix: 

Tomcat deployment steps are not configured correctly in Jenkins job. Click on post build action, add deploy to container step and enter tomcat url and credentials per below screenshot:

Also make sure, tomcat credentials are entered correctly per below screenshot in tomcat-users.xml
Execute the below command
sudo vi /var/lib/tomcat9/conf/tomcat-users.xml 

make sure your file looks like below file:

make sure you restart tomcat as well:
sudo systemctl restart tomcat9

Issue # 5 - SonarQube server can not be reached Error

Root cause and Fix:
When ever you stop EC2 instance and re-start, public DNS name always changes. So you will have to change it every time you restart.
You need to re-enter new sonarqube server URL under Jenkins --> Manage jenkins --> Configure system. scroll down and under sonarQube configuration 

enter SonarQube URL. no need to create token again. old token is enough.

Issue # 6  - Not able to publish into GitHub or Bitbucket using git push

root cause and fix:

This could be due to SSH keys are not set up correctly from the machine you are trying to push the code.

Issue # 7 - Unable to push into Bitbucket or GitHub, getting rejected error

root cause and fix:

When ever you do git push, if changes are not pushed and getting above messages such as rejected. 
You need to perform git pull first by syncing your local repository with remote repo changes.

Issue # 8 - Unable to create EC2 instance

root cause and fix:

Re-name the security group to something else, or use the same group instead of creating another one.

Issue # 9 - Not authorized. Please check the properties sonar.login and sonar.password

root cause and fix:

You need to login in SonarQube as an administrator and create token. Copy the token in Jenkins --> Manage Jenkins--> sonarqube section.

Issue # 10 - Puppet Login credentials error.

root cause and fix:
Make sure you have your access keys and secret keys in the below file and also no space before [default]

sudo vi ~/.aws/credentials

aws_access_key_id = ?

aws_secret_access_key = ?

Issue # 11 - Unable to upload WAR file into Nexus,  WAR file is not uploading into Nexus

Root cause and fix:
You need to choose as Nexus 3 from drop down instead of Nexus2

Issue # 12 - Unable to access Jenkins or Tomcat from the browser

Root cause and Fix:

You need to enter public dns name from step # 4 in below screen. 
Go to AWS console, select EC2 instance, click to connect and copy public dns name value from step #4.

Issue # 13 - Tomcat Deployment failed in Jenkins - Invalid username and password

Root cause and Fix:
It could be due to invalid tomcat user name and password. You need enter right username and password and also select it from drop down.

Also make sure, tomcat credentials are entered correctly per below screenshot in tomcat-users.xml
Execute the below command 

sudo vi /var/lib/tomcat9/conf/tomcat-users.xml 
Make sure you have the below entries properly per screenshot:
<role rolename="manager-script"/>

<user username="tomcat" password="password" roles="manager-gui,manager-script"/>

Make sure you restart tomcat  once you make the above changes:
sudo systemctl restart tomcat9

Issue # 14 - Unable to upload WAR file into Nexus,  Not Authorized error

Root cause and Fix:
Make sure you are using right credentials for Nexus which is admin/admin123, and also you need to select from drop down

Issue # 15 - Invalid Credentials ID when applying TerraForm

Root cause and Fix:
Make sure your AWS access keys and secret keys are correct in your file.

Issue # 16 - SonarQube does not match any configured installation

Root cause and fix:

Go to Jenkins ->Manage Jenkins- > Configure system, sonarqube server installation, make sure the variable name correct. It should match with below name:

stage ('Code quality scan') {

withSonarQubeEnv('SonarQube') {

sh "${mvnHome}/bin/mvn sonar:sonar -f MyWebApp/pom.xml"


Issue # 17 - SonarQube is stopped Error 

Root cause:

Your may not be setup correctly. It could be due to three different reasons:

1. verify below sonar.proeprties
sudo vi /opt/sonarqube/conf/
uncomment the below lines by removing # and add values highlighted yellow

Next, uncomment the below line, removing #

2. Switch to sql shell by entering

Execute the below lines (one by one)

ALTER USER sonar WITH ENCRYPTED password 'password';

CREATE DATABASE sonarqube OWNER sonar; 

type exit tom come out of postgresSQL.

3. Make sure you have setup sonarqube as a service properly.

Execute the below command:
sudo vi /etc/systemd/system/sonar.service

# copy the below green highlighted commands in the above file
Description=SonarQube service


ExecStart=/opt/sonarqube/bin/linux-x86-64/ start
nux-x86-64/ stop

Issue # 18 - Unable to do git clone - make sure you have correct access rights error

root cause and fix

When ever you have this error, it means you have not uploaded your SSH keys correctly into bitbucket or github. Make sure you generate ssh keys by executing the below command:
(enter four times)
Copy the ssh-keys like below and upload into Bitbucket or GitHub.
cat ~/.ssh/

Issue # 19 - Non-Readable POM - No such file or directory
When you try to build in Jenkins, you may get this issue.

Root cause and fix
Java project may not be setup in BitBucket properly. You will have to setup MyWebApp in Bitbucket on the repo you are using in Bitbucket.

Re-visit lab # 2 on how to set up java project or use the repo that has already MyWebApp setup correctly. 

Issue # 20 - sudo: no tty present and no askpass program specified Error

Root cause and fix:
You need to give permission for Jenkins to do deployment.
Login to instance where you installed Jenkins, type the below command

sudo visudo

look for root and add the changes highlighted in yellow at the below line of root:

# User privilege specification
After making the changes, Ctrl O, press enter to save. Ctrl X to exit the file.

Now with the above changes, you are able to execute the build jobs which require sudo permissions.

Issue # 21 -  Unable to connect to GitHub in Jenkins

Root cause and fix:

There are two issues in the above picture.
1. you need to select add button and add your GitHub credentials and select it from drop down after adding it.
2. Do not add Jenkins credentials, it should be GitHub credentials.

Issue # 22 -  Maven Path issue 

Root cause and fix:
1.Remove space before or after /usr/share/maven

Issue # 23 -  Puppet Error: Could not run: Could not find type ec2_instance

Root cause and fix:
You need to install puppet-labs-aws module
sudo /opt/puppetlabs/bin/puppet module install puppetlabs-aws
Issue # 24 -  SonarQube Quality gate issue Error

hudson.remoting.ProxyException: net.sf.json.JSONException: Invalid JSON Stringat org.sonarsource.scanner.jenkins.pipeline.WaitForQualityGateStep$Execution.checkTaskCompleted(

Root cause and fix:
Make sure you fix SonarQube url by removing / in the end under Manage Jenkins->configure system 
you need to remove / in the end of URL. there should not be any / after 9000, it should be something like below:

right url - http://sonarqube_url:9000
Wrong url - http://sonarqube_url:9000/

Issue # 25 -  Error can not perform an interactive login from a non TTY device

Root cause and fix:
remove existing access + secret keys if you have attached in EC2 instance by performing below command:
sudo rm ~/.aws/config
sudo rm ~/.aws/credentials

Then you need to create an IAM role with AmazonEC2ContainerRegistryFullAccess policy.