Provision Ubuntu 18.0.4 EC2 Instance | How to create EC2 instance in AWS console | Launch Ubuntu 18.0.4 instance in AWS

 How to create EC2 instance in AWS console using new UI experience?

What is EC2 instance? 

It is virtual server provided by AWS. We will be using this EC2 to setup both Jenkins and Tomcat. Please follow the below steps to create an EC2 instance.

Watch here for live demo:

Steps:
1: Login to AWS console by clicking this link -->  https://aws.amazon.com/console/

click on All services, Click on Compute -->  Click on EC2

2. Click on Launch instance

3. Enter Name as EC2 and enter 2 as number of instances

4. Select Ubuntu and choose Ubuntu server 18.0.4 as AMI

5. Enter t2.small as instance type

6. Click on Create new Key Pair

7. Choose the existing key pair if you have one, otherwise create new one, give some name as myJenkinsKey. Make sure you download the key in your local machine. Please do NOT give space or any character while naming the key.

8. Under Network settings, Click Edit

Add port range as 8080 and select AnyWhere as Source Type, that should enter 0.0.0.0/0 as Source

9. Enter 10 GB as storage 

And then make sure in Summary, values appear as below:

10. Click on Launch Instance.

Click on View instances

Now you should be able to view instances in AWS console.

Connect to EC2 instance from local machine:
Please click the below link to understand the steps for connecting to EC2 instance from your local machine - windows or Apple laptop.

http://www.cidevops.com/2018/02/how-to-connect-to-ec2-instance-from.html

Create Amazon EKS cluster by eksctl | How to create EKS cluster in AWS cloud using eksctl | Create EKS Cluster in command line using IAM Role

What is Amazon EKS

Amazon EKS is a fully managed container orchestration service. EKS allows you to quickly deploy a production ready Kubernetes cluster in AWS, deploy and manage containerized applications more easily with a fully managed Kubernetes service.

EKS takes care of master node/control plane. We need to create worker nodes.

EKS cluster can be created in following different ways

1. AWS console

2. AWS CLI

3. eksctl command

4. using Terraform

We will create EKS cluster using eksctl command line tool.

Please watch the steps in YouTube channel:

Pre-requistes:

This Lab is using Jenkins EC2 instance. Jenkins EC2 instance needs to have following configured:

• Install AWS CLI – Command line tools for working with AWS services, including Amazon EKS.

• Install eksctl – A command line tool for working with EKS clusters that automates many individual tasks.

• Install kubectl  – A command line tool for working with Kubernetes clusters. 

Create IAM Role with Administrator Access

You need to create an IAM role with AdministratorAccess policy.
Go to AWS console, IAM, click on Roles. create a role

 

Select AWS services, Click EC2, Click on Next permissions.

 
 Now search for AdministratorAccess policy and click

Skip on create tag.
Now give a role name and create it.

Assign the role to EC2 instance
Go to AWS console, click on EC2, select EC2 instance, Choose Security.
Click on Modify IAM Role

Choose the role you have created from the dropdown.
Select the role and click on Apply.

Switch to Jenkins user

sudo su - jenkins

Create EKS Cluster with two worker nodes using eksctl

eksctl create cluster --name demo-eks --region us-east-2 --nodegroup-name my-nodes --node-type t3.small --managed --nodes 2

the above command should create a EKS cluster in AWS, it might take 15 to 20 mins. The eksctl tool uses CloudFormation under the hood, creating one stack for the EKS master control plane and another stack for the worker nodes. 

Once EKS cluster is created, kubeconfig file be created under /var/lib/jenkins/.kube folder.

you can view the kubeconfig file by entering the below command:

cat  /var/lib/jenkins/.kube/config

Connect to EKS cluster using kubectl commands

To view the list of worker nodes as part of EKS cluster.

kubectl get nodes

kubectl get ns

Deploy Nginx on a Kubernetes Cluster
Let us run some apps to make sure they are deployed to Kubernetes cluster. The below command will create deployment:

kubectl create deployment nginx --image=nginx

View Deployments
kubectl get deployments

Delete EKS Cluster using eksctl

eksctl delete cluster --name demo-eks --region us-east-2 

the above command should delete the EKS cluster in AWS, it might take a few mins to clean up the cluster.

Errors during Cluster creation

If you are having issues when creating a cluster, try to delete the cluster by executing the below command and re-create it.

eksctl delete cluster --name demo-eks --region us-east-2 

or Login to AWS console --> AWS Cloud formation --> delete the stack manually.

you can also delete the cluster under AWS console --> Elastic Kubernetes Service --> Clusters

Click on Delete cluster

Automate Docker builds using Jenkins Pipelines | Dockerize PHP App | Upload Images into Nexus Docker Registry

We will learn how to automate Docker builds using Jenkins pipeline. We will use PHP based application. I have already created a repo with source code + Dockerfile. We will see how to create Docker image and upload into Nexus Docker registry successfully. 

- Automating builds
- Automating Docker image builds
- Automating Docker image upload into Nexus docker registry

- Automating Docker container provisioning

 

Watch here for YouTube channel:

Pre-requisites:
1. Jenkins is up and running
2. Docker installed on Jenkins instance. Click here to for integrating Docker and Jenkins
3. Docker and Docker pipelines plug-in are installed
4. Nexus is up and running and docker registry is configured. Click here to know how to do that.
5. port 80 is opened up in firewall rules to access phpApp running inside Docker container

Create an entry in Manage Credentials for connecting to Nexus

Go to Jenkins --> Manage Jenkins--> Click on Manage Credentials.

 

Enter Nexus user name and password with ID as nexus

Click on Save.

Step # 1 - Create a pipeline in Jenkins, name can be anything

Step # 2 - Copy the pipeline code from below
Make sure you change red highlighted values below:
Your account_d should be updated and repo should be updated.

pipeline {
    
    agent any
    
    environment {
        imageName = "myphpapp"
        registryCredentials = "nexus"
        registry = "ec2-13-58-223-172.us-east-2.compute.amazonaws.com:8085/"
        dockerImage = ''
    }
    
    stages {
        stage('Code checkout') {
            steps {
                checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: '', url: 'https://bitbucket.org/ananthkannan/phprepo/']]])                   }
        }
    
    // Building Docker images
    stage('Building image') {
      steps{
        script {
          dockerImage = docker.build imageName
        }
      }
    }

    // Uploading Docker images into Nexus Registry
    stage('Uploading to Nexus') {
     steps{  
         script {
             docker.withRegistry( 'http://'+registry, registryCredentials ) {
             dockerImage.push('latest')
          }
        }
      }
    }
    
    // Stopping Docker containers for cleaner Docker run
    stage('stop previous containers') {
         steps {
            sh 'docker ps -f name=myphpcontainer -q | xargs --no-run-if-empty docker container stop'
            sh 'docker container ls -a -fname=myphpcontainer -q | xargs -r docker container rm'
         }
       }
      
    stage('Docker Run') {
       steps{
         script {
                sh 'docker run -d -p 80:80 --rm --name myphpcontainer ' + registry + imageName
            }
         }
      }    
    }
}

Step # 3 - Click on Build - Build the pipeline

Once you create the pipeline, click on Build now.

Steps # 4 - Check Docker images are uploaded into Nexus Registry

Login to Nexus, click on your repo, now you should see the image got uploaded.

Steps # 5 - Access PHP App in the browser which is running inside docker container

Once build is successful, go to browser and enter http://public_dns_name

You should see page like below:

Automate Docker builds using Jenkins Pipelines | Dockerize Python App | Upload Docker Images into AWS ECR

We will learn how to automate Docker builds using Jenkins. We will use Python based application. I have already created a repo with source code + Dockerfile. We will see how to create Docker image and upload into AWS ECR successfully. We will not be using AWS access keys to upload image into ECR, we will be using IAM role and attach to Jenkins instance to access ECR.

- Automating builds
- Automating Docker image builds
- Automating Docker image upload into AWS ECR
- Automating Docker container provisioning

 

Watch here for YouTube channel:
 

Pre-requisites:

1. Jenkins is up and running
2. Docker installed on Jenkins instance. Click here to for integrating Docker and Jenkins
3. Docker and Docker pipelines plug-in are installed
4. Repo created in ECR, Click here to know how to do that.
5. Make sure port 8096 is opened up in firewall rules. 

6. Create an IAM role with AmazonEC2ContainerRegistryFullAccess policy, attach role to Jenkins EC2 instance

7. Make sure AWS cli is installed in Jenkins instance.

Code for this video is here:

Make sure you fork my repo https://github.com/akannan1087/myPythonDockerRepo

and make changes in the pipeline accordingly.

Step # 1 - Create a pipeline in Jenkins, name can be anything

Step # 2 - Copy the pipeline code from below
Make sure you change red highlighted values below:
Your account_d should be updated and repo should be updated.

pipeline {
    agent any
    environment {
        registry = "acct_id.dkr.ecr.us-east-1.amazonaws.com/your_ecr_repo"
    }
   
    stages {
        stage('Cloning Git') {
            steps {
                checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: '', url: 'https://github.com/akannan1087/myPythonDockerRepo']]])     
            }
        }
  
    // Building Docker images
    stage('Building image') {
      steps{
        script {
          dockerImage = docker.build registry
        }
      }
    }
   
    // Uploading Docker images into AWS ECR

    stage('Pushing to ECR') {
     steps{  
         script {
                sh 'aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin acct_id.dkr.ecr.us-east-1.amazonaws.com'
                sh 'docker push acct_id.dkr.ecr.us-east-1.amazonaws.com/your_ecr_repo:latest'
         }
        }
      }
   
         // Stopping Docker containers for cleaner Docker run
     stage('stop previous containers') {
         steps {
            sh 'docker ps -f name=mypythonContainer -q | xargs --no-run-if-empty docker container stop'
            sh 'docker container ls -a -fname=mypythonContainer -q | xargs -r docker container rm'
         }
       }
      
    stage('Docker Run') {
     steps{
         script {
                sh 'docker run -d -p 8096:5000 --rm --name mypythonContainer acct_id.dkr.ecr.us-east-1.amazonaws.com/your_ecr_repo:latest'
            }
      }
    }
    }
}

Step # 3 - Click on Build - Build the pipeline

Once you create the pipeline and changes values per your ECR account ID, click on Build now.

Steps # 4 - Check Docker images are uploaded into ECR

Login to ECR, click on your repo, now you should see the image got uploaded.

Steps # 5 - Access PythonApp in the browser which is running inside docker container

Once build is successful, go to browser and enter http://public_dns_name:8096

You should see page like below:

How to create an Elastic IP Address in AWS and assign to your EC2 instance | Associate Elastic IP address to EC2 instance

Elastic IP address is static IP address provided by AWS, you should avoid using public ip address as it changes every stop/start of EC2 instance.
How to create Elastic IP address:

Go to AWS console, Click on EC2, Elastic IPs.

Click on Allocate Elastic IP address

Now it should create Elastic IP address.

Click on Actions, Associate Elastic IP address and choose your instance from Instances textbox and pick up the Private ip address automatically.

That's it! Elastic IP(static) address have been assigned to your EC2 instance.

How to setup Kubernetes Cluster in Ubuntu | Install Kubernetes Cluster in Ubuntu | Setup Master Node and one worker Node in AWS

Kubernetes  is an open source container platform that eliminates many of the manual processes involved in deploying and scaling containerized applications. We will learn how to setup Kubernetes Cluster in Ubuntu 18.0.4. You can setup Kubernetes Cluster in many ways. One of the ways is to use Kubeadm. 

Kubeadm is a tool built to provide kubeadm init and kubeadm join as best-practice “fast paths” for creating Kubernetes clusters.

Pre-Requistes:

1. Ubuntu instance with 4 GM RAM - Master Node - (with port open to all traffic

2. Ubuntu instance with at least 2 GM RAM - Worker Node - (with ports open to all traffic)

Kubernetes Setup using Kubeadm

###Start - Execute the below commands in both Master/worker nodes##########

Login to both instances execute the below commands:
sudo apt-get update -y  && sudo apt-get install apt-transport-https -y

Change to root user

sudo su -
sudo curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -

cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF

sudo apt-get update

#Disable swap memory
swapoff -a
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
Enable IP tables
#We need to enable IT tables for pod to pod communication.

modprobe br_netfilter
sysctl -p
sudo sysctl net.bridge.bridge-nf-call-iptables=1

Install Docker on both Master and Worker nodes
apt-get install docker.io -y

Add ubuntu user to Docker group

usermod -aG docker ubuntu
systemctl restart docker
systemctl enable docker.service

Install Kubernetes Modules
sudo apt-get install -y kubelet kubeadm kubectl kubernetes-cni

sudo systemctl daemon-reload
sudo systemctl start kubelet
sudo systemctl enable kubelet.service

sudo systemctl status docker
#End - Execute the above commands in both Master/worker nodes##########

Initialize Kubeadm on Master Node(only on Master Node)

#Execute the below command as root user
sudo su -
kubeadm init

#exit from root user and execute as normal user

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Installing the Weave Net Add-On
kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"

It make take a few mins to execute the above command

You will get below messages.

serviceaccount "weave-net" created

clusterrole "weave-net" created

clusterrolebinding "weave-net" created

role "weave-net" created

rolebinding "weave-net" created

daemonset "weave-net" created

Now execute the below command to see the pods.

kubectl get pods  --all-namespaces

Now login to Worker Node

Join worker node to Master Node

The below command will join worker node to master node.

sudo kubeadm join <master_node_ip>:6443 --token xrvked.s0n9771cd9x8a9oc \
    --discovery-token-ca-cert-hash sha256:288084720b5aad132787665cb73b9c530763cd1cba10e12574b4e97452137b4a

Go to Master and type the below command

kubectl get nodes

the above command should display both Master and worker nodes.

It means Kubernetes Cluster - both Master and worker nodes are setup successfully and up and running!!!

Deploy Nginx on a Kubernetes Cluster
Let us run some apps to make sure they are deployed to Kuberneter cluster. We will do this in master node. The below command will create deployment:

kubectl create deployment nginx --image=nginx

View Deployments
kubectl get deployments

Create as a service 

kubectl create service nodeport nginx --tcp=80:80

kubectl get svc

run the above command to see a summary of the service and the ports exposed.

Now go Master or worker node, enter public dns and access it with port exposed

You should see the welcome page!