Showing posts with label Infrastructure. Show all posts
Showing posts with label Infrastructure. Show all posts

Tuesday, January 23, 2024

Create Ansible Role to create a new EC2 instance | Ansible Role for provisioning infrastructure in AWS | Refactor Ansible playbook into Ansible Role

We will learn how to create Ansible Role for provisioning a new EC2 instance in AWS cloud. We will pick a playbook which has all the logic and we will refactor into reusable ansible role.


What is Ansible Role?
Ansible also lets you organize tasks in a directory structure called a Role. Using Ansible roles you can break down complex playbooks into smaller and manageable chunks. Ansible role enables reuse and share our Ansible code efficiently.

How to create Ansible Role?

Using ansible galaxy command, we can create Ansible role. This will create the below directory with all the files. 

directory structure of Ansible role
aws-infra-role/
├── README.md
├── create.yml
├── defaults
│   └── main.yml
├── handlers
│   └── main.yml
├── meta
│   └── main.yml
├── tasks
│   ├── create-ec2.yml
│   └── create-sg.yml
├── tests
│   ├── inventory
│   └── test.yml
└── vars
    └── main.yml

Directory structure explained
tasks - contains the main list of tasks to be executed by the role.
handlers - handlers are typically used to start, reload, restart, and stop services.
defaults - default variables for the role.
vars - other variables for the role. Vars has the higher priority than defaults.
meta - defines some data / information about this role (author, dependency, versions, examples, etc,.)

tests - test cases if you have any.

Pre-requisites:
Steps to create EC2 instance using Ansible Role:

Login to EC2 instance using Git bash or ITerm/putty where you installed Ansible. Execute the below command:

Create an Inventory file first

sudo mkdir /etc/ansible

Edit Ansible hosts or inventory file
sudo vi /etc/ansible/hosts

Add the below two lines in the end of the file:
[localhost]
local


cd ~
mkdir roles  
cd roles

Create Ansible Role

ansible-galaxy role init aws-infra-role


We will convert this playbook into ansible role.
So all the variables will go inside vars folder.

vars
    └── main.yml

sudo vi aws-infra-role/vars/main.yml
(copy below content)
keypair: myNov2023Key
instance_type: t2.small
image: ami-007855ac798b5175e
wait: yes
group: webserver
region: us-east-1
security_group: my-jenkins-security-grp1

Save the file and come out of it.

So all the tasks will go inside tasks folder. let's create security group first.

sudo vi aws-infra-role/tasks/create-sg.yml
---
  - include_vars: "vars/main.yml"
    tags: create

# tasks file for security group
  - name: configuring security group for the instance
    ec2_group:
        name: "{{ security_group }}"
        description: my-ajenkin-security_groAup
        region: "{{ region }}"
        rules:
            - proto: tcp
              from_port: 22
              to_port: 22
              cidr_ip: 0.0.0.0/0
            - proto: tcp
              from_port: 80
              to_port: 80
              cidr_ip: 0.0.0.0/0
            - proto: tcp
              from_port: 8080
              to_port: 8080
              cidr_ip: 0.0.0.0/0
        rules_egress:
            - proto: all
              cidr_ip: 0.0.0.0/0

Let's create a task for ec2 instance creation.

sudo vi aws-infra-role/tasks/create-ec2.yml

---
  - include_vars: "vars/main.yml"
    tags: create
  - name: creating ec2 instance
    ec2_instance:
        security_group: "{{ security_group }}"
        name: test-stan
        key_name: "{{ keypair }}"
        instance_type: "{{ instance_type}}"
        image_id: "{{ image }}"
        region: "{{ region }}"
        wait_timeout: 2   


Let's create a task for creating s3 bucket.

sudo vi aws-infra-role/tasks/create-s3.yml                                                                                                     ---
  - include_vars: "vars/main.yml"
    tags: create
  - name: creating s3

    s3_bucket:
      name: myansibles3bucket1234
      state: present
      region: "{{ region }}"
      versioning: yes
      tags:
        name: myansiblebucket
        type: example
    register: s3_url

  - name: Display s3 url
    debug: var=s3_url       

Let's create Ansible main playbook.
sudo vi aws-infra-role/main.yml
---
# This Playbook creates infra in aws cloud

- hosts: local
  connection: local
  gather_facts: False
  tags: ec2_create

  tasks:
  - include: tasks/create-sg.yml
  - include: tasks/create-ec2.yml
  - include: tasks/create-s3.yml 

now execute the ansible playbook by
ansible-playbook aws-infra-role/main.yml


If everything is good, you should see the new instance, S3 bucket created on AWS console. make sure you are able to connect to that instance.

That's it!! That is how you create a new EC2 instance using Ansible role in AWS cloud. 
Please watch steps in YouTube channel:

Wednesday, January 4, 2023

How to run Ansible playbook from Jenkins pipeline job | Automate EC2 provisioning in AWS using Jenkins and Ansible Playbook | Create new EC2 instance in AWS cloud using Ansible Playbook and Jenkins Pipeline

We will learn how to create new EC2 instance using Ansible playbook and automate using Jenkins Pipeline. 


Watch Steps in YouTube Channel:

Pre-requisites:

  • Ansible is installed and Boto is also installed on Jenkins instance
  • Ansible plug-in is installed in Jenkins. 
  • Make sure you create an IAM role with AmazonEC2FullAccess policy and attach the role to Jenkins EC2 instance.
  • Playbook for creating new EC2 instance needs to be created but you can refer my GitHub Repo
Steps:

Create Ansible playbook for provisioning EC2 instance

(Sample playbook is available in my GitHub Repo, you can use that as a reference)

Create Jenkins Pipeline 
pipeline {
    agent any

    stages {
        
        stage ("checkout") {
            steps {
                        checkout([$class: 'GitSCM', branches: [[name: '*/master']], extensions: [],                                                     userRemoteConfigs: [[url: 'https://github.com/akannan1087/myAnsibleInfraRepo']]])         
            }
        }
        stage('execute') {
            steps {
                //to suppress warnings when you execute playbook    
                sh "pip install --upgrade requests==2.20.1"
                // execute ansible playbook
                ansiblePlaybook playbook: 'create-EC2.yml'
            }
        }
    }
}

Execute Pipeline


Pipeline Console output




Tuesday, April 14, 2020

How to install Puppet on Ubuntu 16.0.4 - How to configure Puppet Master

Puppet is an Infrastructure provisioning tool, similar to Ansible, Chef. We will see how to create EC2 instances in AWS using Puppet in this article.


Pre-requistes:
install puppet master on new Ubuntu with medium instance
port 8140 needs to be opened.

Steps:
First let us how to install Puppet on your Ubuntu machine.

Steps for Puppet Master
 Installation

1. Download puppet installable

curl -O https://apt.puppetlabs.com/puppetlabs-release-pc1-xenial.deb
sudo dpkg -i puppetlabs-release-pc1-xenial.deb
sudo apt-get update
sudo apt-get install puppetserver -y

sudo ufw allow 8140
sudo systemctl enable puppetserver
 

(the above command is to start the service during starting the Ubuntu instance)

sudo systemctl start puppetserver          

(The above command is for starting the server and this may take some time)
sudo systemctl status puppetserver 

       you should see a message like
       puppet systemd[1]: Started puppetserver Service.
    
That's it puppet master is up and running.

Now press q to come out of window.


This confirms that Puppet Master is installed successfully.

2. you need to install the aws-sdk-core and retries gems as root (or superuser):
sudo /opt/puppetlabs/puppet/bin/gem install aws-sdk-core retries
Done installing documentation for retries after 0 seconds
6 gems installed
3. Also install AWS SDK

sudo /opt/puppetlabs/puppet/bin/gem install aws-sdk retries

(The above command may take 3 to 5 mins to get gems installed)











4. Now you can install puppet-labs-aws module 
sudo /opt/puppetlabs/bin/puppet module install puppetlabs-aws

That's it. Puppet Master is setup successfully!!!!

Friday, April 3, 2020

How to install Ansible on Mac OS | Ansible Installation Steps on Apple Mac OS

Ansible is one of the popular configuration management tools. Ansible can also be used for infrastructure provisioning as well in AWS. Please find steps for installing Ansible on Apple Mac OS or laptop.

Ansible can be installed on Mac OS many ways, but preferred way is using pip which is a Python package manager.

Install Pip

If pip isn’t already available on your system of Python, run the following commands to install it:
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py

sudo python3 get-pip.py --user
pip3 --version

Install Ansible using Pip

sudo pip3 install ansible --user
Once Ansible is installed, you can verify the version.

ansible --version
You should be able to see the version.

ansible 2.10.5

Install Boto framework
pip3 install boto --user
pip3 install boto3 --user
 
Create Ansible hosts file
when you install Ansible on Mac OS, it does not create ansible hosts file. so you need to create it.
sudo mkdir /etc/ansible
 

Thursday, April 11, 2019

Ansible Vs Puppet - What is the difference between Ansible and Puppet

This is one of the common DevOps interview questions. What is the difference between Ansible and Puppet? When will you choose Ansible over Puppet?


Ansible Puppet
Introduced 2012 2005
Written in? Python Java & Ruby
Syntax Playbooks(YAML file) Domain specific language
Model Push Pull
Architecture Agent-less Client/server
Deployments Fast, simple deployments Larger, complex deployments
Commercial version Ansible Tower Puppet Enterprise
Scheduling not supported in free version Default time 30 mins in Open Source puppet

Friday, February 8, 2019

How to create EC2 instances using Terraform | Automate EC2 instance Creation using Terraform on AWS Cloud | Terraform With AWS Cloud

Hashicorp's Terraform is an open-source tool for provisioning and managing cloud infrastructure. Terraform can provision resources on any cloud platform. 

Terraform allows you to create infrastructure in configuration files(tf files) that describe the topology of cloud resources. These resources include virtual machines, storage accounts, and networking interfaces.
 
We will see how you can use Terraform to provision EC2 instance. Please do the below steps for provisioning EC2 instances on AWS:



Watch the steps in YouTube channel:

Pre-requistes:



2. Create a new access key if you don't have one. Make sure you download the keys in your local machine. Login to AWS console, click on username and go to My security credentials.
 Continue on security credentials, click on access keys

Perform below commands in MacOS/EC2 where you have installed Terraform:

First setup your access keys, secret keys and region code locally.
aws configure

cd ~
mkdir project-terraform
cd project-terraform

Create Terraform Files
sudo vi variables.tf

variable "aws_region" {
       description = "The AWS region to create things in." 
       default     = "us-east-2
}

variable "key_name" { 
    description = " SSH keys to connect to ec2 instance" 
    default     =  "myJune2021Key
}

variable "instance_type" { 
    description = "instance type for ec2" 
    default     =  "t2.micro" 
}

variable "security_group" { 
    description = "Name of security group" 
    default     = "my-jenkins-security-group-2022" 
}

variable "tag_name" { 
    description = "Tag Name of for Ec2 instance" 
    default     = "my-ec2-instance" 
variable "ami_id" { 
    description = "AMI for Ubuntu Ec2 instance" 
    default     = "ami-0b9064170e32bde34
}


Now create main.tf file

sudo vi main.tf

provider "aws" {
  region = var.aws_region
}

resource "aws_vpc" "main" {
  cidr_block = "172.16.0.0/16"
  instance_tenancy = "default"
  tags = {
    Name = "main"
  }
}

#Create security group with firewall rules
resource "aws_security_group" "jenkins-sg-2022" {
  name        = var.security_group
  description = "security group for jenkins"

  ingress {
    from_port   = 8080
    to_port     = 8080
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

 ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

 # outbound from Jenkins server
  egress {
    from_port   = 0
    to_port     = 65535
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags= {
    Name = var.security_group
  }
}

resource "aws_instance" "myFirstInstance" {
  ami           = var.ami_id
  key_name = var.key_name
  instance_type = var.instance_type
  vpc_security_group_ids = [aws_security_group.jenkins-sg-2022.id]
  tags= {
    Name = var.tag_name
  }
}


Now execute the below command:
terraform init
you should see like below screenshot.


Execute the below command
terraform plan
the above command will show how many resources will be added.
Plan: 3 to add, 0 to change, 0 to destroy.


Execute the below command
terraform apply
Plan: 3 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
Now login to EC2 console, to see the new instances up and running

List Resources created by Terraform
Execute the below command to view list of the resources created by Terraform.
terraform state list
The above command will list three resources created.


You should be able to see EC2 instance up and running in AWS console.

How to push Terraform files into GitHub
All Terraform files should be checked into version control systems such as GitHub, BitBucket or GitLab. Let us see how to push code changes into GitHub. Make sure you are in the directory where Terraform files are created.

Create Remote repo in GitHub
Create a new repo with below name, make sure it is a private repo. Also do not click on initialize this repository with a README option.

 Note down the remote url as highligted below:




Note:
If you have any issues in uploading tf files, you may not have created ssh-keys and uploaded into GitHub. Create ssh keys using ssh-keygen command:

ssh-keygen
This should generate both public and private keys.
Copy the public keys by executing the below command:
sudo cat ~/.ssh/id_rsa.pub

Initialize the directory first
git init

The above command will create local git repository.
Now add terraform files.
git add *.tf

git commit -m "Added terraform files"

Copy the below red highlighted url from above screenshots circled in red.
git remote add origin your remote repo url per above screenshot

Now push the code into GitHub
git push -u origin master

Now Login to GitHub to view the Terraform files

Wednesday, January 16, 2019

How to destroy a specific resource using Terraform - Destroy specific resource using Terraform

If you would like to destroy all the resources you had created using Terraform, all you have to do is perform the below command:
 
terraform destroy
 
The above command will destroy all the resources. 
 
But let's say you would like to destroy only a specific resource you have created using Terraform. You can do it by passing a an argument in terraform destroy command. if you would like to delete EC2 instance, you can mention like below:

Destroy specific resources
terraform destroy -target aws_instance.myInstanceName

if you would like to delete a security group, you can mention like below:
terraform destroy -target aws_security_group.security_group_name
 
Please watch in YouTube on how to do the above steps:


Saturday, December 15, 2018

Ansible Playbook for provisioning a new EC2 instance in AWS - Create a new EC2 Using Ansible Playbook

Please find the Ansible Playbook for provisioning a new EC2 instance. Please follow the below steps in the machine where you installed Ansible.

Steps to create EC2 instance using Ansible:


1. Login to AWS console, click on username and go to My security credentials.
2. Continue on security credentials, click on access keys
3. Create a new access key if you dont have one. Make sure you download the keys.
4. Login to EC2 instance using Git bash or ITerm where you installed Ansible.

execute the below command

sudo vi ~/.boto

add below three lines in the above file, replace the ?? with access key and secret key values.
[Credentials]
aws_access_key_id = ??
aws_secret_access_key = ??





5. Edit Ansible hosts or inventory file
sudo vi /etc/ansible/hosts 
Add the below two lines in the end of the file:
[localhost]
local

6. cd ~
7. mkdir playbooks  
8. cd playbooks

Create Ansible playbook
9. sudo vi create_jenkins_ec2.yml 
(copy the below content in green color)
edit the create_jenkins_ec2.yml to make sure you update the key which is red marked below:
---
 - name:  provisioning EC2 Lab Exercises using Ansible
   hosts: localhost
   connection: local
   gather_facts: False
   tags: provisioning

   vars:
     keypair: MyEC2Key
     instance_type: t2.small
     image: ami-07c1207a9d40bc3bd
     wait: yes
     group: webserver
     count: 1
     region: us-east-2
     security_group: my-jenkins-security-grp
   
   tasks:

     - name: Create my security group
       local_action: 
         module: ec2_group
         name: "{{ security_group }}"
         description: Security Group for webserver Servers
         region: "{{ region }}"
         rules:
            - proto: tcp
              from_port: 22
              to_port: 22
              cidr_ip: 0.0.0.0/0
            - proto: tcp
              from_port: 8080
              to_port: 8080
              cidr_ip: 0.0.0.0/0
            - proto: tcp
              from_port: 80
              to_port: 80
              cidr_ip: 0.0.0.0/0
         rules_egress:
            - proto: all
              cidr_ip: 0.0.0.0/0
       register: basic_firewall
     - name: Launch the new EC2 Instance
       local_action:  ec2 
                      group={{ security_group }} 
                      instance_type={{ instance_type}} 
                      image={{ image }} 
                      wait=true 
                      region={{ region }} 
                      keypair={{ keypair }}
                      count={{count}}
       register: ec2
     - name: Add Tagging to EC2 instance
       local_action: ec2_tag resource={{ item.id }} region={{ region }} state=present
       with_items: "{{ ec2.instances }}"
       args:
         tags:
           Name: MyTargetEc2Instance




10. now execute the ansible playbook by
sudo ansible-playbook create_jenkins_ec2.yml



Fix the warnings by executing below command
pip install --upgrade requests==2.20.1

If everything is good, you should see the new instance created on AWS console. make sure you are able to connect to that instance.

That's it!! That is how you create a new EC2 instance using Ansible.