How to Create a Docker Image for Springboot App and Push Docker image into Amazon ECR from Azure DevOps Pipelines | Azure DevOps Pipelines to Build and Push a Docker image to AWS ECR

Amazon Elastic Container Registry (ECR) is a fully managed Docker container registry that makes it easy to store, share, and deploy container images. We will learn how to build docker image for a springboot microservices app using Azure DevOps(ADO) build pipeline and push docker image into AWS ECR.

What are we going to do in this lab?

1. Create a Repository in AWS ECR for storing docker images

2. Create an IAM user and AmazonEC2ContainerRegistryFullAccess policy.

3. Create access keys for IAM user in AWS console

4. Create service connection in AzureDevOps to connect to AWS using IAM user access & secret keys.

5. Create Azure DevOps Build pipeline with below tasks:

• maven build for building JAR
• build docker image
• push docker image into ECR

6. Verify if docker image has been pushed to AWS ECR

Pre-requisites:

• Azure DevOps Account setup and project is created in ADO
• Install AWS toolkit for Azure DevOps
• Setup springboot app in GitHub (you can fork this repository)
• Create service connection in ADO to connect to GitHub personal access Token

Watch Steps in YouTube channel:

Step 1 - Create a repo in ECR 

Go to AWS console, type ECR

Click on Create Repository

Enter name for your repo - all lower case and Click create repository

Create an IAM user

Go to AWS console --> IAM --> Add Users

Enter name for the user

Search for EC2 and choose AmazonEC2ContainerRegistryFullAccess

Click on Create User

Create security credentials

Click on user name ecr-user

Click on Security credentials

Create Access key

Create Service connection 

Go to Project settings --> Service Connections

Enter Access keys and Secret keys

Enter Connect name and select Grant access to all pipelines

Click on save

Create a classic Azure Build pipeline

Click on use the classic editor

Select GitHub and choose your spring-boot project and click continue

Choose a template for the pipeline, type docker and select docker container

Click on Apply

Select build Agent for the Pipeline

Choose Ubuntu latest as build agent

Add Pipeline variables

imageName as springboot-app

repoName as my-springboot-repo

Add Maven task for building springboot JAR file

Make sure Maven task is moved up..it should be a first task

enter maven goas as install

Customize build an image Task 

start customizing the task, choose the version as 2.0

Enter $(imageName) as  Container repository 

Select build as command from drop down

Tags as it have shown below

Remove push an image task

Add ECR push task

now configure the task

Select as shown in screenshot

Choose aws service connection from drop down

select region as per your settings.

select Image ID

enter as $(imageName):$(Build.BuildId)

Repo Name as $(repoName)

$(Build.BuildId) as tar repo tag

Save and Queue

Select ubuntu latest as build agent..do NOT select window agents.

Now make sure build is successful.

Verify if Docker image has been pushed into AWS ECR

Now login to AWS console --> Go to ECR--> select your repo. verify if image has been uploaded successfully.

How to create Azure Container Registry using Terraform in Azure Cloud | Setup Azure Container Registry using Terraform

Hashicorp's Terraform is an open-source tool for provisioning and managing cloud infrastructure. Terraform can provision resources on any cloud platform. 

Terraform allows you to create infrastructure in configuration files(tf files) that describe the topology of cloud resources. These resources include virtual machines, storage accounts, and networking interfaces. The Terraform CLI provides a simple mechanism to deploy and version the configuration files to Azure.

Watch the steps in YouTube:

Advantages of using Terraform:

• Reduce manual human errors while deploying and managing infrastructure.
• Deploys the same template multiple times to create identical development, test, and production environments.
• Reduces the cost of development and test environments by creating them on-demand.

How to Authenticate with Azure?

Terraform can authenticate with Azure in many ways, in this example we will use Azure CLI to authenticate with Azure and then we will create resources using Terraform.

Pre-requisites:

Azure CLI needs to be installed.

Terraform needs to be installed.

Logging into the Azure CLI

Login to the Azure CLI using:

az login

The above command will open the browser and will ask your Microsoft account details. Once you logged in, you can see the account info by executing below command:

az account list

Now create a directory to store Terraform files.

mkdir tf-acr

cd tf-acr

Let's create a terraform file to use azure provider. To configure Terraform to use the Default Subscription defined in the Azure CLI, use the below cod.

Now initialize the working directory

sudo vi create-acr.tf

provider "azurerm" {
  features {}
}
resource "azurerm_resource_group" "rg" {
  name     = "rg-tf-acr"
  location = "southcentralus"
}
resource "azurerm_container_registry" "acr" {
  name                     = "azcontainerregistry321"
  resource_group_name      = azurerm_resource_group.rg.name
  location                 = azurerm_resource_group.rg.location
  sku                      = "Basic"
  admin_enabled            = true
}
output "admin_password" {
  value       = azurerm_container_registry.acr.admin_password
  description = "The object ID of the user"
sensitive = true
}

Perform the below command to initialize the directory.

terraform init

Once directory is initialized, you can start writing code for setting up the infrastructure. Now Perform the below command to validate terraform files.

terraform validate

perform plan command to see how many resources will be created.

terraform plan

terraform apply

Do you want to perform these actions?

type yes 

Now Login into Azure Cloud to see the resources created.

How to destroy the resources ?

Execute terraform destroy

The above command to destroy both resource group and container registry created before.

How to create AKS cluster in the Azure portal | How to connect to AKS cluster in Azure Portal

 Azure Kubernetes Service (AKS)

AKS allows you to quickly deploy a production ready Kubernetes cluster in Azure, deploy and manage containerized applications more easily with a fully managed Kubernetes service. 

AKS reduces the complexity and operational overhead of managing Kubernetes by offloading much of that responsibility to Azure. As a hosted Kubernetes service, Azure handles critical tasks like health monitoring and maintenance for you. The Kubernetes masters are managed by Azure. You only manage and maintain the agent nodes. As a managed Kubernetes service, AKS is free - you only pay for the agent nodes within your clusters, not for the masters.

We will see how to create AKS cluster in portal Azure.

1. go to https://portal.azure.com/#home

2. Click on Kubernetes services

3. Click on Add Kubernetes Cluster

 

4. Click on Add Kubernetes Cluster

 Enter information below:

 

 

5. Enter node count as 2, choose standard size

6. Click on Create after making sure validations are passed:
 

This is will take at least 2 to 5 mins to create AKS clusters. If all goes well, you should be able to see the cluster.

Once it is created, go to Kubernetes Services, click on cluster name.

 

 

How to connect to AKS cluster using Azure CLI

Now click on the cluster name, click on connect.

Make sure you have Azure CLI is installed on your local machine in order to connect to AKS cluster.

Connect to Azure portal using Azure CLI first.

az login

Enter your Microsoft credentials

az account set --subscription subscription_id

az aks get-credentials --resource-group AKS-POC --name AKS-POC

# List all deployments in a specific namespace
kubectl get deployments --all-namespaces=true

# List all deployments in a specific namespace
kubectl get deployments --namespace kube-system

 

Please watch the above steps in YouTube video:

How to setup Elastic Container Registry (ECR) for Docker on AWS | How to Create a Repo in ECR for Hosting Docker images | How to Push Docker image into Amazon ECR

Amazon Elastic Container Registry (ECR) is a fully managed container registry service provided by Amazon Web Services (AWS). It allows users to store, manage, and deploy Docker container images. Amazon ECR uses Amazon S3 for storage to make your container images highly available and accessible, allowing you to reliably deploy new containers for your applications. We will learn how to create ECR in AWS, will create a docker image and upload docker image into ECR.

You can watch the steps on YouTube as well:

What are we going to do in this lab?

1. Create a Repository in AWS ECR

2. Create an IAM role with ContainerRegistryFullAccess

3. Attach the IAM role to EC2 instance

4. Download pythonApp from Bitbucket

5. Build docker image for the Python App

6. Tag docker image

7. Login to AWS ECR using aws cli 

8. Push docker image into ECR

9. Run python app in Docker container

Pre-requisites:

• EC2 instance up and running with Docker installed
• Make sure you open port 8081
  
• Install aws cli

Create a repo in ECR 

Go to AWS console and search for ECR

Click on Create Repository

Enter name for your repo - all lower case and Click create repository:

Once repo is created, choose the repo and click on view push commands. Note down the account ID

Note the URL from step # 3 below, this will be used for tagging and pushing docker images into ECR.

That's it, you have created repo successfully. Let us create docker images and push it to above repo in ECR.

Create an IAM role
You need to create an IAM role with AmazonEC2ContainerRegistryFullAccess policy.
Go to AWS console, IAM, click on Roles. create a role

 

Select AWS services, Click EC2, Click on Next permissions.

 
 Now search for AmazonEC2ContainerRegistryFullAccess policy and click

Skip on create tag.
Now give a role name and create it.

You need to assign the role to EC2 instance you have installed docker.

Go to AWS console, click on EC2, select EC2 instance, Go to Actions --> Security--> Modify IAM role.

Choose the role you have created from the dropdown.
Select the role and click on Apply.

Now Login to EC2 instance where you have installed Docker. You must be able to connect to AWS ECR through AWS CLI which can be installed by

sudo apt install unzip -y

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"

unzip awscliv2.zip

sudo ./aws/install

Once AWS CLI is installed, you can verify the installation:
aws --version

Now you can login to AWS ECR using CLI:

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin your_acct_id.dkr.ecr.us-east-1.amazonaws.com

Where your_acct_id is from AWS ECR in the above picture.

You must get a message says Login succeeded. Now let's build a docker image, I have already created a public repo in Bitbucket. All you need to do is perform the below command to clone my repo:

git clone https://bitbucket.org/ananthkannan/mydockerrepo; cd mydockerrepo/pythonApp

docker build . -t mypythonapp
the above command will build a docker image.

 

Now tag Docker image you had build
docker tag mypythonapp:latest your_acct_id.dkr.ecr.us-east-1.amazonaws.com/devops/your-ecr-repo-name:1.0

You can view the image you had built.


docker push your_acc_id.dkr.ecr.us-east-1.amazonaws.com/devops/your-ecr-repo-name:1.0

Now you should be able to login to ECR and see the images already uploaded.

 

How to run a Docker container from a Docker image?

sudo docker run -p 8081:5000 -d --rm --name myfirstApp1  your_acc_id.dkr.ecr.us-east-1.amazonaws.com/devops/your-ecr-repo-name:1.0

where -d is for running with detached mode.

Access the application from the browser

Access the container using browser by using public_dns_name 

http://public_dns_name_your_ec2:8081

How to run another Docker container from a Docker image?

sudo docker run -p 8085:5000 -d --rm --name myfirstApp2  your_acc_id.dkr.ecr.us-east-1.amazonaws.com/devops/your-ecr-repo-name:1.0

make sure you open port 8085 in AWS security firewall rule.

where -d is for running with detached mode.

 Note: You can also create a repo through CLI command in AWS ECR.

aws ecr create-repository --repository-name myawesome-repo --region us-east-1