Tuesday, April 25, 2023

How to Monitor AKS Cluster using Azure Monitor and Grafana | Setup monitoring on AKS Cluster using Azure Log Analytics Workspace and Grafana

We can monitor AKS cluster using many ways. We will monitor AKS cluster using Azure Log Analytics workspace which collects log data and metrics from AKS cluster and stores them inside workspace. We will use Grafana for visualizing the data from Log Analytics workspace.

Watch steps in YouTube channel:

    Different ways of Monitoring AKS Cluster

    What is Azure Log Analytics Workspace?

    Azure Log Analytics workspace is a logical storage unit in Azure where all log data generated by Azure Monitors are stored.  Log Analytics workspace collects log data from various Azure resources such as Azure Virtual machine or Azure Kubernetes Service. You can think of the workspace as a folder where all your monitoring data is stored and managed.

    What is Grafana?

    • Grafana is an open source visualization and analytics tool. 
    • It allows you to query, visualize, alert on, and explore your metrics no matter where they are stored.


    You can also monitor existing AKS cluster as well, but make sure monitoring is enabled. once enabled, you can associate default workspace to AKS cluster or create a new workspace, associate it to cluster.

    But we will try creating a new AKS cluster and monitor.

    Create AKS Cluster with Monitoring Enabled

    Make sure you are login to Azure portal first.

    az login

    enter your Microsoft credentials.

    Create a resource group first

    az group create --name myResourceGroup --location southcentralus

    Create Log Analytics workspace

    az monitor log-analytics workspace create --resource-group myResourceGroup \
                                               --workspace-name my-loganalytics-workspace \
                                               --query id \
                                               -o tsv

    Output of the above command will display log analytics Id which is needed for next command while creating AKS cluster. we will be associating Log Analytics Workspace ID with AKS Cluster during creation.

    Create AKS cluster with 2 worker nodes with Monitoring Enabled

    az aks create --resource-group myResourceGroup --name myAKSCluster --node-count 2 --enable-addons monitoring --workspace-resource-id /subscriptions/XXXXX/resourceGroups/myResourceGroup/providers/Microsoft.OperationalInsights/workspaces/aks-loganalytics-workspace

    Verify all the resources are created in Azure Portal

    Click on resource group name, you will see AKS cluster, log analytics workspace

    Display Details of Cluster

    az aks show --name myAKSCluster --resource-group myResourceGroup

    The above command will display AKS Cluster details.

    Connect to the cluster

    az aks get-credentials --resource-group myResourceGroup --name myAKSCluster --overwrite-existing

    To verify the connection to your cluster, use the kubectl get command to return a list of the cluster nodes.

    kubectl get nodes



    Permission needed (SPN) to pull metrics Data from Azure Log Analytics 

    We need to create Service principal(SPN) and assign Log Analytics Reader Role on the AKS Cluster ResourceGroup. Execute below command to first get resource group ID where AKS cluster is running.

    First Let's get Resource Group ID 

    az group show --name  myResourceGroup --query id --output tsv

    output of the above command will be resource group ID. 

    Create a Service Principal and Assign Role

    az ad sp create-for-rbac --role="Log Analytics Reader" --scopes="/subscriptions/xxxxxx-xxxx-xxxx-xxxxx/resourceGroups/myResourceGroup"

    Creating 'Log Analytics Reader' role assignment under scope '/subscriptions/XXX/resourceGroups/myResourceGroup'

      "appId": "xxx",
      "displayName": "azure-cli-2023-04-12-22-06-28",
      "password": "xxx",
      "tenant": "xxx"

    Note all the information above and save it.

    Grafana Installation steps on AKS cluster using Helm

    Install Grafana

    Once Helm is set up properly, add the repo as follows:

    helm repo add grafana https://grafana.github.io/helm-charts

    helm search repo grafana

    Create Monitoring namespace

    kubectl create namespace monitoring

    Install Grafana alone(NOT kube-prometheus-stack)

    helm install stable grafana/grafana -n monitoring

    Lets check if grafana pods are running

    kubectl get pods -n monitoring

    Expose Grafana

    # Port forward the Grafana service

    kubectl port-forward  -n monitoring stable-grafana-XXX 3000

    How to get Grafana admin password by querying pod

    kubectl get secret --namespace monitoring stable-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo

    Login to Grafana

    Now Open localhost:3000 in browser to access Grafana

    UserName: admin
    Password: <from the above command>

    Create Data Source in Grafana

    Go to Grafana, configuration and click Data Sources. Click on Add Data Source and search for Azure Monitor.

    Enter tenant id, app id and secret information. click on Load subscriptions. Click on Save and Test.

    Create Dashboard in Grafana

    In Grafana, we can create various kinds of dashboards as per our needs.

    How to Create Azure Monitor For Containers Dashboard?

    Click '+' button on left panel and select ‘Import’.

    Enter 10956 dashboard id under Grafana.com Dashboard.

    Click ‘Load’.

    Click ‘Import’.

    This will show monitoring dashboard for all cluster nodes

    How to Create POD Metrics for AKS Dashboard?

    Click '+' button on left panel and select ‘Import’.

    Enter 14891 dashboard id under Grafana.com Dashboard.

    Click ‘Load’.

    This will show monitoring dashboard for all pods, you can also select namespaces as well.

    Clean up Resources

    Let's see how to clean up the resources that were created. We can use the az group delete command to remove the resource group, AKS cluster, and all related resources. 

    az group delete --name myResourceGroup --yes --no-wait

    Click here for Script for Creating AKS cluster with Monitoring Enabled. 

    Monday, April 10, 2023

    Provision Ubuntu 22.0.4 EC2 Instance | How to create EC2 instance in AWS console | Launch Ubuntu 22.0.4 instance in AWS

    How to create new EC2 instance in AWS console using new UI experience?

    What is EC2 instance? 

    It is virtual server provided by AWS. We will be using this EC2 to setup both Jenkins and Tomcat. Please follow the below steps to create an EC2 instance.

    Watch in YouTube for a demo:

    1: Login to AWS console by clicking this link -->  https://aws.amazon.com/console/
    click on All services, Click on Compute -->  Click on EC2

    2. Click on Launch instance

    3. Enter Name as EC2 and enter 2 as number of instances (one for Jenkins and another for Tomcat)

    4. Select Ubuntu

     and choose Ubuntu server 22.0.4 as AMI

    5. Enter t2.small as instance type
    6. Click on Create new Key Pair

    7. Choose the existing key pair if you have one, otherwise create new one, give some name as myJenkinsKey. Make sure you download the key in your local machine. Please do NOT give space or any character while naming the key.

    8. Under Network settings, Click Edit

    Add port range as 8080 and select AnyWhere as Source Type, that should enter as Source

    9. Configure Storage
    Enter 15 GB as root volume 

    And then make sure in Summary, values appear as below:

    10. Click on Launch Instance.

    Click on View instances

    Now you should be able to view instances in AWS console. Now you can re-name as Jenkins-EC2 and Tomcat-EC2

    Connect to EC2 instance from local machine:
    Please click the below link to understand the steps for connecting to EC2 instance from your local machine - windows or Apple laptop.


    Wednesday, April 5, 2023

    SonarQube Server Cannot be reached Error Resolution | SonarQube Troubleshooting | Fix for max virtual memory areas vm.max_map_count [65530] is too low

     How to Resolve SonarQube Server cannot be reached Error?


    refer this page for pre-requisites to install SonarQube


    Make sure EC2 instance or server where you are installing SonarQube has enough virtual memory and open file limits for  Linux OS.

    How to change default value for vm.max_map_count temporarily
    sudo sysctl vm.max_map_count=262144
    sudo sysctl fs.file-max=65536
    The above command will load the max_map_count values and open file limits till the next system restart.

    How to update vm.max_map_count directly in sysctl.conf permanently
    Login to instance where you will be installing SonarQube, perform the below command to configure virtual memory permanently for SonarQube to function:
    sudo vi /etc/sysctl.conf

    Add the following lines to the bottom of that file:


    To make sure changes are getting into effect:
    sudo sysctl -p

    Make sure SonarQube is up and running by checking the logs
    sudo docker-compose logs --follow

    Once you see the message, that's it. SonarQube have been configured successfully. press control C and enter.

    Watch Steps in YouTube channel: