Author: Hitesh Jethva
Last Updated: Wed, Feb 1, 2023Metrics server is a scalable cluster add-on for the Kubernetes built-in autoscaling pipelines. It is a critical element to scale pods in the Kubernetes cluster. It is designed to automatically scale up or down your application workloads using the Kubernetes Horizontal Pod Autoscaler.
Metrics server works by retrieving kubelet metrics from each worker node, collecting CPU and memory usage for each application workload then use the Kubernetes API server to expose CPU and memory usage metrics. Then, Horizontal Pod Autoscaler fetches CPU and memory usage metrics and scales up or down your application deployment pods based on metrics observation and target threshold.
The main features of using Kubernetes metrics server are:
Scale up and down resources automatically.
Support up to 5,000 node clusters.
Collect resource metrics such as CPU and memory utilization in horizontal autoscaling.
Fewer resources requirement.
Monitors cluster metrics that are not Kubernetes-specific.
This guide shows you how to install Metrics server on Vultr Kubernetes Engine.
Deploy a Kubernetes cluster at Vultr with at least three nodes.
The kubectl CLI installed and configured in your local machine.
Helm client installed in your local machine.
First, add the Helm repository for the Metrics server.
# helm repo add metrics-server https://kubernetes-sigs.github.io/metrics-server
Next, update the Helm repository to the latest version.
# helm repo update
Then, verify the added repository.
# helm repo list
Sample output.
NAME URL
metrics-server https://kubernetes-sigs.github.io/metrics-server
After that, create a namespace for the Metrics server.
# kubectl create ns metrics-server
Next, set default namespace to metrics-server.
# kubectl config set-context $(kubectl config current-context) --namespace=metrics-server
Before installing the Metrics server, obtain the default configuration value from the Helm repo and save it to the file metrics-server.values.
# helm show values metrics-server/metrics-server > ~/metrics-server.values
Next, edit the file metrics-server.values.
# nano metrics-server.values
Change the desired configuration to file.
replicas: 4
defaultArgs:
- --cert-dir=/tmp
- --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
- --kubelet-use-node-status-port
- --metric-resolution=15s
- --kubelet-insecure-tls
Now, deploy the Metrics server to Kubernetes.
# helm install metrics-server metrics-server/metrics-server -n metrics-server --values ~/metrics-server.values
Sample output.
NAME: metrics-server
LAST DEPLOYED: Fri Jan 13 10:02:19 2023
NAMESPACE: metrics-server
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
***********************************************************************
* Metrics Server *
***********************************************************************
Chart version: 3.8.3
App version: 0.6.2
Image tag: k8s.gcr.io/metrics-server/metrics-server:v0.6.2
***********************************************************************
After the successful deployment, verify it using the following command.
# helm ls -n metrics-server
Sample output.
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
metrics-server metrics-server 1 2023-01-13 10:02:19.083957787 +0530 IST deployed metrics-server-3.8.3 0.6.2
To verify other components, run.
# kubectl get all -n metrics-server
Sample output.
NAME READY STATUS RESTARTS AGE
pod/metrics-server-54c4d8c9df-6z25l 1/1 Running 0 31s
pod/metrics-server-54c4d8c9df-7t5bn 1/1 Running 0 31s
pod/metrics-server-54c4d8c9df-rjdt2 1/1 Running 0 31s
pod/metrics-server-54c4d8c9df-tz54s 1/1 Running 0 31s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/metrics-server ClusterIP 10.96.76.218 <none> 443/TCP 32s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/metrics-server 4/4 4 4 33s
NAME DESIRED CURRENT READY AGE
replicaset.apps/metrics-server-54c4d8c9df 4 4 4 33s
The simple and easiest way to verify the metrics data using the kubectl top command. You can verify the resource usage at the cluster level and at the Pods level.
To verify the resource usage at the cluster level, run:
# kubectl top nodes
This command will show you the resource usage for both CPU and memory on each worker node.
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
metrics-kubernetes-226bb1c797fe 77m 9% 853Mi 50%
metrics-kubernetes-7588a6dad02f 46m 5% 765Mi 45%
metrics-kubernetes-cd72d509b5a3 46m 5% 736Mi 43%
To verify the resource usage at the Pod level, run:
# kubectl top pods -n metrics-server
This command will show you the CPU usage in millicores and memory usage in Mebibytes.
NAMESPACE NAME CPU(cores) MEMORY(bytes)
metrics-server metrics-server-54c4d8c9df-6z25l 4m 13Mi
metrics-server metrics-server-54c4d8c9df-7t5bn 2m 18Mi
metrics-server metrics-server-54c4d8c9df-rjdt2 3m 18Mi
metrics-server metrics-server-54c4d8c9df-tz54s 3m 18Mi
In this section, you will configure Horizontal Pod Autoscaling to grow and shrinks Pods automatically based on resource utilization.
First, create a deployment to test Horizontal Pod Autoscaling.
# nano deployment.yml
Add the following configurations.
apiVersion: apps/v1
kind: Deployment
metadata:
name: sample-app
spec:
selector:
matchLabels:
run: sample-app
replicas: 1
template:
metadata:
labels:
run: sample-app
spec:
containers:
- name: busybox
image: busybox
resources:
limits:
cpu: 50m
requests:
cpu: 20m
command: ["sh", "-c"]
args:
- while [ 1 ]; do
echo "Test";
sleep 0.01;
done
Apply the above manifest to the Kubernetes cluster.
# kubectl apply -f deployment.yml
Verify the deployment.
# kubectl get deployment
Sample output.
NAME READY UP-TO-DATE AVAILABLE AGE
metrics-server 4/4 4 4 2m30s
sample-app 1/1 1 1 25s
Next, create the Horizontal Pod Autoscaler CRD to configure Horizontal Pod Autoscaler (HPA) on your deployment
# nano crd.yml
Add the following configurations.
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: my-app-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: sample-app
minReplicas: 2
maxReplicas: 6
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 50
The above configuration will create a Horizontal Pod Autoscaler on the sample-app deployment with CPU max value set to 50 and scaling between 2-6 replicas.
Apply the manifest to the Kubernetes cluster.
# kubectl apply -f crd.yml
If you don't want to run Horizontal Pod Autoscaler using the above method then you can deploy it via the kubectl autoscale command.
# kubectl autoscale deployment sample-app --cpu-percent=50 --min=2 --max=6
Now, verify the HPA creation using the following command.
# kubectl get hpa
You should the the current usage%/target usage% in the TARGET column.
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
my-app-hpa Deployment/sample-app 30%/50% 2 6 2 7m19s
To see the detailed information of your HPA, run:
# kubectl describe hpa my-app-hpa
You should see the detail overview of your HPA deployment in the following output.
Name: my-app-hpa
Namespace: metrics-server
Labels: <none>
Annotations: <none>
CreationTimestamp: Fri, 13 Jan 2023 10:06:40 +0530
Reference: Deployment/sample-app
Metrics: ( current / target )
resource cpu on pods (as a percentage of request): 22% (4m) / 50%
Min replicas: 2
Max replicas: 6
Deployment pods: 2 current / 2 desired
Conditions:
Type Status Reason Message
---- ------ ------ -------
AbleToScale True ScaleDownStabilized recent recommendations were higher than current one, applying the highest recent recommendation
ScalingActive True ValidMetricFound the HPA was able to successfully calculate a replica count from cpu resource utilization (percentage of request)
ScalingLimited False DesiredWithinRange the desired count is within the acceptable range
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal SuccessfulRescale 42s horizontal-pod-autoscaler New size: 2; reason: Current number of replicas below Spec.MinReplicas
Your sample-app is now autoscaled as per resource usage. You can verify it using the following command.
# kubectl get deployment
Sample output.
NAME READY UP-TO-DATE AVAILABLE AGE
metrics-server 4/4 4 4 12m
sample-app 2/2 2 2 10m
You can also verify your Pods using the following command.
# kubectl get pods
Sample output.
NAME READY STATUS RESTARTS AGE
metrics-server-54c4d8c9df-6z25l 1/1 Running 0 12m
metrics-server-54c4d8c9df-7t5bn 1/1 Running 0 12m
metrics-server-54c4d8c9df-rjdt2 1/1 Running 0 12m
metrics-server-54c4d8c9df-tz54s 1/1 Running 0 12m
sample-app-967cff66f-qjfq2 1/1 Running 0 10m
sample-app-967cff66f-tnqcd 1/1 Running 0 103s
You can visit the metrics-server official release page to check the latest available versions.
To upgrade the Metrics server stack to the latest version, run the following command.
# helm upgrade metrics-server metrics-server/metrics-server --version "metrics-server-new-version" -n metrics-server --values ~/metrics-server.values
You can check the helm upgrade command documentation for more information.
You can remove your complete Metrics server installation from your system using the helm uninstall command.
# helm uninstall metrics-server -n metrics-server
The above command will remove all metrics-server related components from your server. You will also need to delete the metrics-server namespace from your system. To delete the namespace, run the following command.
# kubectl delete ns metrics-server
You've finished deploying the Metrics server on the Vultr Kubernetes Engine. Metrics server is a powerful tool for monitoring Kubernetes autoscaling metrics based on CPU utilization or memory usage. For more information, check out the Metrics server official GitHub page.