Cluster Monitoring

Part of series: Kubernetes

4. Cluster Monitoring

We are going to install Prometheus and Grafana to monitor our cluster.

4.1 Install Helm

On the master node, install Helm:

curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh

4.2 Install Prometheus and Grafana using Helm

# Add the Prometheus repository
sudo helm repo add prometheus-community https://prometheus-community.github.io/helm-charts --kubeconfig /etc/rancher/k3s/k3s.yaml
sudo helm repo update --kubeconfig /etc/rancher/k3s/k3s.yaml

# Create namespace for monitoring
sudo kubectl create namespace monitoring --kubeconfig /etc/rancher/k3s/k3s.yaml

# Install Prometheus and Grafana
sudo helm install prometheus prometheus-community/kube-prometheus-stack --namespace monitoring --kubeconfig /etc/rancher/k3s/k3s.yaml

4.3 Configure an Ingress for Grafana (optional)

If you want to access Grafana without using port-forward, you can configure an Ingress:

# Install Nginx Ingress Controller
sudo kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.8.2/deploy/static/provider/cloud/deploy.yaml --kubeconfig /etc/rancher/k3s/k3s.yaml

# Wait for the controller to be ready
sudo kubectl wait --namespace ingress-nginx \
  --for=condition=ready pod \
  --selector=app.kubernetes.io/component=controller \
  --timeout=120s \
  --kubeconfig /etc/rancher/k3s/k3s.yaml

# Create an Ingress for Grafana
cat <<EOF | sudo kubectl apply -f - --kubeconfig /etc/rancher/k3s/k3s.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: grafana-ingress
  namespace: monitoring
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx
  rules:
  - host: grafana.local
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: prometheus-grafana
            port:
              number: 80
EOF

Add grafana.local to your hosts file in Windows (C:\Windows\System32\drivers\etc\hosts):

192.168.1.127 grafana.local

Now you can access Grafana from http://grafana.local

Default user: admin
Default password: prom-operator

4.4 Predefined Dashboards

Prometheus and Grafana already come with predefined dashboards for Kubernetes. You can explore them from the Grafana interface:

Log in to Grafana
Click on “Dashboards” (squares icon in the sidebar)
Click on “Browse”
Explore the available dashboards:
- Kubernetes / Compute Resources / Cluster
- Kubernetes / Compute Resources / Namespace (Pods)
- Kubernetes / Compute Resources / Node (Pods)

← Previous