GRAFANA LOKI

 

 

                  GRAFANA LOKI 

This is an open-source tool with the ability to scale horizontally, and it can handle large volume of log data. It is integrated with prometheus and configured in a YAML file (loki.yaml)

HOW DOES GRAFANA LOKI COLLECT LOGS:

Loki stores data in S3, locally on the filesystem as well as built ONLY with indexing metadata of your logs. In the course of collecting data, Grafana Loki manages the index storage using DynamoDB Table. 

HOW LOG IS FILTERED?

We can use "LOGQL", a query language that allows you to filter logs using LABEL which is attached to the log streams.

FILTERING STAGES IN GRAFANA LOKI:

• Stream selector: This allows users to a unique keywords for example, you may be searching for error in a key value pair via the logs from an application within a certain period. 

• Line filters: This makes use of log line and each line makes use of "filter operator" which uses "strings"

• Label filters: This uses LOGQL tool to query and analyze metrics within prometheus.

MAJOR TYPES OF LOGQL

Log queries: It returns all the content of the log lines.

Metrics queries: It extends the log queries and calculate the average duration of a request. 

TWO RULES OF LOKI:

• Alert rules.

• Recording rules. 

GRAFANA LOKI & HTTP:

Every user must authenticate to get that permission to access the token after being authorized you can send HTTP requests to the API endpoints to carry out the task. 

ENDPOINTS:

There are few endpoints components;

GET /ready

GET /metrics

GET /config

GET /services

GRAFANA LOKI WITH KUBERNETES:

Grafana Loki can be used to collect pod logs in Kubernetes clusters, and in order to collect the logs, "Fluentd and Promtail" can be used by installing an agent that runs on each node within the Kubernetes cluster. Promtail then sends the logs to LOKI for storage. 

HOW DOES GRAFANA LOKI IS CONFIGURED IN KUBERNETES MANIFEST FILE? 

 

apiVersion: extensions/v1beta1

kind: Deployment

metadata:

labels:

name: query-frontend

name: query-frontend

namespace: <namespace>

spec:

minReadySeconds: 10

replicas: 2

selector:

matchLabels:

name: query-frontend

template:

metadata:

labels:

name: query-frontend

spec:

containers:

- args:

- -config.file=/etc/loki/config.yaml

- -log.level=debug

- -target=query-frontend

image: grafana/loki:latest

imagePullPolicy: Always

name: query-frontend

ports:

- containerPort: 3100

name: http

protocol: TCP

resources:

limits:

memory: 1200Mi

requests:

cpu: "2"

memory: 600Mi

volumeMounts:

- mountPath: /etc/loki

name: loki-frontend

restartPolicy: Always

terminationGracePeriodSeconds: 30

volumes:

- configMap:

defaultMode: 420

name: loki-frontend

name: loki-frontend

 Next slide we will look at prometheus . Happy Learning !!! 😊

Referencing : https://grafana.com/docs/loki/latest/logql/log_queries/#log-stream-selector