In an earlier blog post I provided the steps to install elastisearch using helm and setting it up for logging using fluent-bit. With the introduction of elasticsearch operator the experience of managing the elasticsearch cluster in kubernetes has improved greatly. We will cover the same goal of setting up elastisearch and configuring it for logging as the earlier blog, with the same ease but much better experience.
Using operator allows you benefits in the area of security, upgrades and scalability. To learn more read the ealstic blog
Let’s look at the steps that we will be following:
- Install the Elasticsearch operator
- Install Elasticsearch using operator
- Install Kibana using operator
- Install fluent-bit
Step 1 - Install the Elasticsearch operator
Just run the below command. It will install teh CRDs and the controller that will help in managing the clusters.
kubectl apply -f https://download.elastic.co/downloads/eck/0.9.0/all-in-one.yaml
Step 2 - Install Elasticsearch using operator
Create a namespace logs using the below command:
kubectl create ns logs
Next prepare the below elasticsearch.yaml definition file.
|
|
You can also install the above using the single below line.
kubectl apply -f https://raw.githubusercontent.com/prabhatsharma/elasticsearch-using-operator/master/elasticsearch.yaml
-
Test the installation using the below command:
kubectl -n logs get pods
Output:
|
|
Get the password for elasticsearch using the below command. You will need this later to setup fluent-bit and login to your cluster.
kubectl -n logs get secret es-logs-es-elastic-user -o=jsonpath='{.data.elastic}' | base64 --decode
Step 3 - Install Kibana using operator
Create a below kibana.yaml definition file.
|
|
You can also apply it using the below 1 line command.
kubectl apply -f https://raw.githubusercontent.com/prabhatsharma/elasticsearch-using-operator/master/kibana.yaml
_
Test the installation using the below command:
kubectl -n logs get pods
Output:
|
|
To log on to kibana using port forwarding use below command:
kubectl -n logs port-forward svc/kibana-logs-kb-http 5601:5601
Now go to https://localhost:5601 and login using below credentials User ID: elastic Password: Output of command ($ kubectl get secret quickstart-es-elastic-user -o=jsonpath=’{.data.elastic}’ | base64 –decode)
Step 4 - Install fluent-bit
We will use helm to install fluent-bit
Download the fluent-bit helm values file using below command:
wget https://raw.githubusercontent.com/prabhatsharma/elasticsearch-using-operator/master/fluent-bit-values.yml
_
|
|
Set the http_passwd value to what you got in step 2
Now install fluentbit and configure it using below command
helm install stable/fluent-bit --name=fluent-bit --namespace=logs -f fluent-bit-values.yml
Test the installation using the below command:
kubectl -n logs get pods
|
|
After this step you should be able to access logs using kibana.