containers

Build Docker images without Docker - using Kaniko, Jenkins and Kubernetes

Thursday, Aug 8, 2019 by Prabhat Sharma kubernetes, containers, docker, kaniko, jenkins, CICD

Note: I will be using an Amazon EKS cluster on AWS and using Amazon ECR for storing images. There will be minor differences for handling other repositories. Jenkins is a hugely popular build tool that has been around for ages and used by many people. With huge shift to Kubernetes as a platform you would naturally want to run jenkins on Kubernetes. While running Jenkins in itself on Kubernetes is not a challenge, it is a challenge when you want to build a container image using jenkins that itself runs in a container in the Kubernetes cluster.

Continue Reading

Installing Jenkins on Kubernetes

Thursday, Aug 8, 2019 by Prabhat Sharma kubernetes, containers, docker, jenkins, CICD

In this post I will cover the steps to install Jenkins in a Kubernetes cluster. While Jenkins can be installed on standard VMs, if you are running a Kubernetes cluster then its much easier to install and run Jenkins in it. We will be using Helm to install Jenkins. We will follow the below steps to get up and running: Get the Jenkins values file for configuring our Jenkins installation Install Jenkins Access Jenkins Let’s rock ‘n’ roll:

Continue Reading

Installing Airflow on Kubernetes Using Operator

Sunday, Jul 28, 2019 by Prabhat Sharma kubernetes, containers, docker, airflow, helm, data engineering

Operator - “A Kubernetes Operator is an abstraction for deploying non-trivial applications on Kubernetes. It wraps the logic for deploying and operating an application using Kubernetes constructs.” –Richard Laub, staff cloud engineer at Nebulaworks Note: I will be using an EKS cluster on AWS. You could use the same steps on other cloud providers too. Well created Kubernetes Operators pack a lot of power and help run and manage stateful applications on kubernetes.

Continue Reading

Installing Airflow on Kubernetes

Friday, Feb 1, 2019 by Prabhat Sharma kubernetes, containers, docker, airflow, helm, data engineering

Data engineering is a difficult job and tools like airflow make that streamlined. Let’s take a look at how to get up and running with airflow on kubernetes. Prerequisites A kubernetes cluster - You can spin up on AWS, GCP, Azure or digitalocean or you can start one on your local machine using minikube Helm - If you do not already have helm installed then follow this tutorial to get it installed Installing airflow using helm 1.

Continue Reading