Getting started with Microservices and Kubernetes

In this I want to showcase how you can keep things simple with microservices and highlight everything you need to know about getting started with microservices and kubernetes.

With this tutorial I am going to take my blog, wordpress php monolith, and rip apart the only components I need and make them microservices. Then, we will go through setting up a local development kubernetes with minikube.

First things first, lets think about the services we need.

Decoupling the monolith backend

It’s a blog so I think its fairly easy to decouple what we need. Here’s a simple list of the services we are going to create:

  • Posts
  • Tags
  • Categories
  • API Gateway

Services I did not mention

Yeah, that was a short list huh? Well this is by design. Those are the bare minimum pieces I need for my blog to work. The reason is when possible I try to offset things to other services. Think in hash form here:

  1. Stats => Google analytics
  2. Cache => CDN
  3. Comments => Disqus
  4. Pages => Static routes on frontend
  5. etc… etc…

This is a key component to microservices. Don’t reinvent the wheel like I am doing here in this tutorial haha. But in general if there is a service that is affordable or free and available you can spend more time on writing business logic.

Writing our Services

Posts

The posts service should be pretty simple. Lets think, we need basic CRUD (Create, Read, Update, Delete) operations. For this we are going to do a simple Python flask service.

app.py
"""
Post Service App Setup
"""
from flask import Flask, request
from flask_restful import Resource, Api
from flask_sqlalchemy import SQLAlchemy


# Handle Requests
class Posts(Resource):
    """
    Handles all HTTP methods for Post Service
    """
    def get(self, post_uuid=None):
        """
        Retreive all posts or a given post on uuid
        :param post_uuid: UUID
        :return: Dict
        """
        if post_uuid:
            return get_post(post_uuid)
        return get_all_posts()

    def post(self):
        """
        Create a new post
        """
        post_data = request.get_json(force=True)
        return create_post(post_data)

    def put(self, post_uuid):
        """
        Update a post
        :param post_uuid: UUID
        :return: Boolean
        """
        return update_post(post_uuid)

    def delete(self, post_uuid):
        """
        Delete a post
        :param post_uuid: UUID
        :return: Boolean
        """
        return delete_post(post_uuid)


# Functions
def get_post(post_uuid):
    """
    Pull data from db for a blog post
    :param post_uuid: UUID
    :return: Dict
    """
    # Mock Data
    return {
        "post_uuid": post_uuid,
        "title": "I'm a blog post!",
        "content": "Yes, this is a blog post about things and stuff but mainly about showing how to do microservices in a simple and easy way",
        "author": "Matthew Harris",
        "created": "12-03-2016 12:15:00",
        "updated": ""
    }

def get_all_posts():
    """
    Pull all posts from db
    :return: List of Dicts
    """
    # Mock Data
    return [
        {
            "post_uuid": "1234-5678-1234-5678",
            "title": "I'm a blog post!",
            "content": "A post about foo",
            "author": "Matthew Harris",
            "created": "12-03-2016 12:15:17",
            "updated": ""
        },
        {
            "post_uuid": "2345-6789-2345-6789",
            "title": "I'm a different blog post!",
            "content": "A different post about bar",
            "author": "Matthew Harris",
            "created": "12-03-2016 12:18:33",
            "updated": ""
        }
    ]

def create_post(post_data):
    """
    Create a new post
    :param post_data: Dict
    :return: Boolean
    """
    # Mock Data
    return True

def update_post(post_uuid):
    """
    Update a given post
    :param post_uuid: UUID
    :return: Boolean
    """
    # Mock Data
    return True

def delete_post(post_uuid):
    """
    Delete a given post
    :param post_uuid: UUID
    :return: Boolean
    """
    # Mock Data
    return True

if __name__ == "__main__":
    # Setup App
    app = Flask(__name__)
    app.config.from_object('config')
    api = Api(app)
    db = SQLAlchemy(app)
    # Setup Routes
    api.add_resource(
        Posts,
        '/v1/post',
        '/v1/post/<post_uuid>',
    )

    # Run the App
    app.run(host="0.0.0.0")
models.py
from datetime import datetime

import uuid


class Posts(db.Model):
    """
    Our Database Model
    """
    id = db.Column(db.Integer, primary_key=True)
    post_uuid = db.Column(db.String(50))
    title = db.Column(db.String(100))
    content = db.Column(db.Text)
    author = db.Column(db.String(50))
    created = db.Column(db.DateTime)
    updated = db.Column(db.DateTime)

    def __init__(self, title, content, author):
        self.post_uuid = uuid.uuid4()
        self.title = title
        self.content = content
        self.author = author
        self.created = datetime.utcnow()

    def set_update_time(self):
        self.updated = datetime.utcnow()
config.py
"""
All of our App Configuration
"""

class Config(object):
    """
    Configuration Object
    """

    # DB Vars
    db_user = 'dbuser'
    db_pass = 'dbpassword'
    db_host = 'localhost'
    db_name = 'posts'

    # DB Settings
    SQLALCHEMY_TRACK_MODIFICATIONS = True
    SQLALCHEMY_DATABASE_URI = 'mysql://{db_user}:{db_pass}@{db_host}/{db_name}'.format(
  db_user=db_user,
  db_pass=db_pass,
  db_host=db_host,
        db_name=db_name
    )
    SQLALCHEMY_POOL_SIZE = 10
    SQLALCHEMY_POOL_TIMEOUT = 10
    SQLALCHEMY_POOL_RECYCLE = 500
requirements.txt
aniso8601==1.2.0
click==6.6
Flask==0.11.1
Flask-RESTful==0.3.5
Flask-SQLAlchemy==2.1
itsdangerous==0.24
Jinja2==2.8
MarkupSafe==0.23
python-dateutil==2.6.0
pytz==2016.7
six==1.10.0
SQLAlchemy==1.1.4
Werkzeug==0.11.11

Tags

The great thing about microservices is you are not limited by a given technology, language or infrastructure. So below is a mock tags microservice implemented in node.js.

tags.js
'use strict';

const Hapi = require('hapi');
const Boom = require('boom');

/*
 * Setup Server
 */
var server = new Hapi.Server();
server.connection({port: 5001});

/*
 * Setup Routes
 */
server.route({
  method: ['GET', 'POST', 'PUT', 'DELETE'],
  path: '/v1/tag/{tag_uuid?}',
  handler: function(request, reply) {
    /*
     * Request logging
     */
    var timestamp = new Date(request.info.received);
    console.log(`${timestamp} - ${request.info.remoteAddress} - ${request.method.toUpperCase()} ${request.info.host}${request.path}`);

    /*
     * Handle each method
     * with mock responses
     */
    switch ( request.method ) {

      /*
       * GET Handling
       */
      case 'get':
        if ( request.params.tag_uuid ) {
          return reply({
            "tag_uuid": request.params.tag_uuid,
            "name": "Sample",
            "id": 1,
            "type": 1
          })
        }
        return reply([
            {
              "tag_uuid": "1234-5667-1234-1234",
              "name": "Test",
              "id": 2,
              "type": 1
            },
            {
              "tag_uuid": "4321-1234-4321-1234",
              "name": "Javascript",
              "id": 3,
              "type": 1
            }
        ]);
        break;

      /*
       * POST Handling
       */
      case 'post':
        if ( request.params.tag_uuid ) {
          return reply(Boom.badRequest('Unsupported method'));
        }
        return reply(true);
        break;

      /*
       * PUT Handling
       */
      case 'put':
        if ( request.params.tag_uuid ) {
          return reply(true);
        }
        return reply(Boom.badRequest('Unsupported method'));
        break;

      /*
       * DELETE Handling
       */
      case 'delete':
        if ( request.params.tag_uuid ) {
          return reply(true);
        }
        return reply(Boom.badRequest('Unsupported method'));
        break;
    }
  }
});

/*
 * Start Server
 */
server.start(() =&gt; {
  console.log(`started at ${server.info.uri}`);
});

/*
 * Shutdown Server
 */
function shutdown() {
  server.stop(() =&gt; console.log('shutdown complete'));
}

process
.once('SIGINT', shutdown)
.once('SIGTERM', shutdown);
package.json
{
  "name": "tags",
  "version": "1.0.0",
  "description": "A Sample Tags Microservice",
  "main": "tags.js",
  "dependencies": {
    "boom": "^4.2.0",
    "hapi": "^16.0.1"
  },
  "devDependencies": {},
  "scripts": {
    "start": "node tags.js",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "Matthew Harris",
  "license": "ISC"
}

Categories

Yes! That’s right folks. You can write microservices in PERL!!! Perl is not dead, in fact it is alive and well. See: Perl6

categories.pl
use Mojolicious::Lite;
# A Perl Microservice Sample
# perl categories.pl daemon -m production -l http://*:8080

# 
# Setup Routes
#
get '/v1/category/:category_uuid' =&gt; 
sub {
    shift-&gt;render(
            json =&gt; {
                "id" =&gt; 1,
                "name" =&gt; "Perl",
                "category_uuid" =&gt; "1234-1234-1234-1234"
            }
        );
};

put '/v1/category/:category_uuid' =&gt;
sub {
    shift-&gt;render(
        text =&gt; 'true'
    ); 
};

del '/v1/category/:category_uuid' =&gt;
sub {
    shift-&gt;render(
        text =&gt; 'true'
    );
};

post '/v1/category' =&gt;
sub {
    shift-&gt;render(
        text =&gt; 'true'
    );
};

get '/v1/category' =&gt; 
sub {
    shift-&gt;render(
        json =&gt; [
            {
                "id" =&gt; 1,
                "name" =&gt; "Perl",
                "category_uuid" =&gt; "1234-1234-1234-1234"
            },
            {
                "id" =&gt; 2,
                "name" =&gt; "Python",
                "category_uuid" =&gt; "4321-4321-4321-4321"
            }
        ]
    );
};
#
# Exception Handling
#
sub handle404 {
    my $self = shift;
    $self-&gt;rendered(404);
    $self-&gt;render(
        json =&gt; {
            "statusCode" =&gt; 404,
            "message" =&gt; "404 Not Found"
        }
    );
}

any '*' =&gt; &handle404;
any '/' =&gt; &handle404;

API Gateway

An API Gateway is a single source that is going to sit in front of all of our services and and handle routing requests to the proper service. Typically it would also handle common things such as authentication and authorization; however that is outside the scope of this tutorial.

main.go

Golang, a fast performant language. It can also handle a crap ton of requests! Maybe not the most elegant implementation but it works 🙂.

package main

import "log"
import "net/http/httputil"
import "net/http"
import "net/url"
import "regexp"

func main() {
    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {

        uri := r.RequestURI

        // Pass request to post service
        post_matched, _ := regexp.MatchString("/v1/post.*", uri)
        if post_matched {
            proxy := reverseProxy("http://post-service/")
            proxy.ServeHTTP(w, r)
        } 
        
        // Pass request to category service
        category_matched, _ := regexp.MatchString("/v1/category.*", uri)
        if category_matched {
            proxy := reverseProxy("http://category-service/")
            proxy.ServeHTTP(w, r)
        } 
        
        // Pass request to tag service
        tag_matched, _ := regexp.MatchString("/v1/tag.*", uri)
        if tag_matched {
            proxy := reverseProxy("http://tag-service/")
            proxy.ServeHTTP(w, r)
        }

        log.Printf("%s %s %s%s", r.Header.Get("X-Forwarded-For"), r.Method, r.Host, r.RequestURI)
    })
    log.Fatal(http.ListenAndServe(":80", nil))
}

func reverseProxy(target string) *httputil.ReverseProxy {
    url, _ := url.Parse(target)
    return httputil.NewSingleHostReverseProxy(url)
}
build.sh
CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o gateway .

Now we have all the pieces of our mock application. Time to containerize and setup our development environment on minikube.

Setup Development Environment

Dockerizing Apps

Now lets setup our services with docker find dockerfiles below:

Post Service
FROM python:2-onbuild
CMD ["python", "./app.py"]
Tag Service
FROM node:5-onbuild
Category Service
FROM perl:5.20
RUN curl -L https://cpanmin.us | perl - -M https://cpan.metacpan.org -n Mojolicious
COPY categories.pl /usr/src/app/categories.pl
WORKDIR /usr/src/app
CMD ["perl", "./categories.pl", "daemon", "-m", "category-service", "-l", "http://*:80"]
API Gateway
FROM scratch
ADD gateway /
CMD ["/gateway"]

Now that were all dockerized; let go ahead and setup are configuration files for kubernetes for each service.

Configure Deployments and Services
post.yml
apiVersion: v1
kind: Service
metadata: 
  name: post-service
spec:
  ports:
    - port: 80
      targetPort: 5000
      name: http
  selector:
    name: post-service
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: post-service
spec:
  template:
    metadata:
      labels:
        name: post-service
    spec:
      containers:
        - name: post-service
          image: morissette/post-service
          ports:
            - containerPort: 5000
tag.yml
apiVersion: v1
kind: Service
metadata: 
  name: tag-service
spec:
  ports:
    - port: 80
      targetPort: 5001
      name: http
  selector:
    name: tag-service
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: tag-service
spec:
  template:
    metadata:
      labels:
        name: tag-service
    spec:
      containers:
        - name: tag-service
          image: morissette/tag-service
          ports:
            - containerPort: 5001
category.yml
apiVersion: v1
kind: Service
metadata: 
  name: category-service
spec:
  ports:
    - port: 80
      name: http
  selector:
    name: category-service
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: category-service
spec:
  template:
    metadata:
      labels:
        name: category-service
    spec:
      containers:
        - name: category-service
          image: morissette/category-service
          ports:
            - containerPort: 80
gateway.yml
apiVersion: v1
kind: Service
metadata: 
  name: gateway-service
spec:
  type: NodePort
  ports:
    - port: 80
      name: http
  selector:
    name: gateway-service
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: gateway-service
spec:
  template:
    metadata:
      labels:
        name: gateway-service
    spec:
      containers:
        - name: gateway-service
          image: morissette/gateway-service
          ports:
            - containerPort: 80
Installing minikube

minikube, is a command that allows you to have a kubernetes cluster on your local machine.

Linux
curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.12.2/minikube-linux-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/
OSX
curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.12.2/minikube-darwin-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/
Create your local cluster

Note: You will need to have VirtualBox w/ the kernel modules

[[email protected] devops]$ minikube start
Starting local Kubernetes cluster...
Kubectl is now configured to use the cluster.
Launch with kubectl
[[email protected] devops]$ ls | while read file; do kubectl create -f $file; done
service "category-service" created
deployment "category-service" created
You have exposed your service on an external port on all nodes in your
cluster.  If you want to expose this service to the external internet, you may
need to set up firewall rules for the service port(s) (tcp:30647) to serve traffic.
See http://releases.k8s.io/release-1.3/docs/user-guide/services-firewalls.md for more details.
service "gateway-service" created
deployment "gateway-service" created
service "post-service" created
deployment "post-service" created
service "tag-service" created
deployment "tag-service" created
Wait for pods to start

Pod, is the term kubernetes uses for containers run on its platform. You interact with your cluster via the kubectl command. You should see:

[[email protected] devops]$ kubectl get pods
NAME                               READY     STATUS              RESTARTS   AGE
category-service-367250328-xxqhx   0/1       ContainerCreating   0          1m
gateway-service-3188116052-b34ev   0/1       ContainerCreating   0          1m
post-service-1383581485-g6ec9      0/1       ContainerCreating   0          1m
tag-service-746243472-1t71s        0/1       ContainerCreating   0          1m
[[email protected] devops]$ kubectl get pods
NAME                               READY     STATUS              RESTARTS   AGE
category-service-367250328-xxqhx   1/1       Running             0          2m
gateway-service-3188116052-b34ev   1/1       Running             0          2m
post-service-1383581485-g6ec9      0/1       ContainerCreating   0          2m
tag-service-746243472-1t71s        0/1       ContainerCreating   0          2m

As the docker images are pulled from docker hub it can take a couple minutes depending on the size of your images.

[[email protected] devops]$ kubectl get pods
NAME                               READY     STATUS    RESTARTS   AGE
category-service-367250328-xxqhx   1/1       Running   0          4m
gateway-service-3188116052-b34ev   1/1       Running   0          4m
post-service-1383581485-g6ec9      1/1       Running   0          4m
tag-service-746243472-1t71s        1/1       Running   0          4m

Alright, it took four minutes for all the containers to be pulled from the internet and started. I’m on wifi so you will probably have faster download than me.

Test our microservice backend

To make sure our application is working as expected we need to get the exposed IP and port of the gateway-service, we do that with this command:

[[email protected] devops]$ minikube service -n default --url gateway-service
http://192.168.99.100:30647

Now lets test some of our routes and look at the logs.

Some simple curl tests
[[email protected] devops]$ curl http://192.168.99.100:30647/v1/category
[{"category_uuid":"1234-1234-1234-1234","id":1,"name":"Perl"},{"category_uuid":"4321-4321-4321-4321","id":2,"name":"Python"}]
[[email protected] devops]$ curl http://192.168.99.100:30647/v1/tag
[{"tag_uuid":"1234-5667-1234-1234","name":"Test","id":2,"type":1},{"tag_uuid":"4321-1234-4321-1234","name":"Javascript","id":3,"type":1}]
[[email protected] devops]$ curl http://192.168.99.100:30647/v1/post
[{"updated": "", "post_uuid": "1234-5678-1234-5678", "author": "Matthew Harris", "created": "12-03-2016 12:15:17", "content": "A post about foo", "title": "I'm a blog post!"}, {"updated": "", "post_uuid": "2345-6789-2345-6789", "author": "Matthew Harris", "created": "12-03-2016 12:18:33", "content": "A different post about bar", "title": "I'm a different blog post!"}]

Reviewing logs
[[email protected] devops]$ kubectl logs -f gateway-service-3188116052-b34ev
2016/12/04 00:19:45 172.17.0.1 GET 192.168.99.100:30647/v1/category
2016/12/04 00:19:56 172.17.0.1 GET 192.168.99.100:30647/v1/tag
2016/12/04 00:20:00 172.17.0.1 GET 192.168.99.100:30647/v1/post
[[email protected] devops]$ kubectl logs -f tag-service-746243472-vqzv8
npm info it worked if it ends with ok
npm info using [email protected]
npm info using [email protected]
npm info lifecycle [email protected]~prestart: [email protected]
npm info lifecycle [email protected]~start: [email protected]
&gt; [email protected] start /usr/src/app
&gt; node tags.js
started at http://tag-service-746243472-vqzv8:5001
Sun Dec 04 2016 00:19:56 GMT+0000 (UTC) - 172.17.0.5 - GET 192.168.99.100:30647/v1/tag
Learning More

This is meant to be a mere primer. There is a lot more involved with microservices but hopefully this allows you to get your feet wet.

Interested in learning more? Comment here.

Or checkout some of these resources:
http://microservices.io/

Write a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.