Using mcrouter with memcached on Kubernetes

Cloud Platform (intermediate level) posted on 17th August 2018


This is a part of the series of posts on Getting memcache up and running on Kubernetes which explained how to create your first cluster and Installing memcache with Kubernetes which installed some memcache instances on your cluster and Exposing a memcache loadbalancer which makes it available externally. You'll also have created a test app in your favorite node environment, and tested the memcache integration as described in Creating a test app for memcache on Kubernetes.  You also should have got the app running on Kubernetes as described in Getting a simple app running on Kubernetes

What we discovered was that although memcache works properly with each platform, the cache results are not reliably available across platforms, so we need to implement a router to synchronize them.

Helm install

Luckily there is already a Helm chart to install mcrouter. We used a similar process to install memcached in the first place as described in Installing memcache with Kubernetes. (which you should refer to if you don't have Helm installed any longer).

First though, we'll get rid of the memcached instances we already have.
$ helm delete mycache

and the exposed loadbalancer service
$ kubectl delete service mc

An now install the mcrouter service
$ helm install stable/mcrouter --name=mcrouter

Expose the service
$ kubectl expose service mcrouter-memcached --port=11211 --target-port=11211 --name=mcrouter --type=LoadBalancer

You should now have these pods
$ kubectl get pods
NAME                  READY STATUS RESTARTS AGE
mcdemo-566dfddd79-9d446 1/1 Running       0 16m
mcdemo-566dfddd79-pjw7p 1/1 Running       0 15m
mcrouter-mcrouter-4zc8l 1/1 Running       0 38m
mcrouter-mcrouter-f4v8x 1/1 Running       0 38m
mcrouter-mcrouter-phw6z 1/1 Running       0 38m
mcrouter-memcached-0    1/1 Running       0 38m
mcrouter-memcached-1    1/1 Running       0 37m
mcrouter-memcached-2    1/1 Running       0 37m

and these services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes         ClusterIP    10.27.240.1   <none>       443/TCP          3d
mcdemo-expose      LoadBalancer 10.27.243.143 xx.xxx.xxx.x 8081:31143/TCP   3h
mcdemo-service     ClusterIP    10.27.250.152 <none>       8081/TCP         4h
mcrouter           LoadBalancer 10.27.241.197 xx.xxx.xxx.x 11211:32737/TCP 39m
mcrouter-mcrouter  ClusterIP    None          <none>       5000/TCP        42m
mcrouter-memcached ClusterIP    None          <none>       11211/TCP       42m

Endpoints

You'll need to modify your secrets file with the new dns name (for inside the cluster) and the mc-demo loadbalancer external ip address (outside)
module.exports = ((ns) => {

    ns.memcached = {
        defExpires: 30 * 60 * 2,
        maxExpires: 30 * 60 * 24,
        c9: {
            host: 'xxxxxx:11211', // temporarly exposed for testing 
            silo: 'some value',
            verbose: true,
        },
        ku: {
            host: 'mcrouter-memcached.default.svc.cluster.local:11211',
            silo: 'some value',
            verbose: true
        }
    };

    return ns;
})({});

Does it work?

What we are hoping now is that a search from Kubernetes will cause its cached result to be visible if the same result is done on the standalone node app.

searching
http://35.xxx.xxx.xxx:8081/starwars/people?search=chewbacca

time without cache
1132 'ms to complete'
timing with cache
hit: http://swapi.co/api/people?search=luke 25960109ae793d5f3e351a5fb946726963a35f7c
1 'ms to complete'

The same search using the standalone node App
hit: http://swapi.co/api/people?search=chewbacca 05041f462435653b85a5bc386cfd72b50fb8e4fb
10 'ms to complete'

Wonderful - it works!

Next steps

Let's see if we can get Apps Script working with the same memcached router







Why not join our forum, follow the blog or follow me on twitter to ensure you get updates when they are available.
Comments