Continuous Deployment to Google Cloud Platform with Drone

Source:- nytimes.com

Over the course of the last year, the software development teams at The New York Times have been evaluating Google Cloud Platform for use in some of our future projects. To do this, we’ve surveyed a wide variety of software management techniques and tools, and we’ve explored how we might standardize building, testing and deploying our systems on GCP.

Our newly formed Delivery and Site Reliability Engineering team came up with two methods of deployment with Google Container Engine and Google App Engine for computing environments and the open source implementation of Drone as a continuous integration tool. As a result of this work, we are open sourcing two plugins for Drone: drone-gke and drone-gae respectively.

Container Engine is Google’s managed Kubernetes container orchestration platform. Kubernetes is an open source project that provides a declarative approach to managing containerized applications, enabling automatic scaling and healing properties. It encourages a common standard of how our applications are designed, deployed, and maintained across many independent teams. And because Kubernetes pools compute resources, developers can run many isolated applications in the same cluster, maximizing its resource usage density.

App Engine is a mature serverless platform Google has offered since 2008. It is capable of quickly scaling up and down as traffic changes, which is ideal for many scenarios at The New York Times when you consider possible sudden spikes from breaking news alerts or the publication of the new daily crossword every weekday at 10 p.m.

Drone is an open source continuous integration and delivery platform based on container technology, encapsulating the environment and functionalities for each step of a build pipeline inside an ephemeral container. Its flexible yet standardized nature enables our teams to unify on a plugin-extensible, ready-to-use CI/CD pipeline that supports any custom build environment with isolation, all with a declarative configuration similar to commercial CI/CD services. The result is the ability for developers to confidently ship their features and bug fixes into production within minutes, versus daily or weekly scheduled deployments. As a containerized Go application, it is easy to run and manage, and we hope to contribute to the core open source project.

Google provides an excellent set of command line utilities that allow developers to easily interact with Google Container Engine and Google App Engine, but we needed a way to encapsulate those tools inside of Drone to simplify the workflow for developers. Luckily, plugins for Drone are simple to create as they can be written in Go and are easily encapsulated and shared in the form of a Docker container. With that in mind, the task of creating a couple reusable plugins was not that daunting.

drone-gke is our new plugin to wrap the gcloud and kubectlcommands and allow users to orchestrate deployments to Google Container Engine and apply changes to existing clusters. The Kubernetes yaml configuration file can be templated before being applied to the Kubernetes master, allowing integration with Drone’s ability to encrypt secrets and injecting build-specific variables.

Here is an example Drone configuration to launch a Go application in a container via a Kubernetes Deployment resource into Google Container Engine:

# test and build our binary
build:
  image: golang:1.7
 
  environment:
    - GOPATH=/drone
 
  commands:
    - go get -t
    - go test -v -cover
    - CGO_ENABLED=0 go build -v -a
 
  when:
    event:
      - push
      - pull_request
 
# build and push our container to GCR
publish:
  gcr:
    storage_driver: overlay
    repo: my-gke-project/my-app
    tag: "$$COMMIT"
    token: >
      $$GOOGLE_CREDENTIALS
 
    when:
      event: push
      branch: master
 
# create and apply the Kubernetes configuration to GKE
deploy:
  gke:
    image: nytimes/drone-gke
 
    zone: us-central1-a
    cluster: my-k8s-cluster
    namespace: $$BRANCH
    token: >
      $$GOOGLE_CREDENTIALS
 
    vars:
      image: gcr.io/my-gke-project/my-app:$$COMMIT
      app: my-app
      env: dev
    secrets:
      api_token: $$API_TOKEN
 
    when:
      event: push
      branch: master

And the corresponding Kubernetes configuration:

kind: Deployment
apiVersion: extensions/v1beta1
 
metadata:
  name: {{.app}}-{{.env}}
 
spec:
  replicas: 3
  template:
    metadata:
      labels:
        app: {{.app}}
        env: {{.env}}
    spec:
      containers:
        - name: app
          image: {{.image}}
          ports:
            - containerPort: 8000
          env:
            - name: APP_NAME
              value: {{.app}}
            - name: API_TOKEN
              valueFrom:
                secretKeyRef:
                  name: secrets
                  key: api-token
---
kind: Service
apiVersion: v1
 
metadata:
  name: {{.app}}-{{.env}}
 
spec:
  type: LoadBalancer
  selector:
    app: {{.app}}
    env: {{.env}}
  ports:
    - port: 80
      targetPort: 8000
      protocol: TCP

And the corresponding Kubernetes secrets configuration:

kind: Secret
apiVersion: v1
 
metadata:
  name: secrets
 
type: Opaque
 
data:
  api-token: {{.api_token}}

drone-gae is our new plugin to wrap the gcloud and appcfgcommands and allow users to make deployments to Google App.

Engine standard environment with Go, PHP or Python or to the flexible environments with any language.

Here’s a very basic example of all the configuration required to launch a new version of a Go service to Google App Engine’s standard environment with a second step to migrate traffic to that version:

deploy:
  # deploy new version to GAE
  gae:
    image: nytimes/drone-gae
 
    environment:
      - GOPATH=/drone
 
    action: update
    project: my-gae-project
    version: "$$COMMIT"
    token: >
      $$GOOGLE_CREDENTIALS
 
    when:
      event: push
      branch: master
 
  # set new version to 'default', which migrates 100% traffic
  gae:
    image: nytimes/drone-gae
 
    action: set_default_version
    project: my-gae-project
    version: "$$COMMIT"
    token: >
      $$GOOGLE_CREDENTIALS
 
    when:
      event: push
      branch: master

Deploying new versions to the flexible environment requires a little more work, but it’s straightforward when using the plugin. We first use a build step to test and compile the code, then a publish step to build and publish a Docker container to Google Container Registry (via the drone-gcr plugin) and finally, we kick off the deployment via our new plugin.

# test and build our binary
build:
  image: your-dev/golang:1.7
 
  environment:
    - GOPATH=/drone
 
  commands:
    - go test -v -race ./...
    - go build -o api .
 
  when:
    event:
      - push
      - pull_request
 
# build and push our container to GCR
publish:
  gcr:
    storage_driver: overlay
    repo: my-gae-project/api
    tag: "$$COMMIT"
    token: >
       $$GOOGLE_CREDENTIALS
 
    when:
      branch: [develop, master]
      event: push
 
deploy:
  # deploy a new version using the docker image we just published and stop any previous versions when complete.
  gae:
    image: nytimes/drone-gae
 
    action: deploy
    project: my-gae-project
    flex_image: gcr.io/my-gae-project/api:$$COMMIT
    version: "$${COMMIT:0:10}"
    addl_flags:
     - --stop-previous-version
    token: >
      $$GOOGLE_CREDENTIALS
 
    when:
      event: push
      branch: develop

We hope open sourcing these tools helps other engineers who want to leverage Drone as a continuous delivery solution. We are also looking to the community to take a look and help us harden our systems. Please raise an issue if you find any problems and follow the contributing guidelines if you make a pull request.