Search

CI/CD Enabler Framework for Google Cloud Platform

October 19, 2020


Introduction

Software systems in a typical enterprise environment are quite complex (& grows more complex over time).  It becomes a humongous task to coordinate different activities such as code & configuration changes especially when a large team is involved in development and operations. DevOps in software is seen as a cultural movement to ease out operational challenges related to integration and deployment. 

Continuous Integration (CI) is an underlying technique for DevOps. According to the CI principle, if a task takes a lot of time and energy, it should be done more often, making it significantly less painful at each step. Continuous Deployment (CD) is the last leg to make software releases available to staging/production environments. Enterprises are finding it crucial to employ a CI/CD pipeline in a software project(s) to ensure quality and timely deliveries.

Features of GCP’s Cloud Build Service:

Google Cloud Platform (GCP) provides Cloud Build service as a toolchain to kick start the CI for a project. Some of its features are listed below:

  • Integrates with existing source control repository i.e. GitHub, Bitbucket, Cloud Source Repositories, etc.
  • Executes builds on GCP infrastructure i.e. VMs (GCE), serverless compute, Kubernetes (GKE), or Firebase
  • Supports customization and control for the build, test, deploy workflows

Limitations of GCP’s Cloud Build Service:

However, Cloud Build does not support deployment/ setup creation natively and build artifacts management. It needs to be extended (or coupled with other services) to provide Continuous Deployment and hence have a complete CI/ CD pipeline. Some of the GCP services that are useful in this regard are Container Registry (for build artifact management) and Deployment Manager (Infrastructure as Code/Infrastructure deployment service).

Building a CI/CD Pipeline for a GCP Project:

To build a complete CI/CD pipeline for a GCP based project, some work is required for deployment/ setup creation and artifact management. This is a cumbersome task and DevOps engineers may find it hard to make quick progress using the available services (out of the box). In such a situation, having an enabling framework that helps in building the CI/CD pipeline without significant effort is very useful.

HSC’s team has been working on the GCP based projects and have developed a CI/CD pipeline enabler framework. This framework additionally supports notification for sharing the progress of the pipeline. The developed framework helps in the realization of the CI/CD pipeline shown below:

CI-CD-Pipeline

CI/CD Pipeline enabler framework

The framework helps in executing the workflow shown below:

CI-CD-Pipeline-Enablement-Framework

The key components of a typical workflow are briefly discussed below:

  • Cloud Source Repo: This is part of the target project while enabling the CI/CD.
  • Terminal: DevOps engineers need to apply a tag in a defined format after code check-in.
  • Cloud Build: This framework creates two Cloud Build triggers (one for Dev/Staging & one for Prod) and a configuration file as expected by Cloud Build. Cloud Build service triggers match the tag and invoke the framework provided CI/CD program. This program takes care of the complete CI/CD pipeline with the help of other GCP services.
  • Cloud Pub/Sub: This communicates the result of the pipeline through a Pub/Sub cloud resource with the help of an MS Teams plugin to the MS Teams channel.

The project lead/manager is responsible for enabling the CI/CD pipeline using this framework and the DevOps team simply triggers the pipeline by applying a git TAG after they check-in their changes.

HSC has been using this framework in multiple projects and has thereby helped in enabling CI/CD. These projects encompass complete AI/ML application lifecycle i.e. ETL, model training, testing/ validation, evaluation, deployment, monitoring, and a web app to see the application results.

Implementing a CI/CD pipeline helps reduce the risks in each build and makes a software application available to the prospective customers faster than ever before. Interested in deploying a CI/CD pipeline. Early bug detection to avoid any last-minute surprises, greater visibility of new builds, test cases, and issues arising due to the same, and achieving faster feedback to ensure flawless code delivery is a few added advantages of deploying the CI/CD pipeline framework.

Explore these benefits in depth in connection with your Google Cloud Project. Connect with us to take this forward.

 

References:

  1. CI principle: https://martinfowler.com/articles/originalContinuousIntegration.html
  2. Cloud Build: https://cloud.google.com/cloud-build/docs
  3. Reference CI Pipeline: https://cloud.google.com/solutions/continuous-integration
  4. Reference CD Pipeline: https://cloud.google.com/solutions/continuous-delivery
  5. DM Best Practices: https://cloud.google.com/deployment-manager/docs/best-practices





No Comments




Add Comment

 
 

We use cookies (including third party cookies) to ensure you get the best experience while visiting our website. Click "Accept Cookies" to accept the cookie usage. Click "Cookie Settings" to adjust cookie settings.

Mandatory Cookies

These cookies cannot be disabled

These cookies are necessary for the website to function and cannot be switched off.

Cookies:
  • .ASPXANONYMOUS
  • .DOTNETNUKE
  • __RequestVerificationToken
  • authentication
  • dnn_IsMobile
  • language
  • LastPageId
  • NADevGDPRCookieConsent_portal_0
  • userBrowsingCookie

Analytics Cookies

These cookies allow us to monitor traffic to our website so we can improve the performance and content of our site. They help us to know which pages are the most and least popular and see how visitors move around the site. All information these cookies collect is aggregated and therefore anonymous. If you do not allow these cookies we will not know when you have visited or how you navigated around our website.

Cookies:
  • _ga
  • _gat
  • _gid

Functional Cookies

These cookies enable the website to provide enhanced functionality and content. They may be set by the website or by third party providers whose services we have added to our pages. If you do not allow these cookies then some or all of these services may not function properly.

Cookies:
  • __atuvc
  • euconsent

Targeting Cookies

These cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Cookies:

Not used.