GS DevOps Mate Series: Intro

Over the last decade, Salesforce has emerged as the CRM leader and one of the PaaS of choice for Enterprise Apps. We can attribute this success to how quickly an organization can build production-ready business applications, typically in a few months, if not days!  So much can be achieved through mere configurations & minimum code customizations, which is the platform’s core strength. 

However, with large enterprises having complex business processes, you see the need for a lot more code customizations and a more mature DevOps process.

Traditionally, the DevOps processes in such organizations get built on the central idea that all of your metadata, be it code or config, is utilizing a Version/Source Control System. A Build system then uses this version control system to perform various steps like compilation, bundling/packaging, static code analysis, automated testing, etc., whenever there is code/config check-in by any of the Devs. This process is typically known as Continuous Integration(CI). Once the CI is successful, there can be automated ways to prepare the Servers and deploy the Software, commonly known as Continuous Deployment/Delivery. The challenge arises when organizations need to leverage their existing, well-established processes to solve the same problem for Salesforce.

Historically, DevOps on Salesforce and especially automated Deployments have always been a challenge, or I should say it is a bit different from the traditional Software development stack. While DevOps core principles are applicable on the Salesforce platform, due to the prior org-driven nature of development, fully automated deployments are still far-fetched.

Unlike a traditional Software application where devs develop pretty much everything locally—perform the build, that typically includes steps like compilation/transpilation, static code analysis, bundling, unit-testing, and packaging—in Salesforce, compilation and unit-testing of backend code are only possible once you deploy your code to Salesforce servers. 

Although Salesforce has addressed some of these concerns with the launch of SFDX, let’s first look at some of the challenges organizations face while trying to follow the industry standard CI/CD processes as mentioned above for their Salesforce deployments.

1. Bringing all the config and code into source control: Salesforce refers to its code and config as metadata. Although Salesforce has covered good ground here, managing 100% of your metadata in source-control is not possible yet. There is also a lack of parity across the various deployment & packaging options. You can find more details here.
2. Local builds are not possible: Once the code and config are in source control, the next challenge is the build and deployment systems. Unlike traditional builds, the build on Salesforce needs to happen on the Salesforce servers, which means until you initiate a deployment/validation on the actual server, you can’t be sure if your build will successfully compile and deploy. 
3. Slow CI/CD: The source control based Deployments can become slow as the code and config increases. There is no out of the box way to deploy only the latest committed changes.
4. Maintaining Package Manifest: Developers also need to make sure that the package.xml, a manifest file, has a list of all the components to be deployed. There is no standard way in MDAPI, which takes care of this. 
5.  Support for Granular Metadata: Some of the core metadata tend to encapsulate a lot of information, making it challenging to independently deploy granular components. An example of one such metadata would be Objects (Encapsulating Validation Rules, Search Layout Options, Quick Actions, Fields, Record-Types etc.) & Profiles in the MDAPI format.

Due to all the challenges mentioned above, there is a thriving ecosystem of Deployment tools available that are only meant for Salesforce Deployments. Even if a company’s IT or Engineering team is utilizing Standard Version Control and Build tools – for example, BitBucket Pipelines, Circle CI, and Azure DevOps, there is a high chance you would see their Salesforce team following a different process and using one of those specialized Salesforce Deployment tools.
 

Identifying the gaps in this space, Salesforce introduced Salesforce Developer Experience, popularly known as SFDX, under which it offers the following core features:
 SFDX or Source format for metadata files: This format, unlike the old MDAPI format, is inherently capable of managing multi-folder source code like a traditional Software project. It also manages some of the metadata, like Objects, in a much more modular fashion.

A Command-Line Tooling: Also known as SFDX CLI, it enables devs to perform critical tasks related to source-driven development straightforwardly, unlike the old Force.com migration toolkit.

Scratch Orgs: Short-lived orgs (max 30 days), that can be spun up with relevant metadata using a few commands from SFDX CLI.
The overall idea is to make the Salesforce development experience close to a traditional source control driven development as possible.

Unlocked Packages and Second Generation Managed Packaging: As part of this, Salesforce wanted to make modular deployments a reality. With the help of SFDX commands tailored toward Second Gen Packaging, automating the packaging is a breeze.

MDAPI to New Source(SFDX) format conversion and vice versa: One of the most noticeable things in SFDX CLI is the commands for converting the old MDAPI format source code to the new SFDX format source code and vice versa. 

Commands for deploying and retrieving metadata and data: There are commands to deploy and extract the code in any of these formats, and then there are commands to retrieve and load data.

We specifically want to focus on the CLI part here. 

SFDX CLI is basically a NodeJS program that you install locally on your system. You can also install it on your Build Server of choice, thus enabling you to have an automated deployment or a CI/CD pipeline without the need for a specialized paid tool just meant for Salesforce deployments. 

You can stick to your standard CI/CD processes and tooling (which is easier said than done), and while doing this in some of the projects earlier on, we realized that there should be an easier way for various DevOps teams to adopt to SFDX based CI/CD. With our learnings from more than a year and a half of work, we are super stoked to announce that we will be open-sourcing the first of its kind Deployment tooling to the awesome Salesforce Community: GS DevOps Mate!

In the next part, we will be talking about what GS DevOps Mate is, the tech stack behind it, some of its core features, what it is not, and why we have chosen to build it the way it is today.