Posted onJul 21, 2022

CircleCI Split Config Orb!

#circleci #cue

I created a CircleCI Orb named Split Config. This is my private fun coding :)

https://circleci.com/developer/orbs/orb/bufferings/split-config

Do you want to split your configuration file?

Have you ever thought if you could split theconfig.yml file of your CircleCI projects? Especially it could be bigger when the repository is a type of MonoRepo which includes multiple services in one repository.

Good news! Split Config Orb is here for you!

Split Config Orb

You can split yourconfig.yml into multiple configuration files. This Orb merges the split configs into one YAML when CircleCI starts and execute a pipeline with the merged configuration.

Example

Let's say you have multiple services in your repo like this:

Well, it's empty but please imagine the source codes of each services are there :)

Each service has aconfig.yml for the service. For example. the config for service1 is as follows:

version:2.1jobs:service1-say-hello:docker:-image:cimg/base:stablesteps:-checkout-run:name:"Sayhello"command:"echoHello,World!1"workflows:service1-say-hello-workflow:jobs:-common-say-hello-service1-say-hello

service2:

version:2.1jobs:service2-say-hello:docker:-image:cimg/base:stablesteps:-checkout-run:name:"Sayhello"command:"echoHello,World!2"workflows:service2-say-hello-workflow:jobs:-common-say-hello-service2-say-hello

etc...

The Orb merges these config files and it will be like this:

version:2.1jobs:common-say-hello:docker:-image:cimg/base:stablesteps:-checkout-run:name:Say hellocommand:echo Hello, World! commonservice1-say-hello:docker:-image:cimg/base:stablesteps:-checkout-run:name:Say hellocommand:echo Hello, World!1service2-say-hello:docker:-image:cimg/base:stablesteps:-checkout-run:name:Say hellocommand:echo Hello, World!2service3-say-hello:docker:-image:cimg/base:stablesteps:-checkout-run:name:Say hellocommand:echo Hello, World!3workflows:service1-say-hello-workflow:jobs:-common-say-hello-service1-say-helloservice2-say-hello-workflow:jobs:-common-say-hello-service2-say-helloservice3-say-hello-workflow:jobs:-common-say-hello-service3-say-hello

How to use

It's super simple. You just need to prepare thisconfig.yml

version:2.1setup:trueorbs:split-config:bufferings/split-config@0.1.0workflows:generate-config:jobs:-split-config/generate-config:find-config-regex:.*/\.circleci/config\.yml

Then the Orb finds the config files according to the regex you specified, and merge them into one YAML file. After that, the Orb executes CircleCI pipeline with the generated YAML file.

It uses theDynamic Config feature of CircleCI to start a new pipeline. By default, the feature is disabled, so pleaseenable the feature to get started.

Then you will see something like this:

That's all!

Are you interested in how it merges the configs?

How does it merge the configs?

The Split Config Orb uses CUE to merge the YAML files.

CUEhttps://cuelang.org/

CUE can import YAML file, convert them into CUE format, and merge them into one CUE config. In addition, it has a feature to export CUE config as YAML.

I used these CUE features to merge your YAML configs. CUE is so powerful but I only use a small part of it. If you want to know more about CUE or how it merges multiple CUE configs, please check CUE specification!