Maintaining a fixed number of machines for Kraken CI can be costly. If there is no high traffic of builds, then some machines sit idle and waste power. The solution to this problem is autoscaling ie. spawning machines with Kraken Agents dynamically depending on the current needs in the Kraken CI. If there is submitted a new flow with many building or testing jobs, then new machines are spawned. When the flow completes, machines are terminated and no costs are incurred.
Currently, Kraken CI autoscaling is supporting AWS EC2 virtual machines, AWS ECS containers and Azure virtual machines.
The following chapters show how to configure access to AWS, then how to configure parameters and behaviour of spawned machines in case of AWS EC2. In the end, the usage in job definitions is presented.
Details about configuration for AWS as well as for Azure VMs can be found in the Kraken docs.
Global Cloud Settings
First, global settings have to be set to allow access to a given cloud provider. In Web UI, on Kraken -> Settings page, in Cloud tab, there is a form for collecting credentials to cloud providers:
In the case of AWS, there are required
Access Key and
Key. After filling in the values and saving settings, it is possible
to check if it works by clicking
Test Access button.
Preparing Cloud Environment
In the case of AWS it is also required to assign proper permissions so that Kraken can create or destroy EC2 instances. The list of all requires permissions looks as follows:
Configuration in Agents Groups
Having set credentials to cloud providers, it is possible now to
configure aspects of spawning new machines. This can be done on Kraken
-> Agents -> Groups page. Let's create a new Agents Group by clicking
Add New Group button and naming it
aws-t2-micro. The newly created
details will be presented on a separate tab. On this tab, there is a
Agents Deployment. So the deployment can be manual (default)
or automated to a particular cloud provider. In the case of Amazon Web
Services, there are the following options that can be set:
Select any region that fits you. Limit of instances can be set to 2. This will not allow having more running machines than 2. For now, the default AMI can be empty.
Then in VM options, let's stick to AWS free tier and select t2.micro instance type. The rest of the checkboxes leave empty. Init script also can be blank.
To ensure that dangling instances do not eat up our budget, let's set the options in Destruction section to 1 job and 10 minutes. So a spawned machine will be terminated after executing 1 job. And even then, if it sits idle for more than 10 minutes, it will be destroyed as well.
Now, to use the defined
aws-t2-micro Agents Group, we need to prepare
a project with a branch and a stage. More details about that can be
found in Introductory Guide. So let's concentrate
now on defining a job.
"cmd": "echo 'hello world'"
There isn't much difference compared to regular Kraken jobs. The job
has a defined "environments" section where we are pointing to our
aws-t2-micro Agents Group. What's new is the
system field, which
points to the
AMI identifier. This indicates the AWS EC2 image. The
system field can be set to
any as well. In such a case, the AMI ID
will be taken from the Agents Group definition, from the
Now when a job is assigned to an agents group with configured Agents Deployment then a new machine will be spawned for that job if agents are no available in the Kraken.
Let's change the view to Branch Results view and trigger a new flow by
Run Flow button. On the run page, the list of jobs shows our