Skip to main content

Autoscale in Cloud

Kraken CI allows for creating executing machines dynamically in the cloud when they are needed. They can be either virtual machines or containers. When new jobs are triggered and there are no agents available for them, new machines with Kraken agents are spawned. The configuration of the way of spawning new machines is located in agent groups.

Currently, Kraken CI autoscaling is supporting:

  • AWS EC2 virtual machines
  • AWS ECS containers
  • Azure virtual machines
  • Kubernetes

Preparing Cloud Environment

In the case of AWS EC2, it is required to assign proper permissions to the accessing user so that Kraken can create or destroy EC2 instances. The list of all requires permissions looks as follows:

"Version": "2012-10-17",
"Statement": [
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"Resource": "*"

Global Cloud Settings

First, global settings must be set to allow access to a given cloud provider. In Web UI, on Kraken -> Settings page, in Cloud tab, there is a form for collecting credentials to cloud providers:

In the case of AWS, Access Key and Secret Access Key are required.

Preparing VM Image

In the case of VM environments, an operating system image needs to be prepared. For example, in the case of AWS EC2, this image is called AMI (Amazon Machine Image). There are certain requirements that this image needs to meet in order to run a Kraken Agent. It should have Python and Git preinstalled. Additionally, the generated image should be capable of running a user data script during the initial boot. This script is used to install the Kraken Agent and your own init script (which can be passed via Kraken).

This is an example Packer script for preparing AMI with Ubuntu:

And here, you can find an example script for Ansible that prepares an AWS AMI image for Windows with all the dependencies included:

Configuration in Agents Groups

Having set credentials to cloud providers, it is possible now to configure the aspects of spawning new machines. This can be done on Kraken -> Agents -> Groups page. When a particular group is selected, then its details will be presented on a separate tab. On this tab, there is a section Agents Deployment. So the deployment can be manual (default) or automated to the particular cloud provider.

In the case of AWS EC2, jobs are executed in AWS EC2 VMs. There are the following options that can be set:

The meaning of the settings is as follows.

In General section:

Region - AWS region where a machine will be spawned

Instances Limit - maximum number of spawned machines at a time

Default AMI Image - if AMI image is not provided in job definition then this one is used

In VM Options section:

Instance Type - EC2 instance type that should be used for spawned machines, the full list is available on AWS web page

Spot Instance - indicates where Spot instance should be used (AWS docs)

Monitoring - indicates if monitoring should be enabled (AWS docs)

CPU Credits Unlimited - this allows having more burst power, more in AWS docs

Disk Size - the size of root disk, if 0 then default size is used otherwise provided number is counted in GB

Init Script - a value of this text is passed as UserData to EC2 instance, generally, this is a script that is executing during the first boot of instance (AWS docs)

In Destruction section:

Destroy VM after N jobs - this indicates that the machine should be destroyed (terminated) after it is used in N Kraken jobs, default is 1, ie. machine is used once and then it is destroyed so each time a new fresh machine is used for each job; 0 means that this mechanism is disabled

Destroy VM after N minutes of idle time - this indicates that the machine should be destroyed (terminated) after it is not used in any Kraken jobs for a given number of minutes, default is 10 minutes, ie. a machine is destroyed if it sits idle for 10 minutes; 0 means that this mechanism is disabled


Now when a job is assigned to an agents group with configured Agents Deployment then a new machine will be spawned for that job if there are no available agents.