Kraken CI Architecture

class: center, middle

# Kraken CI
## Architecture

---

# Agenda

1. Architecture
2. Workflow

---

# Introduction

- Kraken composed of:
 - web UI
 - several backend services
 - distributted agents
- Backend in Python
- UI in Typescript/HTML
- Development based on Docker

---

# Kraken Architecture

![Kraken Architecture](arch-overview.svg)

---
.imgover[![Kraken Architecture](arch-overview-ui.svg)]

# Web UI

- Angular framework
- PrimeNG components

---
.imgover[![Kraken Architecture](arch-overview-srv.svg)]

# API Server

- Serves API requests from:
 - users (via web UI or programmatically)
 - other backend services
- Serves webhooks
- API defined in Swagger

---
.imgover[![Kraken Architecture](arch-overview-ctrl.svg)]

# Controller

- Single instance
- Planner
- Scheduler
- Watchdog
- Storage

---
.imgover[![Kraken Architecture](arch-overview-ctrl.svg)]

# Controller / Planner

- Remembers planned job triggers
- Fires them at indicated time:
 - periodically
 - on given time
 - according CRON rule

---
.imgover[![Kraken Architecture](arch-overview-ctrl.svg)]

# Controller / Scheduler

- Assigns triggered jobs to idle agents
- FIFO

---
.imgover[![Kraken Architecture](arch-overview-ctrl.svg)]

# Controller / Watchdog

- Monitors Runs and their Jobs
 - Run timeout
 - Job timeout
- Monitors Agents
 - if not responding then disabled to avoid assigning

---
.imgover[![Kraken Architecture](arch-overview-ctrl.svg)]

# Controller / Storage

- Serves storage for agents
- Agents can put or get artifacts

---
.imgover[![Kraken Architecture](arch-overview-clry.svg)]

# RQ

- Background tasks
- processes results of a job
- results history analysis
- and more

---
.imgover[![Kraken Architecture](arch-overview-agnt.svg)]

# Agent

- Agent runs on a host
- Jobs are run in executor via tool
- Executors
 - Bare
 - Docker
 - LXD

---
.imgover[![Kraken Architecture](arch-overview-agnt.svg)]

# Agent / Tool

- Run in executor
- Provided operations:
 - shell
 - git operations
 - storing/retrieving artifacts
 - pytest
 - pylint
 - ngling
 - cloc

---
.imgover[![Kraken Architecture](arch-overview-elk.svg)]

# ELK

- Logstash
 - backend services and all agents log to Logstash
 - log entries are assigned to agents/jobs/etc
- Elasticsearch
 - allows querying logs
 - eg. shows all logs associated with given job or stage
- Kibana
 - UI to Elasticsearch

---
.imgover[![Kraken Architecture](arch-overview-psql.svg)]

# PostgreSQL

- main database

---
.imgover[![Kraken Architecture](arch-overview-sml.svg)]

# Kraken Workflow

- Planner or UI trigger a Job
- Job is scheduled for execution to idle Agent
- Agent executes the Job
- Agent can store or fetch artifacts from Storage
- Whole Job execution log goes to ELK
- After completing the Job its results are processed by RQ

![Kraken Workflow](workflow.svg)