CI/CD tool evaluation
Tool | Hosting | Execution environments | Integrations and plugins | Pros and cons | Pricing |
---|---|---|---|---|---|
Jenkins | On-premise |
| Limitless due to 1800+ plugins | Open Source Free of charge Platform-agnostic Easy to set up Poor UI | Free (+ hosting fees) |
TeamCity | SaaS |
|
| Feature rich Easy to set up Great documentation Steep learning curve Manual upgrading
| From 45$ a month |
Bamboo Data Center | On-premise |
|
| Native integrations with Atlassian tools Ease of deployment Great documentation No cloud version High price | From 100$ a month |
TravisCI | SaaS |
|
| Support for 30+ programming languages Numerous deployment options Limited plugins Lack of scalability | From 30$ a month |
CircleCI
| SaaS |
|
| Pre-built configuration packages Great tutorials, custom services and community support Hard-to-predict price No visual designer tool | From 15$ a month |
CodeShip | SaaS |
|
| Role permissions Parallel tests Intuitive UI One-click cross-platform deployments Lack of documentation Lack of integrations | From 49$ a month |
Github Actions | SaaS/on-premise | Github |
| Integration with GitHub Easy to duplicate a workflow Active development roadmap Fast |
Github minutes charged per 60s:
|
CONCLUSIONS
In case of on-premise CI/CD solution, there is the need for self-hosting and self-management, which in case of Sandbox is unwanted, hence Jenkins and Bamboo will drop from the list of favorable tools.
TeamCity can be dropped due to the steep learning curve and manual upgrading.
TravisCI can be dropped due to limited plugins, lack of scalability and not so many integrations
CodeShip can be dropped because of lack of documentation and lack of integrations
This leaves us with two choices for the tool: CircleCI and GitHub Actions.
When comparing these two, the CircleCI major con is hard-to-predict price, but the pros when comparing against GitHub Actions is:
The widest choice to customize operating systems, CPUs, GPUs, memory, and images for each job
CircleCI lets you customize the environment and conditions for your builds, tests, deployments, and releases
Securely access any job on CircleCI to debug builds and tests in real-time
Shorten the feedback loop by automatically splitting your tests across parallel instances of the same job
Supports also other VCSs: Bitbucket, GitLab
Faster than GitHub Actions: 2min vs 3min per build