Competition Platforms
This is a companion page for chapter 10 of the AI Competitions and Benchmarks book.
Platform Comparison
| AIcrowd | Codabench | DrivenData | EvalAI | Kaggle | Tianchi | Zindi | |
|---|---|---|---|---|---|---|---|
| Design Support | ✓ | - | ✓ | - | ✓ | - | ✓ |
| Data Support | ✓ | - | ✓ | ✓ | ✓ | - | ✓ |
| Code sharing | ✓ | - | - | ✓ | ✓ | ✓ | - |
| Code evaluation | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | - |
| Custom metrics | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Staged contests | ✓ | ✓ | ✓ | ✓ | - | - | - |
| Private evaluation | - | ✓ | - | ✓ | - | - | - |
| RL-friendly | ✓ | ✓ | - | ✓ | ✓ | - | - |
| Judging panel | ✓ | - | ✓ | - | ✓ | ✓ | ✓ |
| HITL evaluation | ✓ | - | - | ✓ | - | - | - |
| Run for free | - | ✓ | - | - | ✓ | - | ✓ |
| Open-source | - | ✓ | - | ✓ | - | - | - |
| Year launched | 2017 | 20131 | 2014 | 2017 | 2010 | 2014 | 2018 |
| Users2 | 400k | 50k | 150k | 40k | 29m | 1.5m | 100k |
For an annual overview of these and other platforms, see the State of ML Competitions report.
For definitions of each of the feature criteria, refer to the book chapter. Indication of features is based on our knowledge at the time of writing. For corrections, please email feedback@mlcontests.com.