Migrating from Terminal-Bench
Migrating from Terminal-Bench to Harbor
Harbor is created by the same team as Terminal-Bench. The Harbor task format is an iteration on the format proposed by Terminal-Bench and addresses some of the limitations of the Terminal-Bench format.
The difference in file trees is shown below:
Terminal-Bench Task Format
docker-compose.yaml
task.yaml
solution.sh
run-tests.sh
test_outputs.py
...
...
Harbor Task Format
instruction.md
task.toml
Dockerfile
...
solve.sh
...
test.sh
...
Migration Guide
Harbor provides a mapper from the Terminal-Bench task format to the Harbor task format. Note that this mapper is not perfect, and some particularly custom tasks may require manual migration.
harbor tasks migrate -i "<path/to/terminal-bench/task(s)>" -o "<path/to/harbor/task(s)>"To understand the differences between the two task formats, see the differences page.