Cloud Deployments

Containerized agentic tasks can be slow when performing rollouts. This is due to container startup and teardown overhead, waiting for LLM API calls, and waiting for command execution. Horizontal scaling becomes the only viable way to accelerate experimentation, so we recommend using a cloud sandbox provider like Daytona.

Using a cloud sandbox provider shifts command execution to the cloud, making trials I/O bounded rather than compute bounded. This means you can typically parallelize far above your CPU count.

Using a cloud sandbox provider

There are many cloud sandbox providers to choose from. We recommend Daytona, because we have found them to be the most flexible. Other good options are Modal and E2B.

harbor run -d "<dataset@version>" \
  -m "<model>" \
  -a "<agent>" \
  -e daytona \
  -n "<n-parallel-trials>"

We run up to 100 trials in parallel on a MacBook Pro with 14 cores.

Multi-container deployments

Daytona supports multi-container deployments. To use multi-container tasks with Daytona, include an environment/docker-compose.yaml file in your task definition.

Other cloud sandbox providers (Modal, E2B) do not currently support multi-container environments. For those providers, you will need to use single-container tasks or switch to Daytona or the local Docker environment.