Sandboxed Application Review

Let Spec Reviewer launch the application from the Change branch inside an isolated sandbox and validate the running implementation against the specification.

Sandboxed application review allows Spec Reviewer to launch and inspect the application associated with a Change.

Instead of relying only on source code or an existing preview deployment, Baz creates an isolated execution environment containing the repository, its dependencies, the application process, and the tools required for browser validation.

How it works

When Spec Reviewer identifies requirements that need runtime or visual validation, Baz performs the following sequence:

Baz creates a short-lived sandbox for the Change.
The sandbox checks out the source branch associated with the Change.
Baz restores the repository dependency cache when one is available.
The configured setup command installs dependencies and prepares the environment.
The configured start command launches the application.
Baz waits for the application health check to succeed.
Spec Reviewer opens the application in a browser and validates the implementation.
Baz stores the resulting findings and selected review artifacts.
The sandbox is deleted automatically.

The sandbox agent and the application run in the same isolated workload. They use separate internal endpoints so Baz can control the agent independently from the application being reviewed.

Availability

Sandboxed application review is currently supported for GitHub-connected repositories with a configured sandbox.

The application must be able to run in one of the supported Baz runtime images using repository-defined setup and start commands.

Private application review inside a customer Kubernetes environment is available as a limited technical preview.

Repository configuration

Each repository can define the environment Baz needs to run the application.

Setup command

The setup command prepares the repository for execution.

Common examples include:

npm ci

pnpm install --frozen-lockfile

pip install -r requirements.txt

The setup command should be non-interactive and safe to run in a clean environment.

Start command

The start command launches the application and keeps the process running.

Examples include:

npm run dev

pnpm start

python -m app

The application should listen on the configured application port and become reachable from inside the sandbox.

Environment variables

Environment variables can be configured per repository.

Use them for non-code configuration required to install dependencies, connect to development services, select application modes, or complete authentication setup.

Sensitive values are stored separately from the setup and start commands and are injected only when the sandbox starts.

Runtime

Baz runtime images include commonly used language and browser tooling.

The selected runtime should match the repository architecture and the tools required by the setup command. Baz-maintained runtime images are recommended for the initial setup.

Customer-provided images, private base images, and privileged Docker workloads are not currently part of the standard sandboxed review flow.

Dependency cache

Baz can restore and save dependency caches using short-lived signed URLs.

Caching reduces startup time for repositories with large dependency trees. A cache failure does not block the review. Baz continues with a clean installation.

Browser validation

After the application becomes healthy, Spec Reviewer can open it through a browser session.

The browser may be used to validate:

Visual implementation against design references.
Navigation and route behavior.
Form and interaction behavior.
Authenticated application states.
Session or local storage configuration.
Requirements that cannot be verified reliably from code alone.

Headless browser sessions support screenshots and recording artifacts when enabled.

Authentication

Sandboxed application review supports the authentication methods configured for Spec Reviewer.

These can include username and password authentication, session token authentication, and session storage overrides.

For applications that require custom login flows, certificates, host mappings, or internal gateways, additional repository or private-runner configuration may be required.

Application health

Baz begins browser validation only after the application responds successfully to its configured health target.

Startup time should account for dependency installation, compilation, asset generation, and any application initialization required before the page becomes available.

Repositories with long startup times should use dependency caching and an appropriate startup timeout.

Network access

Baz-managed sandboxes run in an isolated Kubernetes environment.

Outbound access is controlled through the sandbox network policy and allowlist. Applications that require access to private customer services should use private application review rather than exposing those services publicly.

Private application review

Private application review runs the sandbox inside the customer Kubernetes environment.

A Baz controller deployed in the customer cluster receives review jobs over an outbound connection, provisions short-lived workloads, and returns findings and approved artifacts to Baz.

The application and internal network traffic remain inside the customer environment.

The private runner is designed to support:

Private services and staging environments.
Internal DNS and networking.
Customer-managed certificates.
Customer-owned secrets.
Private package and container registries.
Customer-controlled compute and storage.

Private application review is currently available as a limited technical preview.

Operational behavior

Each sandbox receives a unique execution identifier and a configured lifetime.

Baz automatically removes sandbox resources after completion, failure, cancellation, or expiration.

Execution logs distinguish between repository checkout, dependency setup, application startup, browser connection, review execution, artifact upload, and cleanup. This makes it possible to identify whether a failure belongs to the application configuration or the review itself.

Troubleshooting

The application does not start

Confirm that the setup and start commands work from a clean checkout without interactive input.

Verify that all required environment variables are configured.

Check that the application listens on the configured port and on an address reachable outside the application process.

Increase the startup timeout for repositories that require long installation or compilation steps.

Dependencies cannot be installed

Confirm that private registry credentials are available as sandbox environment variables.

Check whether the runtime architecture is compatible with any native dependencies.

Remove local-machine assumptions from the setup command, including absolute paths or preinstalled tools that are not part of the selected runtime.

The browser cannot reach the application

Confirm that the application health check succeeds inside the sandbox.

Check for fixed hostnames, self-signed certificates, host-header restrictions, or local hosts-file dependencies.

Applications with private DNS or internal service dependencies may require private application review.

Authentication fails

Confirm that the selected Spec Reviewer authentication method matches the application login flow.

For session storage overrides, verify that the configured command produces the expected browser state.

For private applications, confirm that the runner can reach the required identity and authentication services.

Limitations

The current managed flow supports GitHub-connected repositories.
Docker-in-Docker and other privileged workloads are not supported.
Applications requiring fixed public hostnames, custom local DNS, or private network access may require additional configuration or the private runner.
Arbitrary customer container images are not yet supported in the standard configuration.
Private application review remains in technical preview.

FAQ

What does the sandbox add over a preview environment?

Baz launches the application from the change branch itself, so validation does not wait on a preview deployment. See Preview Environments for the alternative.

Which repositories can use it?

GitHub-connected repositories with a configured sandbox, where the application can run in one of the supported Baz runtime images using repository-defined setup and start commands.

Can we run it inside our own environment?

Private application review inside a customer Kubernetes environment is available as a limited technical preview. Applications needing fixed public hostnames, custom local DNS, or private network access typically need that private runner.

What is not supported?

Docker-in-Docker and other privileged workloads, and arbitrary customer container images in the standard configuration.