Artifacts and evaluation

Daydream records each run as files that support review, recovery, and corpus work.

Agent Trajectory Interchange Format (ATIF) is the trajectory data format. Continuous integration (CI) is an unattended automation environment. Hugging Face (HF) is the optional remote dataset service. A pull request (PR) is a proposed change in a GitHub repository.

Live run files

The default main trajectory path is:

TARGET/.daydream/runs/SESSION_ID/trajectory.json

The trajectory uses ATIF version 1.7. The file contains messages, reasoning content, tool calls, tool results, tokens, cost data, and phase data. Parallel agent work creates child trajectories below trajectories/ in the same run directory.

Use --trajectory PATH to select another main trajectory path. Daydream still copies that file into the archive as trajectory.json.

Daydream handles the interrupt signal (SIGINT) and termination signal (SIGTERM) before shutdown. The handler appends .partial to the selected trajectory path when the trajectory contains at least one step. The partial trajectory sets extra.partial to true. Daydream also archives the partial run when archive assembly is active. Daydream does not upload a partial run to HF.

Working artifacts

Daydream writes these files in or below TARGET/.daydream/:

Path Content
diff.patch The tracked input diff that Daydream reviewed.
deep/ Intent, stack reviews, parsed findings, merge data, supervision data, coverage data, and recovery data.
exploration/ Cached exploration context and its exact-match cache key.
recommended.patch The worktree changes proposed by the fix phase.
runs/SESSION_ID/ The main trajectory, child trajectories, and a possible handoff.

Daydream captures recommended.patch immediately after the fix agents finish. The capture occurs before fix-failure cleanup and before tests. The patch can contain a change that Daydream later reverts. Check fix-failures.json, the manifest status, and the final Git diff before applying the patch.

The root .review-output.md file contains the rendered review. --cleanup can remove the root file after a successful run. The copy below .daydream/deep/ remains available.

Central archive

Daydream archives recorded agent runs by default. The default archive root is:

~/.daydream/archive

Set DAYDREAM_ARCHIVE_DIR to use another root:

export DAYDREAM_ARCHIVE_DIR="$PWD/daydream-archive"
daydream .

Each completed archive uses this layout:

ARCHIVE_ROOT/
├── index.db
└── runs/
└── SESSION_ID/
├── trajectory.json
├── trajectories/
├── diff.patch
├── recommended.patch
├── review-output.md
├── findings.json
├── deep/
├── evaluation.json
└── manifest.json

Some files are conditional. For example, findings.json requires --findings-out. The archive skips a missing optional file.

manifest.json records run settings, Git context, PR context, status, cost, timing, and evaluation summaries. The manifest also records fix failures and leftover untracked files when those values exist.

index.db is a SQLite index across archived runs. The index stores run metadata and label observations. Corpus commands use this index.

Archive assembly failures produce warnings. An archive failure does not change the primary run result. An evaluation failure can leave the archive without evaluation.json.

Archive and export controls

Disable central archive output

Use --no-archive to disable normal archive creation:

daydream --no-archive .

The live trajectory and working artifacts still use the target worktree. Daydream removes a temporary worktree after the run when cleanup succeeds. A temporary-worktree run can lose live artifacts when no archive or dump exists.

Disable deterministic evaluation

Use --no-eval to skip evaluation during archive assembly:

daydream --no-eval .

The manifest keeps token, cost, and elapsed-time data that the recorder supplies. Evaluation fields remain empty when evaluation does not run.

Copy a complete bundle

Use --dump-artifacts DIR to copy the assembled bundle into one directory:

daydream --dump-artifacts "$PWD/daydream-artifacts" .

Use a separate destination for each concurrent run. Daydream merges files into an existing destination directory.

--dump-artifacts activates archive assembly even with --no-archive. Daydream first assembles and indexes the run below the central archive root. Daydream then copies the assembled bundle to the dump directory. Do not combine these options when central archive storage is prohibited.

Upload an archive bundle

Use --trajectory-hub-repo OWNER/DATASET to upload each complete archive bundle. Daydream uploads the bundle into a folder named with the session identifier.

The setting uses this precedence:

  1. --trajectory-hub-repo
  2. DAYDREAM_TRAJECTORY_HUB_REPO

Daydream ignores a trajectory_hub_repo key in the target checkout's file configuration. This restriction keeps the upload destination under operator control.

The upload requires the optional huggingface_hub package and HF_TOKEN.

export HF_TOKEN="HF_TOKEN_VALUE"
daydream --trajectory-hub-repo OWNER/DATASET .

Daydream creates a missing dataset repository with private visibility. Daydream keeps the visibility of an existing dataset repository. Daydream warns before an upload to an existing public repository.

A missing token, missing package, or upload error produces a warning. An HF upload error does not fail the Daydream run. Daydream retries a concurrent-commit conflict up to three total attempts. --no-archive also disables the HF upload.

Sensitive data

Artifact bundles can contain proprietary source code and PR data. Bundles can also contain prompts, reasoning, diffs, file content, and tool output.

Daydream redacts common secret patterns before it writes trajectory text. Redaction cannot identify each sensitive value. Inspect a bundle before you copy or upload the bundle.

Restrict access to the archive root and dump directory. Confirm the visibility of an existing HF dataset before upload. Do not publish an artifact bundle as a public CI artifact without inspection.

Findings artifacts

--findings-out FILE writes a strict version 1 findings artifact. The artifact identifies the repository, PR, head commit, findings, and fingerprints. The review flow stops before posting, fixing, or testing.

When archiving is active, Daydream copies the artifact into the bundle as findings.json. Use daydream post-findings for the separate posting phase.

See Automation and integration for the posting contract.

Deterministic evaluation

Evaluation runs by default during archive assembly. The evaluator reads trajectory and run artifacts from disk. The evaluator does not start a model or ask for a human verdict.

evaluation.json contains these groups:

  • Cost and token totals
  • Elapsed time and per-agent timing
  • Tool-call counts
  • Changed-file review coverage
  • Finding counts and confidence data
  • Evidence grounding data
  • Exploration-use data
  • Training-signal quality checks
  • Python complexity-concentration and verbosity measurements
  • Cost per finding

The analysis can return unavailable values when required data is absent. The grounding rate is the share of findings whose cited files the review agent read. The calculation also checks that a rationale does not cite an unread file. An empty finding set has no grounding rate.

For Python files, erosion is the share of weighted complexity in functions with cyclomatic complexity above 10. Cyclomatic complexity measures independent paths through a function. The weight uses cyclomatic complexity and the number of source lines in the function. The verbosity value is the share of nonblank lines that match selected unnecessary-code patterns.

Daydream has no daydream eval command. Use the archived evaluation.json file or the corpus commands for analysis.

Summarize a trajectory

Use daydream summarize PATH for a trajectory file or an archive run directory:

daydream summarize "$DAYDREAM_ARCHIVE_DIR/runs/SESSION_ID"

The command prints Markdown to standard output. The output contains a run summary and a per-phase table. The command reads files only. The command does not post to GitHub or calculate a diff.

The command returns status 1 for an invalid path or an empty run directory. The command returns status 2 when all selected trajectory files fail to parse.

Cost calculation

Daydream uses backend-reported cost when the backend supplies that value. Otherwise, Daydream calculates cost from token counts and a price table.

Daydream selects a price in this order:

  1. A backend-reported cost
  2. A model entry in the user price file
  3. A model entry in the built-in price table
  4. An unavailable cost marker

The default user price file is ~/.daydream/prices.toml. DAYDREAM_PRICES_FILE selects another file. An unknown model does not receive an estimated price.

Prices use United States dollars (USD) per one million tokens. Each user entry requires input and output values. The optional cached_input value defaults to input.

See Configuration for the price-file format.

Back to Daydream