AMD Developer Hackathon ACT II / Track 1

Accuracy first. Remote tokens only when needed.

ProofRoute combines proof-first deterministic computation, embedded FunctionGemma assessment, selective Gemma E2B inference and evaluator-authorized Fireworks fallback.

Final scorecard Pareto calibration Fuzz report Neural audit Fireworks arena Release decision README SUBMISSION

Why this wins

The hard part is not calling the strongest model. The hard part is knowing when not to call it.

4 guarded stages

Live score

The final submitted image answered 18 of 19 evaluator tasks correctly.

94.7% / 3,051 tokens

Previous evaluator run: 94.7% / 4,723 tokens. The final run preserved accuracy while using 35.4% fewer scored Fireworks tokens.

Submission shape

Public, immutable, no-download image with audited OCI labels and an exact-image evaluator gate.

2.94 GB

Pull the submitted image

docker pull --platform linux/amd64 \
  ghcr.io/rvbernucci/track1-token-router:v3.12.3-proof-pull-retry

This public tag exposes a linux/amd64 manifest and passed clean pull, exact-image execution and immutable-label audits.

Expected route

solver_arithmetic, because code can answer this safely.

Expected answer

42, with final validation preserving the strict numeric format.

Remote spend

0 Fireworks tokens. The remote model is held for tasks where risk justifies it.

Run the submitted container

Create the official input file, then pass the three evaluator variables from your shell.

mkdir -p input output
printf '%s\n' '[{"task_id":"t1","prompt":"What is 6 * 7? Return only the number."}]' > input/tasks.json

docker run --rm --platform linux/amd64 \
  --memory=4g --cpus=2 \
  -e FIREWORKS_API_KEY \
  -e FIREWORKS_BASE_URL \
  -e ALLOWED_MODELS \
  -v "$PWD/input:/input:ro" \
  -v "$PWD/output:/output" \
  ghcr.io/rvbernucci/track1-token-router:v3.12.3-proof-pull-retry

cat output/results.json

Calibration status

The runtime remains fail-closed. The established E2B policy achieved 80/87 correct releases on protected holdout. Sprint 80 added a post-response Extra Trees gate for code debugging that contributed 13/13 additional correct releases, raising the union to 93/100 (93.00%) without adding an observed error. The new gate is hash-pinned and can only add candidates inside its validated intent and contract boundary.

Fireworks accuracy safeguards

An 800-task paired arena retained Kimi as the token-efficient default and selected MiniMax only for logic puzzles, its statistically supported advantage. Completion budgets remain contract-aware. If a response ends because of the length limit, or reaches 95% of its budget with clear incomplete-output evidence, ProofRoute retries once with a larger ceiling and accounts for tokens from both attempts.

Answer Contract Engine

Models receive only the raw prompt, never the evaluator's JSON wrapper. ProofRoute validates the returned content and reconstructs results.json deterministically. A controlled ablation reduced Fireworks input tokens by 51.9% while preserving byte-identical Kimi answers.

Run source checks

These commands validate the repository; they are separate from running the submitted image.

python3 -m pip install -e .
scripts/offline_release_check.sh
python3 scripts/export_public_report.py --check

Verified: release CI completed 774 tests with three environment-dependent skips.

Immutable release evidence

Release v3.12.3-proof-pull-retry was built from revision 76df565. Its OCI digest is sha256:ec0d62c4c08489e8b8f06abf26087d1c1bfa43128d330b591f8588976b333c59. The exact public image completed the official I/O smoke under 4 GB RAM, 2 vCPU, linux/amd64 and disabled networking in five seconds. Fireworks calls share an absolute 28-second task deadline, and factual questions are never answered from a lookup table.

Final official result

The final submitted image scored 94.7% accuracy (18/19) with 3,051 Fireworks tokens. Documentation was corrected after scoring with organizer approval; the submitted source revision, Docker tag and OCI digests remain unchanged.