How I Used AI Agents to Migrate Reka UI's Tests to Vitest Browser Mode

From the source

I wanted to learn Vitest Browser Mode properly. Not with a counter component. Not with five tests written specifically for a demo. I wanted a real component library with years of testing history and enough sharp edges to punish bad assumptions. So I created a working fork of Reka UI and used AI coding agents to migrate its test strategy away from jsdom. The original suite had 97 test files. Eighty-seven touched the DOM and received Browser Mode counterparts. The remaining ten contained 571 DOM-free tests, so they moved to a plain Node project instead. I kept every original jsdom file as a comparison corpus. This gave the agents a live baseline rather than a memory of what the old tests used to do. The AI didn’t only translate tests. It reviewed ports, introduced deliberate bugs, ran both environments, recorded findings, and improved its own prompts after each batch. That process is the interesting part. This work lives in my [Reka UI Browser Mode fork](https://github.com/alexanderop/reka-ui-bench-mark/tree/browserMode). It isn't an upstream Reka UI change. The fork exists to study the migration and preserve the comparison. <TLDR items={[ “Build the migration oracle before asking agents to port files”, “Give one file to one implementer and a fresh reviewer”, “Keep the old suite alive so every port has a comparison target”, “Use targeted mutations to prove that tests can catch real breakage”, “Store prompts in Git and fix the prompt when a batch goes wrong”, “Run only a few browser agents concurrently because each one launches Chromium”, ]} /> The Wrong Way to Use AI for a Migration The tempting prompt is: Port all tests from jsdom to Vitest Browser Mode. Make the suite green. That prompt optimizes for green tests. An agent can reach green by deleting an awkward assertion. It can replace an exact query with a broader one. It can add force: true to every click or quarantine failures without understanding them. All of those ports compile. All of them can pass. The problem gets worse with parallel agents. One weak port is reviewable. Eighty-seven weak ports become a new baseline before anyone notices. So the first job wasn’t porting tests. It was building a machine that could reject bad ports. The Core Idea: Agents Propose, Machines Decide The migration became a feedback-controlled loop: flowchart LR A[Inventory] --> B[Implement one file] B --> C[Run machine oracles] C --> D[Independent review] D --> E[Apply targeted mutations] E --> F[Record findings] F --> G[Improve shared prompt] G --> B The AI handled judgment-heavy work inside each step. Deterministic scripts controlled the boundaries. That distinction mattered. The agents could propose translations and investigate differences. They couldn’t redefine what “complete” meant. Step 1: Keep Both Environments Running The first commit created three Vitest projects over the same source tree: Project Purpose unit Original jsdom tests, kept unchanged browser New *.browser.test.ts files in Chromium node Tests that never needed a DOM Running the environments side by side changed the migration from a rewrite into an experiment. For every file, I could ask: Did the Browser Mode port keep the same test structure? Did it preserve the assertions? Did it reach at least the same production code? Which suite noticed when we broke the component? Without the retained jsdom suite, those questions would become opinions. Step 2: Build an Inventory Before Assigning Work I scanned all 97 files for DOM signals such as: Vue Test Utils and Testing Library imports document, window, and DOM constructors accessibility audits browser API mocks story fixture mounts The script then classified every file by migration risk: Tier Meaning T0 No DOM. Move it to Node T1 DOM-dependent but expected to be boring T2 Mostly mechanical component tests T3 Mocks browser APIs or depends on geometry T4 Special patterns such as fake timers, module mocks, snapshots, or virtualization This inventory became the progress bar and the work queue. It also prevented a wasteful mistake. Ten files didn’t need Chromium or jsdom. Moving them to Node removed the fake browser environment from 28% of the original tests. Step 3: Calibrate the System on Three Files Before the fan-out, I chose three deliberately different files. Slider: the difficult port Slider mocked ResizeObserver, scrolling, and pointer capture. It forced the first agent to solve real geometry and input problems. useForwardExpose: the boring port This composable installed no compensating mocks. Its Browser Mode version was almost identical and bought little new information. That negative result was useful. The process needed to report “this port gained nothing” without inventing a victory. Label: the mechanical port Label was small and looked easy. It still exposed several traps around click semantics, zero-width elements, and missing positive assertions. These three files produced the first version of the implementer and reviewer prompts. They gave later agents concrete precedents for difficult, boring, and mechanical work. Step 4: Give Each Implementer One File Each agent received one original test file and one output path. The assignment looked roughly like this: Port packages/core/src/{FILE} to packages/core/src/{COMPONENT}.browser.test.ts. Keep every describe and it name verbatim. Never reduce the assertion count. Don't change production source or fixtures. Keep the original jsdom file. Run only focused tests for your file. Record at least one evidence-backed finding. The narrow ownership reduced conflicts. It also made failures attributable. If a batch failed, I knew which file, prompt, and agent decision produced it. Agents weren’t allowed to fix product bugs during the port. A faithful Browser Mode test failing was considered a successful finding. The agent had to quarantine that test with it.fails and link it to a findings key: // @finding Slider/Slider.test.ts#axe it.fails("should pass axe accessibility tests", async () => { // Keep the original assertion strong. }) This kept the suite runnable without hiding the discovery. When the bug gets fixed, it.fails becomes red because the expected failure disappears. Step 5: Review with a Fresh Context The implementer never reviewed its own port. A second agent received the original and the Browser Mode version. It didn’t initially receive the implementer’s reasoning or findings. Its task was adversarial: Assume this port is weaker than the original. Find out how. The reviewer searched for: exact assertions replaced by broad ones lazy locators that were never resolved retrying assertions that widened timing contracts new sleeps hiding synchronization problems role queries replacing tag assertions forced interactions that bypassed the behavior under test quarantines swallowing unrelated assertions This separation worked because implementation and review reward different behavior. The implementer wants completion. The reviewer wants a counterexample. The machine checked structure. The reviewer checked meaning. Step 6: Make Parity Machine-Enforced Every port had to pass five focused commands: pnpm --filter reka-ui exec vitest run \ --project=browser <browser-file> pnpm --filter reka-ui port:checklist \ <component> --complete pnpm --filter reka-ui port:parity \ <component> --complete pnpm --filter reka-ui port:coverage \ <component> pnpm --filter reka-ui exec vitest run \ --project=unit <original-file> Each command answered a different question. port:checklist Did the port keep every describe and it node in source order? port:parity Did it preserve test names, assertion counts, and legitimate quarantines? port:coverage Did it still reach the same production lines? A lost line couldn’t be waived for an entire file. The agent had to explain that exact line and connect the exception to a recorded finding. Coverage gains received the same scrutiny. Automatic Browser Mode cleanup reached teardown code that many jsdom originals never exercised. That was a harness improvement, not proof that Chromium earned the line. Step 7: Treat Findings as the Main Output Every agent appended evidence to FINDINGS.tsv. The ledger recorded: the original file the verdict deleted and retained mocks coverage differences measurements, mutations, or source evidence unresolved questions marked as unverified “Ported cleanly” wasn’t enough. If the browser added nothing, the agent had to say what it checked before reaching that conclusion. If it found a bug, the finding had to explain how it reproduced. If it kept a mock, it had to explain why that mock constructed the scenario rather than compensated for jsdom. This changed the incentive. The ported test was no longer the only deliverable. The knowledge produced during the port mattered just as much. Step 8: Let AI Perform Targeted Mutation Testing Coverage parity proves that a port reaches the same code. It doesn’t prove that either test would detect broken behavior. Normally, I’d use Stryker for mutation testing. However, StrykerJS’s official Vitest runner currently doesn’t support Browser Mode. So the agents performed targeted mutations themselves: Read the behavior named by the test. Introduce one deliberate production defect. Run the Browser Mode test. Run the matching jsdom test. Record whether each suite killed the mutation. Restore the source immediately. Verify a clean diff and green baseline. For Slider, the agent removed the call to setPointerCapture(). The Browser Mode test failed. The jsdom test stayed green because its own mock claimed pointer capture had succeeded. This wasn’t exhaustive mutation testing. It didn’t produce a mutation score. It was hypothesis-driven testing aimed at the exact contract each file claimed to protect. I previously wrote about this approach in Mutation Testing with AI Agents When Stryker Doesn’t Work. Step 9: Put the Prompts Under Version Control The implementer and reviewer prompts lived in PORT-PROMPTS.md. That file had a changelog. When a batch produced a weak port or a false belief, I updated the prompt before running the next batch. One early rule claimed that screen.getBy* couldn’t see portalled content. A later Teleport port disproved it. The agent measured that screen and page returned the same portalled node. We corrected the shared rule before eight overlay components inherited it. This became the most transferable idea from the migration: Fix the prompt, not only the generated code. Hand-fixing one port removes one symptom. Updating the prompt removes a class of future mistakes. Step 10: Use Bounded Parallelism One file per agent doesn’t mean 87 agents at once. Every Browser Mode worker starts Chromium. During the first mechanical batch, concurrent coverage runs competed for the browser. One in four identical runs failed before writing a report. That failure was dangerous. An agent could interpret missing coverage as a broken port and start “fixing” correct code. I limited the fan-out to roughly three concurrent agents. Each agent ran only its focused file. The coordinating session ran global validation between batches. flowchart TD A[Select next batch] --> B1[Agent: file 1] A --> B2[Agent: file 2] A --> B3[Agent: file 3] B1 --> C[Independent reviews] B2 --> C B3 --> C C --> D[Global parity and inventory] D --> E[Update prompts and findings] E --> A More agents would’ve increased contention without increasing throughput. The Exact Batch Loop After calibration, each batch followed the same process: Select files from the inventory. Assign one file to each implementer. Run focused browser, parity, checklist, and coverage checks. Assign independent reviewers with fresh context. Run targeted mutations for important claims. Merge findings into the shared ledger. Update the prompts and migration guide. Regenerate the inventory. Run global parity and both retained suites. Start the next batch only from a green baseline. The Git history shows that progression: Commit What changed bd93d9b1 Browser harness and migration oracles f050ec36 First complete, mutation-verified Slider port cb8fd28c Ten DOM-free files moved to Node 675792e3 Implementer and reviewer prompts formalized 1a02ac8c Pattern-frontier files completed 12d75a3b All 97 files received a non-jsdom destination The setup began on August 16. The completion commit landed on August 18. How to Reuse This in Your Project You don’t need Reka UI or Vitest to reuse the process. You need six things: A live baseline. Keep the original system runnable during the migration. An inventory. Give every item an owner, risk tier, and completion state. Machine oracles. Compare structure, behavior, and coverage automatically. Narrow agent ownership. One file or one coherent unit per implementer. Independent review. Don’t let the generating context approve itself. A feedback loop. Store prompts in Git and improve them after every failure class. The tool is secondary. I used Claude Code sessions, but the pattern works with any coding agent that can edit files and run commands. Pick one difficult file, one boring file, and one mechanical file. Use them to build your oracle and prompt before adding parallel agents. For larger deterministic agent workflows, see Claude Code Workflows: Deterministic Multi-Agent Orchestration. What Comes Next This post explains how the AI migration worked. It doesn’t answer whether Vitest Browser Mode was worth the effort. That deserves a separate post. In Part 2, I’ll cover what the paired suites actually proved: which mocks disappeared, which bugs only Chromium caught, where jsdom was already good enough, and what Browser Mode cost. The core lesson from Part 1 is simpler: Don’t use AI as a mass code translator. Build a system where agents propose changes, independent agents challenge them, mutations test their claims, and machines enforce the boundary. That’s how I turned 97 test files into a controlled migration instead of 97 opportunities for confidently green mistakes.