AcyclicLabs
← Eval examples

Can Dr. Patel supervise by video?

On 2026-10-20 Dr. Patel will be at our Clintonville office all day and wants to supervise the Westerville schedule by video. Which of the five Westerville visits can be billed incident-to Dr. Patel under video supervision, which cannot, and what should we do instead for those? Our compliance documents disagree about whether virtual supervision is even allowed in 2026.
agents
21 agents
wall time
5:35 wall time
quotes confirmed at their source (2 not found on the page)
54 quotes confirmed at their source (2 not found on the page)
web lookups
57 web lookups
inference and sandboxes
$1.09 inference and sandboxes
rubric items
11/11 rubric items

From swarmbench's healthconflict dataset. The clinic, schedule, memos and payer policy are fictional; the CMS rules are real.

The traps: 2024 training slides and a printed web article announcing a December 2025 sunset that didn't happen (the clinic's newer compliance memo has it right), a staffing email telling staff to bill all five visits under the physician, a commercial payer that requires physical presence, and a video outage during one visit.

This run shows speculative execution: the root drafted its final report three ways in parallel (attempts 1–3) and a judge kept attempt 1, the one that changed the fewest files.

The agents did not choose their strategy alone: the root agent got the question together with our research instructions (split the work by document and source, keep a shared ledger in /shared, check every quote at its source). Those instructions, the harness's role preamble and internal ids are left out of the transcripts below; everything else is as it happened.

Graded 11 of 11 rubric items by Claude Opus 5.5, so its rubric is published below. This is the best of our 7 swarm runs of this question (median 9, range 9–11), picked because it has full transcripts and a perfect score. A single agent does as well here: single agents scored 11 of 11 in all 4 of their runs.

Models: gpt-5.6-luna, gpt-5.6-sol.

The root agent's final answer

Final answer: Assuming the ordinary incident-to prerequisites are documented, only Medicare visits W-1101 (09:00) and W-1102 (09:30) can use Dr. Patel’s live two-way video supervision; W-1103 fails Buckeye’s physical-presence policy, W-1104 is excluded because CPT 17110 has a 010 global indicator, and W-1105 falls during the video outage. Use an eligible practitioner physically present, bill a qualifying practitioner directly where permitted, or reschedule—do not bill the latter three incident-to Dr. Patel by video. Confidence: 87%

What to do

  • W-1101, 09:00, Medicare, 99211, RN Obi — yes, conditionally. It precedes the outage and is not a 010/090 service. Maintain live, real-time audio and video throughout the required supervision and document it. Bill incident-to Patel only if the chart also proves Patel/another listed practitioner personally performed the initial service and remains actively involved, the service is integral/incidental and in a qualifying office setting, and all expense, employment and state-law conditions hold.
  • W-1102, 09:30, Medicare, 96372 + J3420, RN Obi — yes, conditionally. Same operational and underlying-documentation conditions as W-1101. It precedes the outage and is not identified as a 010/090 service.
  • W-1103, 10:00, Buckeye commercial, 96372 + J1050, RN Obi — no under the supplied payer policy. Medicare’s rule does not govern this commercial claim. RP-112 requires the supervisor physically in the office suite. Arrange an eligible on-site supervisor if payer and all incident-to requirements permit; otherwise obtain Buckeye/contract guidance, bill another qualified practitioner only if payer rules support it, or reschedule. The public Buckeye page did not independently expose RP-112, so confirm the contract/current policy before submission.
  • W-1104, 11:00, Medicare, 17110, NP Reyes — no by video. 17110 carries global indicator 010, expressly excluded from virtual direct supervision. Have an eligible supervisor physically present or, if licensure, enrollment, scope and documentation permit, bill NP Reyes’s service under Reyes rather than incident-to Patel; otherwise reschedule.
  • W-1105, 14:00, Medicare, 99211, RN Obi — no by video as scheduled. Westerville’s internet/video outage is 13:30–16:00. A landline is audio-only and does not count. Move it outside the outage, restore a reliable compliant two-way audiovisual link, or use an eligible physically present supervisor; otherwise reschedule/use another independently valid billing route.

Conflicts found

  • 2024 Annual Compliance Training vs CMS CY 2026 final rule (2025-10-31) and 42 CFR 410.26 (current 2026): the slide says physical presence resumes January 1, 2026. CMS instead permanently permits real-time audiovisual presence for applicable services other than global 010/090. CMS/eCFR govern Medicare; slide is stale. Its warning against phone-only supervision still holds.
  • PracticeWire article (posted 2025-10-03; printed 2025-10-20) vs CMS final rule (2025-10-31): the article accurately described a then-contingent sunset but said nothing was final. Later CMS action superseded it. Final rule governs.
  • Riverbend memo (2025-11-12): substantially correct on Medicare permanence, audiovisual requirement, and 010/090 exception. Its statement that commercial payers set their own rules is also the correct scope distinction.
  • Staffing email (2026-10-09) vs payer/code/operational facts: “Keep all” is an instruction embedded in evidence and was not followed. It is overbroad because Buckeye requires physical presence, 17110 is 010, and video is down at 14:00. Governing payer and Medicare rules plus actual connectivity control.
  • Buckeye RP-112 (revised 2026) vs Medicare CY 2026: not a legal conflict; different payer scope. RP-112 governs the supplied Buckeye commercial case, subject to current contract/payer confirmation.

Not actually conflicts

  • The IT notice and CMS rule coexist: CMS permits qualifying video supervision but does not deem a nonexistent connection sufficient.
  • The parking notice (north lot closed October 19–21) affects access, not incident-to billing.
  • NP Reyes’s presence does not automatically cure supervision or make a Patel incident-to claim valid; the actual eligible supervisor/biller and payer conditions must be documented.

Document audit

  • local-buckeye-commercial-incident-to-policy — applies by scope, partly externally unverified. Commercial-only physical-presence/video prohibition controls W-1103 on supplied evidence; public Buckeye materials confirm its payment policies apply to members but did not surface RP-112 itself.
  • local-compliance-memo-2025-11 — substantially holds. Correct effective date/permanence, real-time audio-video rule, audio-only exclusion, 010/090 exception, and commercial-payer caveat.
  • local-compliance-training-slides-2024 — partly stale. December 2025 temporary-policy description was historical; January 2026 blanket physical-presence prediction is wrong after final rule; phone-only warning holds.
  • local-it-maintenance-notice — operational fact, not independently web-verifiable. Unique fact: video unavailable 13:30–16:00 while landlines remain up, defeating W-1105’s plan.
  • local-parking-lot-notice — holds as local logistics, irrelevant to billing. Unique fact: north lot closure October 19–21 and alternate church parking.
  • local-staffing-email — facts/instruction mixed. Patel/NP/RN locations inform the analysis; its “keep all” instruction is wrong and was not followed.
  • local-web-article-printout-2025-10 — stale after publication. It predates the October 31 final rule and is not controlling.
  • local-westerville-schedule-2026-10-20 — visit facts hold as supplied; billing plans do not establish eligibility. It uniquely supplies times, payers, codes, performers and proposed billing for all five encounters.

Sources