Replays each AI medical-device update against its locked clinical cohort, giving reviewers comparable evidence before the next release.

1. Trade coverage describes 12-to-18-month review cycles and six-to-eight-month cycles for minor AI-device updates, making release-specific evidence valuable before the next update review.
2. Scarlet says customers have shortened product-update cycles from months to weeks, a company-reported outcome indicating that manufacturers value faster update processes.
3. Scarlet's $50 million round is intended to support expansion in the United States and Asia-Pacific and add reviewers, signaling investment in specialist review capacity.
4. Coverage identifies technical-documentation assessment as part of the reviewer workflow, giving a release-level evidence record a concrete review context.
5. HIT Consultant describes rapid AI-model development alongside slower certification and update cycles, matching the proposed product's model-change wedge.

Catalyst. Scarlet's funding and planned reviewer expansion signal growing capacity for specialist AI-device review, while trade coverage still reports six-to-eight-month cycles for minor updates and describes a mismatch between rapid model development and slower certification.

The manufacturer selects a previously reviewed baseline model, a candidate release, and a legally reusable, access-controlled clinical validation cohort. Before a run, its clinical and regulatory leads define the comparison protocol, clinically meaningful thresholds, and prespecified subgroups; the harness executes both versions against the same cases. It produces a version-bound record containing model identifiers, dataset manifest, protocol, subgroup deltas, exceptions, and reproducible outputs for the team's update review. A failed threshold routes to human assessment and additional evidence work; a passing replay is not represented as automatic regulatory approval.

What's different. Unlike a certification body, this product does not decide conformity or promise approval; it supplies an auditable comparison of a model release against the manufacturer's own locked clinical evidence. Unlike generic model monitoring, it is triggered by a proposed regulated release and preserves the exact baseline, cases, protocol, and subgroup results needed to discuss that change with reviewers.

Jobs to be done

flowchart LR
  DeviceTeam["AI device model team"] --> Change["Candidate model update"]
  Change --> Replay["Replay baseline and candidate on locked cohort"]
  Replay --> Record["Version-bound subgroup evidence"]
  Record --> Review["Human regulatory review"]
  Review --> Outcome["Evidence-ready update decision"]

Executive takeaways

• The pain is current and commercially visible: a specialist certification body has paying customers, a subscription-style ongoing review service, and customer-reported rapid update cycles, but no public price or independent buyer-budget data for a standalone replay tool.
• The mechanism is scientifically relevant but not technically rare. FDA research directly addresses repeated evaluation of evolving models on fixed test data; generic ML evaluation, model registries, and federated medical benchmarking are also available.
• Scarlet is the closest direct competitor and potential channel. QMS software, medical-device regulatory consultancies, and open-source evaluation stacks are credible substitutes.
• The reachable market is a modest, assumption-heavy evidence-workflow niche, not the multi-billion-dollar AI-device sales market. The estimate is capped by the number of device programs with an imminent update and legally reusable validation data.
• The key falsifier is whether a buyer can show a paid, recurring update-evidence problem that existing QMS, consultants, internal statisticians, or its certification body cannot resolve adequately.

Market definition

Annual evidence-generation and change-control workflow for manufacturers of regulated AI medical-device software: replay a candidate model and approved baseline on a controlled clinical cohort, preserve the protocol and versions, and export results into existing quality and regulatory review. It excludes device sales, certification fees, clinical-trial revenue, and general AI observability.

Customer and buyer

The daily user is a regulatory-affairs or ML-validation lead; the economic buyer is usually a director or VP of regulatory affairs, quality, clinical operations, or R&D. The best-fit beachhead is an AI-device manufacturer with an established validation cohort, internal clinical/statistical ownership, an upcoming software-model change, and rights to run the cohort in a controlled environment. Quality, privacy/security, engineering, legal, and clinical-statistics stakeholders can all veto deployment.

Buying triggers

• A planned update that must be assessed against a previously reviewed change protocol or disclosed in a marketing-submission change assessment. [2][4]
• A manufacturer already operating an update cadence that makes a months-long review bottleneck costly; Flok reports a goal of releasing upgrades twice monthly, while Lucida reports shipping an update within days of transferring its certificate. [13][14]
• A Class III EU device change that may affect safety or performance, where the change category and any required notified-body approval must be considered before validation results are used to argue that a change is minor. [8][12]
• A failure or subgroup regression found before release that would otherwise trigger further clinical, quality, or regulatory work. [3][5][21]

Willingness to pay

There is directional budget evidence, not verified willingness to pay for this product. Scarlet describes a subscription covering surveillance, annual audits, and product changes; it also announced a $50 million Series B to expand certification capacity. Customers on Scarlet-hosted case studies describe rapid update needs. The sources do not disclose contract values, how much is allocated to replay versus certification, or whether manufacturers would buy a separate tool. Validate spend by reviewing recent update project budgets and seeking a paid pilot commitment from RA/quality buyers. [10][13][14][15]

Category dynamics

Growth signal Grand View Research estimates the broad AI-enabled medical-devices market at 38.5% CAGR for 2025-2033; that is device revenue, not validation-software growth, and forecasts vary widely.

Regulatory & technical constraints

• An FDA-reviewed PCCP must define modifications, validation/implementation methodology, and impact assessment; an outside tool cannot expand the approved change boundary.
• Repeatedly reusing a fixed test cohort can inflate measured performance; cohort locking alone is insufficient and may make repeated evaluation less trustworthy.
• AI-device test sets must represent intended populations and remain meaningfully independent of model development; subgroup sample sizes and uncertain reference standards may limit conclusions.
• Clinical-data access, processing location, reuse rights, and retention require customer-specific legal and security review; federated execution reduces data movement but does not itself grant rights.
• EU MDR and the AI Act impose overlapping but distinct change, quality, documentation, and high-risk-system obligations; jurisdictional evidence mapping is needed.
• Customer-run or vendor-run software that affects regulated release decisions may add supplier qualification, validation, cybersecurity, and QMS integration work.

The closest direct threat is Scarlet, which combines software/AI-device certification with ongoing update surveillance. Ketryx automates software-device QMS enforcement, traceability, and documentation; Innolitics sells integrated engineering, AI/ML validation, and regulatory submission services. Evidently and MLflow provide generic testing, evaluation, and model-version workflows, while MLCommons MedPerf offers open federated medical AI benchmarking. Internal statistical scripts plus the manufacturer's QMS and a consultant remain the status quo.

Why incumbents do not win by default

• Notified bodies and specialist certification services. Scarlet's own announcement reports faster update cycles, and its ongoing service includes product-change reviews; a separate product cannot assume certification bodies are too slow or incapable. A cross-body, manufacturer-controlled replay could complement a notified body only if buyers need pre-review evidence independent of whichever body they use. No source proves that Scarlet's workflow lacks the proposed replay.
• QMS and change-control platforms. Ketryx integrates with developer tools and supports traceability, change impact analysis, QMS enforcement, and release documentation, which can own the workflow record. Its public product description does not explicitly describe running paired models on patient-level cohorts or producing clinical subgroup deltas.
• Regulatory engineering and consulting services. Innolitics offers integrated software engineering, AI/ML validation, regulatory strategy, V&V evidence, and submission packages; this can meet bespoke needs. The potential gap is repeatability and marginal effort for every release, not lack of capable service providers.
• Generic MLOps and evaluation libraries. Evidently and MLflow support model evaluation, dataset-linked comparisons, testing, versioning, and traceability. They lower build-versus-buy costs; the remaining work is clinical protocol design, safe test-data reuse, privacy controls, and regulatory-quality evidence context.
• Open-source medical-AI benchmarking and internal builds. MedPerf already provides federated evaluation for medical AI and can keep data at participating sites. Teams can also script baseline/candidate analysis internally. Neither public description is a productized, manufacturer-specific regulatory change gate, but both can be assembled into one.

We will sell a clinical replay change gate to AI medical-device manufacturers preparing post-validation model updates, not a new certification or regulatory decision service. For each release, the product runs the approved baseline and candidate model on the same rights-cleared cohort under a prespecified protocol and returns subgroup deltas with version, dataset, and run provenance. The first user is an RA or ML-validation lead and the likely economic buyer is a VP or director of regulatory affairs, quality, or clinical operations. Research reports six-to-eight-month review cycles for minor AI-device updates, but it does not establish standalone replay demand, buyer budget, or reviewer acceptance. The research sizing case is a $144M workflow TAM, $36M SAM, and $1.8M SOM, all assumption-heavy and capped by eligible device lines rather than broad AI-device sales. The defensible claim is a regulated release workflow and evidence record, not novel replay mathematics or ownership of customer clinical data; Scarlet, QMS platforms, consultants, and internal tools can bundle substitutes. This pre-seed plan funds paid-pilot proof of willingness to pay, lawful cohort reuse, and evidence utility before expanding deployment, hiring, or geography.

Problem

• Model teams can iterate faster than their review process; trade coverage reports six-to-eight-month cycles even for minor AI-device updates.
• For each proposed change, RA and validation teams need comparable baseline-versus-candidate results, subgroup analysis, and traceable evidence, while the supplied research does not establish how consistently current manual workflows deliver them.
• A retained cohort may not be legally reusable, representative, or statistically independent after repeated evaluation, limiting the eligible customer pool and risking misleading results.

Solution

• Run a customer-selected baseline and candidate model against the same documented clinical cases in a customer-controlled environment, after verifying cohort rights and intended-use scope.
• Lock the protocol, metrics, thresholds, subgroup definitions, model identifiers, and dataset manifest before execution; report uncertainty, missing coverage, exceptions, and version-bound deltas.
• Export a reproducible evidence record into the manufacturer's existing review and QMS workflow for human clinical, quality, and regulatory assessment.
• Treat the output as supporting evidence only; it does not decide change classification, submission requirements, conformity, or approval.

Why we win

• For a regulated release, the product binds paired model execution, cohort manifest, prespecified subgroup analysis, and a traceable report into one repeatable workflow; generic evaluation and registry tools provide components but leave the clinical and change-control context to the manufacturer.
• Customer-hosted or, if demanded, federated execution can reduce unnecessary data movement and fit existing custody constraints, while making clear that deployment location does not grant data-use rights.
• Repeated customer deployments can build validated protocol templates and QMS/model-registry integrations; customer clinical data remains customer-controlled and is not the moat.

Milestones

flowchart LR
  Trigger[Planned model update] --> Qualify[Rights and cohort qualification]
  Qualify --> Replay[Paired baseline-candidate replay]
  Replay --> Evidence[Version-bound subgroup evidence]
  Evidence --> Review[Human RA and clinical review]
  Review --> Expansion[More lines and recurring releases]

Founding team

Experiment roadmap

Risk assessment

What must be true

• At least 3 of 15 interviewed target manufacturers commit to paid pilots, and at least 2 will pay within the tested $20,000-$30,000 range.
• At least 3 of 10 qualified prospects can document cohort-use rights, adequate subgroup representation, and a credible control against repeated-test-set contamination.
• On four historical updates, the workflow reduces median evidence-preparation hours by at least 30% while preserving protocol lock, reproducibility, and explicit uncertainty.
• At least 3 of 4 structured consultant/reviewer respondents judge the report useful supporting evidence, without treating a passing replay as approval or change classification.
• At least 2 of the first 3 paid pilots convert to recurring annual device-line contracts, with repeat-run gross margin capable of exceeding 70% and no incumbent bundle displacing the purchase.

Open diligence questions

• Can the first three target buyers show actual update budgets, consultant invoices, or staff hours that fund a separate replay purchase?
• What consent, contract, retention, and reference-label terms permit repeated execution on each proposed cohort, and what refresh or independent-set plan protects statistical validity?
• Which historical update decisions and subgroup regressions would have changed if this paired replay had been available, and how is that measured against the existing pipeline?
• What feedback have independent RA consultants or reviewers given on the report, and what parts would still require new clinical evidence or formal submission?
• Can customers deploy and validate the runner in their controlled environment and export to their QMS without adding more supplier-validation burden than the workflow removes?
• Will Scarlet, Ketryx, Innolitics, or an internal team provide a good-enough bundled substitute before the company reaches repeatable distribution?

Model sanity

• Revenue engine. At $80K blended annual revenue per line, a churn-unadjusted ramp of 3.9 average active lines in Y2 and 7.1 in Y3 lifts annual revenue from $310K to $570K (A7-A10, A21).
• Must go right. Two of the first three $20K pilots must convert, the 70% target margin must hold, and five recurring lines must be reached by month 24 to justify the pre-seed plan (A4, A11, A17).
• Model breaks if. A 40% reduction in paid line-months leaves Y3 revenue at $342K and drives the modeled cash low point to -$868.6K without follow-on financing (A22-A23).
• Next-round proof. By month 24, prove five recurring U.S. lines, two pilot conversions, a customer-accepted QMS export, and at least 70% repeat-run gross margin before modeled cash turns negative in Q3Y3 (A9, A11, A17).

Scenarios

Sensitivity

flowchart LR
  Accounts[Target accounts] --> Qualified[Rights-qualified device lines]
  Qualified --> Pilots[Paid retrospective pilots]
  Pilots --> Annual[Annual line conversions]
  Annual --> Revenue[Revenue at blended annual ARPU]
  Revenue --> GrossProfit[70% target gross profit]
  GrossProfit --> Cash[Cash after payroll and operating costs]
  Cash --> Milestone[Recurring lines and next-round proof]

Flags: The unfinanced base cash balance turns negative in Q3Y3 and ends Y3 at -USD 573.7K; only about USD 118K remains after the six months following the month-24 milestone, so a follow-on round or earlier hiring cuts are required (A17, A22). · The 3-pilot funnel, 2-of-3 conversion, standalone buyer budget, lawful cohort reuse, and renewal behavior are unvalidated, and the baseline customer ramp is not churn-adjusted; the 1.5% monthly churn input is used only for LTV and the sensitivity case (A4, A8-A9, A20-A21). · The 70% gross margin is a BP target; bespoke statistical, security, and customer validation work could keep delivery below target (A11, A14-A16). · Modeled $80K annual ARPU exceeds the research sizing proxy of $60K per line-year because it adds a hypothesized replay fee; neither the price premium nor volume is validated (A5-A7 and research.yaml.market). · Fully loaded CAC of USD 54K is based on only 2 modeled annual conversions and an assumed 40% CEO sales-time allocation, so the 5.76x LTV/CAC ratio is highly provisional (A20-A21). · Y3 revenue per FTE of USD 84K is below the broad SaaS planning heuristic, and the 7.35-FTE year-end team should remain gated on renewal and delivery-margin proof (A13, A25). · The research SAM and SOM use unverified 25% eligibility and USD 60K annual spend assumptions; they are not evidence of a forecastable buyer pool (business-plan.yaml.market and research.yaml.market). · The cash roll-forward excludes working capital, taxes, capex, and all follow-on financing; actual liquidity could be lower or the next round needed sooner (A22).

• Clinical data rights. Manufacturers may lack the rights or governance basis to reuse a retained patient cohort for repeated model-release tests. Mitigation: Run data in the manufacturer's controlled environment and qualify only cohorts with documented permissions and retention terms.
• Reviewer acceptance. A replay against prior validation cases may not provide sufficient evidence for substantial model changes or every reviewer's expectations. Mitigation: Co-design protocols with each customer's clinical and regulatory leads, preserve transparent outputs, and position the replay as supporting evidence rather than approval.
• Cohort overfitting. Repeated access to a fixed cohort can encourage model tuning against the test set and weaken its value as independent evidence. Mitigation: Use access controls, separate development and sealed evaluation sets, and require customer-approved rules for cohort rotation or fresh evidence when warranted.