“A re-run is a new event that produces a new output. It is not proof of what the system did.”
In an ordinary commercial dispute, the record of what happened is email, documents, and the memory of the people involved. When an autonomous agent took the action, little of that exists. There is no author to depose about why the agent did what it did, and the people who built the system frequently cannot say either.
What exists instead is a different category of evidence—system prompts, tool-call logs, model identifiers, retrieval context, and configuration state—and a great deal of it is discarded by default within days. By the time a complaint is filed, the record that would have explained the conduct is often gone, and it was deleted by a retention policy nobody thought of as a litigation-hold problem.
Key Takeaways
- The evidence is configuration, not correspondence. What the agent was instructed to do, what it was permitted to call, and what it retrieved matter more than any single output.
- The default retention posture is destructive. Prompt and tool-call logs are commonly kept for days, not years, and inference providers often retain nothing on the customer’s behalf.
- The model version is a discrete, load-bearing fact. Behavior is not reproducible without it, and providers deprecate versions on their own schedule.
- Your duty reaches vendor-held data as far as your control does, and handing data to a third party after the duty attaches, without securing its preservation, can breach it. Much of the record sits with an inference provider or an orchestration platform under contract, and the contract usually does not require them to keep it.
- Reproduction is not verification. Running the agent again produces a new output, not proof of the old one, and offering your own runs can open the failed ones to discovery.
- A person still sits for the deposition. Human-in-the-loop practices have to be honored and documented, not just asserted.
The Categories That Actually Matter
System prompts and instruction state. The standing instructions defining the agent’s role, constraints, and authority. This is the closest thing to a corporate policy the agent has, and it is the first thing to request. It also changes—often frequently, and often without version control—so the operative question is what the prompt said at the time, not what it says now. We found no reported decision compelling a deployer’s system prompt, so the request rests on ordinary relevance and proportionality, and a demand drawn too broadly can fail.1
Tool and function-call logs. What the agent actually invoked: which API, with what arguments, and what came back. This is the transactional record, and it is where the conduct is visible. It is also the category most likely to have been truncated or sampled for cost reasons.
Model identity and version. Which model, at which version, with which inference parameters. Two versions of the same model behave differently, and providers retire versions on their own timeline. A dispute that takes eighteen months to reach discovery may concern a model that is no longer available.
Retrieval and context state. For agents that retrieve documents or carry memory across sessions, what was in the context window is a substantive fact—the agent acted on that, and not on whatever the same query would retrieve today. Persistent memory raises its own problems, which we have written about in AI agent memory and privacy compliance.
Authorization and scope records. Who authorized the agent to act, within what limits, and what the counterparty was told. This is the same record that answers the liability question in deployer liability, which is why building it once serves both purposes.
Human-in-the-loop artifacts. Where a person reviewed, approved, or overrode the agent, that is ordinary evidence of ordinary decision-making, and it is often the most useful material in the case. It also answers the question no log can. Data helps, but a model does not sit for a deposition. A person does, and that person needs actual knowledge of what the agent was permitted to do and who reviewed it. Human-in-the-loop practices have to be honored and documented, not just asserted.
What Courts Have Already Ordered
Courts have ordered model developers to preserve and produce user prompts, outputs, and logs, and the contest has been over relevance, proportionality, and privacy. In the OpenAI copyright litigation, a magistrate judge directed OpenAI in May 2025 “to preserve and segregate all output log data that would otherwise be deleted on a going forward basis until further order of the Court.”2 That going-forward obligation later ended by stipulated order. The court then compelled a sample of 20 million de-identified ChatGPT logs, reasoning that privacy considerations “are only one factor in the proportionality analysis, and cannot predominate where there is clear relevance and minimal burden,” and the district judge affirmed in January 2026.3 In Concord Music Group v. Anthropic, the court set a sample of 5 million prompt-output pairs.4
Those are disputes with model developers over output at scale, not with an operator over one transaction. The closest operator-side ruling went the other way, on a ground most operators will not have. In the OpenAI litigation the court refused to compel the New York Times’s logs from an internal tool built on OpenAI’s API, because the uses at issue were the defendants’ and other users’, not the Times’s, and the production would not have been proportional in any event; with the logs irrelevant, it concluded that their inadvertent loss “does not implicate Fed. R. Civ. P. 37(e).”4 We found no decision yet applying these rules to an autonomous agent’s telemetry. An operator whose agent’s conduct is the dispute should plan for the developer cases.
Why Preservation Is the Whole Problem
Federal Rule of Civil Procedure 37(e) applies where electronically stored information that should have been preserved in the anticipation or conduct of litigation is lost because a party failed to take reasonable steps to preserve it, and it cannot be restored or replaced through additional discovery.5 That third element is what makes agent telemetry a live problem rather than an unfortunate one: a deleted log may have no second custodian, and even a copy held elsewhere may not show how the system used it.6
The fact pattern is already in the reporters. In Meta Platforms v. BrandTotal, a debugging log sat on a subscription with 30-day auto-deletion, and the court observed that “it is not hard to imagine that BrandTotal’s employees might have overlooked their transient diagnostic log as a source of relevant and discoverable information.”7 In Gregory v. Montana, surveillance footage auto-deleted on a 17-day cycle that had been assumed to be 30 days, and the Ninth Circuit held that “failing to promptly determine how long the surveillance system preserved its information was unreasonable.”8
The remedies then run in two tiers. On a finding of prejudice, the court may order measures no greater than necessary to cure it. Only on a finding that the party acted with the intent to deprive another party of the information’s use may the court presume the lost information was unfavorable, instruct the jury that it may or must so presume, or dismiss the action or enter default judgment.5 In the Ninth Circuit that line holds: Gregory reversed an adverse sanction because recklessness is not intent, and it held that Rule 37(e)(2) displaces a court’s inherent authority over this kind of loss.8 Even so, one magistrate judge has ordered the loss put in front of the jury, leaving intent for the jury to decide. In Flexport v. Freightmate AI, in an order that does not discuss Gregory, deleted shipping documents, operational files, and “prompt engineering files” drew an order for a modified jury instruction under Rule 37(e)(1); the case settled before trial.9
Two features make this worse than ordinary ESI loss.
The first is timing. Rule 37(e) presupposes a duty that can attach before a complaint: it reaches information that should have been preserved “in the anticipation or conduct of litigation.”5 The test is objective. It asks whether “a reasonable party in the same factual circumstances would have reasonably foreseen litigation,” and “the mere existence of a potential claim” is not enough.10 That line can fall later than teams fear: Flexport put the trigger, on that record, at a claim letter, not at an earlier consultation with a litigator.9 But telemetry does not wait for the line. In an agent dispute the operator often learns something went wrong within hours—a chargeback, a complaint, an anomalous transaction—while the telemetry that explains it may expire within days. A short retention period is not itself the violation; the failure is not stopping the deletion once the duty attaches.11
The second is custody. A meaningful portion of the record sits with third parties: the inference provider, an orchestration platform, an observability vendor. A litigation hold directed at your own systems does not reach them, and standard commercial terms rarely obligate a provider to retain anything for a customer’s litigation. Whether that material is within a party’s “possession, custody, or control”12 depends on the forum. In the Ninth Circuit it is a contract question: control is “the legal right to obtain documents upon demand,” and courts read the vendor agreement to decide.13 Courts elsewhere may also ask whether the party has the practical ability to get it.14 One court has treated turning held records over to a third party, without preserving them or ensuring the third party would, as a failure of the duty to preserve.14 We found no decision applying either test to an inference provider’s prompt logs, so the question should be answered before it is a discovery dispute.
Reproduction Is Not Verification
There is an intuitive and wrong move available in these cases: run the agent again with the same input and treat the result as proof of what happened.
A re-run does not show that. It is a new event: the model version may have changed, the retrieval corpus has almost certainly changed, and the system prompt may have been edited since—and inference is not reliably reproducible even holding those constant, though the variation can be slight.15 One court ran the same prompt on three computers, got three values, and wrote that “the fact there are variations at all calls into question the reliability and accuracy of Copilot to generate evidence to be relied upon in a court proceeding.”16 We found no decision ruling on a re-run as proof of a past event. The output may be useful evidence of how a system behaves; it is not proof of what the system did.
It also has a price. A party that offers its own runs through a witness can open the failed ones to discovery. One court ordered production of all prompts and outputs, “negative as well as positive,” from investigators and client-witnesses in the post-suit investigation, though not counsel’s own prompts never given to them, and ordered only counts for pre-suit testing: “if Publishers are supplying the numerator, Anthropic is entitled to discover the denominator.”17
The corollary is that contemporaneous logging is the only reliable record, which returns the question to preservation.
Practical Guidance
For operators, before anything happens. Find out what your retention periods actually are, in writing, because assuming them is how Gregory happened. Set retention for prompts, tool calls, model identifiers, and context state to a period measured against your realistic limitations exposure rather than your storage budget. Version your system prompts and keep the history. Ask your inference provider, in writing, what it retains and for how long, and negotiate a preservation obligation if the answer is nothing, and a right to obtain the records on demand; in the Ninth Circuit that right, not the retention promise, creates control. Write an incident trigger that issues a hold on agent telemetry automatically, because a manual process will lose the race.
For operators, once something happens. Issue the hold immediately and extend it to vendors in writing the same day. Capture the model version before the provider deprecates it. Do not edit the system prompt to fix the problem before preserving what it said—the fix is usually right and the sequence is usually wrong. Decide now who would testify to how the system was configured and reviewed, and make sure that person actually knows.
For claimants. Request the configuration, not just the output.18 Ask for the system prompt as of the date of the conduct, the tool-call log for the transaction, the model identifier and parameters, the retrieval context, and the retention policy itself. The retention policy is worth requesting early: it establishes what existed, which is the predicate for everything that follows if it is gone. Expect a fight over prompts a lawyer wrote for the case, which courts have protected as work product, subject to waiver.19
For boards. Retention design for agent telemetry is an oversight question, not only an engineering one, for the reasons set out in the Caremark duty in the managed agents era. A company that cannot say what its agents did is a company whose board cannot supervise them.
Work with Astraea Counsel
Astraea Counsel litigates disputes arising from AI systems, including the preservation and discovery fights that decide what the record will show. See our AI litigation attorney page, or contact us to discuss a dispute or a preservation question.
Notes and sources
Footnotes
-
Fed. R. Civ. P. 26(b)(1), available at https://www.law.cornell.edu/rules/frcp/rule_26; cf. Concord Music Grp., Inc. v. Anthropic PBC, No. 24-cv-03811-EKL (SVK), 2025 U.S. Dist. LEXIS 99068, at *11—12 (N.D. Cal. May 23, 2025) (denying without prejudice, as an overbroad work-product waiver argument, requests reaching the system prompts and temperature settings an opponent used in its pre-suit investigation). PDF ↩
-
In re OpenAI, Inc., Copyright Infringement Litig., No. 25-md-3143 (SHS) (OTW), 2025 U.S. Dist. LEXIS 97943, at *54 (S.D.N.Y. May 13, 2025) (emphasis omitted). PDF ↩
-
In re OpenAI, Inc., Copyright Infringement Litig., No. 25-md-3143 (SHS) (OTW), 2025 U.S. Dist. LEXIS 252574, at *60, *62 n.2, *69 (S.D.N.Y. Dec. 2, 2025) (denying reconsideration of Nov. 7, 2025 order compelling production); see In re OpenAI, 2026 U.S. Dist. LEXIS 1986, at *31—32, *36 (S.D.N.Y. Jan. 5, 2026) (affirming Nov. 7 and Dec. 5, 2025 orders). PDF PDF ↩
-
Concord, 2025 U.S. Dist. LEXIS 99068, at *15; In re OpenAI, Inc., Copyright Infringement Litig., 800 F. Supp. 3d 602, 606 n.2, 608—13 & n.19 (S.D.N.Y. 2025). PDF PDF ↩ ↩2
-
Fed. R. Civ. P. 37(e), available at https://www.law.cornell.edu/rules/frcp/rule_37. ↩ ↩2 ↩3
-
See Flexport, Inc. v. Freightmate AI, Inc., No. 25-cv-02500-RFL (PHK), 2026 U.S. Dist. LEXIS 211541, at *18 (N.D. Cal. Sept. 16, 2026) (“[E]ven if the actual content could somehow be restored from a different location, this does nothing to show how this ESI was used by Defendants or whether the files were altered in any way while in Defendants’ possession.”); Meta Platforms, Inc. v. BrandTotal Ltd., 605 F. Supp. 3d 1218, 1238 (N.D. Cal. 2022). PDF PDF ↩
-
Gregory v. Montana, 118 F.4th 1069, 1072, 1074, 1078—80 (9th Cir. 2024). PDF ↩ ↩2
-
Flexport, 2026 U.S. Dist. LEXIS 211541, at *11—15, *19, *28—29. The plaintiff moved for relief from the order, ECF No. 299 (N.D. Cal. Oct. 1, 2026), and the case ended in a stipulated judgment on the parties’ settlement, ECF No. 304 (N.D. Cal. Oct. 8, 2026). PDF PDF PDF ↩ ↩2
-
Micron Tech., Inc. v. Rambus Inc., 645 F.3d 1311, 1320 (Fed. Cir. 2011). PDF ↩
-
In re Google Play Store Antitrust Litig., 664 F. Supp. 3d 981, 992 (N.D. Cal. 2023). PDF ↩
-
Fed. R. Civ. P. 34(a)(1), available at https://www.law.cornell.edu/rules/frcp/rule_34. ↩
-
In re Citric Acid Litig., 191 F.3d 1090, 1107 (9th Cir. 1999) (quoting United States v. Int’l Union of Petroleum & Indus. Workers, 870 F.2d 1450, 1452 (9th Cir. 1989)); Albornoz v. Wal-Mart Assocs., Inc., No. 1:22-cv-01229-JLT-CDB, 2023 U.S. Dist. LEXIS 116428, at *4—7 (E.D. Cal. July 6, 2023). PDF PDF ↩
-
In re NTL, Inc. Sec. Litig., 244 F.R.D. 179, 195, 197 (S.D.N.Y. 2007). PDF ↩ ↩2
-
United States v. Deleon, 116 F.4th 1260, 1275 (11th Cir. 2024) (Newsom, J., concurring) (observing that large language models “aren’t even designed to produce the exact same answer every time,” and that when a model’s “creativity dial is turned down, its successive responses to identical questions should be very similar, if not perfectly repetitive”). PDF ↩
-
In re Accounting by Weber, 220 N.Y.S.3d 620, 633 (Sur. Ct. 2024). PDF ↩
-
Concord Music Grp., Inc. v. Anthropic PBC, No. 24-cv-03811-EKL (SVK), 2025 U.S. Dist. LEXIS 262233, at *9—10, *12—14, *16—18 (N.D. Cal. Dec. 18, 2025) (emphasis omitted) (quoting Order, Dkt. 478 at 5). PDF ↩
-
See Fed. R. Civ. P. 26(b)(1), available at https://www.law.cornell.edu/rules/frcp/rule_26; Fed. R. Civ. P. 34, available at https://www.law.cornell.edu/rules/frcp/rule_34. ↩
-
Tremblay v. OpenAI, Inc., No. 23-cv-03223-AMO, 2024 U.S. Dist. LEXIS 141362, at *7—10 (N.D. Cal. Aug. 8, 2024); Concord, 2025 U.S. Dist. LEXIS 99068, at *8—9; Concord, 2025 U.S. Dist. LEXIS 262233, at *9, *12—13; Disney Enters., Inc. v. Midjourney, Inc., No. 2:25-cv-05275-JAK-AJR, 2026 U.S. Dist. LEXIS 158796, at *48, *53—54 (C.D. Cal. June 15, 2026). PDF PDF ↩
