Proceedings of the 2025 Workshop on Law-Following AI

Introduction

Law-following AI (LFAI)^1[1] denotes three closely related concepts. First, it denotes a plausibly desirable form of AI technology: agentic AI systems^2[2] designed to follow the law by refusing to follow illegal orders or use illegal means to accomplish lawful orders.^3[3] Second, it denotes an accompanying policy proposal: imposing legal requirements that agentic AI systems in certain settings be LFAIs.^4[4] Third, it refers to an emerging field of inquiry aimed at enabling the development of law-following AI agents and the implementation of LFAI policies.^5[5]

To catalyze further research into LFAI, the Institute for Law & AI, with support from the Leverhulme Centre for the Future of Intelligence at the University of Cambridge and the United Kingdom’s Advanced Research & Innovation Agency, hosted the inaugural Workshop on Law-Following AI at the University of Cambridge. Held from August 6–8, 2025, the workshop convened more than forty scholars from law, computer science, and related disciplines to discuss LFAI and, hopefully, make progress on aspects of the LFAI research agenda. This proceedings report compiles some of the key insights from the sessions and discussions at the workshop.

Given the number and diversity of participants, individual authors or participants may disagree with many of the ideas or arguments presented herein. Rather than attempting to capture any particular consensus, this document generally attempts to summarize key ideas and discussions from the workshop, so that those who were unable to attend can nevertheless benefit from them and continue the academic discussion on this topic.

The Case for Law-Following AI in Brief

While we encourage readers interested in the topic of LFAI to read the full article,^6[6] we summarize some of its key ideas here for readability.^7[7] LFAI is a set of propositions about AI agents. While the exact definition of AI agency remains debated, AI agents are generally AI systems that can “pursue more complex goals in more complex environments, exhibiting independent planning and adaptation to directly take actions in virtual or real-world environments.”^8[8] The Law-Following AI article uses “AI systems ‘that can do anything a human can do in front of a computer’ as competently as a human expert”^9[9] as its definition for “AI agents.” Importantly, this definition is illustrative only: a more formal (and likely broader) definition of AI agent would be necessary for actual policymaking.^10[10]

AI agents are often analyzed using familiar principal–agent frameworks.^11[11] A major goal of AI policy and safety research has been ensuring that AI agents are intent-aligned: that is, that the agent reliably acts as its principal intended.^12[12] Intent-alignment is an unsolved technical problem: AI developers currently do not know how to make AI systems that are intent-aligned.^13[13]

However, intent-alignment is likely inadequate in many ways. Principals might include bad actors, or actors that are simply indifferent to the interests of others. To address this class of problems, AI safety researchers have proposed value-alignment—wherein AI agents have extralegal normative constraints on their actions that principals cannot override—as a complement to intent-alignment.^14[14]

LFAI can be thought of as an alternative to value-alignment, and therefore a complement to intent-alignment.^15[15] While the law is certainly not a complete guide to moral behavior, aligning AI systems to the law has important advantages over value-alignment. One major advantage is legitimacy: democratically enacted laws are a much more legitimate source of constraints than extralegal moral principles.^16[16] Another advantage is authoritativeness. The law is generally drawn from a small set of well-specified authoritative sources, such as constitutions, statutes, and case law.^17[17] There is no comparable authoritative list of written moral principles that commands near-universal recognition. A related benefit is precision. While legal drafting is often vague or inartful, legal prohibitions tend to be much more clear about what, exactly, they require than moral injunctions.^18[18] Finally, the law is more easily resolvable than morality: “when there is disagreement or ambiguity, the law contains established processes for authoritatively resolving disputes over the applicability and meaning of laws.”^19[19]

LFAI might be particularly important for AI agents controlled by governments.^20[20] LFAI is an ex ante means of preventing lawless government actions: LFAIs by definition refuse to violate applicable laws in the first place. Preventing lawless action by governmental AI agents ex ante is important for several reasons. First, ex post remedies, such as tort suits and criminal prosecutions for civil rights violations, play a limited role in constraining governmental abuse.^21[21] Immunity doctrines,^22[22] indemnification practices,^23[23] the prospect of pardons for criminal abuses, resource asymmetries, misaligned financial incentives,^24[24] and limited opportunities for self-help^25[25] all mean that ex post liability (whether criminal or civil) is a significantly weaker check against government agents than private actors. Accordingly, we largely use ex ante legal tools—injunctions,^26[26] nullification,^27[27] multiple independent veto points,^28[28] oaths, responsible hiring practices, supervision, and disqualification rules^29[29]—to prevent lawless governmental action in the first place. LFAI can be thought of as just such an ex ante tool: one that takes advantage of the designable nature of AI systems^30[30] to stop lawless action at its source. 

Indeed, ex ante constraints might be more important for governmental AI agents than for human agents. Notwithstanding all of the limitations of ex post mechanisms, they still play a very important role in preventing lawless governmental action.^31[31] But AI agents that are merely intent-aligned would, by default, lack many of the incentives to follow the law that human agents have, such as fear of imprisonment,^32[32] reputational damage, and the unpleasantness of litigation or congressional oversight, not to mention innate personal morality. Ex ante law-following design can compensate for these ex post weaknesses.^33[33]

Major Themes from the Workshop

In this section, we summarize some of the major themes from presentations and discussions at the workshop. Since the goal of the workshop was to inspire further research on LFAI, many discussions took on a critical posture, such as identifying weaknesses or limitations in the LFAI article. The discussion herein reflects many of those critical themes. These critical discussions, however, should not be mistaken for an overall evaluation of LFAI; participants generally found the topic intellectually generative and meritorious.

The Promises and Limits of Liability

LFAI is hypothesized as a desirable complement to liability. In many cases where an AI agent behaves illegally and thereby causes harm, it will be possible to hold the agent’s principal liable for that harm in tort. LFAI does not aim to remove the possibility of such actions.^34[34] However, as mentioned above,^35[35] LFAI is also premised on the assumption that tort will prove inadequate in many cases, such as where the principal is a government actor,^36[36] or where an AI system behaves much more culpably than its principal (for example, by using criminal means to satisfy an innocuous command), such that it would be unjust to hold the principal fully responsible.^37[37] Since LFAI is justified in part as a solution to the shortcomings of liability, an accurate appraisal of LFAI depends on an accurate understanding of how liability is likely to work—or fail—when AI agents are widely deployed.

The workshop contained lively discussion on these questions, drawing on a large and growing body of scholarly literature.^38[38] As with many discussions in technology law,^39[39] discussions about appropriate analogies for AI agents in existing law loomed large.^40[40] Some participants expressed optimism that liability frameworks adapted from either respondeat superior^41[41] or the closely related law of corporate liability^42[42] could provide adequate incentives to private actors deploying AI agents. Participants also noted that even absent regulation, AI systems and the corporations deploying them will be constrained by existing market and legal constraints.^43[43]

Others were more skeptical that ex post mechanisms were up to the task. One major source of skepticism starts from the observation that insurance will play a large role in determining how the legal rules established in AI liability regimes actually affect the behavior of AI developers and deployers.^44[44] Since AI developers and deployers are likely to be (and indeed, already are) insured against third-party claims, liability can promote safer behavior only if insurers are able to either price AI-related risks accurately or actively reduce those risks through measures such as offering consulting and risk-management services.^45[45] However, such “regulation by insurance” in cybersecurity—an arguably structurally similar domain to AI—seems not to have effectively incentivized improved security practices.^46[46] Although AI-specific insurance policies are beginning to emerge in the market, it remains to be seen whether the insurers offering these policies will be able to gather data that allows them to better design incentives for their policyholders.^47[47]

Of course, this debate cannot be resolved here. However, we note that, despite these widely varying perspectives on the efficacy of liability for harms caused by AI agents controlled by private actors, few participants contested the premise that liability will be a weaker deterrent for governmental AI agents than for governmental human agents. Accordingly, even if new or existing tort doctrines can properly address many risks from private actors, the case for LFAI in the public sector stands on firmer ground.^48[48]

The Nuances of Automated Legal Reasoning

For LFAI to work, AI agents must be able to perform reasonably reliable automated legal reasoning:^49[49]

[A]n LFAI must be able to determine whether it is obligated to refuse a command from its principal or whether an action it is considering runs an undue risk of violating the law. Without the ability to reason about its own legal obligations, an LFAI would have to outsource this task to human lawyers. While an LFAI likely should consult human lawyers in some situations, requiring such consultation every time an LFAI faces a legal question would dramatically decrease its efficiency. If law-following design constraints were, in fact, a large and unavoidable tax on the efficiency of AI agents, then LFAI as a proposal would be much less attractive.^50[50]

Relatedly, the LFAI policy proposal requires some method for determining whether a given AI agent is sufficiently law following.^51[51] Thus, a significant portion of the workshop was dedicated to discussion of automated legal reasoning,^52[52] providing more thorough coverage of the topic than is available in the LFAI article.^53[53] As an understanding of the rationale for an AI agent decision is critical for the attribution of liability and the ability to contest such a decision, mechanisms for discerning and recording the reasoning processes of AI agents were also considered.

This area is particularly fast-moving. GPT-5 performs almost twenty percentage points better than GPT-3.5 on Legal Bench,^54[54] a leading legal AI^55[55] benchmark. Whereas GPT-3.5 earned no better than a B on law school exams in 2022,^56[56] OpenAI’s o3 earned several A+s this spring.^57[57] Accordingly, future developments in legal AI (and better understanding of current models’ capabilities) could render much of the below discussion dated in the near future.

The Fundamentals of AI Evaluation

A key concept in this area is evaluation: “the science of measuring AI behaviors, impacts, and performance.”^58[58] Evaluations might aim to assess the performance of (1) an AI model on its own, (2) an AI model when integrated into some larger AI system, or (3) a human with access to some AI model or system.^59[59] Evaluations might assess performance relative to some benchmark: some normatively desirable level of performance, such as objective correctness or human performance.^60[60] 

Figure 1: An illustration of AI benchmarks^61[61]

Not all benchmarks are created equal. When assessing an AI benchmark generally, it is important to assess: (1) whether the inputs are representative of real-world distributions, (2) how subjective the normative outputs are, (3) the temporal stability of the benchmark, (4) the risk that the benchmark “leaks” and contaminates future training data, and (5) the risk that the benchmark is otherwise “gameable.”^62[62]

These factors can both contextualize what, exactly, the benchmark is measuring and help researchers translate benchmark performance into a prediction of real-world performance and impacts.

A major frontier in legal AI evaluation is developing methods of evaluation more nuanced than hallucination rates, particularly for non-objective AI outputs.^63[63] For example, legal AI systems can be evaluated for, inter alia: (1) accuracy on a benchmark; (2)robustness “against adversarial attacks and perturbations”;^64[64] (3) factuality: whether the output “originates from a verifiable and citable source”; (4) comprehensiveness: whether the output “coherently and concisely addresses all aspects of the task”; (5) the fairness of outputs; (6) the understandability of the outputs; and (7) the transparency of how the system was developed and how it produced the outputs.^65[65]

Evaluations of legal AI systems face significant challenges. Since law and legal practice vary by jurisdiction, an inherent limitation of many legal AI evaluations is jurisdiction-dependence.^66[66] However, there may be much deeper problems. One is data quality. It is sometimes assumed that law is a promising domain for the creation of expert AI systems because of the widespread availability of legal texts (e.g., statutes, case law, court documents, contracts, and legal scholarship). However, there are significant data quality issues for many of these texts.^67[67] Even legal briefs from the largest law firms frequently contain errors.^68[68] Recent scholarship has also noted important limitations on the use of judicial opinions as data, since such opinions often intentionally omit the parts of the reasoning process that produced an opinion.^69[69] Another key limitation is that measuring the relative quality of many of the most important legal outputs, such as briefs, memos, or contracts, is an inherently subjective endeavor notwithstanding high-level objective criteria for quality.

The Role of Explanation in Legal AI

When discussing the potential promise of legal AI systems, care must be taken to identify which legal tasks, exactly, the AI system will be performing. For LFAI, one crucial task is judgment prediction:^70[70] we may wish for LFAIs to base their judgment of whether a contemplated action is legal on a prediction of what some court(s) will do.^71[71]

Of course, while judgment prediction may be useful, it is not the whole picture:^72[72] explanation of legal conclusions plays a crucial role in the legal system.^73[73] Thus, legal AI systems that provide some form of explanation are also crucial in many contexts.^74[74] One method of providing such explanations has been the use of argument graphs in which legal arguments (and the relationships between them and legal conclusions) are represented symbolically, enabling logical analysis.^75[75] Proponents of such techniques argue that it enables provision of “a step-by-step justification” of the overall structure of a legal argument, which can be further supplemented by sources of law or normative rationales justifying each argumentative step.^76[76] This technique has been used to analyze US trade secret law,^77[77] the ownership rules for wild animals,^78[78] the automobile exception to the Fourteenth Amendment,^79[79] and Article 6 of the European Convention on Human Rights.^80[80] Efforts to combine these symbolic approaches with neural systems like LLMs—and thus reap the benefits of both—are ongoing.^81[81]

Risks in Automated Compliance

LFAI might be seen as a form of automated compliance: LFAIs comply with applicable laws automatically, without the need for ex post enforcement. Due to the opaque and ubiquitous nature of AI systems, it can be costly, even unfeasible, for a regulator to monitor compliance. In such a context, embedding compliance protocols within AI systems can form an efficient mechanism to advance lawful AI agent behavior.^82[82] Another major theme of the workshop explored the possible downsides of this approach to law.

Automated compliance is similar to perfect enforcement of the law.^83[83] But people often chafe at perfect enforcement even of laws they approve of, such as traffic laws.^84[84] Scholars have noted that the increasing automation of monitoring and enforcement could fundamentally change how laws operate in practice.^85[85] In the field of privacy and data protection, for example, recent work examined the effects if the internet shifted from selective enforcement to comprehensive automated monitoring of GDPR compliance.^86[86] Such a shift could make rarely enforced rules bite, even where the legislators never intended them to be applied so aggressively.

LFAI policies must therefore contemplate how rigorously LFAIs must obey the law.^87[87] Per the perfect enforcement literature, the answer should probably not be “perfectly,” at least for many laws.^88[88] Yet even beyond the question of rigor, LFAIs raise deeper challenges. Should an LFAI be a textualist or a purposivist? And when legal rules embody distributive choices (for instance between consumers and corporations) how should those trade-offs be resolved? Law-following requires explicit design decisions about legal interpretation and values,^89[89] and implicitly requires us to be comfortable with the automatic, large-scale execution of such choices.

Another concern might be that LFAIs could run afoul of antidiscrimination law if not designed carefully.^90[90] Suppose that an LFAI, through its training process, learns (without being instructed) to use a proxy metric for some protected classification to decide whether to refuse an order.^91[91] For example, perhaps the model has learned that men are more likely to commit violent crimes than women.^92[92] In some cases, the LFAI might deny a request from a man that it would comply with if requested by a woman. This could raise antidiscrimination issues,^93[93] especially in regimes that use a “disparate impact”^94[94] or “indirect discrimination”^95[95] standard.

Standard of Care for AI Agents

The workshop featured a rich discussion of the proper standard of care for AI agents, focusing especially on the reasonableness standard.^96[96] The study of AI reasonableness is attractive for LFAI as a project because reasonableness standards are pervasive in the law;^97[97] creation of AI agents that behave reasonably, in the legal sense, would thus represent significant progress towards LFAI. The reasonableness standard also recognizes a pluralism of goods anchored in the existing human values, thus making it both a normatively attractive target for study and an exciting technical challenge, especially when compared to the unidimensional measures of performance often used in machine learning.^98[98] 

We can start with the simple question of how well LLMs can already match the reasonableness judgments of humans^99[99]—a strategy called “silicon sampling” in social science.^100[100] While LLM outputs are known to imperfectly reflect the views of most populations,^101[101] the silicon sampling literature nevertheless finds that they can produce views that correlate remarkably well with human populations.^102[102] A recent study^103[103] comparing LLM and human perceptions of reasonableness found remarkable similarities. For example, when evaluating whether failure to take some precaution was negligent, human jurors tend to weigh social factors (that is, how common that precaution is) much more heavily than economic factors (that is, how expensive the precaution would be and how effective it would be at preventing loss), contrary to the general scholarly consensus favoring the latter.^104[104] Remarkably, LLMs share this bias.^105[105] Other replications find many other cases in which LLMs mirror human judgments, but also cases in which they differ, sometimes depending on the model used.^106[106] This line of research suggests that comparing AI agents’ and humans’ reasonableness judgments is already informative. These comparisons, in turn, might eventually form the basis for concluding that certain AI systems can be trusted to make legally relevant reasonableness judgments as well as humans can.

Yet there are also substantial reasons to doubt that the human baseline is the correct one.^107[107] “Assessing algorithms by reference to how reasonable people behave [may] set too low of a bar—AI can and should outperform humans on many tasks.”^108[108] This has led to suggestions for the development of a standard of care that incorporates AI agents’ unique (and possibly superhuman) competencies. One proposal would hold that an AI behaved unreasonably if it “causes injury more frequently . . . than the combined incident rate for all actors—both human and AI—engaged in the same type of conduct.”^109[109] Another proposal would interrogate both the reasonableness of the AI itself and the reasonableness of the AI developer: the more reasonably the AI itself behaved, the lower the standard of care imposed on its developer.^110[110]

Interplay Between AI Agents and Principals

“Two-pronged” tort standards that consider the reasonableness of both the AI itself and its developer point to a broader theme: the interplay between AI agents and their principals. Indeed, one way of motivating LFAI and related proposals is to ask: what is the optimal allocation of legal responsibility among the various actors in the chain of causation (e.g., AI developers, deployers, users, and AI itself)? 

Another discussion within this theme is the extent to which fiduciary principles can accomplish much of what LFAI aims to achieve.^111[111] Participants overwhelmingly agreed that AI chatbots satisfied each of the elements traditionally used to justify imposition of fiduciary duties.^112[112] This suggests a strong pro tanto case for treating the developers and providers of AI systems as fiduciaries of their users^113[113]—or treating AI systems as fiduciaries themselves.^114[114] 

Of course, since fiduciary law is law, we might consider fiduciary AIs as one type of law-following AI. Indeed, fiduciary duties are sometimes understood as entailing an obligation to comply with positive law.^115[115] But one might extend the fiduciary AI concept further and argue that AI systems with a fiduciary duty to a sufficiently broad set of stakeholders (possibly including broadly conceived stakeholders like “the general public” or even “the Constitution”) might, if charged with balancing those duties, achieve the goals of LFAI more effectively, while also reflecting a richer and more nuanced set of considerations than LFAI would allow.^116[116] The thought here is that agentic AI systems are in fact enmeshed in a network of principal–agent relationships much more complex than the simple principal–agent relationship that LFAI imagines.^117[117] According to this argument, a networked approach to understanding an AI system’s duties—and the inherent conflicts and tensions therein—is more realistic than the command-and-control approach to legal compliance that LFAI imagines.^118[118]

It is also worth considering how AI agents’ “soft skills” will affect how AI agents are actually deployed. In particular, real-world lawyering is more than just logical reasoning: empathizing with clients is a core skill for many forms of legal work.^119[119] Accordingly, legal AI systems that use empathetic language with their users are perceived as more helpful and trustworthy.^120[120] This could have numerous implications for LFAI. For example, designers of LFAIs may wish to engineer LFAIs to empathetically deescalate the situation when the user requests that the AI break the law on their behalf. AI developers already have to steer clear of both “overrefusal” (which can degrade user experience and utility without supplying any safety benefit)^121[121] and “underrefusal” (which can lead to the AI enabling behavior that is foreseeably destructive to the user or others).^122[122] How well AI developers navigate this tradeoff for illegal requests may significantly influence whether LFAI is perceived by stakeholders as a legitimate and reasonable constraint.^123[123]

Evaluating AI Mental States

Many laws have a mental state as an element. To be able to say that an AI agent violated the law, we likely need some way of evaluating whether it acted with the requisite mental state.^124[124] To that end, the workshop featured a session on different approaches to thinking about AI intentionality.^125[125]

Drawing on a distinction in philosophy of mind, a session at the workshop distinguished between realist and interpretivist approaches to inferring or imputing mental states.^126[126] Realists try to assess the subject’s actual mental state as subjectively experienced by that subject.^127[127] Interpretivists, on the other hand, conclude that the subject’s mental state is that mental state that best serves as a “coherent explanation for future behavior.”^128[128] Interpretivism is often associated with Daniel Dennett’s “intentional stance”:^129[129] “the strategy of prediction and explanation that attributes beliefs, desires, and other ‘intentional’ states to systems—living and nonliving—and predicts future behavior from what it would be rational for an agent to do, given those beliefs and desires.”^130[130]

When polled on their views as to the appropriate approach for humans, workshop participants were evenly split between realism and interpretivism.^131[131] For AI systems, by contrast, most participants endorsed some form of interpretivism.^132[132] Views on the latter varied widely, of course, with many participants remaining skeptical of attributing mental states to AI systems.^133[133] 

Legal systems around the world are already grappling with AI mental states, and how they relate to the mental states of AI systems’ developers and users. The European Commission’s Guidelines on Prohibited AI Practices forbids “purposefully manipulative techniques,” including “AI systems that manipulate individuals without any human intending them to do so.”^134[134] The district court in Garcia v. Character Technologies, Inc.,^135[135] in which a chatbot is alleged to have caused a teen’s suicide,^136[136] had to confront the question of how AI speech related to developers’ intentions. The court was asked to hold that the chatbot’s output was protected speech.^137[137] It declined to do so,^138[138] giving serious weight to the proposition that protected “speech” must come from “a human being with First Amendment rights [who has] made an inherently expressive ‘choice’” to publish that content.^139[139] We should expect questions of AI mental states to become increasingly pressing for courts and litigants as AI systems become increasingly capable of generating content and taking actions that could give rise to legal actions. 

Legal Status for AI Agents

The Law-Following AI article proposes that LFAIs be treated as a new type of legal entity: a “legal actor” on which the law imposes duties but not rights.^140[140] A number of conversations at the workshop explored the implications of this aspect of LFAI. A recurring concern with this proposal was that it was obfuscatory, possibly deflecting attention away from the developers of AI systems.^141[141] 

But there was also skepticism about the legal actor proposal from the other direction, holding that it did not go far enough. One repeated concern was that giving LFAIs legal duties without legal rights of some sort would be unworkable: to fairly adjudicate whether an AI system violated a legal duty, we might need to give that AI agent some procedural (e.g., the right to counsel, the right to appeal) and substantive (e.g., rights that give rise to an affirmative defense) rights.^142[142] This seems correct. Future work on LFAI should more carefully explore which rights AI agents will need to be given if we are to impose legal duties on them. Nevertheless, it still seems possible to give AI agents many fewer rights than most legal persons.^143[143]

An even more radical set of proposals argued for giving AI agents a more fulsome set of private-law rights.^144[144] The thought is that granting AI agents secure property rights could encourage positive-sum trade between humans and AI agents with their own misaligned goals, thus mitigating the risk of conflict between them.^145[145] Since LFAI relies, in part, on ensuring that AI agents are aligned,^146[146] these proposals might be seen as an alternative to LFAI, in case LFAI is not timely and effectively implemented.

Case Study: Law-Following AI and International Humanitarian Law

LFAI owes a significant intellectual debt to the rich scholarly literature about whether and how AI-enabled systems, especially autonomous weapons, might be made to comply with international humanitarian law (IHL).^147[147] Autonomous weapons present a rich domain for work on LFAI for obvious reasons. Assessments in identifying proper targets, for example, may reflect the pinnacle of high-stakes legal analysis. Autonomous weapons systems with excessively permissive targeting standards will impose unnecessary harm to civilian persons and objects. On the other hand, an excessively cautious approach to automated IHL compliance could have catastrophic tactical consequences, such as an autonomous weapon system erroneously refusing to engage a lawful target, thereby endangering human allies relying on its functionality. 

Many States have already endorsed LFAI-like principles for autonomous weapons, through instruments such as the Political Declaration on Responsible Military Use of Artificial Intelligence and Autonomy, which declares that “use of AI in armed conflict must be in accord with States’ obligations under international humanitarian law, including its fundamental principles.”^148[148] Yet, IHL rules and principles are unmistakably difficult to formally represent. For example, IHL’s proportionality principle hinges on an assessment of “military advantage” a concept for which it has been asserted that “no abstract calculations [are] possible.”^149[149] Likewise, the scope of behaviors that render civilians “direct participants” in hostilities and thus lawful targets is often considered by States to be “undefined and largely undefinable.”^150[150] These difficulties and tensions are emblematic of the challenges posed by the introduction of LFAs for governmental functions more generally.

One session at the Workshop explored the implications of this literature for LFAI, especially in light of the current wave of autonomous weapons being deployed in ongoing conflicts.^151[151] One key background observation is that IHL primarily regulates the use of weapons, rather than their design and development.^152[152] For example, the International Court of Justice has held that the legality of nuclear weapons—the most destructive technology ever created by humans—rested primarily on the context of their use rather than the intrinsic destructiveness of their design.^153[153] There are, of course, exceptions. “Weapons that are, by their nature, indiscriminate” can be proscribed on the basis of their design.^154[154] An example of this is biological weapons, which by their nature cannot be used consistent with the principle of distinction.^155[155] Another example is weapons that are designed such that they cause superfluous injury or unnecessary suffering in contravention of the principle of necessity, such as poisoned weapons.^156[156] Finally, states must assess weapons for legality prior to deployment, accounting for both intended and foreseeable misuse.^157[157]

These distinctions are relevant to LFAI insofar as LFAI primarily proposes that AI agents be regulated through design rather than their use.^158[158] While there is ample precedent for such design-based regulation within IHL, IHL, like most of international law,^159[159] typically defaults to technology-neutral rules of state conduct. However, there is reason to believe that some of the distinctive technical properties associated with AI—such as model opacity, emergent behavior, vulnerability to counter-AI, and propensity to systematic error—create obstacles to IHL compliance that cannot be effectively neutralized at the point of use. This suggests that LFAI would fit most comfortably in IHL in circumstances where reliance on use regulations cannot sufficiently ensure compliance with IHL when considering a State’s envisioned use of the AI systems in question.^160[160] It seems plausible that, given AI agents’ high degree of autonomy and deploying States’ limited supervisory capabilities, it may indeed be hard to achieve such assurances with use-based restrictions alone, therefore possibly justifying design-based regulation like LFAI.^161[161]

Conclusion

As the above discussion reveals, the LFAI research agenda still contains many open questions. Many of these questions relate to the proper scope of LFAI: When can other tools address the harms LFAI aims to prevent? Would LFAI exhibit the same flaws as prior attempts at automated enforcement? Does LFAI objectionably deflect responsibility from the developers of AI systems? Other questions relate to the design of LFAIs and related policies: To what extent is symbolic representation of legal rules necessary in the age of LLMs? How would a “reasonable” AI agent behave? What is the exact bundle of duties and rights we should give to LFAIs?

The workshop did not generate widespread consensus on answers on these questions. But that is to be expected. LFAI, at its most ambitious, contemplates the restructuring of the legal order to account for the introduction of a powerful new type of actor. Such a restructuring is bound to raise thorny questions, many of which will have no obvious answer.

Nevertheless, the overall tenor of discussion at the workshop was optimistic. The questions identified may be difficult, but participants generally felt that the topic was important and generative. Society must address, one way or another, novel (or newly amplified) risks from AI agents. Designing AI agents to respect legal rules will almost certainly be one method for doing so; indeed, it already is.^162[162] As long as AI companies and policymakers continue to treat the law as a source of values for AI systems, scholars will need to grapple with the core questions that LFAI raises. This line of inquiry seems likely to, in the fullness of time, yield significant and actionable insights, just as scholarship on the nature and treatment of business entities has enabled market economies to capitalize on their enormous benefits while managing, however imperfectly, some of their most serious risks. The main uncertainty, then, is not whether further research on LFAI will be useful, but rather whether such insights will keep pace with advances in AI technology. It is our hope that, through events like the workshop and documents like this one, we can increase the odds that humanity will develop the ideas we need to safeguard human welfare and the rule of law in this transformative era.