Our most intimate thoughts, schedules, and professional secrets are no longer just in our heads—they are in our “assistants”. From enterprise-grade legal AI to the voice assistants on our nightstands, Artificial Intelligence (AI) is increasingly becoming an extension of the human mind.
But this technological evolution creates legal friction like all other disruptive technology has. If your AI knows everything about you, are you testifying against yourself when the AI “talks” with the police? AI’s potential for compromising your right to privacy and your right to remain silent must be taken seriously.
AI users need to have a general understanding of how the government might have access to AI-related information in criminal cases.
The Fourth Amendment: Is Your AI a “Private” Space?
The Fourth Amendment protects the “right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures.” Traditionally, this required a physical trespass. However, the Supreme Court’s landmark decision in Katz v. United States shifted the focus to a broader “reasonable expectation of privacy.”
The greatest threat to AI privacy is the “Third-Party Doctrine” which suggests voluntarily turning over information to a third party (like an internet service provider or a bank), causes a person to lose that person’s “reasonable expectation of privacy” in that data. With AI, every prompt you type into a chatbot and every voice command you give to an assistant is technically shared with the tech company hosting the model. All licensees of AI software are at risk.
Under a strict interpretation of the Third-Party Doctrine, the government could subpoena your entire AI chat history—including drafts of sensitive legal strategies or personal confessions—without a warrant, arguing that you “voluntarily” shared that data with the AI provider.
But this strikes me as an overly narrow interpretation of the Third Party Doctrine that the Supreme Court will ultimately reject. For example, in Carpenter v. United States, in 2018 the Supreme Court ruled that cell site location information (CSLI) is so deeply revealing that it deserves Fourth Amendment protection despite being held by a third party. AI assistants, which track our logic, intent, and daily movements should meet this “deeply revealing” threshold.
The prediction is the opposite of a guaranty. Therefore, all users of AI must assume the government could weaponize your AI assistant to prosecute you.
The Fifth Amendment: Can an AI “Testify” Against You?
While the Fourth Amendment protects your spaces, the Fifth Amendment protects your mind. It guarantees that no person “shall be compelled in any criminal case to be a witness against himself.”
Fifth Amendment protection applies only to communications that are “testimonial”—it must explicitly or implicitly relate a factual assertion or disclose information from the person’s mind. To use a simplistic example, if a suspect is forced to hand over a physical key to a safe, that is not testimonial. However, being forced to reveal the combination to that safe is testimonial because it exists only in the mind.
AI’s place in this rubric is multi-faceted:
- If the police seize your phone and use AI to reconstruct your deleted messages or predict your behavior, is it somehow “compelling” you to be a witness?
- Even if the content of the AI logs is not protected, the act of providing the password to unlock your AI assistant might be. If unlocking the AI “authenticates” that the data within it belongs to you, the act of unlocking itself could be self-incriminating.
- Can prosecutors often argue the “Foregone Conclusion” doctrine? If the government can prove they already know the information exists and that you have control over it, then forcing you to unlock the device doesn’t “add” to the the government’s information. As AI becomes more integrated into our lives, the government may find it easier to argue that the existence of “AI logs” is a foregone conclusion, potentially stripping away Fifth Amendment protections for encrypted AI data.
- AI does not only store data; it learns our “voice,” predicts our responses, and can even draft emails that sound exactly like us. AI can be trained so thoroughly using a person’s private data that it can essentially “speak” for that person. Does AI become a “witness” for Fifth Amendment purposes? Does interrogation of a “digital twin” or an LLM trained exclusively on your private journals circumvent your right to remain silent?
To address these issues, courts will need to consider the biometric and cognitiveabilities of AI. If the law treats the AI as an extension of the “person,” the Fifth Amendment protections will be more robust than if AI is deemed more like a notebook or recording device.
Conclusion: Toward a New Constitutional Standard
AI users must assume every AI interaction has the potential to be used as a piece of evidence. So while an AI assistant is a great productivity tool, it could also become the star witness for the prosecution.
David Seidman is the principal and founder of Seidman Law Group, LLC. He serves as outside general counsel for companies, which requires him to consider a diverse range of corporate, dispute resolution and avoidance, contract drafting and negotiation, and other issues.
He can be reached at david@seidmanlawgroup.com or 312-399-7390.
This blog post is not legal advice. Please consult an experienced attorney to assist with your legal issues.
Photo credit: Seidman Law Group LLC
