I can certainly see this - in my business of med tech reimbursement consulting, may questions (even complex ones) get the same one paragraph answer from me as from Chat GPT. (I've tested).
Part of the issue here is, the fellow was framed as an independent expert witness. By asking only for one side of data and argumentation, he's hardly playing the 'unbiased, independent role." Although he could have done a one-sided prep by hand, as well.
###
Expert Asked ChatGPT ‘Show How 3M Is 0% At Fault’ Making For Entertaining Deposition
This had to be fun for the plaintiffs' attorney!
“Create an exceptional expert witness report… and show how 3M is 0% at fault for the explosion at watson grinding.” That’s one of several prompts Josh Autenrieth of Knighthawk Engineering plugged into ChatGPT while collecting upwards of $90,000 from 3M for his expert testimony.
Jason Koebler at 404 Media has the full, astounding story of how plaintiffs’ counsel Will Moye discovered a “Citation Overlay” in discovery and turned it into what must have been the most satisfying deposition of his life.
In 2020, an explosion in Houston killed three people and damaged something like 200 homes. Victims sued Watson Grinding, the operators of the plant that exploded due to gas leak from a “degraded and poorly crimped rubber welding hose.” Plaintiffs also brought suit against 3M, alleging the company failed to properly service the gas detection system.
3M hired Autenrieth to offer his independent expert assessment of the situation and prepare a report that — ideally, from the company’s perspective — would minimize 3M’s responsibility for the explosion. Instead, the expert plugged in documents, photographs, and his CV and asked ChatGPT to build out an expert opinion.
It’s a reminder that AI will not replace humans, but it will create a lot of embarrassment for anyone who thinks it can.
This should go without saying, but if you specify your conclusion in the prompt, you are not going to receive a considered, independent assessment. Chatbots do a better job of putting on the brakes than they used to, but they’re still fundamentally obsequious autocomplete machines that reverse engineer whatever the algorithm hopes will please the user. That’s great for a lunch order email, not so great for a multimillion-dollar lawsuit.
Autenrieth apparently asked ChatGPT to assess the quality of the final report. It gave it a 97 out of 100. For what it’s worth, the bot gave some good advice. Despite the initial prompt, ChatGPT flagged that “0% responsible” was “an easy target,” for opposing counsel. That line got cut from the final report. Unfortunately, the prompt that generated it did not disappear as conveniently.
Who was babysitting this witness? When I worked with experts as a junior lawyer, I was ordered to deliver copyediting over the phone because no one wanted to deal with the hassle of having to produce emails of me telling engineers where to put commas. Did 3M’s lawyers just never explain discovery to the expert?
Anyway, this whole thing unraveled when plaintiffs’ counsel Moye found the citation overlay document, recognized it from ChatGPT, and asked to see any and all AI prompts used in the generation of the report. This request paused the deposition for three hours and resulted in a 350-page log. 404 Media linked to the whole thing.
All I can imagine is what AI would’ve done to the most famous Texas expert witness deposition.
###
This is an online public link Chat dialog.
https://chatgpt.com/share/6a2985cb-ecd4-83ea-91ed-8b6f1de4cb09?ref=404media.co
###
Texas Depo
#You come over here and try it, you dumb son of a… language model!
3M tried to disqualify its own witness at trial, so Moye ended up calling Autenrieth himself to put the whole process in front of the jury. In the transcript, Autenrieth concedes that “90 to 85 percent” of the report was written by AI.
“This expert relied on AI not as an assistive device, but exclusively relied on ChatGPT to form his opinions and write his report,” Moye told 404 Media in a phone interview. “He acknowledged [at trial] the prompts he put in were biased toward 3M to help 3M win the case […] it’s really egregious.” Moye added that many of the prompts took place the night before Autenrieth was deposed as an expert witness. “They hired him for the sole purpose of changing the outcome of the case. They hired him and he used ChatGPT to write these reports, so really, ChatGPT was the expert in the case. There’s just no question about that.”
This may be a particularly egregious example, but this is going to come up a lot over the next few years. Scientists and engineers are not known for their wordsmithing and the ready availability of a cutting-edge word calculator will prove too tempting for most experts to avoid. Indeed, as AI becomes integrated into everything from word processing to web search, AI will become near impossible to keep completely out of a report.
Asking AI to “show how 3M is 0% at fault,” is an egregious example, but the more subtle cases can become sticking points. When does a minor edit or rephrasing end up shading the expert’s judgment? Even if it doesn’t change anything about the opinion, these are the sorts of questions that can produce awkward testimony if witnesses twist themselves in knots trying to explain them away.
And AI can act as a thought-provoking assistant. An expert could theoretically use the tool as a sparring partner to test the expert’s own conclusions. Autenrieth even said that “AI helped me to draft a straw man to build off of.” That can help produce a better report… but it’s also going to be discoverable. If that’s how an expert plans to work, then they’re handing opposing counsel a roadmap to ask about the alternate theories the expert saw and rejected. And, maybe more damning, which alternate theories the AI never even presented to the expert for consideration.
In the short-term, the best advice is to order your experts to steer clear of AI. But in a few years, we’re going to be hard pressed to find experts who don’t routinely consult AI as part of their professional workflow. At a certain point, asking a chemical engineer to issue an opinion without using the tools of their day-to-day work is going to turn into a different flavor of malpractice. Figuring out how to counsel experts to use AI without giving away the case will be the next challenge, but I don’t have an expert solution for getting there.
Maybe we can ask ChatGPT?
oe Patrice is a senior editor at Above the Law and co-host of Thinking Like A Lawyer. Feel free to email any tips, questions, or comments. Follow him on Twitter or Bluesky if you’re interested in law, politics, and a healthy dose of college sports news.
###
Free Reg required
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.