I have been evaluating the best practices for using AI and AI detection tools in writing, including how to approach plagiarism checks. In my research, I came across a recent article in the SMU Law Review Forum that offers a comprehensive list of bullet points on the standard of care. Proper attribution, disclosure, and transparency top their list.[1]
Similarly, this advice from an article in the Chronicle of Higher Education will sound familiar to law librarians: “The best way to use …AI tools responsibly is to constantly check sources…Researchers should also be transparent in their papers about when they’ve used AI, so readers can determine for themselves if they trust the information.”[2]
AI detection tools will probably fail us
Since traditional plagiarism detection and comparison tools are unreliable, I only use them as a starting point for traditional paper checks.[3] I follow up online checks with manual reviews. I firmly believe that a paper check requires careful reading by a human to be done correctly. Although I have sometimes used AI detection programs, they have problems similar to those of older tools. They are not dependable. Librarians are more reliable.
Given this lack of reliability with AI detection tools, I am not surprised that university administrators in the Big Ten decided not to make detection tools available in course management systems.
There are no ethical norms in AI design
According to a Pew study[4], no norm has been adopted for ethical AI design . Therefore, academics and scholars must develop standards on their own for how AI can be used in writing. Perhaps we need to adopt the Three Laws of Robotics.
The Fourth Law of Robotics
If, like me, you are a fan of science fiction and Isaac Asimov, then you know his Three Laws of Robotics. (You may also enjoy this interview from the BBC archives[5] in which he discusses their creation.) For non-science fiction fans, the laws are:
1. A robot may not harm a human being or through inaction allow a human to be harmed.
2. A robot must obey the orders given to it by human beings except where it might conflict with the first law.
3. A robot must protect its own existence if such protection doesn’t conflict with the first or second laws.
Of course, these laws have a problem: robots in the real world don’t follow codes. For example, yesterday my husband typed truck into his phone, which uses autocorrect, a grammar AI, and it suggested a swear word that rhymed with truck, which didn’t even fit into his sentence. A surprising choice? Maybe. Or perhaps autocorrect is part gremlin. But examples like an accidentally humorous autocorrect may explain why folks have suggested that we need a fourth law:
“Perhaps Asimov missed an essential Fourth Law: A robot must identify itself. We have the right to know if we’re interacting with a human or AI.”[6]
When AI is used, we have the right to know. Then we can decide whether to trust it. Intriguingly, in my search for publisher policies on AI in writing, I discovered that some are adopting affirmative disclosure requirements for author submissions. The Fourth Law, the transparent use of AI, has found a home in the world of publishing.
Meanwhile others, including some educators, are addressing AI use in writing by drawing a different line, NEVER.[7] They insist students and scholars cannot use AI.
“Never” as a policy seems challenging. I am not sure how well it will hold up. To me, a disclosure policy seems like a more reasonable compromise, placing the responsibility for the appropriate use of AI on the author.
It would certainly make my plagiarism check work easier if the author disclosed AI use in advance. I assume it would be a relief for editors as well. Detecting plagiarism is difficult, and generative AI in writing doesn’t help.
I’ll conclude with an article that examines the positive uses of AI in writing as well as the risks. It is a critical literature review that examined AI’s impact on a writer’s self-efficacy, discovering that AI tools “aid in overcoming writing challenges, inspiring creativity, and overcoming writer’s block.”[8] I hope that AI makes every scholar a better writer, that AI’s pitfalls are fixed, and my services as a plagiarism checker go completely out of fashion.
Sources
[1] Bill Tomlinson, Andrew W. Torrance and Rebecca W. Black, ChatGPT and Works Scholarly: Best Practices and Legal Pitfalls in Writing with AI, SMU Law Review Forum, 108, 117, (vol. 76, 2023).
[2] Maggie Hicks, No, ChatGPT Can’t Be Your New Research Assistant, Chronicle of Higher Education, (August 23, 2023), https://www.chronicle.com/article/no-chatgpt-cant-be-your-new-research-assistant
[3] The literature that evaluates paper check tools is often critical when examining the tools, and warns of the pitfalls of becoming dependent on the reports they generate.
[4] Experts doubt ethical AI Design Will be Broadly Adopted as the Norm Within the Next Decade, https://www.pewresearch.org/internet/2021/06/16/1-worries-about-developments-in-ai/
[5] Bing Videos– 1965: ISAAC ASIMOV’s 3 laws of robotics, Horizon, Past Predictions, BBC Archive.
[6] Jonny Thompson, 3 Rules for Robots from Isaac Asimov — And One Crucial Rule he Missed, April 9, 2023, https://bigthink.com/the-future/3-rules-for-robots-isaac-asimov-one-rule-he-missed/
[7] “AI-assisted technologies [such as large language models (LLMs), chatbots, and image creators] do not meet the Science journals’ criteria for authorship and therefore may not be listed as authors or coauthors, nor may sources cited in Science journal content be authored or coauthored by AI tools, Science Journals.” See, Editorial Policies, https://www.science.org/content/page/science-journals-editorial-policies#authorship
[8] Jerry W. Washington, The Impact of Generative Artificial Intelligence on Writer’s Self-Efficacy: A Critical Literature Review, SSRN, (Aug. 2023) https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4538043.