Potential inaccuracies, biases and plagiarism
Daniel Acuña, Associate Professor of Computer Science, University of Colorado Boulder
I am a regular user of GitHub Copilot, a tool for helping people write computer code, and I’ve spent countless hours playing with ChatGPT and similar tools for AI-generated text. In my experience, these tools are good at exploring ideas that I haven’t thought about before.
I’ve been impressed by the models’ capacity to translate my instructions into coherent text or code. They are useful for discovering new ways to improve the flow of my ideas, or creating solutions with software packages that I didn’t know existed. Once I see what these tools generate, I can evaluate their quality and edit heavily. Overall, I think they raise the bar on what is considered creative.
But I have several reservations.
One set of problems is their inaccuracies – small and big. With Copilot and ChatGPT, I am constantly looking for whether ideas are too shallow – for example, text without much substance or inefficient code, or output that is just plain wrong, such as wrong analogies or conclusions, or code that doesn’t run. If users are not critical of what these tools produce, the tools are potentially harmful.
Recently, Meta shut down its Galactica large language model for scientific text because it made up “facts” but sounded very confident. The concern was that it could pollute the internet with confident-sounding falsehoods.
Another problem is biases. Language models can learn from the data’s biases and replicate them. These biases are hard to see in text generation but very clear in image generation models. Researchers at OpenAI, creators of ChatGPT, have been relatively careful about what the model will respond to, but users routinely find ways around these guardrails.
Another problem is plagiarism. Recent research has shown that image generation tools often plagiarize the work of others. Does the same happen with ChatGPT? I believe that we don’t know. The tool might be paraphrasing its training data – an advanced form of plagiarism. Work in my lab shows that text plagiarism detection tools are far behind when it comes to detecting paraphrasing.
These tools are in their infancy, given their potential. For now, I believe there are solutions to their current limitations. For example, tools could fact-check generated text against knowledge bases, use updated methods to detect and remove biases from large language models, and run results through more sophisticated plagiarism detection tools.