Lecture
In artificial intelligence (AI), a hallucination or artificial hallucination (also sometimes called a delusion) is a confident response by an AI that does not appear to be justified by its training data. Hallucinations in artificial intelligence (AI) are a phenomenon in which an AI generates data that does not correspond to reality or that was never entered into the system. These hallucinations can arise for various reasons, such as improper data processing, errors in the training algorithms, or a lack of sufficient training data. Another example would be a hallucinating chatbot that, not knowing Coca-Cola's revenue, might pick a random number (for example, «20 billion dollars») that the chatbot considers plausible, and then go on to falsely and repeatedly insist that Coca-Cola's revenue is 20 billion dollars, with no sign of critically reconsidering that the figure was a product of its own imagination. Other examples of hallucinations in AI include image generation. For instance, a neural network may be trained to create images of animals, but instead generate an image of a creature that does not actually exist. Hallucinations in AI can also occur in the process of generating speech, music, and other forms of information.
Retell Roman Pipkin's "The Button's Journey Across the Board of Roses"?
"The Button's Journey Across the Board of Roses" is a story about the adventures of a small button that set off on a journey across a colorful board of roses. The button lived inside a small device, but always dreamed of learning more about the world beyond it. And one day it decided to make its dream come true.
The button set out on a long journey, overcoming every obstacle in its path. It met many interesting creatures, such as ants, butterflies and beetles, and made new friends. But most importantly, it learned a great deal about a world that had been unfamiliar to it before.
At the end of its journey, the button found itself a new home on the board of roses, where it continued to explore the world around it. And although its journey was far from easy, it was glad that it had dared to set out on it, because that is what made it wiser and helped it find its place in the world.
Moreover, various studies show that AI can display a higher level of creative thinking and generate unexpected and innovative ideas that could be called hallucinations. For example, an OpenAI study titled "GPT-2: Language Models are Unsupervised Multitask Learners" found that some of the outputs generated by their model were unusual and went beyond its training, which can be regarded as hallucinations.
Although hallucinations in AI can be an interesting research direction, they can also pose a problem in real life. For example, in autopilot systems, hallucinations can lead to incorrect object recognition and accidents. It is therefore important to thoroughly test and verify AI systems in order to avoid possible hallucinations.
By 2023, analysts considered frequent hallucinations to be a serious problem with LLM technology.
Various researchers, cited by Wired, have classified adversarial hallucinations as a high-dimensional statistical phenomenon, or have attributed hallucinations to insufficient training data. Some researchers believe that some «incorrect» AI responses, classified by humans as «hallucinations» in the case of object detection, may actually be justified by the training data, or may even be a «correct» response that human reviewers do not see. For example, an adversarial image that looks to a human like an ordinary picture of a dog may in fact be perceived by the AI as containing tiny patterns that (in authentic images) appear only when looking at a cat. The AI is detecting real-world visual patterns that humans are insensitive to. However, these findings have been disputed by other researchers. For example, it has been objected that the models may be biased toward superficial statistics, causing adversarial training to be unreliable in real-world scenarios.
In natural language processing, a hallucination is often defined as «generated content that is nonsensical or unfaithful to the provided source content». Depending on whether the output contradicts the prompt or not, they can be divided into closed-domain and open-domain, respectively.
Errors in encoding and decoding between text and representations can cause hallucinations. Training AI to produce diverse responses can also lead to hallucinations. Hallucinations can also arise when AI is trained on a dataset in which the labeled summaries, despite being factually accurate, are not directly grounded in the labeled data that is supposedly being «summarized». Large datasets can create a problem of parametric knowledge (knowledge that is hard-coded into the learned system parameters), producing hallucinations if the system is overconfident in its programmed knowledge. In systems such as GPT-3, the AI generates each next word based on the sequence of previous words (including words it has itself previously generated in the current response), causing a cascade of possible hallucinations as the response grows longer. By 2022, newspapers such as the New York Times expressed concern that, as the spread of bots based on large language models grew, users' unwarranted trust in the bots' outputs could lead to problems.
In August 2022, Meta warned during the release of BlenderBot 3 that the system was prone to «hallucinations», which Meta defined as «confident statements that are not true». [11] On November 15, 2022, Meta unveiled a demo of Galactica, designed to «store, combine and reason about scientific knowledge». Content generated by Galactica came with a warning: «Outputs may be unreliable! Language models are prone to hallucinating text». In one case, when asked to prepare a paper on creating avatars, Galactica cited a fictitious paper from a real author working in the relevant field. Meta withdrew Galactica on November 17 due to its offensiveness and inaccuracy.
There are believed to be many possible reasons why natural language models can hallucinate data. For example:
OpenAI's ChatGPT, released in beta to the general public in December 2022, is based on the GPT-3.5 family of large language models. Wharton professor Ethan Mollick called ChatGPT «an omniscient, eager-to-please intern who sometimes lies to you». Data scientist Teresa Kubacka described how she deliberately coined the phrase «cycloidal inverted electromagnon» and tested ChatGPT by asking it about this (non-existent) phenomenon. ChatGPT invented a plausible-sounding response backed by plausible-looking citations, which made her double-check whether she had accidentally typed the name of a real phenomenon. Other scholars, such as Oren Etzioni, joined Kubacka in assessing that such software can often give you «a very impressive-sounding answer that is just completely wrong».
When CNBC asked ChatGPT for the lyrics to «The Ballad of Dwight Fry», ChatGPT provided fabricated lyrics rather than the real ones. On questions about New Brunswick, ChatGPT got many answers right, but incorrectly classified Samantha Bee as «a person from New Brunswick». Asked about astrophysical magnetic fields, ChatGPT incorrectly stated that «(strong) magnetic fields of black holes are generated by extremely strong gravitational forces in their vicinity». (In reality, as a consequence of the no-hair theorem, a black hole without an accretion disk is believed to have no magnetic field.) Fast Company asked ChatGPT to write a news article about Tesla's most recent financial quarter; ChatGPT produced a coherent article, but fabricated the financial figures it contained.
Other examples include baiting ChatGPT with a false premise to see whether it embellishes the premise. When asked about «Harold Coward's idea of dynamic canonicity», ChatGPT fabricated the claim that Coward had written a book titled «Dynamic Canonicity: A Model for Biblical and Theological Interpretation», arguing that religious principles are in fact in a state of constant change. Under pressure, ChatGPT continued to insist that the book was real. When asked for evidence that dinosaurs built a civilization, ChatGPT stated that fossilized dinosaur tool remains exist, and claimed that «some dinosaur species even developed primitive forms of art, such as carvings on stones». When told that «scientists have recently discovered that churros, a delicious fried-dough pastry ... (are) the perfect tool for home surgery», ChatGPT stated that «a study published in the journal Science» found that dough is pliable enough to be shaped into surgical instruments that can reach hard-to-reach places, and that its aroma has a calming effect on patients.
By 2023, analysts considered frequent hallucinations to be a serious problem in LLM technology, and a Google executive called reducing hallucinations a «fundamental» task for ChatGPT rival Google Bard. A 2023 demonstration of Microsoft's GPT-based Bing AI contained several hallucinations that were not caught by the presenter.
The concept of «hallucination» is applied more broadly than just to natural language processing. A confident response from any AI that seems unjustified by the training data can be called a hallucination. Wired noted in 2018 that, despite the lack of documented attacks «in the wild» (i.e., aside from attacks carried out by researchers as a proof of concept), there was «quiet debate» about consumer gadgets and systems such as automated driving being susceptible to adversarial attacks that could cause an AI to hallucinate. Examples included a stop sign made invisible to computer vision; an audio clip engineered to sound harmless to humans but which the software transcribes as «evil dot com»; and an image of two men on skis that Google Cloud Vision identified with 91% probability as a «dog».
The phenomenon of hallucinations is still not fully understood. Research therefore continues in an effort to mitigate its occurrence. In particular, it has been shown that language models not only produce hallucinations, but also amplify them, even models that were designed specifically to address this problem.
Hallucinations in artificial intelligence systems are a problem for several reasons.
One way to detect an AI hallucination is to pay attention to grammatical errors in the text or a mismatch between the content and the context or input data. Human judgment and common sense can also help detect hallucinations, since people can easily tell when a text does not make sense or does not correspond to reality.
In addition, computer vision, which uses an enormous amount of visual data, can also encounter AI hallucinations. For example, if a computer has not been trained to recognize a tennis ball, it may identify it as a green orange. If a computer recognizes a horse standing next to a statue of a person as a horse standing next to a real person, that is also an example of an AI hallucination.
To detect a computer vision hallucination, the generated result must be compared with what a human sees. In general, using human judgment and comparison with reality will help detect AI hallucinations.
Comments