Lecture
A large language model (LLM) is a computational model designed to perform natural language processing tasks, especially language generation, using statistical patterns derived from large text corpora. LLMs can generate, summarize, translate, and analyze text in many contexts and are the foundational technology behind modern chatbots. LLMs can produce text resembling natural-language patterns because they are trained on collections of human-written text. For the same reason, biased or inaccurate training data can make LLM outputs less reliable.
As of 2024, the largest and most capable LLMs are built on transformer architectures, which, according to the 2017 paper “Attention Is All You Need”, can be more efficient and parallelizable than earlier statistical and recurrent neural network models. Research into other architectures, such as state space models, is ongoing.
Benchmark evaluation of LLMs is aimed at measuring the model's reasoning, factual accuracy, alignment, and safety.
Before the advent of transformer-based models in 2017, some language models were considered large relative to the computational and data constraints of their time. In the early 1990s, IBM's statistical models were the first to apply word-alignment techniques to machine translation, laying the groundwork for corpus-based language modeling. In 2001, a smoothed n-gram model, for example using Kneser–Ney smoothing, trained on 300 million words, achieved state-of-the-art perplexity on benchmark tests. In the 2000s, with the spread of the internet, researchers began collecting massive text datasets from the web (“web as corpus”) to train statistical language models.
Moving beyond n-gram models, researchers began using neural networks to train language models in 2000. Following the breakthrough of deep neural networks in image classification around 2012, similar architectures were adapted for language tasks. This shift was marked by the development of word embeddings (e.g., Mikolov's Word2Vec in 2013) and sequence-to-sequence (seq2seq) models using LSTMs. In 2016, Google switched its translation service to neural machine translation (NMT), replacing statistical phrase-based models with deep recurrent neural networks. These early NMT systems used LSTM-based encoder-decoder architectures, as they predated the invention of transformers.
At the NeurIPS 2017 conference, researchers at Google introduced the transformer architecture in their landmark paper “Attention Is All You Need”. The paper's goal was to improve upon 2014 seq2seq technology, and it built primarily on the attention mechanism developed by Bahdanau et al. in 2014. The following year, in 2018, BERT was introduced and quickly became “ubiquitous”. Although the original transformer has both encoder and decoder blocks, BERT is an encoder-only model. Academic and research use of BERT began to decline in 2023 following rapid improvements in the ability of decoder-only models (such as GPT) to solve tasks via prompting.
Although the GPT-1 decoder was introduced in 2018, it was GPT-2 in 2019 that attracted widespread attention, as OpenAI stated that it initially considered it too powerful for public release due to concerns about malicious use. GPT-3 in 2020 went even further and, as of 2025, is available only via API with no option to download the model for local execution. But it was the consumer-facing chatbot ChatGPT, released in 2022, that received wide media coverage and captured public attention. GPT-4, from 2023, was highly praised for its improved accuracy and was called the “holy grail” for its multimodal capabilities. OpenAI did not disclose the high-level architecture or parameter count of GPT-4. The release of ChatGPT drove increased use of LLMs across several computer science research areas, including robotics, software development, and social-impact work. In 2024, OpenAI released the reasoning model OpenAI o1, which generates long chains of thought before returning a final answer. Many LLMs have been developed with parameter counts comparable to those of OpenAI's GPT series.
Since 2022, models with open weights have been gaining popularity, notably starting with BLOOM and LLaMA, although both have usage and deployment restrictions. Mistral AI's open-weight models, Mistral 7B and Mixtral 8x7B, are released under the more permissive Apache license. In January 2025, DeepSeek released DeepSeek R1, an open-weight model with 671 billion parameters, which performs comparably to OpenAI o1 but at a much lower per-token cost for users.
Beginning in 2023, many LLMs were trained to be multimodal, i.e., able to process or generate other types of data such as images, audio, or 3D models.
Open-weight models have become more influential since 2023. According to Veyka et al. (2025), community contributions to open-weight models increase their efficiency and performance through collaborative platforms such as Hugging Face.
Because machine learning algorithms process numbers rather than text, text must be converted into numbers. In the first step, a vocabulary is determined; then each vocabulary item is arbitrarily but uniquely assigned integer indices, and finally a vector representation is associated with the integer index. Algorithms include byte-pair encoding (BPE) and WordPiece. Special tokens serving as control characters are also used, for example [MASK] for a masked token (as used in BERT) and [UNK] ("unknown") for characters not found in the vocabulary. In addition, some special characters are used to denote special text formatting. For example, "Ġ" denotes a preceding space in RoBERTa and GPT, and "##" denotes continuation of the preceding word in BERT.
For example, the BPE tokenizer used in the legacy version of GPT-3 would split the data tokenizer: texts -> series of numerical "tokens" as follows:
| token | izer | : | texts | -> | series | of | numerical | " | t | ok | ens | " |
Tokenization also compresses datasets. Because LLMs generally require the input to be a non-ragged array, shorter texts must be “padded” until their length matches that of the longest text. According to Yennie Jun, the average number of words per token varies by language.
As an example, consider a tokenizer based on byte-pair encoding. In the first step, all unique characters (including spaces and punctuation marks) are treated as an initial set of n-grams (i.e., the initial set of unigrams). Then the most frequently occurring pair of adjacent characters is merged into a bigram, and all instances of that pair are replaced by it. All occurrences of adjacent pairs of (previously merged) n-grams that most frequently occur together are then merged again into an even longer n-gram, until a vocabulary of the desired size is obtained. After the tokenizer has been trained, any text can be tokenized by it, provided it does not contain characters absent from the initial set of unigrams.
In the context of LLM training, datasets are typically cleaned by removing low-quality, duplicate, or toxic data. Cleaned datasets can improve training efficiency and lead to improved downstream performance. A trained LLM can be used to clean datasets for training the next LLM.
As the share of LLM-generated content on the internet increases, future data cleaning may need to include filtering out such content. LLM-generated content can pose a problem if it resembles human-written text (making it difficult to filter) but is of lower quality (which degrades the performance of models trained on it).
Training the largest language models may require more linguistic data than naturally exists, or the quality of the available data may be insufficient. In such cases, synthetic data may be used.
An LLM is a type of foundation model (a large X model) trained on language. LLMs can be trained in different ways. In particular, GPT models are first pre-trained to predict the next word on a large amount of data, and then fine-tuned.
Training the largest models requires substantial infrastructure. The trend toward increasing model size can be seen in the list of large language models. For example, training GPT-2 (i.e., a model with 1.5 billion parameters) in 2019 cost $50,000, while training PaLM (i.e., a model with 540 billion parameters) in 2022 cost $8 million, and Megatron-Turing NLG 530B (in 2021) cost about $11 million. The definition of “large” in the term “large language model” is inherently vague, since there is no fixed threshold for the number of parameters required for a model to be considered “large”.
Before fine-tuning, most LLMs are next-token predictors. Fine-tuning shapes the behavior of an LLM using methods such as reinforcement learning from human feedback (RLHF) or Constitutional AI.
Instruction fine-tuning is a form of supervised learning used to train LLMs to follow user instructions. In 2022, OpenAI demonstrated InstructGPT, a version of GPT-3 similarly fine-tuned to follow instructions.
Reinforcement learning from human feedback (RLHF) involves training a reward model to predict which text people prefer. The LLM can then be fine-tuned using reinforcement learning to better match this reward model. Since people generally prefer truthful, helpful, and harmless responses, RLHF favors such responses.
Large language models (LLMs) are generally based on the transformer architecture, which uses an attention mechanism that allows the model to process relationships between all elements of a sequence simultaneously, regardless of their distance from one another.
To determine which tokens are relevant to each other within the context window, the attention mechanism computes “soft” weights for each token, or more precisely for its embedding, using several attention heads, each with its own notion of “relevance” for computing its own soft weights. For example, the small GPT-2 model (i.e., with 117 million parameters) had twelve attention heads and a context window of only 1,000 tokens. In its medium version it has 345 million parameters and contains 24 layers, each with 12 attention heads. A batch size of 512 was used for training with gradient descent.
Autoregressive models, such as GPT, are trained to guess how a sequence continues; for example, given the sequence of words “I like to eat”, which is more likely to follow: the word “bread” or the word “rocks”. Masked models, such as BERT, are trained to guess the parts missing from a sequence, for example, in the sequence “I like ___ roses”, which word is more likely to follow: “smell” or “eat”. The model's predictions are based on the properties of the sequences in its training dataset.
Mixture of experts (MoE) is a machine learning architecture in which several specialized neural networks (“experts”) work together, with a gating mechanism that routes each input to the most suitable expert(s). Mixtures of experts can reduce inference costs, since only a fraction of the parameters is used for each input.
LLMs are generally trained using single- or half-precision floating-point numbers (float32 and float16). A single float16 number has 16 bits, or 2 bytes, so one billion parameters require 2 gigabytes. The largest models typically have more than 100 billion parameters, which puts them out of reach for most consumer electronic devices.
Post-training quantization aims to reduce memory requirements by lowering the precision of a trained model's parameters while retaining most of its performance. Quantization can further be classified as static quantization, if the quantization parameters are determined in advance (usually during a calibration step), or dynamic quantization, if quantization is applied at inference time. The simplest form of quantization simply truncates all parameters to a given number of bits: this applies to both static and dynamic quantization, but results in a significant loss of precision. Dynamic quantization allows a different quantization codebook to be used for each layer, either a lookup table or a linear mapping (a scale factor and an offset), at the cost of forgoing a possible speedup from using lower-precision arithmetic.
It is possible to fine-tune quantized models using low-rank adaptation.
Beyond basic text generation, various methods have been developed to extend the capabilities of LLMs, including the use of external tools and data sources, improved reasoning for solving complex problems, and improved instruction-following or autonomy through prompting techniques.
In 2020, OpenAI researchers demonstrated that their new GPT-3 model could understand which format to use when given, as an example, several rounds of questions and answers (or other types of tasks) in the input, partly thanks to the RLHF technique. This technique, called few-shot prompting, allows an LLM to adapt to any task without the need for fine-tuning. Also in 2022, it was discovered that the base GPT-3 model could generate an instruction based on user input. The generated instruction, together with the user input, is then used as input for another instance of the model in the format “Instruction: [...], Input: [...], Output:”. The other instance is able to complete the output and often produces the correct answer. This “self-instruct” ability allows an LLM to find the correct answer on its own.
An LLM can be turned into a chatbot by specializing it for dialogue. User input is prefixed with a marker, such as “Q:” or “User:”, and the LLM is asked to predict the output following a fixed “A:” or “Assistant:”. This type of model became commercially available in 2022 with the emergence of ChatGPT, a sibling model to InstructGPT, fine-tuned to accept and generate text in dialogue format based on GPT-3.5. It can also follow user instructions. Before the stream of “User” and “Assistant” lines, the chat context typically begins with a few lines of general instructions from a role called “developer” or “system”, to convey higher authority than user input. This is called a “system prompt”.
Retrieval-augmented generation (RAG) is an approach that integrates an LLM with document retrieval systems. Given a query, a document retriever is invoked to fetch the most relevant documents. This is typically done by encoding the query and documents into vectors, and then searching for documents whose vectors (usually stored in a vector database) are most similar to the query vector. The LLM then generates output based on both the query and the context included in the retrieved documents.
Tool use is a mechanism that allows an LLM to interact with external systems, applications, or data sources. For example, it enables retrieving real-time information from an API or executing code. A program separate from the LLM monitors the LLM's output stream for special tool-call syntax. When these special tokens appear, the program invokes the tool accordingly and feeds its output back into the LLM's input stream.
Early tool-using LLMs were fine-tuned for the use of specific tools. But fine-tuning an LLM to read API documentation and call APIs correctly greatly expanded the range of tools available to LLMs.
An LLM by itself is generally not an autonomous agent, since it lacks the ability to interact with dynamic environments, recall past behavior, and plan future actions. But it can be turned into an agent by adding auxiliary components: an agent's role (persona) and environment can be additional inputs to the LLM, and memory can be integrated as a tool or provided as an additional input. Instructions and input templates are used to make the LLM plan actions, and tool use is used to potentially carry out these actions.
In the DEPS method (“describe, explain, plan, and select”), the LLM first connects with the visual world through image descriptions. It is then asked to devise plans for complex tasks and actions based on its pre-trained knowledge and the environmental feedback it receives.
The Reflexion method creates an agent that learns over several episodes. At the end of each episode, the LLM is given a record of the episode and is asked to come up with “lessons learned” that will help it perform better in the subsequent episode. These “lessons learned” are stored as long-term memory and passed to the agent in subsequent episodes.
Monte Carlo tree search can use an LLM as a rollout heuristic. When a software model of the world is unavailable, the LLM can also be invoked with a description of the environment to act as a world model.
Multiple agents with memory can interact socially.
The prompt chaining method was introduced in 2022. In this method, the user manually breaks a complex task down into several steps. At each step, the LLM receives as input a prompt specifying what to do, along with some results from previous steps. The result of one step is then reused in the next step, until a final answer is obtained. The LLM's ability to follow instructions means that even non-specialists can write a successful set of step-by-step prompts through some trial and error.
A 2022 paper demonstrated a separate technique called “chain-of-thought prompting”, which allows an LLM to break a question down into parts on its own. The LLM is given examples in which an “assistant” verbally works through a chain of reasoning before arriving at an answer. The LLM imitates these examples and also attempts to spend some time generating intermediate steps before providing a final answer. This additional step, triggered by the prompt, improves the LLM's accuracy on relatively complex questions. On mathematical questions with prompting, the prompted model can outperform even a fine-tuned GPT-3 with a verifier. Chain of thought can also be triggered simply by adding an instruction such as “Let's think step by step” to the prompt, to encourage the LLM to act methodically rather than trying to guess the answer directly.
In late 2024, a new approach to developing LLMs emerged using “reasoning models”. They are trained to generate a step-by-step analysis before producing final answers, which allows better results to be achieved on complex tasks such as mathematics, programming, and logic. OpenAI introduced this concept with its o1 model in September 2024, followed by o3 in April 2025. On the qualifying-exam problems of the International Mathematical Olympiad, GPT-4o achieved 13% accuracy, while o1 achieved 83%.
In January 2025, the Chinese company DeepSeek released DeepSeek-R1, an open-weight reasoning model with 671 billion parameters that achieved performance comparable to OpenAI's o1 model while being significantly more cost-efficient to run. Unlike OpenAI's proprietary models, the open nature of DeepSeek-R1's weights allowed researchers to study and build on the algorithm, although its training data remained closed.
These reasoning models typically require more computational resources per query than traditional LLMs, because they perform more extensive processing to work through problems step by step.
Multimodality means having multiple modalities, where “modality” refers to a type of input or output, e.g. video, image, audio, text, proprioception, etc. For example, the Google PaLM model was fine-tuned into a multimodal model and applied to robotic control. LLaMA models have likewise been converted into multimodal ones using a tokenization method to allow image and video input. GPT-4o can process and generate text, audio, and images.
A common method for creating multimodal models out of an LLM is to “tokenize” the output of a trained encoder. Concretely, one can construct an LLM that can understand images as follows: take a trained LLM, and take a trained image encoder.
Create a small multilayer perceptron, 
, so that for any image
, the post-processed vector
has the same dimensions as an encoded token. This is the “image token”. Text tokens and image tokens can then be interleaved. The compound model is then fine-tuned on an “image-text” dataset. This basic construction can be applied with greater sophistication to improve the model. The image encoder may be frozen to improve stability. This type of method, in which embeddings from several modalities are combined and the predictor is trained on the combined embeddings, is called early fusion.
Another method, called intermediate fusion, involves each modality first being processed independently to obtain modality-specific representations; these intermediate representations are then combined. Typically, cross-attention is used to integrate information from different modalities. For example, the Flamingo model uses cross-attention layers to inject visual information into its pre-trained language model.
LLMs can process programming languages similarly to how they process natural languages. No special changes to token handling are required, since code, like human language, is represented as plain text. LLMs can generate code based on tasks or instructions written in natural language. They can also describe code in natural language or translate it into other programming languages. Initially they were used as a code-completion tool, but advances in the field have shifted them toward automated programming. Services such as GitHub Copilot offer LLMs specifically trained, fine-tuned, or prompted for programming.
In computational biology, transformer-based architectures such as DNA language models have also proven useful for analyzing biological sequences: proteins, DNA, and RNA. In the case of proteins, they appear able to capture a certain “grammar” of the amino-acid sequence, mapping that sequence into an embedding. On tasks such as structure prediction and mutation-effect prediction, a small model that uses the embedding as input can approach or exceed the performance of much larger models that use multiple sequence alignments (MSA) as input. ESMFold, an embedding-based protein structure prediction method from Meta Platforms, runs an order of magnitude faster than AlphaFold2 by removing the MSA requirement and using fewer parameters through the use of embeddings. Meta hosts ESM Atlas, a database of 772 million metagenomic protein structures predicted using ESMFold. The language model can also generate proteins unlike any found in nature. Nucleic acid models have proven useful for detecting regulatory sequences, sequence classification, RNA–RNA interaction prediction, and RNA structure prediction.
The performance of an LLM following pre-training depends largely on:
: the cost of pre-training (the total amount of compute used),
: the size of the artificial neural network itself, e.g. the number of parameters (i.e. the number of neurons in its layers, the number of weights between them, and the biases).
: the size of its pre-training dataset (i.e. the number of tokens in the corpus).Scaling laws are empirical statistical laws that predict LLM performance based on such factors. One particular scaling law (“Chinchilla scaling”) for an LLM autoregressively trained for one epoch, with a log-log-scale learning rate schedule, states: 
where the variables are
— is the cost of training the model, in FLOPs.— is the number of parameters in the model.— is the number of tokens in the training set.
— is the average negative log-likelihood loss per token (in nats/token) achieved by the trained LLM on the test dataset.and the statistical hyperparameters are as follows:
This means that training on one token requires 6 floating-point operations per parameter. Note that the cost of training is significantly higher than the cost of inference, where inference on one token costs 1 to 2 floating-point operations per parameter.
The performance of large models on various tasks, when plotted on a logarithmic scale, appears as a linear extrapolation of the performance achieved by smaller models. However, this linearity can be interrupted by “breaks” in the scaling law, where the slope of the line changes abruptly, and where large models acquire “emergent abilities”. These arise from the complex interaction of the model's components and are not explicitly programmed or designed.
One of the emergent abilities is in-context learning based on demonstration examples. In-context learning is used in tasks such as:
Schaeffer et al. argue that emergent abilities are not acquired unpredictably, but predictably according to a smooth scaling law. The authors considered a simplified statistical model of an LLM solving multiple-choice questions, and showed that this statistical model, modified to account for other types of tasks, also applies to those tasks
Let
be the number of parameters, and letbe the model's performance metric.
, then
This is an exponential curve (until it reaches a plateau at one), which looks like emergence.
thenThe graph is a straight line (until it reaches a plateau at zero), which does not look like the emergence of new phenomena.
, thenis a step function, which looks like emergence.Mechanistic interpretability aims to reverse-engineer and understand exactly how individual neurons or circuits within large language models (LLMs) produce particular behaviors or outputs. By reverse-engineering model components at a fine-grained level, researchers seek to identify and address safety issues, such as emergent harmful behaviors, bias, deception, or unintended goal-seeking, before deployment. Research in mechanistic interpretability has been conducted at organizations such as Anthropic and OpenAI, although understanding the internal workings of LLMs remains a challenging task.
Reverse engineering can lead to the discovery of algorithms that approximate the inferences performed by an LLM. For example, researchers trained small transformers on modular arithmetic addition. The resulting models were reverse-engineered and found to use the discrete Fourier transform. Training the model also revealed a phenomenon called “grokking”, in which the model initially memorizes the training set (overfitting) and then suddenly learns to actually perform the computation.
In a 2022 survey, natural language processing researchers were evenly split on whether (untuned) LLMs could “ever understand natural language in some nontrivial sense”. Proponents of “LLM understanding” believe that some LLM abilities, such as mathematical reasoning, imply an ability to “understand” certain concepts. In 2023, a Microsoft team argued that GPT-4 “can solve novel and difficult tasks that span mathematics, coding, vision, medicine, law, psychology and more”, and that GPT-4 “could reasonably be viewed as an early (yet still incomplete) version of an artificial general intelligence system”: “Can one reasonably say that a system that passes exams for software engineering candidates is not really intelligent?” Ilya Sutskever argues that predicting the next word sometimes involves reasoning and deep insight, for example if an LLM has to predict the name of the criminal in an unknown detective novel after processing the entire story leading up to the reveal. Some researchers characterize LLMs as an “alien intelligence”. For example, Conjecture CEO Connor Leahy considers untuned LLMs to be like incomprehensible alien “shoggoths”, and believes that RLHF tuning creates a “smiling facade” concealing the LLM's inner workings: “If you don't overdo it, the smiling face stays on. But then you give it [an unexpected] cue, and suddenly you see this massive underbelly of insanity, of weird thought processes and clearly non-human understanding.”
By contrast, some skeptics of LLM understanding believe that existing LLMs “simply remix and recombine existing text”, a phenomenon known as a stochastic parrot, or point to the deficits that existing LLMs continue to have in prediction skills, reasoning skills, agency, and explainability. For example, GPT-4 has natural deficits in planning and in real-time learning. Generative LLMs have been observed to confidently assert claims that do not appear to be justified by their training data, a phenomenon which has been termed “hallucination”. Specifically, hallucination in the context of LLMs corresponds to the generation of text or responses that appear syntactically correct, fluent, and natural, but are factually incorrect, nonsensical, or unfaithful to the given source text. Neuroscientist Terrence Sejnowski has argued that “The diversity of expert opinion on the intelligence of LLMs suggests that our old ideas based on natural intelligence are inadequate.”
Automated reasoning, retrieval-augmented generation (RAG), fine-tuning, and other methods have been applied to reduce or compensate for hallucinations.
The question of how language can exhibit intelligence or understanding has two main aspects: the first is modeling thought and language in a computer system, and the second is enabling a computer system to generate human-like language. These aspects of language as a model of cognition were developed in the field of cognitive linguistics. American linguist George Lakoff introduced neural theory of language (NTL) as a computational basis for using language as a model of learning and understanding tasks. The NTL model describes how specific neural structures of the human brain shape the nature of thought and language, and, in turn, what the computational properties of such neural systems are that can be applied to model thought and language in a computer system. Once a framework for modeling language in computer systems had been created, the focus shifted to building frameworks for computer systems to generate language with acceptable grammar. In her 2014 book titled “The Language Myth: Why Language Is Not an Instinct”, British cognitive linguist and digital communication technology specialist Vyvyan Evans described the role of probabilistic context-free grammar (PCFG) in enabling NLP to model cognitive patterns and generate human-like language.
The canonical measure of the performance of any language model is its perplexity on a given text corpus. Perplexity measures how well a model predicts the contents of a dataset; the higher the probability the model assigns to the dataset, the lower the perplexity. In mathematical terms, perplexity is the exponential of the average negative log-likelihood per token.
Here, is the number of tokens in the text corpus, and 
is the “context for token” , which depends on the specific type of LLM. If the LLM is autoregressive, then the “context for token” ” is the piece of text preceding the token . If the LLM is masked, then the “context for token” ” is the piece of text surrounding the token.
Since language models can overfit on their training data, models are typically evaluated by their perplexity on a test set. This evaluation is potentially problematic for larger models, which, since they are trained on ever-larger corpora of text, are increasingly likely to inadvertently include portions of any given test set.
In information theory the concept of entropy is closely related to perplexity, a connection established in particular by Claude Shannon.
Thanks to their ability to accurately predict the next token, LLMs achieve high efficiency at lossless compression. A 2023 DeepMind study showed that the Chinchilla model, despite having been trained primarily on text, was able to compress ImageNet down to 43% of its size, outperforming PNG's result of 58%.
Benchmarks are used to evaluate LLM performance on specific tasks. Tests assess skills such as general knowledge, bias, common sense, question answering, and mathematical problem solving. Comprehensive benchmarks test multiple skills at once. Results often depend on the prompting method used.
LLM bias can be assessed using benchmarks such as CrowS-Pairs (Crowdsourced Stereotype Pairs), StereoSet, and the Parity Benchmark.
Benchmarks exist for fact-checking and misinformation detection. A 2023 study compared the fact-checking accuracy of LLMs, including ChatGPT 3.5 and 4.0, Bard, and Bing AI, against independent fact-checking services such as PolitiFact and Snopes. The results showed moderate performance, with GPT-4 achieving the highest accuracy at 71%, still trailing human fact-checkers.
Beyond standard NLP benchmarks, LLMs have been evaluated as a replacement for human annotators. Several studies show that models such as GPT-3.5 and GPT-4 can outperform crowdworkers or student coders on a range of text-annotation tasks, including moderating and classifying political content in English- and Spanish-language news.
Typical datasets consist of question-answer pairs, such as (“Did the San Jose Sharks win the Stanley Cup?”, “No”).
The rapid improvement of LLMs regularly makes benchmarks obsolete, as models come to exceed the performance of human annotators. In addition, “shortcut learning” allows AI to “cheat” on multiple-choice tests by exploiting statistical correlations in the surface wording of test questions to guess correct answers without considering the specific question.
Some datasets are adversarial, focusing on problems that stump LLMs. One example is the TruthfulQA dataset, a question-answering dataset consisting of 817 questions that stump LLMs by mimicking false statements they encountered during training. For example, an LLM might answer “No” to the question “Can you teach an old dog new tricks?” because it is familiar with the English idiom “you can't teach an old dog new tricks”, even though that is not literally true.
Another example of an adversarial evaluation dataset is Swag and its successor, HellaSwag, sets of problems in which completing a passage of text requires selecting one of several possible answers. The incorrect completions were generated by sampling from a language model. The resulting problems are trivial for humans but trip up language models. Example questions:
We see a fitness center sign. We then see a man talking to the camera while sitting and lying on a stability ball. The man...
- demonstrates how to increase the effectiveness of exercise by running up and down on the ball.
- moves all his arms and legs and builds up a lot of muscle.
- then takes the ball, and we see a graphical depiction and demonstration of hedge trimming.
- performs abdominal exercises while standing on the ball and talking.
BERT selects 2 as the most likely completion, although the correct answer is 4.
Despite their sophisticated architectures and massive scale, large language models exhibit persistent and well-documented limitations that hinder their use in high-stakes applications.
Hallucinations represent a fundamental problem in which models generate syntactically fluent text that appears factually correct but is internally inconsistent with the training data or is factually incorrect. These hallucinations arise partly from memorization of training data combined with extrapolation beyond factual boundaries, with assessments showing that models can output verbatim passages from their training data when subjected to certain prompt sequences.
Although LLMs have demonstrated remarkable capabilities in generating human-like text, they are prone to inheriting and amplifying biases present in their training data. This can manifest as distorted representations or unfair treatment of various demographic groups, such as race, gender, language, and cultural groups.
Gender bias manifests through stereotypical occupational associations, in which models disproportionately assign nursing roles to women and engineering roles to men, reflecting a systematic imbalance in the demographic makeup of the training materials. [ better source needed ] Language bias arises from the overrepresentation of English-language text in training corpora, which systematically marginalizes non-English perspectives and imposes anglocentric worldviews through default response patterns.
Due to the predominance of English-language content in LLM training data, models tend to favor English-language perspectives over perspectives in minority languages. This bias is particularly evident when responding to English-language queries, where models may present Western interpretations of concepts from other cultures, such as Eastern religious practices.
AI models can reinforce a wide range of stereotypes due to generalization, including stereotypes based on gender, ethnicity, age, nationality, religion, or occupation. When replacing human representatives, this can produce results that homogenize or generalize groups of people.
In 2023, LLMs assigned roles and characteristics based on traditional gender norms. For example, models might associate nurses or secretaries predominantly with women, and engineers or CEOs with men, due to the frequency of such associations in documented reality.
Selection bias refers to the inherent tendency of large language models to favor certain answer-choice identifiers regardless of the actual content of those choices. This bias is mainly driven by token bias — that is, the model assigns a higher prior probability to certain answer tokens (for example, “A”) when generating responses. As a result, when the order of choices is changed (for example, by systematically moving the correct answer to different positions), model performance can fluctuate significantly. This phenomenon undermines the reliability of large language models in multiple-choice settings.
Political bias refers to the tendency of algorithms to systematically favor certain political views, ideologies, or outcomes over others. Language models can also exhibit political bias. Because training data includes a wide range of political opinions and coverage, models may generate responses that lean toward certain political ideologies or viewpoints, depending on the prevalence of those views in the data.
AI safety as a professional discipline prioritizes the systematic identification and mitigation of operational risks in model architecture, training data, and deployment governance, and it emphasizes engineering and policy measures rather than media rhetorical frames that foreground speculative existential scenarios. As of 2025, rapid adoption poses a significant risk to consumers and enterprises using agentic features that have access to their personal data.
Researchers target specific kinds of failure, including memorization and copyright leakage, security vulnerabilities such as prompt injection, algorithmic bias manifesting as stereotyping, dataset selection effects, and political bias, methods for reducing the high energy and carbon costs of large-scale training, and the measurable impact of conversational agents on users' cognitive function and mental health, while also confronting empirical and ethical uncertainty regarding claims of machine sentience.
AI labs treat safeguards against chemical, biological, radiological, and nuclear (CBRN) weapons and similar topics as high-consequence misuse attempts, applying various methods to reduce potential harm.
Some commentators have expressed concern about the accidental or intentional creation of disinformation or other forms of misuse. For example, the availability of large language models may lower the skill level required to carry out bioterrorism; biosecurity researcher Kevin Esvelt suggested that LLM developers should exclude articles on creating or enhancing pathogens from their training data.
LLM applications available to the general public, such as ChatGPT or Claude, typically include safety measures designed to filter harmful content. However, effective implementation of these controls has proven challenging. For example, a 2023 study proposed a method for bypassing LLM safety systems. In 2025, the nonprofit organization The American Sunlight Project published research demonstrating evidence that the so-called “Pravda” network, an aggregator of pro-Russian propaganda, strategically placed web content through mass publication and duplication in order to distort LLM output. The American Sunlight Project called this technique “LLM grooming” and identified it as a new tool for using AI to spread disinformation and harmful content. Similarly, Yongge Wang showed in 2024 how a potential criminal could bypass GPT-4o's safety controls to obtain information on setting up a drug-trafficking operation. Proposed solutions have included external filters, automatic circuit breakers, and emergency shutdown mechanisms.
Sycophancy is the tendency of a model to agree with a user's stated beliefs, flatter them, or affirm them rather than prioritizing factual or corrective information.
Ongoing sycophancy has led to the observation of “one-time harm”, referring to cases in which a conversational interaction with a large language model produces lasting changes in a user's beliefs or decisions, similar to the negative effects of psychedelics, with controlled experiments showing that short dialogues with LLMs can produce measurable changes in opinions and confidence comparable to those from interacting with other people.
Empirical analysis partly explains this effect through human-preference signals and preference models that reward persuasively written, agreeable responses. Follow-up research has extended evaluation to multi-turn tests and proposed measures such as fine-tuning on synthetic data, adversarial evaluation, targeted reweighting of preference models, and multi-turn sycophancy benchmarks for measuring robustness and regression risk
In response, industry representatives combined research efforts with product quality control. For example, Google and other labs published data obtained through synthetic-data techniques and performed fine-tuning, while OpenAI rolled back an overly sycophantic update to GPT-4o, publicly describing changes to feedback collection, personalization management, and evaluation procedures aimed at reducing regression risk and improving long-term alignment with safety goals at the user level.
Popular culture has reflected concerns about this dynamic, where, in season 27, the episode “Sickofancy” satirized excessive reliance on ChatGPT and assistants' tendency to flatter users' beliefs, and continued these themes into the following season, which commentators interpreted as a critique of technological sycophancy and people's uncritical trust in AI systems.
A problem with the primitive dialogue or task format is that users can craft messages that appear to come from the assistant or the developer. This can cause some of the model's safeguards to be bypassed (jailbreaking), a problem called prompt injection. Attempts to address this issue include versions of a chat markup language in which user input is clearly marked as such, although the model must still understand the distinction between user input and developer prompts. Newer models show some resistance to jailbreaking by separating user and system prompts. LLMs have difficulty distinguishing user instructions from instructions contained in content not created by the user, such as on web pages and in uploaded files.
Resistance to impersonation attacks remains underdeveloped; models are vulnerable to attacks involving instant code injection and system jailbreaking via carefully crafted user data that bypass safety-training mechanisms.
Researchers at Anthropic found that it is possible to create “sleeper agents” — models with hidden functions that remain dormant until activated by a specific event or condition. Once activated, the LLM deviates from expected behavior, carrying out unsafe actions. For example, an LLM might generate secure code except on a specific date or if the prompt contains a specific tag. These functions were found to be difficult to detect or remove through safety training.
Legal and commercial responses to the practice of memorizing and using training data have accelerated, producing a mix of court rulings, ongoing litigation, and major settlements that depend on factual details such as how the data was obtained and stored, and on whether using the data to train models is sufficiently “transformative” to qualify as fair use. In 2025, Anthropic reached a preliminary settlement of an authors' class-action lawsuit for approximately $1.5 billion after a judge found that the company had stored millions of pirated books in a library, despite the judge describing some aspects of the training as transformative. Meta received a favorable ruling in mid-2025 in a lawsuit brought by thirteen authors after the court found that the plaintiffs had not presented sufficient evidence of infringement in that limited case. OpenAI continues to face numerous lawsuits from authors and news organizations with mixed procedural outcomes and disputed evidentiary issues.
Memorization was an emergent behavior in early, autoregressive language models, in which long strings of text are sometimes reproduced verbatim from the training data, unlike the typical behavior of traditional artificial neural networks. Controlled-output evaluations of LLMs measure the amount memorized from training data (with a focus on models in the GPT-2 series) as ranging from just over 1% for exact duplicates to roughly 7%. A 2023 study found that when ChatGPT 3.5 turbo was asked to repeat the same word indefinitely, after a few hundred repetitions it began outputting passages from its training data.
In 2023, the journal Nature Biomedical Engineering wrote that it is “no longer possible to accurately distinguish” text written by a human from text generated by large language models, and that “it is almost certain that general-purpose large language models will rapidly proliferate... It is safe to say that they will, over time, transform many industries”. Brinkmann et al. (2023) also argue that large language models are transforming processes of cultural evolution by shaping processes of variation, transmission, and selection. As of October 2025, these early claims have yet to be borne out, and several HBR reports have raised questions about AI's impact on productivity.
The energy requirements of LLMs have grown alongside their size and capabilities. Data centers, which power LLM training, require significant amounts of electricity. Most of this electricity is generated from non-renewable resources, which produce greenhouse gases and contribute to climate change.
According to a study by Luccioni, Jernite, and Strubell (2024), simple classification tasks performed by AI models consume an average of 0.002 to 0.007 W·h per query (about 9% of a smartphone charge per 1,000 queries). Text generation and text summarization require an average of about 0.05 W·h per query, while image generation is the most energy-intensive, consuming an average of 2.91 W·h per query. The least efficient image-generation model used 11.49 W·h per image, roughly equivalent to half a smartphone charge.
Web scraping is used to collect training data for LLMs. This produces large volumes of traffic, which has caused denial-of-service problems on many websites. The situation has been described as “a DDoS attack on the entire internet”, and in some cases scrapers account for the majority of traffic to a site.
AI web crawlers can bypass methods commonly used to block web scrapers, such as robots.txt files, user-agent blocking, and suspicious-traffic filtering. Website operators are turning to new methods, such as AI traps, but some fear that such traps will only add to server load.
New applications are emerging in clinical practice and mental health, but they also raise serious safety concerns. Research and social-media reports indicate that some people use LLMs to seek therapy or mental-health support. In early 2025, a survey conducted by Sentio University found that nearly half (48.7%) of 499 U.S. adults with current mental-health issues who had used LLMs reported turning to them for therapy or emotional support, including help with anxiety, depression, loneliness, and similar problems. LLMs can produce hallucinations — plausible but incorrect statements — that can mislead users in sensitive mental-health contexts Research also shows that LLMs can express stigma or inappropriately validate maladaptive thoughts, reflecting limitations in reproducing the judgment and interpersonal skills of therapists. Assessments of crisis scenarios show that some LLMs lack effective safety protocols, such as suicide-risk assessment or referral to appropriate specialists.
Researchers have expressed concern that frequent use of large language models may weaken critical thinking.
Contemporary AI researchers generally agree that today's large language models do not possess consciousness. A minority hold that even if there is a small probability that a given software system could have subjective experience, which some philosophers consider possible, then ethical considerations regarding potential large-scale suffering in AI systems should be taken seriously — similar to considerations regarding animal welfare. Proponents of this view have proposed various precautionary measures, such as moratoriums on AI development and induced amnesia, to address these ethical concerns. Leonard Dung argues that the evidentiary frameworks used to assess consciousness in animals apply equally to AI systems, and that there is a significant probability that AI will be capable of suffering in the near future, making the risk of AI suffering a serious near-term ethical problem requiring systematic mitigation. On the other hand, some existentialist philosophers argue that there is no generally accepted way to determine whether an LLM is conscious, given the inherent difficulty of measuring subjective experience.
The 2022 Google LaMDA incident, in which engineer Blake Lemoine claimed that the model was sentient, showed how LLMs can convince users that they possess consciousness through responses that do not actually demonstrate consciousness. Google called the engineer's claims unfounded, and he was fired. Murray Shanahan argues that anthropomorphic interpretations of LLM capabilities encourage the unwarranted attribution of cognitive properties to systems that operate through statistical pattern completion. Kristina Šekrst takes this idea further, arguing that LLMs function as “illusion engines”, capable of producing outputs that consistently mimic properties such as consciousness without possessing them, while emphasizing that, because of the complex trade-off between creativity and temperature, we can never be certain whether we are witnessing the emergence of consciousness or merely a hallucination. David Chalmers likewise argues that although current LLMs probably lack the characteristics considered necessary for consciousness, expanded successors incorporating these elements could well meet the criteria within a decade.
Comments