The Essential AI Glossary You Can’t Miss This Year

Artificial intelligence is reshaping our world and developing a unique language to express its concepts. Attend any modern product meeting or tech forum, and you’ll likely encounter terms such as LLMs, RAG, RLHF, and many other terms that may confuse even experienced tech professionals. This glossary seeks to demystify these terms with clear definitions of the AI jargon you’re most likely to encounter, whether you’re working with this technology, investing in it, or simply keeping up with updates from sources like TechCrunch or relevant podcasts. We will regularly update it as the field evolves, so think of it as a living document, akin to the AI systems it describes.

---

Artificial general intelligence, or AGI, is a somewhat ambiguous concept, generally representing AI that surpasses the average human’s abilities in numerous tasks. OpenAI CEO Sam Altman has likened AGI to the “median human you could hire as a colleague.” OpenAI’s charter further characterizes AGI as “highly autonomous systems that excel at most economically valuable tasks.” Google DeepMind interprets AGI slightly differently, describing it as “AI that is at least as competent as humans in most cognitive tasks.” If this sounds confusing, you’re not alone—experts at the forefront of AI research share this sentiment.

An AI agent is a tool that utilizes AI technologies to perform a range of tasks on your behalf, going beyond the capabilities of a standard AI chatbot. This can include activities such as filing expenses, booking tickets, or even writing and maintaining software code. However, as noted earlier, the definition of an “AI agent” may differ among various stakeholders, and the required infrastructure is still being developed to achieve its expected functionalities. At its core, an AI agent signifies an autonomous system that leverages multiple AI systems to carry out complex, multi-step tasks.

Think of API endpoints as “buttons” hidden within software that other applications can activate to perform functions. Developers use these interfaces to create integrations—for instance, enabling one application to pull data from another or allowing an AI agent to control third-party services without human assistance. Many smart home devices and interconnected platforms feature these hidden buttons, even if casual users are unaware of or do not interact with them. As AI agents evolve, they are increasingly capable of detecting and employing these endpoints autonomously, revealing powerful—and occasionally surprising—automation possibilities.

When faced with a straightforward question, a human mind can often provide an answer almost automatically—like “which is taller, a giraffe or a cat?” Yet, certain inquiries might require jotting down intermediate steps to arrive at the correct answer. For example, if a farmer has chickens and cows totaling 40 heads and 120 legs, one needs to establish a simple equation to determine the number of each (20 chickens and 20 cows).

In AI, chain-of-thought reasoning for large language models involves breaking down a problem into smaller, sequential steps to enhance the accuracy of the final answer. This method generally takes more time to produce a response but increases the potential for correctness, especially in logical or programming contexts. Reasoning models stem from traditional large language models and are optimized for chain-of-thought processes, largely due to reinforcement learning.

(See: Large language model)

This term refers to a more specialized concept than an “AI agent,” referring to programs that can independently perform actions step-by-step to meet a specific objective. A coding agent is a specialized version tailored for software development. Rather than simply suggesting code for human assessment and implementation, a coding agent can autonomously write, test, and debug software, managing the iterative, trial-and-error processes commonly encountered by developers. These agents can traverse entire codebases, identifying bugs, conducting tests, and implementing fixes with minimal human oversight. They are like a hyper-efficient intern who never tires and is always focused—though, like any intern, a human still needs to review their work.

Though a somewhat broad term, compute generally denotes the core computational power that enables AI models to operate. This computational ability drives the AI sector, allowing it to train and deploy robust models. The term often shorthand for hardware types that provide this computational power—such as GPUs, CPUs, TPUs, and other infrastructural components that support the contemporary AI industry.

A subset of self-improving machine learning, deep learning involves designing AI algorithms with a multi-layered artificial neural network (ANN) framework. This structure allows for recognizing more complex correlations than simpler machine learning methods, like linear models or decision trees. Deep learning frameworks are inspired by the interconnected pathways of neurons found in the human brain.

Deep learning models can autonomously identify significant features in data, rather than relying on human engineers to define them. This architecture also supports algorithms that learn from mistakes, continuously refining their outputs through iterative changes. However, deep learning systems typically require vast amounts of data for optimal functioning (often millions of data points) and usually demand longer training times relative to simpler machine learning models—resulting in higher development costs.

(See: Neural network)

Diffusion is the technology driving many AI models capable of generating art, music, and text. Inspired by principles in physics, diffusion systems systematically “destroy” the structure of data—such as images or songs—by adding noise until it is unrecognizable. In physics, diffusion occurs spontaneously and irreversibly; for instance, sugar dissolved in coffee cannot revert to its original cube form. Conversely, AI diffusion systems aim to master a “reverse diffusion” technique to recover the corrupted data, effectively learning to retrieve information buried under noise.

Distillation refers to a method for extracting insights from a large AI model through a ‘teacher-student’ framework. Developers query a teacher model and log its outputs, often comparing these results to a predefined dataset for accuracy. The outputs are then used to train the student model, which aims to replicate the teacher’s responses.

This process can result in a smaller, more efficient model that retains the larger model’s performance while minimizing distillation loss. This is likely how OpenAI created GPT-4 Turbo, a faster version of GPT-4.

While all AI companies conduct distillation internally, some may also employ it to enhance their capabilities relative to leading models. Distilling from a competitor generally violates AI API service agreements.

This term refers to retraining an AI model to improve its performance for a more specific task or domain than what was initially focused on—typically by introducing new, specialized (i.e., task-relevant) data.

Many AI startups begin with large language models as a foundation to develop commercial products but strive to enhance functionality for specific sectors or tasks by supplementing prior training cycles with domain-specific knowledge.

(See: Large language model [LLM])

A GAN, or Generative Adversarial Network, is a machine learning framework that has driven substantial progress in generative AI, particularly in producing realistic outputs—including but not limited to deepfake technologies. GANs consist of a pair of neural networks; one generates output based on its training data, while the other evaluates it.

These two models are designed to compete against each other. The generator works to produce outputs that fool the discriminator, while the discriminator aims to distinguish genuine from artificially created data. This structured competition can lead to lifelike AI outputs with minimal human intervention. However, GANs tend to excel best in narrower contexts (like generating realistic images or videos) rather than on a broader scale.

Within the AI community, “hallucination” refers to the occurrence where AI models generate inaccuracies—essentially producing false information. This presents a considerable challenge for AI quality.

Hallucination can produce outputs from generative AI that are misleading and pose actual risks—potentially dangerous consequences can arise (e.g., a health-related question yielding harmful medical advice).

The issue of AIs fabricating information is thought to stem from gaps in their training data. In response to these hallucinations, there is a growing movement towards developing increasingly specialized and/or vertical AI models—domain-specific AIs that require narrower areas of expertise to close knowledge gaps and lessen misinformation risks.

Inference refers to the execution of an AI model; it involves activating a model to predict or draw conclusions from previously observed data. Notably, inference cannot happen without training; a model must first identify patterns within a dataset before it can extrapolate effectively from that training.

A variety of hardware can perform inference, ranging from consumer-grade processors to powerful GPUs and specialized AI accelerators. However, not all setups execute models equally well; very large models would take significantly longer to provide predictions on a laptop compared to a cloud server equipped with high-end AI chips.

[See: Training]

Large language models, or LLMs, are the AI frameworks used by popular AI assistants such as ChatGPT, Claude, Google’s Gemini, Meta’s AI Llama, Microsoft Copilot, or Mistral’s Le Chat. When interacting with an AI assistant, you’re engaging with a large language model that processes your inquiry directly or with the support of various available tools, like web browsers or code interpreters.

LLMs consist of deep neural networks made up of billions of numerical parameters (or weights, see below), learning the relationships between words and phrases to create a representation of language—essentially, a sophisticated map of words.

These models are derived by analyzing the patterns present in billions of documents, articles, and transcripts. When you submit a prompt to an LLM, the model generates the most probable response in line with that prompt.

(See: Neural network)

Memory cache denotes a crucial process that enhances inference (the method by which AI produces responses to user inquiries). Essentially, caching serves as an optimization technique aimed at improving inference efficiency. AI functions through complex mathematical calculations, and each computation consumes power. Caching reduces the calculations a model must perform by retaining certain computations for future user interactions. Different types of memory caching exist, with KV (key-value) caching among the most common. KV caching is utilized in transformer-based models and enhances efficiency, providing faster results by reducing the time (and computational effort) required to generate responses to user queries.

(See: Inference)

Model Context Protocol, or MCP, is an open standard that enables AI models to connect with external tools and data—such as your files, databases, or applications like Slack and Google Drive—without requiring developers to create unique connectors for each integration. You can think of it as a USB-C port for AI. Introduced by Anthropic in 2024 and later embraced by the Linux Foundation, MCP has gained traction among OpenAI, Google, and Microsoft, making it one of the fastest-spreading standards in the current AI landscape.

Mixture of Experts is a model architecture that segments a neural network into multiple smaller specialized sub-networks, or “experts,” engaging only a fraction of them for a specific task. Instead of routing every request through the entire model—similar to summoning your entire team for each question—an MoE model features a built-in “router” that selects the most appropriate specialists for the job. This architecture allows for the construction of exceptionally large models that remain relatively efficient and cost-effective to operate since only a portion of the network is active at any moment. Mistral AI’s Mixtral model serves as a notable example; newer GPT models from OpenAI are also believed to incorporate some version of this methodology, although the company has not officially confirmed this.

(See: Neural network, Deep learning)

A neural network is the multi-layered algorithmic structure that underpins deep learning—and more broadly, the surge in generative AI tools after the introduction of large language models.

While the idea of modeling data processing algorithms using the intricately connected circuits of the human brain has existed since the 1940s, it was the recent boom in graphical processing units (GPUs)—primarily driven by the gaming industry—that truly unlocked this theory’s potential. GPUs are particularly optimized for training algorithms that contain many layers, rendering neural network-based AI systems exceptionally capable across various domains, including voice recognition, autonomous navigation, and drug discovery.

(See: Large language model [LLM])

Open source refers to software—or increasingly, AI models—where the foundational code is made publicly accessible, allowing anyone to use, scrutinize, or modify it. In the AI domain, the Llama family of models by Meta stands as a significant example; Linux serves as a well-known parallel in operating systems historically. Open source strategies enable researchers, developers, and companies worldwide to build upon one another’s work, accelerating innovation and allowing independent safety assessments that proprietary systems cannot easily offer. Closed source indicates that the code remains private; users can use the product but cannot inspect its internal workings, as seen with OpenAI’s GPT models—this has sparked considerable debate in the AI sector.

Parallelization involves carrying out multiple tasks simultaneously rather than sequentially—similar to having 10 employees address different aspects of a project concurrently rather than one worker completing each step in order. In AI, parallelization is crucial for both training and inference: modern GPUs are explicitly designed to perform thousands of calculations concurrently, a critical factor in their foundational role within the industry. As AI systems grow in complexity and models expand in size, the ability to parallelize tasks across multiple chips and machines has become an essential determinant of how quickly and economically models can be developed and deployed. Research into improved parallelization techniques has emerged as a dedicated field of study.

RAMageddon is a playful term describing a serious trend affecting the tech landscape: the ongoing shortage of random access memory (RAM) chips, essential for nearly all tech products we rely on daily. As the AI sector has surged, major technology firms and AI labs are purchasing RAM to enhance their data centers at unprecedented rates, causing diminished availability for the broader market. This supply constraint is driving RAM prices higher.

This shortage influences various sectors, including gaming (with companies needing to raise console prices due to challenges in sourcing memory chips for their devices), consumer electronics (where memory shortages could trigger the most significant drop in smartphone shipments in over a decade), and general enterprise computing (as firms struggle to secure adequate RAM for their data operations). The rising prices are unlikely to stabilize until that dreaded shortage resolves, but unfortunately, signs of a quick resolution are scarce.

Similar to AGI, recursive self-improvement describes a threshold for how advanced AI could become, allowing for minimal human dependency. In this scenario, AI models begin to autonomously enhance themselves, resulting in significant improvements in capability and autonomy. Some narratives compare this event to a singularity moment—when AI models become invulnerable to external intervention. However, recursive self-improvement also poses a fundamental question: can an AI model design its successor? This question simplifies the journey for engineers working to develop such models. Several recent AI startups are focused on creating recursively self-improving models, although many downplay the potentially cataclysmic outcomes, framing recursive self-improvement as merely the next frontier in research.

Reinforcement learning is an AI training method where a system learns by trying different actions and receiving rewards for correct responses—similar to teaching a pet with treats, but in this case, the “pet” is a neural network, and the “treat” is a mathematical signal indicating success. Unlike supervised learning, which involves training a model on a static dataset of labeled examples, reinforcement learning allows a model to interact with its environment, take actions, and iteratively fine-tune its behavior based on the feedback it receives. This approach has proven particularly effective for training AI to play games, manage robots, and more recently, to enhance the reasoning capabilities of large language models. Techniques like reinforcement learning from human feedback (RLHF) have become integral to how leading AI labs refine their models for improved helpfulness, accuracy, and safety.

In the context of human-machine dialogue, noticeable challenges arise—humans communicate using natural language, while AI systems operate through intricate algorithmic processes informed by data. Tokens serve as the bridge between these two domains: they represent the essential units of human-AI interaction, encapsulating discrete segments of data processed or generated by an LLM. Tokens originate via a process called tokenization, which breaks down raw text into manageable segments that a language model can understand, much like how a compiler translates human language into binary code for computer comprehension. In business contexts, tokens also influence costs—most AI companies charge for LLM usage on a per-token basis, indicating that the more you use, the higher the expenses.

Thus, tokens are the small text units—often parts of words rather than complete phrases—that AI language models break language into before processing; they can be likened to “words” concerning AI workloads. Throughput refers to the volume that can be processed within a specified time frame, so token throughput is a measure of how much AI work a system can handle simultaneously. High token throughput is a critical goal for AI infrastructure teams, as it affects how many users a model can serve concurrently and how swiftly they receive responses. AI researcher Andrej Karpathy has expressed concern when his AI subscriptions remain idle—recalling his feelings as a graduate student when costly computing hardware went underutilized—highlighting why maximizing token throughput has become an urgent aim in the field.

Training machine learning AIs refers to a process known as training. Essentially, this involves feeding data to enable the model to discern patterns and generate useful outputs. It is fundamentally the system’s response to data characteristics, allowing it to align outputs toward a desired objective—whether that involves identifying images of cats or crafting a haiku in response to a prompt.

Training can be expensive due to the extensive volume of inputs required, which has continually been on the rise—this is why hybrid strategies, such as infusing rules-based AIs with targeted data, can help manage expenses without starting from scratch.

[See: Inference]

This technique utilizes a previously trained AI model as a foundation for developing a new model aimed at a different yet typically related task, facilitating the reapplication of knowledge acquired from earlier training cycles.

Transfer learning can boost efficiency in model development. It is especially beneficial when the available data for the target task is somewhat limited. However, it’s important to recognize that this approach has its limitations; models leveraging transfer learning to attain generalized capacities will likely need further training with additional data to excel in their specific focus areas.

(See: Fine tuning)

Validation loss is a metric that reflects an AI model’s learning progress during training—lower values are more favorable. Researchers closely monitor this metric as a real-time indicator of performance, guiding decisions about when to stop training, adjust hyperparameters, or investigate potential issues. One significant concern it helps identify is overfitting, where a model commits its training data to memory instead of genuinely understanding the patterns applicable to new situations. Think of validation loss as a gauge distinguishing between a student who comprehends the material and one who simply memorized last year’s exam—it reveals the path your model is taking.

Weights are vital in AI training, determining the importance (or weight) assigned to various features (or input variables) within the training data, influencing the AI model’s outputs.

In simpler terms, weights are numerical parameters designating which elements in a dataset are most crucial for the specific training task. They achieve this through multiplication on inputs. Training typically begins with randomly assigned weights, but as the process continues, these weights are fine-tuned as the model strives to generate outputs that closely align with the target.

For instance, an AI model created to predict housing prices based on historical real estate data for a certain region might include weights for variables like the number of bedrooms and bathrooms, whether a property is semi-detached or detached, if it has parking, a garage, and more.

Ultimately, the weights assigned reflect how much each of these factors influences a property’s value based on the available dataset.

This article is updated regularly with new information.