Summary
A large language model (LLM) is an AI model trained on vast text corpora to process and generate language. Per the overview on Wikipedia, LLMs can “generate, summarize, translate, and analyze text” and underpin assistants such as ChatGPT and Claude. Since ~2017 nearly all of them use the Transformer architecture (“Attention Is All You Need”, Vaswani et al., arXiv June 2017), whose attention mechanism weighs how relevant each token in the context window is to every other.
How an LLM is built
- Pretraining: self-supervised next-token prediction over large text corpora; text is first converted to tokens (e.g. byte-pair encoding).
- Post-training: supervised fine-tuning and reinforcement learning from human feedback (RLHF) to make the model follow instructions and behave as an assistant. Parameter-efficient adaptation such as low-rank-adaptation is common.
- Architecture variants: dense models and mixture-of-experts (only part of the parameters is active per token, which lowers inference cost); model sizes range from billions to trillions of parameters.
- Reasoning models: models such as OpenAI’s o1 that generate step-by-step reasoning before answering, strongly improving maths and coding.
- Multimodality: current frontier LLMs also take images, audio and often video as input (GPT-4 already accepted image and text input, per its 2023 technical report).
- Main limitations: hallucination (plausible but false output), opacity of internal mechanisms, benchmark contamination. See inference-in-generative-ai for serving/inference.
Current landscape (Sept 2026)
Model names change quickly; the vault’s model notes hold the detail and are the source of truth:
- Anthropic: Claude Fable 5.1 (current flagship), Opus 5.5, Sonnet 5.5 - see anthropic-claude, claude-fable-5, claude-opus-5, claude-mythos.
- OpenAI: GPT-6 family (Astra, Sol, Luna) - see gpt-5-and-gpt-6-family.
- Google: Gemini 3.x (latest stable on the API: Gemini 3.8 Flash) and open-weight Gemma 4 - see gemini-3-x-family, gemma-4.
- Others: grok, kimi-k3, nvidia-nemotron, granite; provider overview in ai-model-providers.
- How models are compared: see massive-multitask-language-understanding (MMLU is saturated; newer benchmarks listed there).
LLM MoC
- artificial-intelligence: the overarching field (note is currently a stub).
- machine-learning and neural-networks / artificial-neural-network: the methods LLMs are built on.
- generative-pretrained-transformer: the GPT series and the term itself.
- massive-multitask-language-understanding: the MMLU benchmark and how LLMs are evaluated.
- low-rank-adaptation: cheap fine-tuning.
- artificial-general-intelligence: the long-term goal several labs state.
- ai-research-labs: who builds these models.
- ai-regulation-and-policy, data-privacy-in-ai, ai-literacy-and-education: governance, privacy and skills around LLM use.
- Plain-text topics without a current note: natural language processing, deep learning, BERT, ethics in AI, applications of LLMs.
Sources
- https://en.wikipedia.org/wiki/Large_language_model (accessed 2026-09-30)
- https://arxiv.org/abs/1706.03762 - Attention Is All You Need (accessed 2026-09-30)
- https://arxiv.org/abs/2303.08774 - GPT-4 Technical Report (accessed 2026-09-30)
- Model landscape: KB-AI-AUDIT-2026-09.md Appendix A and Phase 3a corrections, and the linked model notes (each carries its own primary sources) (accessed 2026-09-30)