AI research library
The papers that actually changed what gets built, from the 1958 perceptron to the 2025 frontier reports. Every identifier machine-verified. Each entry says why it mattered, not just what it claimed.
A paper tells you what a lab discovered. It does not tell you whether it changes anything you are paying for. So every entry here is chosen for whether it should change what you build, and every summary below says why it mattered rather than restating the abstract. Full write-ups, covering what each paper means for your business and how it connects to systems we have actually built, are published in batches as each one is finished.
That is also the entry requirement. A paper is here because it is canon a technical buyer expects to find, or because it changes a decision our clients make. Nineteen papers met neither test and were removed rather than published as bare summaries. A shorter library where every page has a reason to exist beats a longer one padded with abstracts you could read at the source.
- Foundations12%9
- The Transformer era7%5
- Scaling and alignment7%5
- Reasoning and test-time compute7%5
- Automation, agents and the evidence28%21
- Efficiency, inference and serving16%12
- Multimodal and generative8%6
- Open-weight frontier reports15%11
Start with Automation, agents and the evidence. It is the section written for the question you actually arrive with — does this work on real business processes, and what does the evidence say. It covers the field studies (5,179 support agents measured in production; 758 consultants randomised, including the task where AI made them worse), the process-automation foundations that predate the AI wave and still decide whether a project succeeds, and the benchmarks that put agents on real company work rather than exams. The best agent in the most realistic of those benchmarks completed under a quarter of its tasks.
It is deliberately the most sceptical section in the library. A vendor will show you the productivity number; this is where the conditions attached to it are written down.
On the publisher’s site, not here. Authors keep copyright and most arXiv submissions are filed under a licence that does not permit republication, so hosting the PDFs would not be ours to do. Every entry links straight to the arXiv abstract, the PDF, or the journal DOI. What we add is the part the paper does not contain: whether it should change what you build.
Language models are very good at producing arXiv identifiers that look correct and resolve to a different paper entirely. Every arXiv ID on this page was queried against the arXiv API and the returned title compared against the title we publish before anything shipped. Journal papers carry a DOI. If a citation here is wrong, it is wrong in the source, not invented here.
Start here
20 papers that carry most of the explanatory weight. Read in this order and the rest of the library becomes much easier to navigate.
Process Mining Manifesto
Established process mining as a discipline: reconstruct how a business process actually runs from the event logs your systems already produce, rather than from how someone says it runs.
Attention Is All You Need
Removed recurrence entirely and replaced it with self-attention. Because attention parallelises across the sequence and recurrence does not, training could finally use GPUs fully — which is what made scale possible.
Language Models are Few-Shot Learners
At 175B parameters, a model started solving tasks from examples in the prompt with no gradient updates. In-context learning was not designed; it appeared.
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Retrieve documents at inference time and condition generation on them, instead of hoping the weights memorised the fact.
LoRA: Low-Rank Adaptation of Large Language Models
Freeze the base weights and train small low-rank matrices alongside them. Trainable parameters drop by orders of magnitude.
Learning Transferable Visual Models From Natural Language Supervision
Train image and text encoders into a shared embedding space on 400M web pairs. Zero-shot classification by comparing to text prompts.
High-Resolution Image Synthesis with Latent Diffusion Models
Run diffusion in a compressed latent space rather than pixel space. Cut the compute by more than an order of magnitude.
Training Compute-Optimal Large Language Models
Showed the field had been building models far too large for the data they were trained on. For a fixed compute budget, parameters and tokens should scale roughly equally.
Training Language Models to Follow Instructions with Human Feedback
RLHF: collect human preference comparisons, train a reward model, optimise the policy against it. A 1.3B InstructGPT was preferred to the 175B base GPT-3.
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Asking a model to show its working improves accuracy on multi-step problems, and the effect only appears above a certain scale.
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
Attention was memory-bandwidth-bound, not compute-bound. Tiling the computation to keep it in SRAM makes it several times faster with identical outputs and no approximation.
Generative AI at Work
A field study of 5,179 real customer-support agents using an AI assistant in production, not a lab task. Average productivity rose about 14% in issues resolved per hour — and the gain was concentrated almost entirely among the least experienced staff, while the most experienced saw little or none.
Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality
758 BCG consultants, randomised. On tasks inside the AI's capability the group using it was substantially faster and rated higher quality. On a task deliberately placed just outside that capability, consultants using AI were MORE likely to reach the wrong answer than those without it.
The Llama 3 Herd of Models
A 92-page account of a frontier training run: data curation, 15T tokens, scaling decisions, infrastructure failures and all.
DeepSeek-V3 Technical Report
671B MoE with 37B active. Introduced auxiliary-loss-free load balancing and multi-token prediction, and documented an FP8 mixed-precision training pipeline in detail.
TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks
Built a simulated software company — codebase, intranet, colleagues to message — and set agents to do real employee tasks end to end. The best agent completed under a quarter of them autonomously.
τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
Tests agents in retail and airline customer-service scenarios where they must follow a written policy, use tools, and talk to a simulated customer. Adds a consistency measure: run the same scenario several times and see whether the agent behaves the same way.
AI Agents That Matter
A critique of how agent capability is reported. Shows that many published gains come from expensive search over many attempts rather than better reasoning, that cost is usually omitted, and that a trivially simple baseline sometimes matches complex agent architectures.
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
Reasoning learned by pure reinforcement learning from verifiable rewards, with no supervised examples of human reasoning at all. Self-verification and backtracking emerged on their own during training.
Qwen3 Technical Report
Dense and MoE models from 0.6B to 235B under Apache 2.0, unifying thinking and non-thinking modes in one model with a controllable thinking budget.
The full library
Foundations
The results the whole field rests on. Mostly pre-2015, mostly not on arXiv, and still worth reading in the original.
The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain
Where it starts. A learnable linear classifier with a convergence proof, and the first working demonstration that a machine could adjust its own weights from examples rather than being programmed.
Learning Representations by Back-Propagating Errors
The answer to the XOR objection: with hidden layers and the chain rule you can train arbitrarily deep networks. Every model on this site is trained by a direct descendant of this algorithm.
Long Short-Term Memory
Solved the vanishing-gradient problem with gated memory cells, making it possible to learn long-range dependencies in sequences.
ImageNet Classification with Deep Convolutional Neural Networks
Won ImageNet by a margin so large it ended the debate about whether deep learning worked. Trained on two consumer GPUs.
Efficient Estimation of Word Representations in Vector Space
Words as dense vectors where geometric relationships encode semantic ones. The origin of the embedding.
Adam: A Method for Stochastic Optimization
Adaptive per-parameter learning rates from first and second moment estimates. Made training deep networks reliable without heroic hyperparameter tuning.
Neural Machine Translation by Jointly Learning to Align and Translate
The first attention mechanism. Instead of squeezing a sentence into one fixed vector, let the decoder look back at all encoder states and weight them.
Deep Residual Learning for Image Recognition
Skip connections let gradients flow through very deep networks, making 100+ layer models trainable for the first time.
Distilling the Knowledge in a Neural Network
A small model trained on a large model's output distribution learns more than one trained on hard labels alone.
The Transformer era
2017-2020. Attention replaces recurrence, and scale becomes the strategy.
Attention Is All You Needessential
Removed recurrence entirely and replaced it with self-attention. Because attention parallelises across the sequence and recurrence does not, training could finally use GPUs fully — which is what made scale possible.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Masked language modelling: hide tokens and predict them from both directions. Established pretrain-then-finetune as the standard recipe.
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Reframed every NLP task as text-in, text-out, then ran a careful ablation over the whole design space.
Language Models are Few-Shot Learnersessential
At 175B parameters, a model started solving tasks from examples in the prompt with no gradient updates. In-context learning was not designed; it appeared.
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
Route each token to one expert instead of running the whole network. Parameter count grows without proportional compute cost.
Scaling and alignment
2020-2023. How big, trained on how much, and how to make it do what was asked.
Scaling Laws for Neural Language Models
Loss falls as a smooth power law in parameters, data and compute. Performance became predictable before training, which made nine-figure training runs a defensible business decision.
Training Compute-Optimal Large Language Modelsessential
Showed the field had been building models far too large for the data they were trained on. For a fixed compute budget, parameters and tokens should scale roughly equally.
Training Language Models to Follow Instructions with Human Feedbackessential
RLHF: collect human preference comparisons, train a reward model, optimise the policy against it. A 1.3B InstructGPT was preferred to the 175B base GPT-3.
Constitutional AI: Harmlessness from AI Feedback
Replace human harmlessness labels with a written set of principles the model critiques itself against.
Direct Preference Optimization: Your Language Model is Secretly a Reward Model
Removed the reward model and the RL loop entirely: optimise preferences directly with a classification loss.
Reasoning and test-time compute
2022 onwards. The shift from bigger models to models that think for longer before answering.
Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelsessential
Asking a model to show its working improves accuracy on multi-step problems, and the effect only appears above a certain scale.
ReAct: Synergizing Reasoning and Acting in Language Models
Interleave reasoning traces with tool calls so the model can look things up mid-thought instead of guessing.
Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
Spending more compute at inference can beat spending it on a bigger model. Quantified the trade-off.
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningessential
Reasoning learned by pure reinforcement learning from verifiable rewards, with no supervised examples of human reasoning at all. Self-verification and backtracking emerged on their own during training.
s1: Simple Test-Time Scaling
Competitive reasoning from 1,000 carefully chosen training examples plus a trick called budget forcing — appending 'Wait' to make the model keep thinking.
Automation, agents and the evidence
What actually happens when this technology meets a business process: the field studies, the process-automation foundations, and the benchmarks that measure agents on real work rather than exams. The most commercially relevant section here, and the most sceptical.
Process Mining Manifestoessential
Established process mining as a discipline: reconstruct how a business process actually runs from the event logs your systems already produce, rather than from how someone says it runs.
Integrating Robotic Process Automation into Business Process Management
Sets out where rule-based RPA belongs inside a managed process, and where it becomes technical debt: brittle bots imitating clicks on interfaces nobody controls.
Generative AI at Workessential
A field study of 5,179 real customer-support agents using an AI assistant in production, not a lab task. Average productivity rose about 14% in issues resolved per hour — and the gain was concentrated almost entirely among the least experienced staff, while the most experienced saw little or none.
Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Qualityessential
758 BCG consultants, randomised. On tasks inside the AI's capability the group using it was substantially faster and rated higher quality. On a task deliberately placed just outside that capability, consultants using AI were MORE likely to reach the wrong answer than those without it.
Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence
A randomised experiment on mid-level professional writing tasks. Time taken fell about 40% and output quality rose about 18%, with the largest gains again going to the weakest writers.
GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models
Mapped US occupations against LLM capability task by task. Around 80% of workers are in jobs where at least 10% of tasks could be affected, and about 19% are in jobs where more than half could be.
AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation
A framework for building systems from multiple conversing agents with distinct roles, including agents that write and execute code and a human participant in the loop.
Toolformer: Language Models Can Teach Themselves to Use Tools
Showed a model could learn, from its own self-generated data, when to call an external tool — a calculator, a search, an API — and how to use the result.
Gorilla: Large Language Model Connected with Massive APIs
Trained a model to call thousands of real APIs correctly and used retrieval over live API documentation so it stays right when those APIs change.
GAIA: a Benchmark for General AI Assistants
Questions that are trivial for a person but require an assistant to chain web browsing, file handling and reasoning. Humans scored around 92%; the best assistant systems at publication scored a small fraction of that.
WebArena: A Realistic Web Environment for Building Autonomous Agents
A self-hosted environment with working e-commerce, forum, CMS and code sites, so agents can be tested on real end-to-end web tasks rather than static pages.
MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework
Gave a multi-agent system explicit human workflows — product manager, architect, engineer roles with standardised documents passed between them — rather than letting agents converse freely.
The Rise and Potential of Large Language Model Based Agents: A Survey
A comprehensive map of the agent field: the brain-perception-action framing, single versus multi-agent designs, and where each approach has been applied.
The Simple Macroeconomics of AI
A deliberately conservative estimate: AI raises total factor productivity by roughly 0.5% and GDP by around 1% over ten years — an order of magnitude below the widely quoted forecasts.
TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasksessential
Built a simulated software company — codebase, intranet, colleagues to message — and set agents to do real employee tasks end to end. The best agent completed under a quarter of them autonomously.
WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?
A benchmark of routine enterprise tasks inside a real ServiceNow instance — raising requests, filtering lists, filling forms, navigating dashboards. Agents performed far below the human baseline on work most office staff consider trivial.
τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainsessential
Tests agents in retail and airline customer-service scenarios where they must follow a written policy, use tools, and talk to a simulated customer. Adds a consistency measure: run the same scenario several times and see whether the agent behaves the same way.
AI Agents That Matteressential
A critique of how agent capability is reported. Shows that many published gains come from expensive search over many attempts rather than better reasoning, that cost is usually omitted, and that a trivially simple baseline sometimes matches complex agent architectures.
Executable Code Actions Elicit Better LLM Agents
Instead of emitting a structured tool call, have the agent write and run actual code. Fewer steps, better success rates, and it composes operations naturally.
OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
Real operating systems, real applications, tasks spanning several programs. Humans completed over 72%; the best agents managed a small fraction.
SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering
Showed that agent performance depends heavily on the interface it is given. Redesigning the tools — how files are viewed, edited and searched — improved results dramatically with no change to the underlying model.
Efficiency, inference and serving
How these things are actually made to run at a price anyone can pay. The least glamorous and most commercially relevant category here.
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
Tensor parallelism: split individual matrix multiplications across GPUs with minimal communication.
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasksessential
Retrieve documents at inference time and condition generation on them, instead of hoping the weights memorised the fact.
LoRA: Low-Rank Adaptation of Large Language Modelsessential
Freeze the base weights and train small low-rank matrices alongside them. Trainable parameters drop by orders of magnitude.
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenessessential
Attention was memory-bandwidth-bound, not compute-bound. Tiling the computation to keep it in SRAM makes it several times faster with identical outputs and no approximation.
Fast Inference from Transformers via Speculative Decoding
A small draft model proposes several tokens; the large model verifies them in one pass. Output is provably identical to the large model alone.
GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
One-shot weight quantisation to 3-4 bits with second-order error correction and no retraining.
Efficient Memory Management for Large Language Model Serving with PagedAttention
Manage the KV cache like operating-system virtual memory, in non-contiguous pages. Cuts the memory waste that limits batch size.
SGLang: Efficient Execution of Structured Language Model Programs
RadixAttention reuses KV cache across requests sharing a prefix — exactly the pattern in agent loops that resend the same system prompt.
QLoRA: Efficient Finetuning of Quantized LLMs
LoRA on top of a 4-bit base model, with 4-bit NormalFloat and double quantisation. Fine-tuned a 65B model on one 48 GB GPU.
AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
Not all weights matter equally. Protect the ~1% with the largest activation magnitudes and quantise the rest hard.
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
Share key and value heads across groups of query heads. Shrinks the KV cache with almost no quality loss.
Mamba: Linear-Time Sequence Modeling with Selective State Spaces
A selective state-space model with linear rather than quadratic scaling in sequence length, and no KV cache.
Multimodal and generative
Vision, audio, images and the models that cross between them.
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Cut an image into patches, treat them as tokens, apply a plain Transformer. Convolutions turned out to be optional given enough data.
Denoising Diffusion Probabilistic Models
Learn to reverse a gradual noising process. Made diffusion competitive with GANs and far more stable to train.
Learning Transferable Visual Models From Natural Language Supervisionessential
Train image and text encoders into a shared embedding space on 400M web pairs. Zero-shot classification by comparing to text prompts.
High-Resolution Image Synthesis with Latent Diffusion Modelsessential
Run diffusion in a compressed latent space rather than pixel space. Cut the compute by more than an order of magnitude.
Robust Speech Recognition via Large-Scale Weak Supervision
680,000 hours of weakly supervised multilingual audio. No fine-tuning needed to beat specialised systems on out-of-distribution speech.
Segment Anything
A promptable segmentation model plus the largest segmentation dataset ever built, released together.
Open-weight frontier reports
2024 onwards. The technical reports from labs that published their weights — the closest thing to documentation for the models on our free-models page.
Evaluating Large Language Models Trained on Code
The Codex paper. Introduced HumanEval and the pass@k metric.
Llama 2: Open Foundation and Fine-Tuned Chat Models
The release that made commercially usable open weights normal, with an unusually candid account of its RLHF process.
GPT-4 Technical Report
Notable as much for what it withholds as what it reports: no architecture, no parameter count, no training data. Extensive capability and safety evaluation.
Gemini: A Family of Highly Capable Multimodal Models
Natively multimodal from pretraining rather than by bolting a vision encoder onto a text model.
The Llama 3 Herd of Modelsessential
A 92-page account of a frontier training run: data curation, 15T tokens, scaling decisions, infrastructure failures and all.
DeepSeek-V3 Technical Reportessential
671B MoE with 37B active. Introduced auxiliary-loss-free load balancing and multi-token prediction, and documented an FP8 mixed-precision training pipeline in detail.
Mixtral of Experts
8 experts, 2 active per token: 47B total parameters at the inference cost of about 13B, under Apache 2.0.
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
Argued that data quality can substitute for scale, using heavily filtered and synthetic 'textbook quality' data.
StarCoder 2 and The Stack v2: The Next Generation
Open code models trained on a fully licence-audited dataset with an opt-out mechanism for developers.
Kimi K2: Open Agentic Intelligence
A 1.04T-parameter MoE with 32B active, pre-trained on 15.5T tokens using MuonClip with no loss spikes — a notable stability result at that scale. Built for agentic tool use rather than chat.
Qwen3 Technical Reportessential
Dense and MoE models from 0.6B to 235B under Apache 2.0, unifying thinking and non-thinking modes in one model with a controllable thinking budget.
Reading AI research, honestly answered
Which paper should I read first?
If you read exactly one, read "Attention Is All You Need" (2017). Every model in production today is a Transformer or a Transformer hybrid, and the paper is unusually short and readable. If you read three, add "Training Compute-Optimal Large Language Models" (the Chinchilla paper) for why small models became good, and "Training Language Models to Follow Instructions with Human Feedback" for why they became usable.
How were these papers chosen?
Chosen for causal influence on systems in production today, not for citation count. Each entry says plainly why it mattered and what it changed. That means a few enormously-cited papers are absent and a few modestly-cited ones are here, because the question we asked was "did this change what gets built?" rather than "how many citations does it have?"
Are the citations reliable?
Yes, and this is not a rhetorical claim. Every arXiv identifier on this page was checked against the arXiv API and the returned title matched before publication. Journal papers carry a DOI. Nothing here is cited from memory. 74 of 74 entries carry a resolvable arXiv ID, DOI or publisher URL. Language models hallucinate plausible-looking arXiv numbers that resolve to entirely different papers, which is why every identifier here was machine-checked rather than written from memory.
Do I need a maths background to read these?
Less than you would expect for most of them. Technical reports from open-weight labs — the Llama 3, DeepSeek-V3 and Qwen3 reports — are largely engineering narrative and are readable by anyone comfortable with software architecture. The optimisation and architecture papers do assume linear algebra and calculus. A practical route in: read the abstract, the introduction and the conclusion of any paper first, and only go into the method section if the result matters to you.
Why are Chinese labs so prominent in the recent entries?
Because they are publishing. DeepSeek, Moonshot AI, Alibaba and MiniMax have released detailed technical reports alongside open weights, while the leading US labs have moved towards releasing capability evaluations without architecture or training details — the GPT-4 technical report explicitly withholds parameter count, architecture and data. If you want to read how a frontier model was actually built in 2024 and 2025, the Chinese reports and Meta's Llama 3 paper are where that information is.
How does this connect to the models I can actually use?
Several papers here are the documentation for models in our free-models directory. The DeepSeek-R1, Kimi K2, Qwen3 and Llama 3 entries all correspond to weights you can download and run today, and the model pages link back to the papers. The efficiency section — FlashAttention, PagedAttention, GPTQ, AWQ, QLoRA — is the reason local inference is possible at all.
How this library is maintained
Every arXiv identifier on this page was checked against the arXiv API and the returned title matched before publication. Journal papers carry a DOI. Nothing here is cited from memory.
Chosen for causal influence on systems in production today, not for citation count. Each entry says plainly why it mattered and what it changed.
Last verification run: 2026-09-06. Summaries are ours. Read the paper before you cite it — a summary is a pointer, not a source.
Turning research into something that runs
Most of what is on this page is three years from production and some of it is already in it. Knowing which is which is the job. That is the part we do.
Where this turns into work: AI strategy and consulting, AI agent development and AI automation development. Or see it done: our case studies.