Most AI engineers do not need to read every paper cover to cover. A small set of papers changes how you reason about the systems you build. They explain why a transformer needs position information, why a RAG stack needs more than a vector database, why fine-tuning can be cheap, and why a capable model may still be unhelpful.

Use this as a reading map. For each paper, focus on the engineering idea, the question it helps you answer today, and the limitation worth remembering.

1. Attention Is All You Need

Attention Is All You Need replaced recurrence and convolution with attention as the central operation for sequence modelling. Its key practical contribution was making training highly parallelizable while still letting every token directly attend to every other token.

If you work with language models, read the attention equations, multi-head attention, residual connections, and positional encodings. Those four pieces explain a surprising amount of modern model behaviour.

Transformers trade sequential computation for a memory-intensive all-to-all interaction. That makes them excellent at learning context, but long context is never free. Context windows, KV caches, sparse attention, and retrieval all manage that bill.

2. LoRA

LoRA: Low-Rank Adaptation of Large Language Models observes that the weight update needed to adapt a large pretrained model often has much lower rank than the full weight matrix. Instead of updating the original weights, LoRA learns two small matrices whose product represents the update.

For engineers, LoRA changed fine-tuning from a heavyweight infrastructure project into a routine deployment option. You keep one base model and store small task-specific adapters for tone, domain language, or behaviour.

Read this paper before choosing a fine-tuning strategy. It helps you ask the right questions: which modules should receive adapters, what rank is enough, and do you need to merge an adapter into the base weights for inference?

3. PEFT

Parameter-Efficient Fine-Tuning of Large-Scale Pre-trained Language Models is a valuable survey rather than a single new architecture. It organizes methods such as adapters, prompt tuning, prefix tuning, and low-rank approaches, and makes their trade-offs explicit.

This is the paper to read when someone says “we should fine-tune the model.” Fine-tuning is not one decision. It is a choice among data quality, trainable parameters, serving complexity, task transfer, and catastrophic forgetting.

In practice, start with prompting and evaluation, move to retrieval when the problem is fresh knowledge, and reach for PEFT when you need stable learned behaviour. A parameter-efficient method can lower the cost of training; it does not lower the cost of poor labels.

4. An Image Is Worth 16×16 Words

An Image Is Worth 16×16 Words treats an image as a sequence of fixed-size patches. Each patch becomes a token, and a transformer processes the resulting sequence much like text.

A general-purpose sequence architecture can work across modalities when the input is represented as tokens. That idea echoes through multimodal models, document understanding, video models, and image generation.

ViT also carries a useful warning: unlike convolutional networks, it has fewer built-in assumptions about locality and translation. The original result depended on large-scale pretraining. Architecture and data regime have to be considered together.

5. Auto-Encoding Variational Bayes

Auto-Encoding Variational Bayes made variational autoencoders practical by introducing the reparameterization trick. Rather than sampling a latent variable in a way that blocks gradients, the model samples noise and transforms it into the latent variable.

VAEs teach the language of latent-variable modelling: an encoder approximates a distribution over hidden causes, and a decoder reconstructs observations. They also explain the KL-divergence term you will encounter in generative modelling and alignment work.

VAEs tend to make smooth, useful latent spaces, but their samples can look softer than GAN or diffusion outputs. Their real value is conceptual: they make uncertainty and representation learning first-class design concerns.

6. Generative Adversarial Nets

Generative Adversarial Nets frames generation as a game. A generator tries to create convincing samples while a discriminator tries to tell real samples from generated ones.

GANs brought remarkable visual sharpness and a hard engineering lesson: adversarial objectives can be unstable. Mode collapse, delicate balance between networks, and evaluation difficulties are not footnotes; they are central to using GANs well.

Even if diffusion has displaced GANs in many image-generation workloads, this paper is worth reading for its game-theoretic framing and its influence on learned perceptual objectives, data augmentation, and synthetic media.

7. BERT

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding showed how masked-language modelling plus a simple task head could produce strong results across language-understanding benchmarks.

BERT is the clearest introduction to the encoder-only side of the transformer family. Use it to understand embeddings, reranking, classification, token labelling, and semantic search models. These workloads remain crucial even in a generative-AI product.

The contrast with GPT-style models matters: BERT sees context on both sides of a masked token; causal decoders predict the next token from the left. That objective choice shapes what each family is naturally good at.

8. High-Resolution Image Synthesis with Latent Diffusion Models

High-Resolution Image Synthesis with Latent Diffusion Models moved diffusion from pixel space into a compressed latent space. That made high-quality diffusion far more tractable and became the basis for systems such as Stable Diffusion.

Read it to understand the modern image-generation stack: an autoencoder compresses images, a denoiser learns to reverse noise in latent space, and cross-attention connects text conditioning to image features.

Model architecture is only part of image generation. The VAE, text encoder, sampler, scheduler, classifier-free guidance, and safety pipeline all affect output quality and cost.

9. Retrieval-Augmented Generation

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks couples a parametric language model with a non-parametric memory: a dense retriever over external documents. The model can ground an answer in information not fully stored in its weights.

Anyone building an enterprise AI application should read this. RAG needs ingestion, chunking, embeddings, retrieval, reranking, context construction, citation handling, and evaluation.

The paper also makes a lasting design point: model weights and external knowledge have different update cycles. If facts change often, storing them only in weights is the wrong operational boundary.

10. Language Models Are Few-Shot Learners

Language Models are Few-Shot Learners documented the surprising ability of a very large autoregressive model to perform tasks from instructions and examples placed in its prompt.

The paper gave engineers a new interface: natural language plus demonstrations. Prompt design, in-context examples, tool schemas, and structured-output instructions all descend from this interface.

Its caution is just as important. Few-shot performance is highly sensitive to examples, ordering, and evaluation setup. Treat a prompt as software: version it, test it against representative cases, and measure failures instead of trusting a good demo.

11. Switch Transformers

Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity uses a router to send each token to one expert feed-forward network. Only a small fraction of parameters are active for a token, allowing capacity to grow faster than compute per token.

This paper explains why parameter count alone is a poor proxy for inference cost. In sparse models, total parameters, active parameters, routing overhead, expert balance, and communication topology all matter.

For production work, the difficult part is often not the math. It is keeping experts balanced and serving them efficiently across hardware. Sparse architectures move complexity from the model layer into the systems layer.

12. Learning to Summarize with Human Feedback

Learning to Summarize with Human Feedback showed how preference comparisons from people can train a reward model, which can then optimize a policy toward outputs people prefer.

This is one of the best entry points to RLHF because the task is concrete. The central pattern is still widely used: collect comparisons, fit a preference or reward model, then optimize a model while constraining it against drifting too far from a reference.

The engineering caveat is reward hacking. A system can become very good at pleasing the reward model without becoming more truthful or useful. Your feedback rubric, annotator instructions, adversarial tests, and evaluation set are part of the model.

13. LLaMA

LLaMA: Open and Efficient Foundation Language Models made an influential case that carefully trained, relatively smaller foundation models can be competitive when given enough high-quality tokens.

For engineers, LLaMA helped make local and self-hosted language-model work realistic. It pushed attention toward token budgets, data mixtures, inference efficiency, and the practical ecosystem around open weights.

Do not read it as a recipe to copy blindly. Training data access, licensing, filtering, compute, and downstream safety requirements can matter more to a product than a clean scaling result.

14. RoFormer

RoFormer: Enhanced Transformer with Rotary Position Embedding introduced rotary position embeddings, commonly called RoPE. It encodes position by rotating query and key representations, making attention scores reflect relative positions naturally.

RoPE is a small implementation detail with outsized consequences. It appears in many modern decoder models and comes up whenever you work on context extension, long-context fine-tuning, or inference compatibility.

Read the derivation once. You do not need to memorize it, but you should know that changing positional encoding is not a harmless configuration edit. It changes how the model interprets distances between tokens.

15. Training Language Models to Follow Instructions with Human Feedback

Training Language Models to Follow Instructions with Human Feedback connected supervised instruction tuning, ranked human preferences, and reinforcement learning into the alignment pipeline that shaped modern assistant behaviour.

The most striking finding is that a much smaller aligned model can be preferred by humans over a far larger base model. Capability matters, but product quality also depends on whether a model follows intent, admits uncertainty, avoids harmful behaviour, and stays useful under ambiguity.

For a product team, this paper turns “alignment” into concrete work: write good tasks, collect consistent labels, define what a helpful refusal looks like, and evaluate against real user requests rather than a single benchmark.

A sensible reading order

If you are new to the field, read them in this order:

1. Transformers, BERT, and GPT-3 for the modern language-model foundation.
2. RoPE, LoRA, and PEFT for adapting and extending those models.
3. RAG, RLHF, and InstructGPT for building useful applications around them.
4. ViT, VAE, GANs, and latent diffusion for the generative-vision track.
5. Switch Transformers and LLaMA for scaling and efficient foundation-model design.

Do not try to retain every result table. For each paper, write down the problem, the mechanism, the resource trade-off, and one way the idea could fail in production. You will get more from that than from memorizing the paper’s year or benchmark score.