Post-Training

Post-training refers to all the techniques applied to an AI model after its initial pre-training phase. Pre-training teaches the model general language understanding from massive datasets. Post-training refines the model to be actually useful, safe, and aligned with human expectations.

Why post-training is necessary

A freshly pre-trained language model is like an encyclopaedia that can finish any sentence but has no idea how to be helpful. It might respond to "How do I make pasta?" by continuing with text from a random cooking website, a chemistry textbook, or a novel — whatever is statistically likely.

Post-training transforms this into an assistant that actually answers your question helpfully, refuses harmful requests, and follows instructions.

Post-training stages

Modern AI models typically go through several post-training stages:

• Supervised fine-tuning (SFT): The model is trained on high-quality examples of ideal conversations. Human annotators write model responses, and the model learns to imitate this behaviour.
• RLHF (Reinforcement Learning from Human Feedback): Human raters compare multiple model responses and rank them. A reward model learns these preferences, and the language model is optimised to produce responses the reward model scores highly.
• RLAIF (RL from AI Feedback): Similar to RLHF but using another AI model to provide feedback instead of humans. Scales better but may carry the feedback model's biases.
• DPO (Direct Preference Optimisation): A simpler alternative to RLHF that directly optimises the model on preference data without needing a separate reward model.
• Safety training: Specific training to refuse harmful requests, avoid biases, and handle sensitive topics appropriately.

The impact of post-training

The difference between a pre-trained and post-trained model is dramatic. Pre-trained models produce coherent text but are unreliable conversational partners. Post-trained models follow instructions, admit uncertainty, refuse harmful requests, and provide structured, helpful responses.

Post-training as competitive advantage

Model providers compete intensely on post-training quality. Two companies using the same base architecture can produce very different products based on their post-training techniques. This is why Claude, ChatGPT, and Gemini feel different despite using similar underlying architectures.