Regression

Regression is a machine learning task where the model predicts a continuous numerical value based on input data. While classification asks "which category?" regression asks "how much?" or "how many?" Predicting house prices, forecasting quarterly revenue, estimating delivery times, and projecting customer lifetime value are all regression tasks.

How regression works

At its simplest, regression finds the mathematical relationship between input variables and an output number:

1. You provide the model with historical data — inputs paired with actual numerical outcomes
2. The model identifies the relationship between inputs and outputs
3. When given new input data, the model predicts the expected numerical outcome

The classic example: predicting house prices based on features like square footage, number of bedrooms, location, and age. The model learns that larger houses in better locations tend to cost more, and it quantifies these relationships mathematically.

Types of regression

• Linear regression: The simplest form. It assumes a straight-line relationship between input and output. "For every additional square metre, the price increases by £3,000." Fast, interpretable, and surprisingly effective for many business problems.
• Polynomial regression: Captures curved relationships. Useful when the relationship between input and output is not a straight line.
• Multiple regression: Uses multiple input variables to predict the output. House price depends on size, location, condition, and age — not just one factor.
• Deep learning regression: Uses neural networks for complex, non-linear relationships. More powerful but less interpretable.

Regression in business

Regression models drive critical business decisions:

• Sales forecasting: Predicting next quarter's revenue based on historical trends, market conditions, and pipeline data
• Pricing optimisation: Estimating the optimal price point based on demand elasticity, competitor pricing, and customer segment
• Demand planning: Predicting how many units of a product will sell in each region
• Customer lifetime value: Estimating how much revenue a customer will generate over their relationship with your company
• Risk scoring: Estimating the probability and severity of insurance claims
• Resource planning: Predicting staffing needs based on expected workload
• Marketing attribution: Estimating the revenue contribution of each marketing channel

Regression vs classification

The distinction is about the type of output:

• Classification: Discrete categories. "Will this customer churn? Yes or no." "Which department should handle this ticket?"
• Regression: Continuous numbers. "How much will this customer spend next year?" "How long will this support ticket take to resolve?"

Sometimes the same question can be framed either way. "Will this loan default?" is classification. "What is the probability this loan will default?" is regression. The choice depends on what decision you need to make.

Evaluation metrics

Regression models are evaluated differently from classifiers:

• Mean Absolute Error (MAE): The average size of prediction errors. "Our house price predictions are off by an average of £15,000."
• Root Mean Squared Error (RMSE): Similar to MAE but penalises large errors more heavily
• R-squared: How much of the output variation the model explains. An R² of 0.85 means the model explains 85% of the variation in the data.

Regression with LLMs

LLMs can perform simple regression-like tasks through prompting ("Based on these data points, estimate the trend"), but for serious numerical prediction, purpose-built regression models remain superior. LLMs are language tools — they are excellent at understanding and generating text but are not optimised for precise numerical computation.