Features, Labels, Weights & Bias

What it is

Every supervised learning model is built from the same four parts. A feature is an input you give the model: for a house, that might be the number of bedrooms, the square footage and the age of the building. A label is the value you want the model to predict, such as the sale price. Features are what you know; the label is what you want to know.

Weights and bias are the numbers the model learns. Each feature gets a weight that says how much it matters: if square footage has a weight of 150, every extra square foot adds 150 to the prediction. The bias is a single extra number added to the result regardless of the inputs. It shifts the whole prediction up or down so the model is not forced to predict zero when all features are zero.

When I trained my first house-price model, this was the whole model: multiply each feature by its weight, add them up, add the bias. Everything else in machine learning, from linear regression to neural networks, is a more elaborate way of choosing those weights and that bias.

Why it's needed

If you cannot name the features, label, weights and bias in a problem, you cannot frame it as a learning task. Deciding what the features are is often the most important modelling decision, more than the choice of algorithm. A model with the right features and a simple algorithm usually beats a fancy algorithm with poor features.

Bias in particular is easy to dismiss as a detail, but it becomes essential once you normalise features into a 0 to 1 range. The smallest house in the dataset then has a feature value of exactly 0, and without bias the model would be forced to predict a price of 0 for it. The bias is what lets the prediction start from a sensible floor, like a taxi fare that charges a base amount before any distance is driven.

How it works

• Write the model as: prediction = (feature1 * weight1) + (feature2 * weight2) + ... + bias
• Store the features as a matrix X with one row per example and one column per feature, and the labels as a vector y
• Initialise weights and bias (zero is fine for linear models; neural networks need random values)
• Compute a prediction for every row, compare it with the label and measure the error with a loss function
• Nudge each weight and the bias in the direction that reduces the loss, then repeat until the error stops shrinking
• At prediction time, feed new feature values through the same formula with the learned weights and bias

Code example

import numpy as np

# Features: [bedrooms, square_feet, age_years] for 4 houses
X = np.array([
    [3, 1200, 20],
    [4, 1500, 5],
    [2, 800, 35],
    [5, 2200, 2],
], dtype=float)

# Labels: the sale price we want to predict (in thousands)
y = np.array([250, 340, 160, 520], dtype=float)

# One weight per feature, plus a single bias term
weights = np.array([20.0, 0.15, -1.0])  # per bedroom, per sqft, per year of age
bias = 30.0                              # base price regardless of features

# prediction = X @ weights + bias, computed for all rows at once
predictions = X @ weights + bias

for features, actual, predicted in zip(X, y, predictions):
    print(f"features={features} actual={actual:.0f} predicted={predicted:.1f}")

# The error tells us how far off the hand-picked weights are.
# Training replaces the guesswork with gradient descent.
print("mean absolute error:", np.mean(np.abs(y - predictions)).round(1))

Use cases

• Every linear and logistic regression model
• The final output layer of a neural network, which is exactly a weighted sum plus bias
• Any model that combines several measured inputs into one numeric prediction
• Framing a new business problem: deciding what to measure (features) and what to predict (label) before writing any code

Common mistakes

• Confusing the label with a feature: if the sale price is in your feature matrix, the model will learn to copy it and look perfect in training while being useless in production
• Leaving out the bias term, then wondering why the model cannot fit data that does not pass through the origin
• Treating categorical values like 'city' as numeric features by assigning them 1, 2, 3; the model will assume city 3 is 'three times' city 1
• Reading a weight as feature importance before scaling: a weight of 0.15 on square feet and 20 on bedrooms says nothing about importance until both features share a scale
• Including features that will not be available at prediction time, such as 'days on market' when predicting the listing price

Interview questions