Linear Regression in Machine Learning: A JavaScript Example

Linear regression models the relationship between a numeric input variable x and a numeric outcome y with a straight line. In supervised machine learning, the model learns the line from paired training examples and can then estimate y for a new value of x.

For a simple regression with one input, the model is:

ŷ = ax + b

• ŷ is the predicted outcome.
• x is the input value.
• a is the slope.
• b is the y-intercept.

Start with a scatter plot

The following data uses house size as x and price as y. A scatter plot helps reveal whether a roughly linear trend is plausible before fitting a model.

const xArray = [50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150];
const yArray = [7, 8, 8, 9, 9, 9, 10, 11, 14, 14, 15];

const data = [{
  x: xArray,
  y: yArray,
  mode: "markers",
  type: "scatter"
}];

const layout = {
  xaxis: { title: "Square meters" },
  yaxis: { title: "Price in millions" },
  title: "House price versus size"
};

Plotly.newPlot("myPlot", data, layout);

A line through two endpoints is not regression

Drawing a line between the smallest and largest observations can illustrate a trend, but it ignores the other data points. It is not a fitted least-squares model and can change dramatically if either endpoint is unusual.

Similarly, calculating sum(y) / sum(x) forces the line through zero. That is appropriate only for a regression model whose intercept is deliberately fixed at zero; it is not the general formula for the slope.

Calculate the least-squares line

Ordinary least squares chooses the slope and intercept that minimize the sum of the squared vertical differences between the observed values and the line's predictions.

const xArray = [50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150];
const yArray = [7, 8, 8, 9, 9, 9, 10, 11, 14, 14, 15];

if (xArray.length !== yArray.length || xArray.length < 2) {
  throw new Error("x and y must contain the same number of observations");
}

const n = xArray.length;
let xSum = 0;
let ySum = 0;
let xxSum = 0;
let xySum = 0;

for (let i = 0; i < n; i++) {
  xSum += xArray[i];
  ySum += yArray[i];
  xxSum += xArray[i] * xArray[i];
  xySum += xArray[i] * yArray[i];
}

const denominator = n * xxSum - xSum * xSum;
if (denominator === 0) {
  throw new Error("Regression requires variation in x");
}

const slope = (n * xySum - xSum * ySum) / denominator;
const intercept = (ySum - slope * xSum) / n;

const predict = x => slope * x + intercept;
console.log({ slope, intercept, priceAt125: predict(125) });

The input arrays must have equal lengths. The denominator is zero when every x value is identical, because the data contains no horizontal variation from which to estimate a slope.

Plot the fitted regression line

Generate two endpoints from the fitted equation and add them as a second Plotly trace.

const minX = Math.min(...xArray);
const maxX = Math.max(...xArray);

const traces = [
  { x: xArray, y: yArray, mode: "markers", type: "scatter", name: "Observed" },
  {
    x: [minX, maxX],
    y: [predict(minX), predict(maxX)],
    mode: "lines",
    type: "scatter",
    name: "Regression line"
  }
];

Plotly.newPlot("myPlot", traces, layout);

Check whether a linear model is suitable

A fitted line does not prove that the relationship is truly linear or causal. Inspect residuals—the differences between observed and predicted values—and evaluate the model on data that was not used to fit it. Large curves or patterns in the residuals suggest that the straight-line assumption is missing structure.

Polynomial regression

Polynomial regression adds terms such as x² or x³ so the fitted relationship can curve. It is still estimated from data, but it does not draw a straight line through the points.

A higher-degree polynomial can follow training data closely and still predict poorly on new observations. Choose model complexity using validation data rather than increasing the degree until the training plot looks perfect.