Learn Machine Learning

Learning is a cycle.

1. A machine learning model is trained by repeatedly iterating through the data.
2. In each iteration, the weight value is adjusted.
3. The training process is complete when the number of repetitions no longer reduces the cost.

Gradient Descent

Gradient Descent is a popular algorithm for solving Artificial Intelligence (AI) problems .

 

A simple linear regression model can be used to illustrate the Gradient Descent algorithm.

The goal of linear regression is to find a linear graph that fits a set of points (x,y). This can be solved using a mathematical formula. But a machine learning algorithm can also solve this.

This is what the example above does.

It starts with a scatter plot and a linear model (y = wx + b).

Next, it trains the model to find a straight line that fits the graph. This is done by changing the weight (slope) and offset (y-axis intersection point) of the line.

 

Below is code for a Trainer Object that can solve this problem (and many others).

Training target

Create a training object that can take any number of values ​​(x,y) from the two arrays (xArr, yArr).

Set the weight to 0 and the bias to 1.

A learning constant (learnc) must be set and a cost variable defined:

For example:

function Trainer(xArray, yArray) { this.xArr = xArray; this.yArr = yArray; this.points = this.xArr.length; this.learnc = 0.00001; this.weight = 0; this.bias = 1; this.cost;

Cost function

A standard approach to solving regression problems is to use a "cost function" to measure how good a solution is.

This function uses the weights and biases from the model (y = wx + b) and returns an error, based on how well the line fits the graph.

This error calculation is done by iterating through all the points (x,y) on the graph and summing the squares of the distances between the y value of each point and the straight line.

The most common approach is to square the intervals (to ensure a positive value) and make the error function differentiable.

this.costError = function() { total = 0; for (let i = 0; i < this.points; i++) { total += (this.yArr[i] - (this.weight * this.xArr[i] + this.bias)) **2; } return total / this.points; }

Another name for the Cost Function is the Error Function.

The formula used in this function is essentially:

1. E is the error (cost)
2. N is the total number of observations (points).
3. y is the value (label) of each observation.
4. x is the value (characteristic) of each observation.
5. m is the slope (weight)
6. b is the intercept (or deviation) coefficient.
7. mx + b is the predicted value
8. ∑1/(N * N) is the square of the mean value.

Training function

Now, we will run the Gradient Descent algorithm.

The Gradient Descent algorithm will move the cost function towards the best straight line.

 

Each iteration will update both m and b toward the line with the lower cost (error).

To do that, we add a training function that iterates through the entire data multiple times:

this.train = function(iter) { for (let i = 0; i < iter; i++) { this.updateWeights(); } this.cost = this.costError(); }

Weight update function

The training function above will update the weights and bias in each iteration.

The direction of movement is calculated using two partial derivatives:

this.updateWeights = function() { let wx; let w_deriv = 0; let b_deriv = 0; for (let i = 0; i < this.points; i++) { wx = this.yArr[i] - (this.weight * this.xArr[i] + this.bias); w_deriv += -2 * wx * this.xArr[i]; b_deriv += -2 * wx; } this.weight -= (w_deriv / this.points) * this.learnc; this.bias -= (b_deriv / this.points) * this.learnc; }

Create your own library

Library code:

function Trainer(xArray, yArray) { this.xArr = xArray; this.yArr = yArray; this.points = this.xArr.length; this.learnc = 0.00001; this.weight = 0; this.bias = 1; this.cost; // Cost Function this.costError = function() { total = 0; for (let i = 0; i < this.points; i++) { total += (this.yArr[i] - (this.weight * this.xArr[i] + this.bias)) **2; } return total / this.points; } // Train Function this.train = function(iter) { for (let i = 0; i < iter; i++) { this.updateWeights(); } this.cost = this.costError(); } // Update Weights Function this.updateWeights = function() { let wx; let w_deriv = 0; let b_deriv = 0; for (let i = 0; i < this.points; i++) { wx = this.yArr[i] - (this.weight * this.xArr[i] + this.bias); w_deriv += -2 * wx * this.xArr[i]; b_deriv += -2 * wx; } this.weight -= (w_deriv / this.points) * this.learnc; this.bias -= (b_deriv / this.points) * this.learnc; } } // End Trainer Object

Now you can integrate the library into your HTML: