Breaking News: Grepper is joining You.com. Read the official announcement!

back propagation

Innocent Iguana answered on April 27, 2023 Popularity 9/10 Helpfulness 4/10

answer back propagation

related back propagation

back propagation

Comment

- When we do forward propagation, the output always have some sort of 
	error (the computed output is way off the real output)
- The difference between the real and computed output is error
- Back propagation is a method of updating weights and bias of inputs 
	through introducing this error from the output to the whole network 
    so that the error is reduced.
- This is actually another way of applying gradient descent to each 
	individual neurons so that the weights and bias are 
    updated for optimized result. 
- slope of output node for the last hidden layer nodes = 2 * (actual value - predicted value in output node, that you get using forward propagation) * incoming input node value * slope of activation function, this is 1 for relu
- slope of weight = input node value * slope of output node * activation function slope (1 for relu)
- updated weight = weight or edge - learning rate * slope of weight
- backpropagated value in nodes other than inputs = last layer node * error * previous layers node value * slope of activation function
- It is common to calculate slopes on only a subset of the data (a batch) for computational efficiency
- Use a different batch of data to calculate the next update
- Start over from the beginning once all data is used
- Each time through the training data is called an epoch
- When slopes are calculated on one batch at a time: stochastic gradient descent
- backpropagation takes the prediction error from output layer and propagates it to the input layers through the hidden layers.
- Thus, it allows gradient descent to update all weights in the neural network (chain rule of calculus)
- Slope of node values are the sum of the slopes for all weights that come out of them
- Previous node value = node value * slope

Popularity 9/10 Helpfulness 4/10 Language whatever

Source: Grepper

Tags: back propagation whatever

Link to this answer
Share Copy Link

Contributed on Apr 27 2023

Innocent Iguana

0 Answers Avg Quality 2/10

Closely Related Answers

back propagation

Comment

weights = np.array([0, 2, 1])
input_data = np.array([1, 2, 3])
target = 0
# Set the learning rate: learning_rate
learning_rate = 0.01

# Calculate the predictions: preds (forward propagation)
preds = (input_data * weights).sum()

# Calculate the error: error
error = preds - target

# Calculate the slope of each input: slope
slope = 2 * error * input_data

# Update the weights: weights_updated (Backpropagation)
weights_updated = weights - learning_rate*slope

# Get updated predictions: preds_updated (forward propagation)
preds_updated = (weights_updated * input_data).sum()

# Calculate updated error: error_updated
error_updated = preds_updated - target

# Print the original error
print(error)

# Print the updated error
print(error_updated)

Popularity 9/10 Helpfulness 1/10 Language python

Source: Grepper

Tags: back propagation pr

Link to this answer
Share Copy Link

Contributed on Apr 27 2023

Innocent Iguana

0 Answers Avg Quality 2/10

back propagation

Contents

More Related Answers

back propagation

Closely Related Answers

back propagation

Grepper

Documentation

Social

Legal

Contact

Oops, You will need to install Grepper and log-in to perform this action.