CS231N Lec. 4 | Backpropagation and Neural Networks

11 Oct 2020 in Studies on Lecture-Review

please find lecture reference from here¹

CS231N Lec. 4 | Backpropagation and Neural Networks

Recap gradient

• Numerical gradient
• Analytic gradient

Computational graphs

linear classification과 같은 예시.

Backpropagation

Gradient

• Concept node f 에 대해서 input(좌측)은 local gradient, output(우측)은 upstream gradient. [local gradient] x [upstream gradient]

실제로 활용 할 때 에는 이렇게 하나의 묶음으로써 자주 사용함. 그렇다면 왜 굳이 하나씩 쪼개가면서 했는가? 에 대한 생각.

• gradient 각 연산의 의미 add : distributor mul : switcher (scaler) max : router

• vectorized example
  

gradient of L2 norm is just simply take derivate it.

gradient of dot product is

신경망

activation function

1. Lecture(youtube) and PDF ↩