CS231N Lec. 8 | Deep Learning Software
please find lecture reference from here1
CS231N Lec. 8 | Deep Learning Software
DeepLearning Software
There are some useful S/W which make us much easer and faster when implementing deep learning.
CPU vs GPU for Deep Learning
During deep learning, many matrix multiplication occurrs. ( e.g. Weight x Input X )
To handle this mat mul much faster, using GPU can be a good option.
Base on below CPU vs GPU comparison, we can find CPU have higher Clock speed without RAM, while GPU have many cores and memory.
This means, GPU is good for simple-parallel tasks. 
In practice, we can observe more than x60 effectiveness.
( But should aware that it’s unfair comparison. These deeplearning algorithms are optimized for GPU ) 
And, cuDNN (which is optimized code for CUDA2) can work much faster then raw CUDA code. 
Actually in practice, below frameworks already use cuDNN for mutliple cases, so in usual case, we barely need to take a look into cuDNN. ( Lecturer also didn’t ) 
Tensorflow is similar to Numpy, while PyTorch is less verbose. 
These framework activley changed. There are no answer for every tasks.
Like, tensor started faster then others so it have many users. But less effective.
While newer frameworks seems to be much effective, but less users.
Tensorflow
We can make tensorflow code as below. 
And could train. But we keep copy new value in “values”.
Meaning, new value calculated from tensor, copied into numpy array “values”.
That is, we used GPU(tensorflow) - CPU(numpy) both. This could be problem in huge model. 
Lecture(youtube) and PDF ↩
CUDA ↩
