TensorFlow — gradient descent optimizers

· 未分类
Authors

Here’s a list of different gradient descent optimizers:

http://sebastianruder.com/optimizing-gradient-descent

The most common one is SDG, which is also the most basic one in TF:

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/training/gradient_descent_test.py

Other versions are listed in the training folder:

https://github.com/tensorflow/tensorflow/tree/master/tensorflow/python/training

Although mentioned in the list, the learning ratio of SGD should be gradually decreasing as iteration # goes up, but there’s no guideline how large the leaning ratio should be at the initialization, at least I didn’t find any.

 

 

 

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: