TensorFlow — gradient descent optimizers

· 未分类

Here’s a list of different gradient descent optimizers:


The most common one is SDG, which is also the most basic one in TF:


Other versions are listed in the training folder:


Although mentioned in the list, the learning ratio of SGD should be gradually decreasing as iteration # goes up, but there’s no guideline how large the leaning ratio should be at the initialization, at least I didn’t find any.





Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: