Search

Advanced Search

Find results with:

Add another field

Search by date

to

Searching collections:

Institutional Repository

Add or remove collections

Home Institutional Repository The Need for Small Learning Rates on Large Problems

Rate

To link to this object, paste this link in email, IM or document

To embed this object, paste this HTML in website

The Need for Small Learning Rates on Large Problems

View Description

Download

small (250x250 max)

medium (500x500 max)

large ( > 500x500)

Full Resolution

Print

PDF
Text

Description

Rating
Permanent URL	http://hdl.lib.byu.edu/1877/2445
Title	The Need for Small Learning Rates on Large Problems
BYU Creator	Martinez, Tony R. Wilson, D. Randall
Keywords	backpropagation; generalization; learning rate; training speed
Description/Abstract	In gradient descent learning algorithms such as error backpropagation, the learning rate parameter can have a significant effect on generalization accuracy. In particular, decreasing the learning rate below that which yields the fastest convergence can significantly improve generalization accuracy, especially on large, complex problems. The learning rate also directly affects training speed, but not necessarily in the way that many people expect. Many neural network practitioners currently attempt to use the largest learning rate that still allows for convergence, in order to improve training speed. However, a learning rate that is too large can be as slow as a learning rate that is too small, and a leaming rate that is too large or too small can require orders of magnitude more training time than one that is in an appropriate range. This paper illustrates how the learning rate affects training speed and generalization accuracy, and thus gives guidelines on how to efficiently select a learning rate that maximizes generalization accuracy.
Source	Wilson, D. R and Martinez, T. R., "The Need for Small Learning Rates on Large Problems", Proceedings of the IEEE International Joint Conference on Neural Networks IJCNN'01, pp. 115-119, 2001.;
Publisher	IEEE
Date Original	2001
Date Digital	2010-09-09
Language	English eng en
Publication Type	Peer-Reviewed Articles Conference Proceedings
Type	Text
Format	application/pdf
Owning Institution	Brigham Young University
BYU College	College of Physical and Mathematical Sciences
BYU Department	Computer Science
Copyright Status	(c) 2001 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.;
Copyright Use Information	http://www.lib.byu.edu/generic_copyright.html

Tags

Add tags for The Need for Small Learning Rates on Large Problems

View as list | View as tag cloud | report abuse

Comments

Post a Comment for The Need for Small Learning Rates on Large Problems

you wish to report:

Your comment:

Your Name:

Submit

Cancel

...

Powered by CONTENTdm®