tensorflow - questions on defining and calling train.momentumoptimizer -
i have questions regarding following code segment
def _optimizer(self,training_iters, global_step, opt_kwargs={}): learning_rate = self.opt_kwargs.pop("learning_rate", 0.2) decay_rate = self.opt_kwargs.pop("decay_rate", 0.95) self.learning_rate_node = tf.train.exponential_decay(learning_rate=learning_rate, global_step=global_step, decay_steps=training_iters, decay_rate=decay_rate, staircase=true) optimizer = tf.train.momentumoptimizer(learning_rate=self.learning_rate_node, **self.opt_kwargs).minimize(self.net.cost, global_step=global_step)
the input pararameter of opt_kwargs
setup opt_kwargs=dict(momentum=0.2)
why need use self.opt_kwargs.pop("learning_rate", 0.2)
assign learning_rate
. guess way can inject learning rate , decay rate information dict structure of opt_kwargs. don't see real usage here.
secondly, regarding tf.train.momentumoptimizer(learning_rate=self.learning_rate_node, **self.opt_kwargs)
, looks **self.opt_kwargs
pass whole opt_kwargs
dict momentumoptimizer
. however, according tf.train.momentumoptimizer.init(learning_rate, momentum, use_locking=false, name='momentum', use_nesterov=false), needs momentum value. here, passing both learning_rate
, decay_rate
included in self.opt_kwargs
. correct way?
1.) argument pop extract learning_rate
, decay_rate
value , feed exponential_decay()
, accepts them individual argument. 2.) it's not clean ok feed in dict entries. makes flexible ex. can swap momentumoptimizer
optimizer takes in decay_rate, etc part of arguments.
Comments
Post a Comment