tensorflow - questions on defining and calling train.momentumoptimizer -
i have questions regarding following code segment
def _optimizer(self,training_iters, global_step, opt_kwargs={}): learning_rate = self.opt_kwargs.pop("learning_rate", 0.2) decay_rate = self.opt_kwargs.pop("decay_rate", 0.95) self.learning_rate_node = tf.train.exponential_decay(learning_rate=learning_rate, global_step=global_step, decay_steps=training_iters, decay_rate=decay_rate, staircase=true) optimizer = tf.train.momentumoptimizer(learning_rate=self.learning_rate_node, **self.opt_kwargs).minimize(self.net.cost, global_step=global_step) the input pararameter of opt_kwargs setup opt_kwargs=dict(momentum=0.2)
why need use self.opt_kwargs.pop("learning_rate", 0.2) assign learning_rate. guess way can inject learning rate , decay rate information dict structure of opt_kwargs. don't see real usage here.
secondly, regarding tf.train.momentumoptimizer(learning_rate=self.learning_rate_node, **self.opt_kwargs), looks **self.opt_kwargs pass whole opt_kwargs dict momentumoptimizer. however, according tf.train.momentumoptimizer.init(learning_rate, momentum, use_locking=false, name='momentum', use_nesterov=false), needs momentum value. here, passing both learning_rate , decay_rate included in self.opt_kwargs. correct way?
1.) argument pop extract learning_rate , decay_rate value , feed exponential_decay(), accepts them individual argument. 2.) it's not clean ok feed in dict entries. makes flexible ex. can swap momentumoptimizer optimizer takes in decay_rate, etc part of arguments.
Comments
Post a Comment