tensorflow - questions on defining and calling train.momentumoptimizer -


i have questions regarding following code segment

def _optimizer(self,training_iters, global_step, opt_kwargs={}):     learning_rate = self.opt_kwargs.pop("learning_rate", 0.2)     decay_rate = self.opt_kwargs.pop("decay_rate", 0.95)     self.learning_rate_node = tf.train.exponential_decay(learning_rate=learning_rate,                                                      global_step=global_step,                                                      decay_steps=training_iters,                                                       decay_rate=decay_rate,                                                      staircase=true)     optimizer = tf.train.momentumoptimizer(learning_rate=self.learning_rate_node,                                                 **self.opt_kwargs).minimize(self.net.cost,                                                                              global_step=global_step)  

the input pararameter of opt_kwargs setup opt_kwargs=dict(momentum=0.2)

why need use self.opt_kwargs.pop("learning_rate", 0.2) assign learning_rate. guess way can inject learning rate , decay rate information dict structure of opt_kwargs. don't see real usage here.

secondly, regarding tf.train.momentumoptimizer(learning_rate=self.learning_rate_node, **self.opt_kwargs), looks **self.opt_kwargs pass whole opt_kwargs dict momentumoptimizer. however, according tf.train.momentumoptimizer.init(learning_rate, momentum, use_locking=false, name='momentum', use_nesterov=false), needs momentum value. here, passing both learning_rate , decay_rate included in self.opt_kwargs. correct way?

1.) argument pop extract learning_rate , decay_rate value , feed exponential_decay(), accepts them individual argument. 2.) it's not clean ok feed in dict entries. makes flexible ex. can swap momentumoptimizer optimizer takes in decay_rate, etc part of arguments.


Comments

Popular posts from this blog

jquery - uncaught exception: DataTables Editor - remote hosting of code not allowed -

java - SSE Emitter : Manage timeouts and complete() -

java - How to resolve error - package com.squareup.okhttp3 doesn't exist? -