emloop_tensorflow.hooks

Module providing a small collection of TF emloop hooks.

The hooks are expected to be created and used by emloop. For additional info about the emloop hooks system please refer to the emloop tutorial.

Classes

  • WriteTensorBoard: Write scalar epoch variables to TensorBoard summaries.
  • DecayLR: Hook for modifying (decaying) the learning rate (or any other variable) during the training.
  • InitLR: Hook for initializing the learning rate (or any other variable) before the training.
  • DecayLROnPlateau: Decay learning rate on plateau.
class emloop_tensorflow.hooks.WriteTensorBoard(model, output_dir, image_variables=None, flush_secs=10, visualize_graph=False, on_unknown_type='ignore', on_missing_variable='error', **kwargs)[source]

Bases: hooks.AbstractHook

Write scalar epoch variables to TensorBoard summaries.

Refer to TensorBoard introduction for more info.

By default, non-scalar values are ignored.

default usage
hooks:
  - emloop_tensorflow.hooks.WriteTensorboard
cast variables with unknown types to strings
hooks:
  - emloop_tensorflow.WriteTensorboard:
      on_unknown_type: str
visualize the computational graph
hooks:
  - emloop_tensorflow.hooks.WriteTensorboard:
      visualize_graph: true
Inheritance diagram of WriteTensorBoard

MISSING_VARIABLE_ACTIONS = {'error', 'ignore', 'warn'}

Action executed on missing variable.

UNKNOWN_TYPE_ACTIONS = {'error', 'ignore', 'warn'}

Possible actions to take on unknown variable type.

__init__(model, output_dir, image_variables=None, flush_secs=10, visualize_graph=False, on_unknown_type='ignore', on_missing_variable='error', **kwargs)[source]

Create new WriteTensorBoard hook.

Parameters:
  • model (BaseModel) – a BaseModel being trained
  • output_dir (str) – output dir to save the tensorboard logs
  • image_variables (Optional[Iterable[str]]) – list of image variable names
  • flush_secs (int) – flush interval in seconds
  • visualize_graph (bool) – include visualization of the computational graph (may be resource-extensive)
  • on_unknown_type (str) – an action to be taken if the variable value type is not supported (e.g. a list), one of UNKNOWN_TYPE_ACTIONS
  • on_missing_variable (str) – an action to be taken if the specified variable is not present, one of MISSING_VARIABLE_ACTIONS
after_epoch(epoch_id, epoch_data)[source]

Log the specified epoch data variables to the tensorboard.

Parameters:
  • epoch_id (int) – epoch ID
  • epoch_data (Mapping[str, object]) – epoch data as created by other hooks
Return type:

None

class emloop_tensorflow.hooks.DecayLR(model, decay_value=0.98, variable_name='learning_rate', decay_type='multiply', **kwargs)[source]

Bases: emloop.hooks.every_n_epoch.EveryNEpoch

Hook for modifying (decaying) the learning rate (or any other variable) during the training.

It expects a variable with the specified name to be present in the TF model being trained.

Each n_epochs, the variable is either multiplied or summed with the specified decay_value.

multiply learning_rate variable by 0.98 after every epoch
hooks:
  - emloop_tensorflow.hooks.DecayLR
multiply learning_rate variable by 0.999 each 5th epoch
hooks:
  - emloop_tensorflow.hooks.DecayLR:
      decay_value: 0.999
      n_epochs: 5
linear decay of my_learning_rate variable
hooks:
  - emloop_tensorflow.hooks.DecayLR:
      decay_value: -0.00001
      variable_name: my_learning_rate
      decay_type: add
Inheritance diagram of DecayLR

LR_DECAY_TYPES = {'add', 'multiply'}

Possible LR decay types.

__init__(model, decay_value=0.98, variable_name='learning_rate', decay_type='multiply', **kwargs)[source]

Create new DecayLR hook.

Parameters:
  • model (BaseModel) – TF model being trained
  • decay_value (float) – the value to modify the learning rate with
  • variable_name (str) – variable name to be modified
  • decay_type (str) – decay type, one of LR_DECAY_TYPES
_after_n_epoch(epoch_id, **_)[source]

Call _decay_variable().

Return type:None
_decay_variable()[source]

Modify the specified TF variable (now saved in self._lr) using self._decay_value.

Parameters:_ – ignore all the parameters
Return type:None
class emloop_tensorflow.hooks.InitLR(model, value, variable_name='learning_rate', **kwargs)[source]

Bases: hooks.AbstractHook

Hook for initializing the learning rate (or any other variable) before the training.

This is useful for setting up a resumed training.

It expects a variable with the specified name to be present in the TF model being trained.

initialize learning_rate variable to 0.001
hooks:
  - emloop_tensorflow.hooks.InitLR:
      value: 0.001
initialize my_variable variable to 42
hooks:
  - emloop_tensorflow.hooks.InitLR:
      variable_name: my_variable
      value: 42
Inheritance diagram of InitLR

__init__(model, value, variable_name='learning_rate', **kwargs)[source]

Create new InitLR hook.

Parameters:
  • model (BaseModel) – TF model being trained
  • value (float) – desired variable value
  • variable_name (str) – variable name to be initialize
before_training()[source]

Set self._variable to self._value.

Return type:None
class emloop_tensorflow.hooks.DecayLROnPlateau(decay_value=0.1, **kwargs)[source]

Bases: emloop.hooks.on_plateau.OnPlateau, emloop_tensorflow.hooks.decay_lr.DecayLR

Decay learning rate on plateau.

After decaying, LR may be decayed again only after additional long_term epochs.

Shares args from both DecayLR and el.hooks.OnPlateau.

multiply the learning rate by 0.1 when the mean of last 100 valid accuracy values is greater than the mean of last 30 accuracy values.
hooks:
  - emloop_tensorflow.hooks.DecayLROnPlateau:
      long_term: 100
      short_term: 30
      variable: accuracy
      objective: max
decay LR by 0.01 when valid loss plateau is detected
hooks:
  - emloop_tensorflow.hooks.DecayLROnPlateau:
      decay_value: 0.01
Inheritance diagram of DecayLROnPlateau

__init__(decay_value=0.1, **kwargs)[source]

Create new OnPlateau hook.

Parameters:
  • long_term – count of last epochs representing long training period
  • short_term – count of last epochs representing short training period
  • stream – name of the processed stream
  • variable – name of the observed variable
  • objective – observed variable objective; one of OnPlateau.OBJECTIVES
  • kwargs – ignored
Raises:

AssertionError – if long_term < short_term

_on_plateau_action(**kwargs)[source]

Call _decay_variable().

Return type:None
after_epoch(**kwargs)[source]

Call _on_plateau_action() if the long_term variable mean is lower/greater than the short_term mean.

Return type:None