emloop_tensorflow.hooks¶

Classes¶

• WriteTensorBoard: Write scalar epoch variables to TensorBoard summaries.
• DecayLR: Hook for modifying (decaying) the learning rate (or any other variable) during the training.
• InitLR: Hook for initializing the learning rate (or any other variable) before the training.
• DecayLROnPlateau: Decay learning rate on plateau.

class emloop_tensorflow.hooks.WriteTensorBoard(model, output_dir, image_variables=None, flush_secs=10, visualize_graph=False, on_unknown_type='ignore', on_missing_variable='error', **kwargs)[source]¶

Bases: hooks.AbstractHook

Write scalar epoch variables to TensorBoard summaries.

Refer to TensorBoard introduction for more info.

By default, non-scalar values are ignored.

default usage¶

hooks:
  - emloop_tensorflow.hooks.WriteTensorboard

cast variables with unknown types to strings¶

hooks:
  - emloop_tensorflow.WriteTensorboard:
      on_unknown_type: str

visualize the computational graph¶

hooks:
  - emloop_tensorflow.hooks.WriteTensorboard:
      visualize_graph: true

MISSING_VARIABLE_ACTIONS = {'error', 'ignore', 'warn'}¶

Action executed on missing variable.

UNKNOWN_TYPE_ACTIONS = {'error', 'ignore', 'warn'}¶

Possible actions to take on unknown variable type.

__init__(model, output_dir, image_variables=None, flush_secs=10, visualize_graph=False, on_unknown_type='ignore', on_missing_variable='error', **kwargs)[source]¶

Create new WriteTensorBoard hook.

Parameters:

model (BaseModel) – a BaseModel being trained output_dir (str) – output dir to save the tensorboard logs image_variables (Optional[Iterable[str]]) – list of image variable names flush_secs (int) – flush interval in seconds visualize_graph (bool) – include visualization of the computational graph (may be resource-extensive) on_unknown_type (str) – an action to be taken if the variable value type is not supported (e.g. a list), one of UNKNOWN_TYPE_ACTIONS on_missing_variable (str) – an action to be taken if the specified variable is not present, one of MISSING_VARIABLE_ACTIONS

after_epoch(epoch_id, epoch_data)[source]¶

Log the specified epoch data variables to the tensorboard.

Parameters:	epoch_id (int) – epoch ID epoch_data (Mapping[str, object]) – epoch data as created by other hooks
Return type:	None

class emloop_tensorflow.hooks.DecayLR(model, decay_value=0.98, variable_name='learning_rate', decay_type='multiply', **kwargs)[source]¶

Bases: emloop.hooks.every_n_epoch.EveryNEpoch

Hook for modifying (decaying) the learning rate (or any other variable) during the training.

It expects a variable with the specified name to be present in the TF model being trained.

Each n_epochs, the variable is either multiplied or summed with the specified decay_value.

multiply learning_rate variable by 0.98 after every epoch¶

hooks:
  - emloop_tensorflow.hooks.DecayLR

multiply learning_rate variable by 0.999 each 5th epoch¶

hooks:
  - emloop_tensorflow.hooks.DecayLR:
      decay_value: 0.999
      n_epochs: 5

linear decay of my_learning_rate variable¶

hooks:
  - emloop_tensorflow.hooks.DecayLR:
      decay_value: -0.00001
      variable_name: my_learning_rate
      decay_type: add

LR_DECAY_TYPES = {'add', 'multiply'}¶

Possible LR decay types.

__init__(model, decay_value=0.98, variable_name='learning_rate', decay_type='multiply', **kwargs)[source]¶

Create new DecayLR hook.

Parameters:	model (BaseModel) – TF model being trained decay_value (float) – the value to modify the learning rate with variable_name (str) – variable name to be modified decay_type (str) – decay type, one of LR_DECAY_TYPES

_after_n_epoch(epoch_id, **_)[source]¶

Call _decay_variable().

Return type:	None

_decay_variable()[source]¶

Modify the specified TF variable (now saved in self._lr) using self._decay_value.

Parameters:	_ – ignore all the parameters
Return type:	None

class emloop_tensorflow.hooks.InitLR(model, value, variable_name='learning_rate', **kwargs)[source]¶

Bases: hooks.AbstractHook

Hook for initializing the learning rate (or any other variable) before the training.

This is useful for setting up a resumed training.

It expects a variable with the specified name to be present in the TF model being trained.

initialize learning_rate variable to 0.001¶

hooks:
  - emloop_tensorflow.hooks.InitLR:
      value: 0.001

initialize my_variable variable to 42¶

hooks:
  - emloop_tensorflow.hooks.InitLR:
      variable_name: my_variable
      value: 42

__init__(model, value, variable_name='learning_rate', **kwargs)[source]¶

Create new InitLR hook.

Parameters:	model (BaseModel) – TF model being trained value (float) – desired variable value variable_name (str) – variable name to be initialize

before_training()[source]¶

Set self._variable to self._value.

Return type:	None

class emloop_tensorflow.hooks.DecayLROnPlateau(decay_value=0.1, **kwargs)[source]¶

Bases: emloop.hooks.on_plateau.OnPlateau, emloop_tensorflow.hooks.decay_lr.DecayLR

Decay learning rate on plateau.

After decaying, LR may be decayed again only after additional long_term epochs.

Shares args from both DecayLR and el.hooks.OnPlateau.

multiply the learning rate by 0.1 when the mean of last 100 valid accuracy values is greater than the mean of last 30 accuracy values.¶

hooks:
  - emloop_tensorflow.hooks.DecayLROnPlateau:
      long_term: 100
      short_term: 30
      variable: accuracy
      objective: max

decay LR by 0.01 when valid loss plateau is detected¶

hooks:
  - emloop_tensorflow.hooks.DecayLROnPlateau:
      decay_value: 0.01

__init__(decay_value=0.1, **kwargs)[source]¶

Create new OnPlateau hook.

Parameters:	long_term – count of last epochs representing long training period short_term – count of last epochs representing short training period stream – name of the processed stream variable – name of the observed variable objective – observed variable objective; one of OnPlateau.OBJECTIVES kwargs – ignored
Raises:	AssertionError – if long_term < short_term

_on_plateau_action(**kwargs)[source]¶

Call _decay_variable().

Return type:	None

after_epoch(**kwargs)[source]¶

Call _on_plateau_action() if the long_term variable mean is lower/greater than the short_term mean.

Return type:	None