Tensorflow: customise LSTM cell with subtractive gating

Question

I want to use subtractive gating which is explained in this paper I'm using Tensorflow, and currently the code is: (Using CPU)

import tensorflow.contrib.rnn as RNNCell 

tgt_cell = RNNCell.LSTMCell(num_units=flags.hidden_size, state_is_tuple=True)
tgt_dropout_cell = RNNCell.DropoutWrapper(tgt_cell, output_keep_prob=self.keep_prob)
tgt_stacked_cell= RNNCell.MultiRNNCell([tgt_dropout_cell] * self.opt.num_layers, state_is_tuple=True)

According to the paper the changes are as follows: where LSTM is:

The gating should be subtractive rather than multiplicative:

when I click on "LSTMCell" in my code, it opens rnn_cells.py and I'm not sure which part should be changed. May someone please help to make changes?

Answer 1

wow thats kind of advanced. Look like RNNCell.LSTMCell and write your own with changes you want. If you look here https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/rnn/python/ops/rnn_cell.py i guess the operation for cells are defined in call like starting from line 220 then find ops you need.