AttributeError：“ Tensor”对象没有属性“ attention”

Question

I'm trying to use dynamic_decode in Tensorflow for an attention model. The original version is provided by https://github.com/tensorflow/nmt#decoder

learning_rate = 0.001
n_hidden = 128
total_epoch = 10000
num_units=128
n_class = n_input = 47

num_steps=8
embedding_size=30


mode = tf.placeholder(tf.bool)
embed_enc = tf.placeholder(tf.float32, shape=[None,num_steps,300])
embed_dec = tf.placeholder(tf.float32, shape=[None,num_steps,300])
targets=tf.placeholder(tf.int32, shape=[None,num_steps])

enc_seqlen = tf.placeholder(tf.int32, shape=[None])
dec_seqlen = tf.placeholder(tf.int32, shape=[None])
decoder_weights= tf.placeholder(tf.float32, shape=[None, num_steps])

with tf.variable_scope('encode'):
    enc_cell = tf.contrib.rnn.BasicRNNCell(n_hidden)
    enc_cell = tf.contrib.rnn.DropoutWrapper(enc_cell, output_keep_prob=0.5)
    outputs, enc_states = tf.nn.dynamic_rnn(enc_cell, embed_enc,sequence_length=enc_seqlen, 
                                            dtype=tf.float32,time_major=True )


attention_states = tf.transpose(outputs, [1, 0, 2])

# Create an attention mechanism
attention_mechanism = tf.contrib.seq2seq.LuongAttention(
    num_units, attention_states,
    memory_sequence_length=enc_seqlen)


decoder_cell = tf.contrib.rnn.BasicLSTMCell(num_units)
decoder_cell = tf.contrib.seq2seq.AttentionWrapper(
    decoder_cell, attention_mechanism,
    attention_layer_size=num_units)

helper = tf.contrib.seq2seq.TrainingHelper(
    embed_dec, dec_seqlen, time_major=True)
# Decoder
projection_layer = Dense(
    47, use_bias=False)
decoder = tf.contrib.seq2seq.BasicDecoder(
    decoder_cell, helper, enc_states,
    output_layer=projection_layer)
# Dynamic decoding
outputs, _ = tf.contrib.seq2seq.dynamic_decode(decoder)

But i got an error when i ran

tf.contrib.seq2seq.dynamic_decode(decoder)

and error shows as below

    Traceback (most recent call last):

  File "<ipython-input-19-0708495dbbfb>", line 27, in <module>
    outputs, _ = tf.contrib.seq2seq.dynamic_decode(decoder)

  File "D:\Anaconda3\lib\site-packages\tensorflow\contrib\seq2seq\python\ops\decoder.py", line 286, in dynamic_decode
    swap_memory=swap_memory)

  File "D:\Anaconda3\lib\site-packages\tensorflow\python\ops\control_flow_ops.py", line 2775, in while_loop
    result = context.BuildLoop(cond, body, loop_vars, shape_invariants)

  File "D:\Anaconda3\lib\site-packages\tensorflow\python\ops\control_flow_ops.py", line 2604, in BuildLoop
    pred, body, original_loop_vars, loop_vars, shape_invariants)

  File "D:\Anaconda3\lib\site-packages\tensorflow\python\ops\control_flow_ops.py", line 2554, in _BuildLoop
    body_result = body(*packed_vars_for_body)

  File "D:\Anaconda3\lib\site-packages\tensorflow\contrib\seq2seq\python\ops\decoder.py", line 234, in body
    decoder_finished) = decoder.step(time, inputs, state)

  File "D:\Anaconda3\lib\site-packages\tensorflow\contrib\seq2seq\python\ops\basic_decoder.py", line 139, in step
    cell_outputs, cell_state = self._cell(inputs, state)

  File "D:\Anaconda3\lib\site-packages\tensorflow\python\ops\rnn_cell_impl.py", line 180, in __call__
    return super(RNNCell, self).__call__(inputs, state)

  File "D:\Anaconda3\lib\site-packages\tensorflow\python\layers\base.py", line 450, in __call__
    outputs = self.call(inputs, *args, **kwargs)

  File "D:\Anaconda3\lib\site-packages\tensorflow\contrib\seq2seq\python\ops\attention_wrapper.py", line 1143, in call
    cell_inputs = self._cell_input_fn(inputs, state.attention)

AttributeError: 'Tensor' object has no attribute 'attention'

I tried installed the latest tensorflow 1.2.1 but it didn't work. Thank you for your help.

UPDATE:

The problem is if I change initial_states of BasicDecoder:

decoder = tf.contrib.seq2seq.BasicDecoder(
      decoder_cell, helper, enc_states,
      output_layer=projection_layer)

into:

decoder = tf.contrib.seq2seq.BasicDecoder(
      decoder_cell, helper,  
      decoder_cell.zero_state(dtype=tf.float32,batch_size=batch_size),
      output_layer=projection_layer)

Then it works. I have no idea if it is a correct solution because initial_states set to zero seems wired. Thank you for your help.

Answer 1

Can you write all your calls to use kwargs everywhere? ie, tf.nn.dynamic_rnn(cell=..., inputs=...) etc. i think your args are misplaced somewhere and using kwargs should resolve it.

Answer 2

Your approach is correct. i added better error messaging in tf master branch for future users. since you're using attention, you probably don't really need to pass anything through to the decoder initial state. however it's still common to feed the encoder final state in. you can do this by creating a decoder cell zero state the way you're doing, and calling its .clone method with arg cell_state=encoder_final_state. use the resulting object as the initial decoder state.