Keras LSTM用于连续输入和连续输出

Question

For example I have binary data, let say: 0, 0, 0, 1, 1, 0, 1, 1. This may continue indefinitely. For each input, there is corresponding output. Let say we use XOR operation. So, output may look like this: 0, 0, 0, 1, 0, 1, 1, 0.

How do I shape Keras input shape? How do I set timesteps? If I declare timesteps 1 are for each 1 timestep considered different case or it can still take account of previous input as sequence or learned memory?

Keras is using LSTM or GRU for it's hidden layer.

I've tried 2 method for this problem, but none seem succeed. Both method stuck at 37.5 acc. In fact, it keep guessing 1.

Method 1:

data = [[[0], [0], [0], [1], [1], [0], [1], [1]]]
output = [[[0], [0], [0], [1], [0], [1], [1], [0]]]

model = Sequential()
model.add(GRU(10, input_shape=(8, 1), return_sequences=True))
model.add(GRU(10, return_sequences=True))
model.add(Dense(1, activation='softmax'))

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['acc'])
model.fit(np.asarray(data), np.asarray(output), epochs=3000)

Method 2:

data = [[[0]], [[0]], [[0]], [[1]], [[1]], [[0]], [[1]], [[1]]]
output = [[0], [0], [0], [1], [0], [1], [1], [0]]

model = Sequential()
model.add(GRU(10, input_shape=(1, 1), return_sequences=True))
model.add(GRU(10))
model.add(Dense(1, activation='softmax'))

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['acc'])
model.fit(np.asarray(data), np.asarray(output), epochs=300)

Answer 1

In fact, it keep guessing 1.

That's because you have used softmax as the activation of last layer. Since the last layer have only one unit and the softmax function normalizes its input such that the sum of elements equals to one, it would always outputs 1. Instead, you need to use sigmoid as the activation function of last layer to have an output between zero and one.

Answer 2

将activation='softmax'更改为activation='sigmoid'