具有Tensorflow后端和Theano后端的Keras使用相同的模型和相同的输入进行不同的预测

Question

I tried to make predictions from a pre-trained model saved as json strings and h5 weights separately, however it seems different backend(Tensorflow and Theano) would give me different output even if the input and model are exactly the same. I found even at the very first layer, which is a 1D convolution, the activations are different, here are the code to print part of the activation from 5th filter of the convolution1D layer:

Theano version:

from keras.models import model_from_json
import numpy as np
import os
os.environ['KERAS_BACKEND'] = 'theano'

model_file = 'model.h5'
x_file = 'x.csv'

model_json = '{"class_name": "Model", "keras_version": "1.2.2", "config": {"layers": [{"class_name": "InputLayer", "config": {"batch_input_shape": [null, 1002, 6], "input_dtype": "float32", "sparse": false, "name": "input_1"}, "inbound_nodes": [], "name": "input_1"}, {"class_name": "Convolution1D", "config": {"batch_input_shape": [null, null, 6], "W_constraint": null, "b_constraint": null, "name": "convolution1d_1", "activity_regularizer": null, "trainable": true, "filter_length": 34, "init": "glorot_uniform", "bias": true, "nb_filter": 128, "input_dtype": "float32", "subsample_length": 1, "border_mode": "valid", "input_dim": 6, "b_regularizer": null, "W_regularizer": null, "activation": "relu", "input_length": null}, "inbound_nodes": [[["input_1", 0, 0]]], "name": "convolution1d_1"}], "input_layers": [["input_1", 0, 0]], "output_layers": [["convolution1d_1", 0, 0]], "name": "model_1"}}'
model = model_from_json(model_json)
model.load_weights(model_file)

x=np.loadtxt(x_file)
x = np.reshape(x,(1,x.shape[0],x.shape[1]))
y = model.predict(x)
y[0,range(230),4] 

The input and output look like: Theano version

Tensorflow version:

from keras.models import model_from_json
import numpy as np
import os
os.environ['KERAS_BACKEND'] = 'tensorflow'

model_file = 'model.h5'
x_file = 'x.csv'

model_json = '{"class_name": "Model", "keras_version": "1.2.2", "config": {"layers": [{"class_name": "InputLayer", "config": {"batch_input_shape": [null, 1002, 6], "input_dtype": "float32", "sparse": false, "name": "input_1"}, "inbound_nodes": [], "name": "input_1"}, {"class_name": "Convolution1D", "config": {"batch_input_shape": [null, null, 6], "W_constraint": null, "b_constraint": null, "name": "convolution1d_1", "activity_regularizer": null, "trainable": true, "filter_length": 34, "init": "glorot_uniform", "bias": true, "nb_filter": 128, "input_dtype": "float32", "subsample_length": 1, "border_mode": "valid", "input_dim": 6, "b_regularizer": null, "W_regularizer": null, "activation": "relu", "input_length": null}, "inbound_nodes": [[["input_1", 0, 0]]], "name": "convolution1d_1"}], "input_layers": [["input_1", 0, 0]], "output_layers": [["convolution1d_1", 0, 0]], "name": "model_1"}}'
model = model_from_json(model_json)
model.load_weights(model_file)

x=np.loadtxt(x_file)
x = np.reshape(x,(1,x.shape[0],x.shape[1]))
y = model.predict(x)
y[0,range(230),4]

The input and output look like: Tensorflow version

After a few experiments, I found Theano tend to have the "wrong" answer, here is an example to calculate the first window of the 5th filter(bias is zero in this model and I have checked that):

l=model.get_layer(index=1)
w1 = l.get_weights()[0]
w2 = l.get_weights()[1]


data1 = w1[:,0,:,4]
data2 = x[0,range(34),:]
ans=0
for i in range(6):
    ans += np.sum(np.multiply(data1[:,i],data2[:,i]))

Ans equals to 0.08544020017143339 Tensorflow gives 0.08544022, identical to what should be from my calculation, however Theano gives 0.0518605. Could anyone comes up with an explanation about that?

Answer 1

It seems the kernel weights in Theano are flipped during calcuation. Code below clearly showed the difference of Tensorflow and Theano:

import numpy as np
import os
os.environ['KERAS_BACKEND'] = 'theano'

from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation,Convolution1D
from keras.optimizers import SGD

model = Sequential()
model.add(Convolution1D(1, 2, border_mode='valid', input_shape=(6, 1),bias=True,activation='relu'))

l=model.get_layer(index=1)
w1 = l.get_weights()[0]
w2 = l.get_weights()[1]
np.random.seed(0)
x = np.random.random((1,6,1))
y = model.predict(x)


a_tf = np.sum(np.multiply(w1[:,0,0,0],x[0,range(2),0])) # y[0,0] would equal to this with Tensorflow backend
a_th = np.sum(np.multiply(np.flip(w1[:,0,0,0]),x[0,range(2),0])) # y[0,0] would equal to this with Theano backend