Keras error when trying to replicate Python results

Question

I am trying to replicate a Python Keras model on some text classification data however I run into an error whilst doing so.

Python code (which works):

# Build the model
model = Sequential()
model.add(Dense(512, input_shape=(max_words,)))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(num_classes))
model.add(Activation('softmax'))

model.compile(loss='categorical_crossentropy',
              optimizer='adam',
              metrics=['accuracy'])

history = model.fit(xx_train, yy_train,
                    batch_size = batch_size,
                    epochs = epochs,
                    verbose = 1,
                    validation_split = 0.1)

R replication (which fails on history ):

num_classes = 3
batch_size = 32
epochs = 10
max_words = 10000

model <- keras_model_sequential() %>%
  layer_embedding(input_dim = max_words, output_dim = num_classes) %>%
  layer_dense(units = 512, activation = "relu") %>%
  layer_dropout(0.5) %>%
  layer_dense(units = num_classes, activation = "softmax")

model %>% compile(
  optimizer = "adam",
  loss = "categorical_crossentropy",
  metrics = c("accuracy")
)


history <- model %>% fit(
  xx_train, yy_train,
  epochs = epochs,
  batch_size = batch_size,
  validation_split = 0.1
)

The only "difference" I see between my attempt at replicating the Python model is that I had to add in output_dim = num_classes - which doesn't seem to be required by the Python version.

I obtain this error when I go to run history on the R code.

Error in py_call_impl(callable, dots$args, dots$keywords) : 
  InvalidArgumentError: Incompatible shapes: [32] vs. [32,10000]
     [[{{node metrics_4/acc/Equal}}]]

Detailed traceback: 
  File "/data/users/msmith/.virtualenvs/r-reticulate/lib64/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 780, in fit
    steps_name='steps_per_epoch')
  File "/data/users/msmith/.virtualenvs/r-reticulate/lib64/python3.6/site-packages/tensorflow/python/keras/engine/training_arrays.py", line 363, in model_iteration
    batch_outs = f(ins_batch)
  File "/data/users/msmith/.virtualenvs/r-reticulate/lib64/python3.6/site-packages/tensorflow/python/keras/backend.py", line 3292, in __call__
    run_metadata=self.run_metadata)
  File "/data/users/msmith/.virtualenvs/r-reticulate/lib64/python3.6/site-packages/tensorflow/python/client/session.py", line 1458, in __call__
    run_metadata_ptr)

I get that the error is something to do with the shape but, the Python code works on the same data.

Thanks in advance for any help.

Edit:

I have followed the advice here: https://github.com/keras-team/keras/issues/11749

I have downgraded to keras 2.2.2 I ran the following pip3 install --user git+https://github.com/keras-team/keras.git -U however I have a few versions of Python installed on the server and not sure if R can find this keras update...

The model works when I set bactch_size = 1 but breaks on every other batch_size .

EDIT:

An additional question regarding the Python implementation. I do something like the following in Python:

tokenize.fit_on_texts(X_train) # only fit on train
xx_train = tokenize.texts_to_matrix(X_train)

However in RI do this:

xx_train <- texts_to_matrix(tokenize, X_train, mode = c("tfidf"
                                                        #"binary", 
                                                        #"count", ,
                                                        #"freq"
                                                        ))

• what is the default Python text_to_matrix mode?

Answer 1

In python version, you are not using an embedding layer while in R version you do. I don't know you're use case so I am not sure if the embedding layer should be there o not.