StellarGraph PaddedGraphGenerator - 如何提供特定的训练、验证和测试集

Question

I'm trying to train a basic Graph Neural Network using the StellarGraph library, in particular starting from the example provided in [0].

The example works fine, but now I would like to repeat the same exercize removing the N-Fold Crossvalidation and providing specific training, validation and test sets. I'm trying to do so with the following code:

# One hot encoding
graph_training_set_labels_encoded = pd.get_dummies(graphs_training_set_labels, drop_first=True)
graph_validation_set_labels_encoded = pd.get_dummies(graphs_validation_set_labels, drop_first=True)

graphs = graphs_training_set + graphs_validation_set

# Graph generator preparation
generator = PaddedGraphGenerator(graphs=graphs)

train_gen = generator.flow([x for x in range(0, len(graphs_training_set))],
                           targets=graph_training_set_labels_encoded,
                           batch_size=batch_size)

valid_gen = generator.flow([x for x in range(len(graphs_training_set),
                                            len(graphs_training_set) + len(graphs_validation_set))],
                          targets=graph_validation_set_labels_encoded,
                          batch_size=batch_size)

# Stopping criterium
es = EarlyStopping(monitor="val_loss",
                   min_delta=0,
                   patience=20,
                   restore_best_weights=True)

# Model definition
gc_model = GCNSupervisedGraphClassification(layer_sizes=[64, 64],
                                            activations=["relu", "relu"],
                                            generator=generator,
                                            dropout=dropout_value)

x_inp, x_out = gc_model.in_out_tensors()
predictions = Dense(units=32, activation="relu")(x_out)
predictions = Dense(units=16, activation="relu")(predictions)
predictions = Dense(units=1, activation="sigmoid")(predictions)

# Creating Keras model and preparing it for training
model = Model(inputs=x_inp, outputs=predictions)
model.compile(optimizer=Adam(adam_value), loss=binary_crossentropy, metrics=["acc"])

# GNN Training
history = model.fit(train_gen, epochs=num_epochs, validation_data=valid_gen, verbose=0, callbacks=[es])

# Calculate performance on the validation data
test_metrics = model.evaluate(valid_gen, verbose=0)
valid_acc = test_metrics[model.metrics_names.index("acc")]

print(f"Test Accuracy model = {valid_acc}")

Where graphs_training_set and graphs_validation_set are lists of StellarDiGraphs.

I am able to run this piece of code, but it provides NaN as result. What could be the problem?

Since it is the first time I am using StellarGraph and in particular PaddedGraphGenerator. I think my mistake rely on the usage of that generator, but providing training set and validation set in different manner didn't produce better results.

Thank you in advance.

UPDATE Fixed I typo in the code, as pointed out here (thanks to george123).

[0] https://stellargraph.readthedocs.io/en/stable/demos/graph-classification/gcn-supervised-graph-classification.html

Answer 1

I found a solution digging in the StellarGraph documentation for PaddedGraphGenerator [0] and GCN Neural Network Class GCNSupervisedGraphClassification [1]. Furthermore, I have found a similar question on StellarGraph Issue Tracker [2] which also points out to the solution.

    # Graph generator preparation
    generator = PaddedGraphGenerator(graphs=graphs)
    
    train_gen = generator.flow([x for x in range(0, num_graphs_for_training)],
                               targets=training_graphs_labels,
                               batch_size=35)
    valid_gen = generator.flow([x for x in range(num_graphs_for_training, num_graphs_for_training + num_graphs_for_validation)],
                               targets=validation_graphs_labels,
                               batch_size=35)
    
    # Stopping criterium
    es = EarlyStopping(monitor="val_loss",
                       min_delta=0.001,
                       patience=10,
                       restore_best_weights=True)
    
    # Model definition
    gc_model = GCNSupervisedGraphClassification(layer_sizes=[64, 64],
                                                activations=["relu", "relu"],
                                                generator=generator,
                                                dropout=dropout_value)
    
    x_inp, x_out = gc_model.in_out_tensors()
    predictions = Dense(units=32, activation="relu")(x_out)
    predictions = Dense(units=16, activation="relu")(predictions)
    predictions = Dense(units=1, activation="sigmoid")(predictions)
    
    # Let's create the Keras model and prepare it for training
    model = Model(inputs=x_inp, outputs=predictions)
    model.compile(optimizer=Adam(adam_value), loss=binary_crossentropy, metrics=["acc"])
    
    # GNN Training
    history = model.fit(train_gen, epochs=num_epochs, validation_data=valid_gen, verbose=1, callbacks=[es])
    
    # Evaluate performance on the validation data
    valid_metrics = model.evaluate(valid_gen, verbose=0)
    valid_acc = valid_metrics[model.metrics_names.index("acc")]
    
    # Define test set indices temporary vars
    index_begin_test_set = num_graphs_for_training + num_graphs_for_validation
    index_end_test_set = index_begin_test_set + num_graphs_for_testing
    
    test_set_indices = [x for x in range(index_begin_test_set, index_end_test_set)]
    
    # Evaluate performance on test set
    generator_for_test_set = PaddedGraphGenerator(graphs=graphs)
    test_gen = generator_for_test_set.flow(test_set_indices)
    result = model.predict(test_gen)

I hope it can be useful to others too.

UPDATE I have updated the links below to point to the master branch.

[0] https://github.com/stellargraph/stellargraph/blob/master/stellargraph/mapper/padded_graph_generator.py

[1] https://github.com/stellargraph/stellargraph/blob/master/stellargraph/layer/graph_classification.py

[2] https://github.com/stellargraph/stellargraph/issues/1980