x and y must have same first dimension, but have shapes (30,) and (1,)

Question

My Python version: Python 3.6.1.

When I test the following program, I get some errors.

Probably the error is caused by the dimension of training_cost since the dimension of training_cost is 1x3.

"""
multiple_eta
~~~~~~~~~~~~~~~
This program shows how different values for the learning rate affect
training.  In particular, we'll plot out how the cost changes using
three different values for eta.
"""

# Standard library
import json
import random
import sys
# My library
sys.path.append('../')
import mnist_loader
import network2
# Third-party libraries
import matplotlib.pyplot as plt
import numpy as np

# Constants
LEARNING_RATES = [0.025, 0.25, 2.5]
COLORS = ['#2A6EA6', '#FFCD33', '#FF7033']
NUM_EPOCHS = 30

def main():
    run_network()
    make_plot()

def run_network():
    """Train networks using three different values for the learning rate,
    and store the cost curves in the file ``multiple_eta.json``, where
    they can later be used by ``make_plot``.
    """
    # Make results more easily reproducible
    random.seed(12345678)
    np.random.seed(12345678)
    training_data, validation_data, test_data = mnist_loader.load_data_wrapper()
    results = []
    for eta in LEARNING_RATES:
        print ("\nTrain a network using eta = "+str(eta))
        net = network2.Network([784,30,10])
        results.append(net.SGD(training_data, NUM_EPOCHS, 10, eta, lmbda = 5.0,
                      evaluation_data=validation_data, monitor_training_cost=True))
    f = open("multiple_eta.json","w")
    json.dump(results,f)
    f.close()

def make_plot():
    f = open("multiple_eta.json", "r")
    results = json.load(f)
    f.close()
    fig = plt.figure()
    ax = fig.add_subplot(111)
    for eta, result, color in zip(LEARNING_RATES, results, COLORS):
        _,_,training_cost,_ = result
        print(training_cost)
    ax.plot(np.arange(NUM_EPOCHS), training_cost, "o-",label = "$\eta$ = "+str(eta),color = color)
    ax.set_xlim([0,NUM_EPOCHS])
    ax.set_xlabel('Epoch')
    ax.set_ylabel('Cost')
    plt.legend(loc = 'upper right')
    plt.show()

if __name__ == "__main__":
    main()

The error information :

Train a network using eta = 0.025
Epoch 0 training complete
Cost on training data: 731.8195315067348

Train a network using eta = 0.25
Epoch 0 training complete
Cost on training data: 2526.705883226454

Train a network using eta = 2.5
Epoch 0 training complete
Cost on training data: 14014.828642157932
[731.8195315067348]
[2526.705883226454]
[14014.828642157932]
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-11-b81c2d8fafc3> in <module>()
     64 
     65 if __name__ == "__main__":
---> 66     main()

<ipython-input-11-b81c2d8fafc3> in main()
     26 def main():
     27     run_network()
---> 28     make_plot()
     29 
     30 def run_network():

<ipython-input-11-b81c2d8fafc3> in make_plot()
     56         _,_,training_cost,_ = result
     57         print(training_cost)
---> 58     ax.plot(np.arange(NUM_EPOCHS), training_cost, "o-",label = "$\eta$ = "+str(eta),color = color)
     59     ax.set_xlim([0,NUM_EPOCHS])
     60     ax.set_xlabel('Epoch')

c:\users\ray\appdata\local\programs\python\python36\lib\site-packages\matplotlib\__init__.py in inner(ax, *args, **kwargs)
   1896                     warnings.warn(msg % (label_namer, func.__name__),
   1897                                   RuntimeWarning, stacklevel=2)
-> 1898             return func(ax, *args, **kwargs)
   1899         pre_doc = inner.__doc__
   1900         if pre_doc is None:

c:\users\ray\appdata\local\programs\python\python36\lib\site-packages\matplotlib\axes\_axes.py in plot(self, *args, **kwargs)
   1404         kwargs = cbook.normalize_kwargs(kwargs, _alias_map)
   1405 
-> 1406         for line in self._get_lines(*args, **kwargs):
   1407             self.add_line(line)
   1408             lines.append(line)

c:\users\ray\appdata\local\programs\python\python36\lib\site-packages\matplotlib\axes\_base.py in _grab_next_args(self, *args, **kwargs)
    405                 return
    406             if len(remaining) <= 3:
--> 407                 for seg in self._plot_args(remaining, kwargs):
    408                     yield seg
    409                 return

c:\users\ray\appdata\local\programs\python\python36\lib\site-packages\matplotlib\axes\_base.py in _plot_args(self, tup, kwargs)
    383             x, y = index_of(tup[-1])
    384 
--> 385         x, y = self._xy_from_xy(x, y)
    386 
    387         if self.command == 'plot':

c:\users\ray\appdata\local\programs\python\python36\lib\site-packages\matplotlib\axes\_base.py in _xy_from_xy(self, x, y)
    242         if x.shape[0] != y.shape[0]:
    243             raise ValueError("x and y must have same first dimension, but "
--> 244                              "have shapes {} and {}".format(x.shape, y.shape))
    245         if x.ndim > 2 or y.ndim > 2:
    246             raise ValueError("x and y can be no greater than 2-D, but have "

ValueError: x and y must have same first dimension, but have shapes (30,) and (1,)

The original code can be found here . How to fix this problem?

Answer 1

Tracing the stack back it looks like the problem is in

ax.plot(np.arange(NUM_EPOCHS), training_cost,  ...

NUM_EPOCHS is probably 30, since that's the shape of the x in the error message. training_cost must have shape (1,). For an ordinary x,y plot, both variables should have the same number of points.

So why is training_cost just one item? Your print shows that you set that in a loop, and that the last value before exiting the loop is

[14014.828642157932]

Why are you trying to plot that one value? against an axis of 30 x points?

What does results = json.load(f) give?

Answer 2

you need matrix "np.arange(NUM_EPOCHS)"

ax.plot(np.mat(np.arange(NUM_EPOCHS)), training_cost, "o-",label = "$\eta$ = "+str(eta),color = color)

if can't solve maybe be need transpose the "matrix" according to "training_cost"

ax.plot(np.mat(np.arange(NUM_EPOCHS)).transpose(), training_cost, "o-",label = "$\eta$ = "+str(eta),color = color)