tf.matmul doesn't works as expected

Question

i try to write and (logic operation) in tensor flow , there are two inputs and two weights multiply them to get one number and add this number to bias, my problem in matmul a send X (input) and W (weight) to method in shape . [[1], [1]] for X (vertical), and [0.49900547 , 0.49900547] for W (horizontal) to get one number as result but it's give me two numbers , how i can make is multiply right ?? this is my code >>

import tensorflow as tf
import numpy
rng = numpy.random

# Parameters
learning_rate = 0.01
training_epochs = 2000
display_step = 50

# Training Data
train_X = numpy.asarray([[[1.0],[1.0]],[[1.0],[0.0]],[[0.0],[1.0]],[[0.0],[0.0]]])
train_Y = numpy.asarray([1.0,0.0,0.0,0.0])
n_samples = train_X.shape[0]

# tf Graph Input
X = tf.placeholder("float",[2,1],name="inputarr")
Y = tf.placeholder("float",name = "outputarr")

# Create Model

# Set model weights
W = tf.Variable(tf.zeros([1,2]), name="weight")
b = tf.Variable(rng.randn(), name="bias")

# Construct a linear model
activation = tf.add(tf.matmul(X,W), b)
mulres = tf.matmul(X,W)

# Minimize the squared errors
cost = tf.reduce_sum(tf.pow(activation-Y, 2))/(2*n_samples) #L2 loss
optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost) #Gradient descent

# Initializing the variables
init = tf.initialize_all_variables()

# Launch the graph
with tf.Session() as sess:
    sess.run(init)

    # Fit all training data
    for epoch in range(training_epochs):
        for (x, y) in zip(train_X, train_Y):
            sess.run(optimizer, feed_dict={X: x, Y: y})

        #Display logs per epoch step
        if epoch % display_step == 0:
            print "Epoch:", '%04d' % (epoch+1),  \
                "W=", sess.run(W), "b=", sess.run(b) , "x= ",x," y =", y," result :",sess.run(mulres,feed_dict={X: x})

    print "Optimization Finished!"
    print  "W=", sess.run(W), "b=", sess.run(b), '\n'


    # Testing example, as requested (Issue #2)
    test_X = numpy.asarray([[1.0,0.0]])
    test_Y = numpy.asarray([0])

    for x, y in zip(train_X, train_Y):
        print "x: ",x,"y: ",y
        print "Testing... (L2 loss Comparison)","result :",sess.run(mulres, feed_dict={X: x})
        print sess.run(tf.matmul(X, W),feed_dict={X: x})
        print "result :"
        predict = sess.run(activation,feed_dict={X: x})
        print predict

Answer 1

As with standard matrix multiplication, if A has shape [m, k] , and B has shape [k, n] , then tf.matmul(A, B) has shape [m, n] ( m rows, n columns in the order TensorFlow uses).

In your program, you are computing tf.matmul(X, W) . X is defined to be a placeholder with shape [2, 1] ; W is defined to be variable initialized to a [1, 2] matrix of zeros. As a result, mulres = tf.matmul(X, W) will have shape [2, 2] , which is what is printed ( result: ... ) when I run your code locally.

If you want to define a hidden layer with a single output, the change is simple:

W = tf.Variable(tf.zeros([1,2]), name="weight")

...should be replaced with:

W = tf.Variable(tf.zeros([2, 1]), name="weight")

(Indeed, initializing your weights to tf.zeros will prevent it from training, because all of the input elements will get the same gradient in backpropagation. Instead, you should initialize them randomly, for example using:

W = tf.Variable(tf.truncated_normal([2, 1], stddev=0.5), name="weight")

This will enable the network to learn different values for each component of the weight.)

Answer 2

matmul operates directly on the tensors which in your case have 2 rows and 1 column.

There is an argument in matmul to transpose either entry like:

matmul(X, W, transpose_a=True)

You can check out the docs here: docs