张量流多项式回归无法训练

Question

I'm trying to do polynomal regression using tensorflow. The fake data using in this code and real data that will applied to this model follow Bivariate Distribution so I want to find the sigma1, sigma2, mux, muy of Bivariate Distribution.

Here is link which describe Bivariate Distribution http://mathworld.wolfram.com/BivariateNormalDistribution.html

import math as m

import matplotlib.pyplot as plt
import numpy as np
import tensorflow as tf
from mpl_toolkits.mplot3d import Axes3D

num_points = 40
vectors_set = []

sigma_x1 = 0.01
sigma_y1 = 0.02
mu_x1 = 30
mu_y1 = 120
weight1 = 0.01
# create fake data

def normalf(x, y):
    return weight1 / (2 * m.pi * sigma_x1 * sigma_y1) * m.exp(
        -(x - mu_x1) * (x - mu_x1) / (2 * sigma_x1) - (y - mu_y1) * (y - mu_y1) / (2 * sigma_y1))



for i in range(num_points):
    x1 = np.random.normal(30, 0.1)
    y1 = np.random.normal(120, 0.1)
    z1 = normalf(x1, y1)
    vectors_set.append([x1, y1, z1])

x_data = [v[0] for v in vectors_set]
y_data = [v[1] for v in vectors_set]
z_data = [v[2] for v in vectors_set]

print('x_data :', x_data)
print('y_data :', y_data)
print('z_data :', z_data)

fig = plt.figure()
ax = Axes3D(fig)

for i in range(40):
    xs = x_data[i]
    ys = y_data[i]
    zs = z_data[i]
    ax.scatter(xs, ys, zs, color='blue')

ax.set_xlabel('X Label')
ax.set_ylabel('Y Label')

plt.show()

# train
sigma_x = tf.Variable(tf.random_uniform([1],0,1), trainable=True)
sigma_y = tf.Variable(tf.random_uniform([1],0,1), trainable=True)
mu_x = tf.Variable(tf.random_uniform([1], 129, 131), trainable=True)
mu_y = tf.Variable(tf.random_uniform([1], 29, 31), trainable=True)
weight = tf.Variable(tf.random_uniform([1],0,1), trainable=True)
var_x_data = tf.Variable(x_data)
var_y_data = tf.Variable(y_data)

z = weight/(2 * m.pi * sigma_x * sigma_y)*tf.exp(-(x_data - mu_x)*(x_data - mu_x)/(2 * sigma_x)-(y_data - mu_y)*(y_data - mu_y)/(2 * sigma_y))
loss = tf.reduce_mean(tf.square(z - z_data))

optimizer = tf.train.GradientDescentOptimizer(0.5)
train = optimizer.minimize(loss)

init = tf.global_variables_initializer()

sess = tf.Session()
sess.run(init)

for step in range(10):
    sess.run(train)
    print('step :', step, '-', sess.run(sigma_x), sess.run(sigma_y), sess.run(mu_x), sess.run(mu_y), sess.run(weight))

result

x_data : [30.019169654005868, 30.04151686293841, 30.003138702178568, 29.948259618114108, 29.973804961551597, 30.106418046307358, 30.02377731407854, 29.995462731965706, 30.060531767570676, 30.016641427558056, 29.79484435714953, 30.002353505751056, 29.93743604963893, 30.088668570426364, 29.789298610893283, 30.003070898711083, 30.07451548627123, 29.828669735419904, 29.928255899050924, 30.175214235408138, 29.930167990278186, 29.90277620297527, 29.91114439856375, 30.050973680551458, 29.881981688838813, 29.886764810970924, 30.12852171960371, 29.976742858188068, 30.102555513370692, 30.08058843864956, 30.01423060105985, 30.1417785666331, 29.94505775236722, 30.036190477298756, 29.999186746912457, 29.801194309808547, 29.86687585345513, 30.17350921771019, 29.84327182238387, 29.978272129448513]
y_data : [120.04311664719668, 120.04728396026447, 120.01752613183059, 119.96947626260086, 120.0074708368806, 119.85338543010528, 119.96354679730872, 120.0944221002095, 120.03622348805933, 119.91071772762156, 120.05201177439841, 119.88085394982495, 120.05182262414606, 119.96638039334746, 119.73626135724012, 120.0385386987263, 120.04360494847882, 119.78820975685764, 119.84457167286291, 119.94155466510361, 119.92755300397495, 120.24845049779589, 119.91834572339475, 120.04973567488224, 120.07455650203526, 120.18076686378147, 120.05745691621354, 120.00766570986937, 119.95423853371014, 120.01967657984122, 120.11763488442516, 120.0020441058596, 120.03321571537539, 120.03342406652649, 119.89723002259565, 119.89898775137291, 120.02497021225373, 120.0325009361513, 119.95872465372054, 120.04806845389801]
z_data : [7.458065212215647, 6.903780231006986, 7.892983766488841, 6.800527070280217, 7.6786335024572026, 2.639290437583994, 7.483173794841493, 6.3612714682233475, 6.41178477092057, 6.4302664723366, 0.9067154836465008, 5.578804230084947, 6.118359074221694, 5.221492189799962, 0.15189593638054374, 7.664073535801603, 5.74875474926795, 0.5975396029452897, 3.3630086876332186, 1.5742013987259473, 5.4690573960257325, 1.0600661981383745, 4.538944615534413, 6.5691812133605465, 3.451341688493068, 1.8517276371989146, 3.208238036016542, 7.734045908198488, 4.463402992774631, 5.695873253187933, 5.5737678967269515, 2.9124171786511046, 6.656749594702553, 7.24802632727239, 6.110882289123907, 0.854596336208196, 3.229916326704241, 1.7202433033042304, 2.233050305489856, 7.335873672423284]
/.local/lib/python3.5/site-packages/matplotlib/figure.py:1999: UserWarning: This figure includes Axes that are not compatible with tight_layout, so results might be incorrect.
warnings.warn("This figure includes Axes that are not compatible "
2018-05-15 09:10:05.847451: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2018-05-15 09:10:05.847481: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2018-05-15 09:10:05.847489: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
step : 0 - [ nan] [ nan] [ nan] [ nan] [ nan]
step : 1 - [ nan] [ nan] [ nan] [ nan] [ nan]
step : 2 - [ nan] [ nan] [ nan] [ nan] [ nan]
step : 3 - [ nan] [ nan] [ nan] [ nan] [ nan]
step : 4 - [ nan] [ nan] [ nan] [ nan] [ nan]
step : 5 - [ nan] [ nan] [ nan] [ nan] [ nan]
step : 6 - [ nan] [ nan] [ nan] [ nan] [ nan]
step : 7 - [ nan] [ nan] [ nan] [ nan] [ nan]
step : 8 - [ nan] [ nan] [ nan] [ nan] [ nan]
step : 9 - [ nan] [ nan] [ nan] [ nan] [ nan]

Answer 1

Looks like you are initializing your sigma_x and sigma_y to zero. This will cause a division by zero on the very first weight update and everything will be set to nan. The best bet is to initialize with high variance and then let it fit from there.