How to specify precision when reading in variables using netCDF4 in Python?

Question

I am trying to read in ADCP (ocean current speed) data and clean/process it. The data are given in x and y component velocities from the .nc file I am using, so I must calculate the resultant speed and angle from the components. I am having precision issues when using np.square to square the individual elements of the component velocity range that are within the same magnitude of the error of the instrument itself.

I suspect that this error is due to the length of the raw data (eg -0.02162790298461914 ). I would like to set the precision of the raw data to 0.001 (the inherent error of the ADCP) and recalculate to see if the calculation error decreases. Is there a way to manually set the precision when reading in the data using infile.variables['variable'][:] , and if so, how would I go about that? Or might there be a numpy method that has less error than np.square(array) ?

An individual example of the error:

x_vel = -0.02162790298461914
y_vel = -0.3665189743041992

manually calculated total_vel = 0.36588029782633846 (this manual calculation was done with python using m.sqrt((-0.3665189743041992) ** 2 + (-0.02162790298461914 ** 2)) )

sript calculated total_vel using np.sqare and np.sqrt on the numpy arrays = 0.3671565353870392

script code: total_v = np.sqrt(np.square(x_vel) + np.square(y_vel))

Answer 1

Well it seems that I simply overlooked a typo for sign convention... The output using the numpy functions is in fact correct. The issue stemmed from -1 ** 2 != (-1) ** 2 and my misuse of parentheses in calculating... Guess we all have to spend a few hours hunting down brain farts every once in a while

The manual input should have been m.sqrt((-0.3665189743041992) ** 2 + (-0.02162790298461914) ** 2)

instead of m.sqrt((-0.3665189743041992) ** 2 + (-0.02162790298461914 ** 2))

(notice the grouping of parentheses on the second term...)