Which method should I use to determine the similarity of 2D, 3D and 4D (quaternions) vectors?

Question

I am writing some simple Unit Tests for math library.

To decide if the library generates good results I have to compare them with expected ones. Because of rounding etc. even good result will differ a bit from expected one (eg 0.701 when 0.700 was expected).

The problem is, I have to decide how similar two vectors are. I want to describe that similarity as an error proportion (for number it would be eg errorScale(3.0f /* generated */, 1.0f /* expected */) = 3.0f/1.5f = 2.0f == 200% ).

Which method should I use to determine the similarity of 2D, 3D and 4D (quaternions) vectors?

Answer 1

There's no universally good measure. In particular, for addition the absolute error is better while for multiplication the relative error is better.

For vectors the "relative error" can also be considered in terms of length and direction. If you think about it, the "acceptable outcomes" form a small area around the exact result. But what's the shape of this area? Is it an axis-aligned square (absolute errors in x and y direction)? That privileges a specific vector base. A circle might be a better shape.