Thin plate spline interpolation of 3D stack python

Question

My problem is as follows. I have a 2d image of some tissue and a 3d stack of the same region of the tissue and plus more tissue that does not go into my 2d image. Now, the 3d stack is slightly rotated with respect to the 2d image, but also has some local deformation, so I can't simply apply a rigid rotation transformation. I can scroll through the 3d stack and find individual features that are common to the 2d image. I want to apply a nonlinear transformation such that in the end I can find my source 2d image as a flat plane in the 2d stack.

My intuition is that I should use thin plate spline for this, may the scipy RBF interpolator, but my brain stops working when I try to implement it. I would use as input arguments let's say 3 points (x1, y1, 0), (x2, y2, 0) and (x3, y3, 0) with some landmarks on the 2d image and then (x1', y1', z1'), (x2', y2', z2') and (x3', y3', z3') for the corresponding points into the 3d stack. And then I get a transformation but how do I actually apply this to an image? The bit that confuses me is that I'm working with a 3D matrix of intensities, not a meshgrid.

Answer 1

scipy RBF is designed to interpolate scattered data, it's just a spline interpolator. To warp a domain, however, you need to find another library or write TPS (thin plate spline) yourself; scipy doesn't do it. I recommend you check VTK, for example. You feed your landmark information of the reference image and the target image to a vtkThinPlateSplineTransform object. Then you can get the transformation matrix and feed it to a vtkImageReslice object, which warps your image accordingly.