Why ifft2 is not working but fft2 is fine?

Question

I want to implement ifft2 using DFT matrix. The following code works for fft2.

import numpy as np

def DFT_matrix(N):
    i, j = np.meshgrid(np.arange(N), np.arange(N))
    omega = np.exp( - 2 * np.pi * 1j / N )
    W = np.power( omega, i * j ) # Normalization by sqrt(N) Not included
    return W

sizeM=40
sizeN=20
np.random.seed(0)
rA=np.random.rand(sizeM,sizeN)

rAfft=np.fft.fft2(rA)


dftMtxM=DFT_matrix(sizeM)
dftMtxN=DFT_matrix(sizeN)

# Matrix multiply the 3 matrices together 
mA = dftMtxM @ rA @ dftMtxN


print(np.allclose(np.abs(mA), np.abs(rAfft)))
print(np.allclose(np.angle(mA), np.angle(rAfft)))

To get to ifft2 I assumd I need to change only the dft matrix to it's transpose, so expected the following to work, but I got false for the last two print any suggesetion please?

import numpy as np

def DFT_matrix(N):
    i, j = np.meshgrid(np.arange(N), np.arange(N))
    omega = np.exp( - 2 * np.pi * 1j / N )
    W = np.power( omega, i * j ) # Normalization by sqrt(N) Not included
    return W

sizeM=40
sizeN=20
np.random.seed(0)
rA=np.random.rand(sizeM,sizeN)

rAfft=np.fft.ifft2(rA)


dftMtxM=np.conj(DFT_matrix(sizeM))
dftMtxN=np.conj(DFT_matrix(sizeN))

# Matrix multiply the 3 matrices together 
mA = dftMtxM @ rA @ dftMtxN


print(np.allclose(np.abs(mA), np.abs(rAfft)))
print(np.allclose(np.angle(mA), np.angle(rAfft)))

Answer 1

I am going to be building on some things from my answer to your previous question . Please note that I will try to distinguish between the terms Discrete Fourier Transform ( DFT ) and Fast Fourier Transform ( FFT ). Remember that DFT is the transform while FFT is only an efficient algorithm for performing it. People, including myself, however very commonly refer to the DFT as FFT since it is practically the only algorithm used for computing the DFT

The problem here is again the normalization of the data. It's interesting that this is such a fundamental and confusing part of any DFT operations yet I couldn't find a good explanation on the internet. I will try to provide a summary at the end about DFT normalization however I think the best way to understand this is by working through some examples yourself.

Why the comparisons fail?

It's important to note, that even though both of the allclose tests seemingly fail, they are actually not a very good method of comparing two complex number arrays.

Difference between two angles

In particular, the problem is when it comes to comparing angles. If you just take the difference of two close angles that are on the border between -pi and pi , you can get a value that is around 2*pi . The allclose just takes differences between values and checks that they are bellow some threshold. Thus in our cases, it can report a false negative.

A better way to compare angles is something along the lines of this function:

def angle_difference(a, b):
    diff = a - b
    diff[diff < -np.pi] += 2*np.pi
    diff[diff > np.pi] -= 2*np.pi
    return diff

You can then take the maximum absolute value and check that it's bellow some threshold:

np.max(np.abs(angle_difference(np.angle(mA), np.angle(rAfft)))) < threshold

In the case of your example, the maximum difference was 3.072209153742733e-12 .

So the angles are actually correct!

Magnitude scaling

We can get an idea of the issue is when we look at the magnitude ratio between the matrix iDFT and the library iFFT .

print(np.abs(mA)/np.abs(rAfft))

We find that all the values in mA are 800 , which means that our absolute values are 800 times larger than those computed by the library. Suspiciously, 800 = 40 * 20 , the dimensions of our data. I think you can see where I am going with this.

Confusing DFT normalization

We spot some indications why this is the case when we have a look at the FFT formulas as taken from the Numpy FFT documentation:

You will notice that while the forward transform doesn't normalize by anything. The reverse transform divides the output by 1/N . These are the 1D FFTs but the exact same thing applies in the 2D case, the inverse transform multiplies everything by 1/(N*M)

So in our example, if we update this line, we will get the magnitudes to agree:

mA = dftMtxM @ rA/(sizeM * sizeN) @ dftMtxN

A side note on comparing the outputs, an alternative way to compare complex numbers is to compare the real and imaginary components:

print(np.allclose(mA.real, rAfft.real))
print(np.allclose(mA.imag, rAfft.imag))

And we find that now indeed both methods agree.

Why all this normalization mess and which should I use?

The fundamental property of the DFT transform must satisfy is that iDFT(DFT(x)) = x . When you work through the math, you find that the product of the two coefficients before the sum has to be 1/N .

There is also something called the Parseval's theorem . In simple terms, it states that the energy in the signals is just the sum of square absolutes in both the time domain and frequency domain. For the FFT this boils down to this relationship:

Here is the function for computing the energy of a signal:

def energy(x):
    return np.sum(np.abs(x)**2)

You are basically faced with a choice about the 1/N factor:

1. You can put the 1/N before the DFT sum . This makes senses as then the k=0 DC component will be equal to the average of the time domain values. However you will have to multiply the energy in frequency domain by N in order to match it with time domain frequency.

    N = len(x) X = np.fft.fft(x)/N # Compute the FFT scaled by `1/N` # Energy related by `N` np.allclose(energy(x), energy(X) * N) == True # Perform some processing... Y = X * H y = np.fft.ifft(Y*N) # Compute the iFFT, remember to cancel out the built in `1/N` of ifft

2. You put the 1/N before the iDFT . This is, slightly counterintuitively, what most implementations, including Numpy do. I could not find a definitive consensus on the reasoning behind this, but I think it has something to do with the implementation efficiency. (If anyone has a better explanation for this, please leave it in the comments) As shown in the equations earlier, the energy in the frequency domain has to be divided by N to match the time domain energy.

    N = len(x) X = np.fft.fft(x) # Compute the FFT without scaling # Energy, related by 1/N np.allclose(energy(x), energy(X) / N) == True # Perform some processing... Y = X * H y = np.fft.ifft(Y) # Compute the iFFT with the build in `1/N`

3. You can split the 1/N by placing 1/sqrt(N) before each of the transforms making them perfectly symmetric. In Numpy , you can provide the parameter norm="ortho" to the fft functions which will make them use the 1/sqrt(N) normalization instead: np.fft.fft(x, norm="ortho") The nice property here is that the energy now matches in both domains.

    X = np.fft.fft(x, norm='orth') # Compute the FFT scaled by `1/sqrt(N)` # Perform some processing... # Energy are equal: np.allclose(energy(x), energy(X)) == True Y = X * H y = np.fft.ifft(Y, norm='orth') # Compute the iFFT, with scaling by `1/sqrt(N)`

---

In the end it boils down to what you need. Most of the time the absolute magnitude of your DFT is actually not that important. You are mostly interested in the ratio of various components or you want to perform some operation in the frequency domain but then transform back to the time domain or you are interested in the phase (angles). In all of these case, the normalization does not really play an important role, as long as you stay consistent.