使用copy = False的numpy.array的内存分配？

Question

If the type of X is numpy.array with dtype='f4'

Then what is the memory behavior of the line below:

X = array(X, dtype=double, copy=False, order='C', subok=True)

Does it allocate new memory space for X (Memory usage * 2)?

Answer 1

The NumPy doc has this to say about the copy argument to the array constructor:

If true (default), then the object is copied. Otherwise, a copy will only be made if array returns a copy, if obj is a nested sequence, or if a copy is needed to satisfy any of the other requirements (dtype, order, etc.).

You have provided an example of one of the "other requirements", dtype . So the answer is that in this case, copy=False is ignored, because a copy must be made to cast the float32 content into higher precision.

Answer 2

I like to run little experiments to see what "really" happens in my system. Here's a simple script:

import numpy as np
import numpy.random as rand
@profile
def test_size(N=20e6):
    x = np.zeros(N, dtype=np.float32)
    x[0] = 200
    x[N-1] = 123.555
    x[0:N] = rand.rand(N)
    x = np.array(x, copy=False, dtype=np.double, order='C', subok=True)
    x[0:N] = rand.rand(N)
    y = np.zeros(N, dtype=np.float32)
    y[0] = 500.0
    y[1000] = 123.4
    y[0:N] = rand.rand(N)
    del(x)
    del(y)

    if __name__ == '__main__':
        test_size()

And the output when profiling with python -m memory_profiler ./testProc.py is:

    Line #    Mem usage    Increment   Line Contents
================================================
     3   17.699 MiB    0.000 MiB   @profile
     4                             def test_size(N=20e6):
     5   17.707 MiB    0.008 MiB       x = np.zeros(N, dtype=np.float32)
     6   17.711 MiB    0.004 MiB       x[0] = 200
     7   17.719 MiB    0.008 MiB       x[N-1] = 123.555
     8   94.031 MiB   76.312 MiB       x[0:N] = rand.rand(N)
     9  170.332 MiB   76.301 MiB       x = np.array(x, copy=False, dtype=np.double, order='C', subok=True)
    10  170.340 MiB    0.008 MiB       x[0:N] = rand.rand(N)
    11  170.344 MiB    0.004 MiB       y = np.zeros(N, dtype=np.float32)
    12  170.344 MiB    0.000 MiB       y[0] = 500.0
    13  170.344 MiB    0.000 MiB       y[1000] = 123.4
    14  246.164 MiB   75.820 MiB       y[0:N] = rand.rand(N)
    15   93.582 MiB -152.582 MiB       del(x)
    16   17.285 MiB  -76.297 MiB       del(y)

As Joe pointed out, the memory profiler's result looks surprising. The profiler is showing memory allocation after each line. The memory footprint enlarges after the lines that fill the arrays, and shrinks when they are deleted. Results from running ps at the shell give a similar result.

Whether the process's deleted memory actually returns to the system stack is another matter.