New at Python and Numpy, trying to create 263-dimensional arrays. I need so much dimensions for Machine Learning model. Of course one way is using numpy.zeros or numpy.ones and writing code as below :
x=np.zeros((1,1,1,1,1,1,1,1,1,1,1)) #and more 1,1,1,1
Is there an easier way to create arrays with many dimensions?
You don't need 263-dimensions . If every dimension had only size 2, you'd still have 2 ** 263
elements, which are: 14821387422376473014217086081112052205218558037201992197050570753012880593911808
You wouldn't be able to do anything with such a matrix : not even initializing on Google servers.
You either need an array with 263 values :
>>> np.zeros(263)
array([ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0.])
or a matrix with 263 vectors of M elements (let's say 3):
>>> np.zeros((263, 3))
array([[ 0., 0., 0.],
[ 0., 0., 0.],
[ 0., 0., 0.],
[ 0., 0., 0.],
...
...
[ 0., 0., 0.],
[ 0., 0., 0.],
[ 0., 0., 0.],
[ 0., 0., 0.],
[ 0., 0., 0.],
[ 0., 0., 0.]])
There are many advanced research centers that are perfectly happy with vanilla Numpy. Having to use less than 32 dimensions doesn't seem to bother them much for quantum mechanics or machine learning.
Let's start with the numpy
documentation, help(np.zeros)
gives
zeros(shape, dtype=float, order='C')
Return a new array of given shape and type, filled with zeros.
Parameters
----------
shape : int or sequence of ints
Shape of the new array, e.g., ``(2, 3)`` or ``2``.
...
Returns
-------
out : ndarray
Array of zeros with the given shape, dtype, and order.
...
The shape argument is just a list of the size of each dimension (but you probably knew that). There are lots of ways to easily create such a list in python, one quick way is
np.zeros(np.ones(263, dtype=int))
But, as others have mentioned, numpy
has a somewhat arbitrary limitation of 32 dimensions. In my experience, you can get similar and more flexible behavior by keeping an index array showing which "dimension" each row belongs to.
Most likely, for ML applications you don't actually want this:
shape = np.random.randint(1,10,(263,))
arr = np.zeros(shape) # causes a ValueError anyway
You actually want something sparse
for i, value in enumerate(nonzero_values):
arr[idx[i]] = value
idx
in this case is a (num_samples, 263)
array and nonzero_values
is a (num_samples,)
array.
ML algorithms usually work on these idx
and value
arrays (usually called X
and Y
) since the actual arrays would be enormous otherwise.
Sometimes you need a "one-hot" array of your dimensions, which will make idx.shape == (num_samples, shape.sum())
, with idx
containting only 0 or 1 values. But that's still smaller than any sort of high-dimetnsional array.
There is a new package called DimPy which can create multi-dimensional arrays in python very easily. To install usepip install dimpy
Use example
from dimpy import *
a=dim(4,5,6) # This is a 3 dimensional array of 4x5x6 elements. Use any number of dimensions within '( ) ' separated by comma
print(a)
By default every element will be zero. To change it use dfv(a, 'New value')
To express it into numpy style array, use a=npary(a)
See in more details here: https://www.respt.in/p/python-package-dimpy.html?m=1
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.