How to store single byte of memory in Python?

Question

How can I allocate/store a single or couple of bytes (eg 2 or 4) bytes of information in Python ?

I am not looking for alternative of malloc/new in Python but may be some datatype which doesn't take huge amount of memory.

I tried the following but as shown below, all are taking huge amount of memory.

Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> i = 1 ; sys.getsizeof(i)
24
>>> i = None ; sys.getsizeof(i)
16
>>> i = 'c' ; sys.getsizeof(i)
38
>>> i = 'good' ; sys.getsizeof(i)
41
>>> i = bytearray(0) ; sys.getsizeof(i)
48
>>> i = bytearray(1) ; sys.getsizeof(i)
50
>>> from struct import *
>>> i = pack('h', 1) ; sys.getsizeof(i)
39
>>> i = array('l', [1]) ; sys.getsizeof(i)
64L

I love Python and am writing an application which will be storing some 100,000 firewall rules. Each rule will be some 500 bytes of information if I use conventional datatypes (integer, string) of Python. I want to save the space and avoid switching to C/C++ too as most of the rest of the application is in Python (2.7).

Also, I can not persist the memory as my application will check for update or modification of rules almost every 2 minutes.

My idea is to save memory by compressing the information. For example, instead of storing the 'direction' of a rule as 'input' or 'output' or 'inout' in a string or integer, I would dedicate 2 or 3 bits for marking the particular direction. With that I am assuming my one rule information can be saved into less than 10 bytes. For this, I want to know a method of storing only 2/4 bytes of information.

Appreciate your feedback / suggestions / pointers.

Answer 1

In measuring your sizes you didn't take care to exclude underlying class overhead from the size of the data stored. For example, below shows bytearray has about 48 bytes of overhead, but then each byte added takes about 1 byte. I presume the jump from 50 bytes to 53 to 56 indicates memory access optimization.

>>> i = bytearray()
>>> sys.getsizeof(i)
48
>>> i = bytearray((1))
>>> sys.getsizeof(i)
50
>>> i = bytearray((1,2))
>>> sys.getsizeof(i)
53
>>> i = bytearray((1,2,3))
>>> sys.getsizeof(i)
53
>>> i = bytearray((1,2,3,4))
>>> sys.getsizeof(i)
53
>>> i = bytearray((1,2,3,4,5))
>>> sys.getsizeof(i)
56