OSError：[Errno 7] ubuntu上的参数列表太长，python用popen调用bitcoind-cli

Question

Running a python script calling bitcoind-cli using popen on ubuntu, on large blocks with many trasactions, when calling getrawtransaction i get the error OSError: [Errno 7] Argument list too long

i understand it's a buffer issue between the shell and the python script? there's a single argument, i guess it's just a very long one

need i check something else? can i make the buffer larger somehow or should i change the method i interact with bitcoind to RPC?

tried it on a local and an AWS ubuntu machines

thanks

Answer 1

It is your OS limitation eg:

>>> import os
>>> os.execl('/bin/ls', 'ls', 'c'*10**7)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/os.py", line 314, in execl
    execv(file, args)
OSError: [Errno 7] Argument list too long

Is it necessary to pass the data on the command line (could you use a pipe/file/socket, etc instead?) in your case? Can you run the command several times with splitted command-line arguments? See Solving “mv: Argument list too long” .

You may get the same error if the passed environment is too large:

>>> os.execle('/usr/bin/env', 'env', {'envvar': 'c'*10**9})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/os.py", line 322, in execle
    execve(file, args[:-1], env)
OSError: [Errno 7] Argument list too long

The solution is to sanitize the passed environment to avoid unused large envvars.

The limits might be hardcoded in the kernel source .

Answer 2

Since you are using Python, the best thing you can do is use RPC, such as:

import base64
import requests

response = requests.post(
    bitcoind_url, 
    data=json.dumps(
        {
            'method': method,
            'params': params,
            'jsonrpc': '2.0',
            'id': 0,
        }
    ), 
    headers={'content-type': 'application/json', 'Authorization': b'Basic ' + base64.b64encode(rpcuser + b':' + rpcpassword)})

where params is a list of the arguments for the specific method .

You can get rpcuser and rpcpassword from the bitcoind configuration file.