使用python从以太坊区块链中提取信息

Question

I am looking to do some analysis on the Ethereum blockchain, particularly, look for correlations in the data between available hash power and transaction confirmation times. However, I am unable to make sense of how to go about downloading either of the blockchains or extract the transaction and worker information from them.

Ideally, I would download the blockchains, then use a python script to extract the relevant information from the blockchain to a CSV file or something like that?

Any pointers on how this can be achieved?

Answer 1

The standard interface

Standard Ethereum nodes can expose a json-rpc interface . It is typically accessible over local sockets (aka IPC), or over HTTP, depending on which node you have and how you start it up.

From the command line, to get the stats of the block 5,000,000 from geth :

$ curl -X POST --data '{"jsonrpc":"2.0","method":"eth_getBlockByNumber","params":["0x4c4b40", false],"id":1}' -H "Content-Type: application/json" http://localhost:8545/
{'id': 1,
 'jsonrpc': '2.0',
 'result': {'difficulty': '0x90c21c56929b2',
  'extraData': '0x743132',
  'gasLimit': '0x7a121d',
  'gasUsed': '0x79fac5',
  'hash': '0x7d5a4369273c723454ac137f48a4f142b097aa2779464e6505f1b1c5e37b5382',
  'logsBloom': '0x8584009c4dd8101162295d8604b1850200788d4c81f39044821155049d2c036a8a00d07f2a10383180984400b0290ba00293400c1d414a5018104a010220101909b918c601251215109755b90003c6a2c23490829e319a506281d9641ac39a840d3aa03e4a287900e0c09641594409a2010543016e966382c02040754030430e2d708316ec64008f0c0100c713b51f8004005bd48980143e08b22bf2262365b8b2658804a560f1028207666d10288144a5a14609a5bcb221280b13da2f4c8800d8422cc27126a46a04f08c00ca9004081d65cc75d10c62862256118481d2e881a993780808e0a00086e321a4602cb214c0044215281c2ccbca824aca00824a80',
  'miner': '0xb2930b35844a230f00e51431acae96fe543a0347',
  'mixHash': '0x94cd4e844619ee20989578276a0a9046877d569d37ba076bf2e8e34f76189dea',
  'nonce': '0x4617a20003ba3f25',
  'number': '0x4c4b40',
  'parentHash': '0xcae4df80f5862e4321690857eded0d8a40136dafb8155453920bade5bd0c46c0',
  'receiptsRoot': '0x6db67db55d5d972c59646a3bda26a39422e71fe400e4cdf9eb7f5c09b0efa7d0',
  'sha3Uncles': '0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347',
  'size': '0x5dd1',
  'stateRoot': '0x6092dfd6bcdd375764d8718c365ce0e8323034da3d3b0c6d72cf7304996b86ad',
  'timestamp': '0x5a70760d',
  'totalDifficulty': '0x7be181d83d2d77d052',
  'transactions': ['0x569c5b35f203ca6db6e2cec44bceba756fad513384e2bd79c06a8c0181273379',
   ...
   '0xaa2703c3ae5d0024b2c3ab77e5200bb2a8eb39a140fad01e89a495d73760297c'],
  'transactionsRoot': '0x91dfce7cc2174482b5ebcf6f4beedce854641982eadb1a8cf538e3206abf7836',
  'uncles': []}}

Python API

There are several "web3" libraries available for different languages, each providing an abstraction layer over json-rpc. Web3.py is an Ethereum-Foundation-funded python interface, for example.

Using Web3.py at version 4 or later (installed with pip install web3 ), you can get the same info this way:

>>> from web3.auto import w3
>>> w3.eth.getBlock('latest')
AttributeDict({'difficulty': 2760989505172940,
 'extraData': HexBytes('0x65746865726d696e652d6177732d7573312d32'),
 'gasLimit': 8000029,
 'gasUsed': 1729027,
 'hash': HexBytes('0xff07c9bba34bf864d144c39b4f99d3fc981afcaab02c3da6456c096aab51eb89'),
 'logsBloom': HexBytes('0x000200000042041002000480000080000000000000001c0400210040100801080000000000000000001040081040000000000000000020000a008000000000100200100004000021001450080000000202002004000000000002000c0041108804000000000000000000000400000111020200090811010000000074001024002000020000010000000000110000140001201044200000100002828000020000000040000040000000060200200080000000140100408000000080400000000010010002000000000010000000800402080000040000028004000080804000012800000120000210000000800020800010040010001080008000980820010001'),
 'miner': '0xEA674fdDe714fd979de3EdF0F56AA9716B898ec8',
 'mixHash': HexBytes('0xe3aeeafccb31673b210c17610d9706a51ad8f9f8bf35a8b71ea8fba5bb260f09'),
 'nonce': HexBytes('0x79d592e01fafd7e9'),
 'number': 5020225,
 'parentHash': HexBytes('0x41e96f6e823dd46f25bb0219c6ff9bccf418879d50f9f426cc40028d115ca785'),
 'receiptsRoot': HexBytes('0xdd7abf25ebb95c9629453b2d287d929b343ea86f52deed83c4a06d64a10137ad'),
 'sha3Uncles': HexBytes('0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347'),
 'size': 6699,
 'stateRoot': HexBytes('0x410941207de8fe6a4ea7841a2f8eb67a101c58d35691bd998ecec9a7e2350813'),
 'timestamp': 1517618294,
 'totalDifficulty': 2338872012321049424432,
 'transactions': [HexBytes('0x5ef662053e3acb450aefdbed9115c81c2562de71cc4907dc3a1647d0810d83ea'),
    ...
    HexBytes('0x23d0cbc238d12c11a5df5cb8f6cf04e89a8d270baf4b5de94c7285750774784e')],
 'transactionsRoot': HexBytes('0xdcaa0c4c4af12e9397e1d93312672e1aeb947262c25111a7ced7a31104135f26'),
 'uncles': []})

Answer 2

Give the RPC (--rpc) option when you start the process. Make sure you have the entire blockchain. The rpc starts a server process on localhost:8545. You can change the port as per your wish by --rpcport option.

Simply send HTTP Get requests (by CURL or some http module) to localhost:8545 and get the necessary info in JSON format. You can also use web3.js or web3.py APIs, which interface with the blockchain, basically execute on the console that is opened by the process.

https://github.com/ethereum/wiki/wiki/JSON-RPC

Answer 3

Here is a guide on how to export Ethereum data to csv https://medium.com/@medvedev1088/exporting-and-analyzing-ethereum-blockchain-f5353414a94e

It uses https://github.com/medvedev1088/ethereum-etl which outputs the data into blocks.csv , transactions.csv , erc20_transfers.csv .

blocks.csv

Column                  | Type               |
------------------------|---------------------
block_number            | bigint             |
block_hash              | hex_string         |
block_parent_hash       | hex_string         |
block_nonce             | hex_string         |
block_sha3_uncles       | hex_string         |
block_logs_bloom        | hex_string         |
block_transactions_root | hex_string         |
block_state_root        | hex_string         |
block_miner             | hex_string         |
block_difficulty        | bigint             |
block_total_difficulty  | bigint             |
block_size              | bigint             |
block_extra_data        | hex_string         |
block_gas_limit         | bigint             |
block_gas_used          | bigint             |
block_timestamp         | bigint             |
block_transaction_count | bigint             |

transactions.csv

Column              |    Type     |
--------------------|--------------
tx_hash             | hex_string  |
tx_nonce            | bigint      |
tx_block_hash       | hex_string  |
tx_block_number     | bigint      |
tx_index            | bigint      |
tx_from             | hex_string  |
tx_to               | hex_string  |
tx_value            | bigint      |
tx_gas              | bigint      |
tx_gas_price        | bigint      |
tx_input            | hex_string  |

erc20_transfers.csv

Column              |    Type     |
--------------------|--------------
erc20_token         | hex_string  |
erc20_from          | hex_string  |
erc20_to            | hex_string  |
erc20_value         | bigint      |
erc20_tx_hash       | hex_string  |
erc20_block_number  | bigint      |