How to extract all Bitcoin Addresses from block files (revxxxxx.dat)

Question

I'm running a full bitcoin node and I've access to all block files (150GB) (my server has 32GB RAM and 400GB SSD)

Any idea how to extract bitcoin addresses or hash160 from block files(revxxxxx.dat)?

Simply I need to find all used bitcoin addresses so far (finding duplicated addresses is fine)

This is my code for doing this but it's extremely slow and useless

from bitcoin.rpc import RawProxy

for blockheight in xrange(0, 543624):
    # Create a connection to local Bitcoin Core node
    p = RawProxy()

    # Get the block hash of block with height blockheight
    blockhash = p.getblockhash(blockheight)

    # Retrieve the block by its hash
    block = p.getblock(blockhash)

    # Element tx contains the list of all transaction IDs in the block
    transactions = block['tx']

    for txid in transactions:
        # Retrieve the raw transaction by ID
        try:
            raw_tx = p.getrawtransaction(txid)
        except:
            with open("error.txt", "a") as f: 
                f.write(str(blockheight) + "," + str(txid) + ",\n" )
            continue

        # Decode the transaction
        decoded_tx = p.decoderawtransaction(raw_tx)

        # Iterate through each output in the transaction
        for output in decoded_tx['vout']:
            try:
                with open('hash160.txt', 'a') as file:
                    file.write(output['scriptPubKey']['asm'].split('OP_HASH160 ')[1].split(' ')[0] + "," + output['scriptPubKey']['addresses'][0] + ",\n")
            except: 
                with open("error.txt", "a") as f: 
                    f.write(str(blockheight) + "," + str(txid) + "," + str(decoded_tx) + ",\n" )

Answer 1

If you want the hash160s , apparently blockparser is a great tool.

https://github.com/znort987/blockparser

You will likely need a lot of disk space, and crucially at least 128GB RAM from my understanding, or a large swap file and a lot of time. It apparently has a nasty habit of crashing/seg-faulting .

In the util.cpp on lines 606 and 729 , apparently the solution to make failing is to comment out // BN_CTX_init(ctx);

Edit: sorry, change to BN_CTX_free(ctx); on lines 606 and 729 .

git clone https://github.com/znort987/blockparser.git

Install deps, and in the directory run ./make and it should work out okay.

Good luck! Don't forget to check ./parser help first.