Reverse engineering a BerkeleyDB database

Question

I have an eBay Turbolister database I am trying to read. Reading the keys via Perl gives:

ItemBLOBs
ItemBLOBs_BLOBIndex
ItemBLOBs_ItemId
Items
Items_FolderId
Items_Modified
Items_Status
ProductVariations
ProductVariations_ITEM_ID
ProductVariations_SYS_SKU
RawDescriptions
VariationPictureBLOBs
VariationPictureBLOBs_BLOBIndex
VariationPictureBLOBs_ItemId
VariationPictureBLOBs_VARIATION_VALUE
VariationPictures
VariationPictures_BLOBOrder
VariationPictures_ItemId
VariationPictures_VARIATION_VALUE
VariationSpecifics
__DATA
__SEQUENCE

Reading the values is not so successful. It gives binary values, whose added length is far less than length of the database file.

The Perl script I am using is:

#!/usr/bin/perl
use strict;
use BerkeleyDB;

my $filename = 'database';
my %o;

tie %o, "BerkeleyDB::Btree", -Filename => $filename,
                             -Flags    => DB_RDONLY
or die "Cannot open database '$filename: $!\n";

foreach (keys %o)
      { print "$_\n" }

My ultimate goal is to use Java, but right now I can't even read the keys with Java.

Any ideas how to proceed?

A note. For any solution to work I had to switch from BDB Java Edition to the native Java binding.

Answer 1

I do it in Java something like the following:

public static void main(final String[] args) throws Exception
{
    final EnvironmentConfig environmentConfig = new EnvironmentConfig();
    //setup EnvironmentConfig

    final DatabaseConfig primaryConfig = new DatabaseConfig();
    //setup DatabaseConfig

    final SecondaryConfig secondaryConfig = new SecondaryConfig();
    //setup SecondaryConfig if necessary

    final Database primary = environment.openDatabase(null,
            pathToDb, null, primaryConfig);

    final DatabaseEntry key = new DatabaseEntry(new byte[ONE_KIBIBYTE]);
    final DatabaseEntry value = new DatabaseEntry(new byte[ONE_MEBIBYTE]);

    final Cursor cursor = primary.openCursor(null, null);

    OperationStatus status = cursor.getFirst(key, value, null);
    int count = 0;
    while (status.equals(OperationStatus.SUCCESS))
    {
        ++count;
        int readId = -1;
        int size = -1;
        try
        {
            final TupleInput input = new TupleInput(value.getData(), value
                    .getOffset(), value.getSize());
            size = value.getSize();
            readId = input.readInt();

            //read other fields from 'input' local variable

            if (status.equals(OperationStatus.SUCCESS))
            {
                status = cursor.getNext(key, value, null);
            }
        }
        catch (final Exception e)
        {
            throw new RuntimeException(e);
        }
    }
}