How can I turn an H5 table query result into a Astropy Table?

Question

Usually I query an online database with SQL, but the database is down. I have an H5 file containing the table I need to query. I queried the table using Table.read_where('condition') , and have a list of numpy.void elements for each row that fit my criteria. Is there any way to take that list of rows and make it into a Astropy table? That's what all of my code previously used, and I'd rather not have to change it. Here is the code I've been using to try and convert it into a Astropy table:

import tables
from astropy.table import Table
import numpy as np

Data = tables.open_file('file_path','r') #opens our .h5 file
DataTable = Data.root.TableName #Points to the table

#Queries the table for rows that meet my 'Condition', and outputs a list of numpy.void's 
#containing integers and floats. Each numpy.void represents a row in my table. 
result = [row for row in DataTable.read_where('Condition')]

#I try to turn the list of rows into a Astropy table to use in the rest of my code.
resultTable = Table(rows=result,names=('Column1','Column2','Column3'))

The error I'm getting is:

Traceback (most recent call last):

  File "<ipython-input-2-aa9501cdbf2a>", line 1, in <module>
    runfile('FilePath', wdir='FilePath')

  File "Spyder File", line 827, in runfile
    execfile(filename, namespace)

  File "Spyder File", line 110, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "File Path", line 27, in <module>
    resultTable = Table(rows=result,names=('Column1','Column2','Column3'))

  File "FilePath/python3.7/site-packages/astropy/table/table.py", line 420, in __init__
    rec_data = recarray_fromrecords(rows)

  File "FilePath/lib/python3.7/site-packages/astropy/table/np_utils.py", line 196, in recarray_fromrecords
    return np.rec.fromarrays(array_list, formats=formats)

  File "FilePath/lib/python3.7/site-packages/numpy/core/records.py", line 645, in fromarrays
    _array[_names[i]] = arrayList[i]

ValueError: Can't cast from structure to non-structure, except if the structure only has a single field.

I tried passing result through np.rec.fromrecords to check if that is valid, since the [Astropy documentation] ( https://docs.astropy.org/en/stable/table/construct_table.html#construct-table ) says it has to be able to pass through that function. It works without any errors. I'm not sure where to go from here.

My alternative plan is to create a PyTables table of the made up of the rows in result , and pull columns as numpy arrays from that. I'd rather stick with just using Astropy since the code I'm using is built around Astropy and it would be easier to stick with that instead of going through and changing it to PyTables.

Answer 1

Based on the documentation for AstroPy Table , the first argument can be a NumPy structured array or 1-d homogeneous array (same type).

The PyTables (tables) function DataTable.read_where('Condition') returns a NumPy record array matching that Table's description (aka dtype in NumPy terminology). So, you want to use the returned array to create your AstroPy Table. You don't need row for row in in the Pytables call; just use result = DataTable.read_where('Condition') .

Note: By default, astropy.table will use the field names from the NumPy array. You can view them with ( print (result.dtype) . Adding the names= parameter overrides the default names with the names you provide.

Updated code showing that change below:

Data = tables.open_file('file_path','r') #opens our .h5 file
DataTable = Data.root.TableName #Points to the PyTable table

#Queries the table for rows that meet my 'Condition', and 
# returns a NumPy record array with dtype matching the table description
result = DataTable.read_where('Condition')

#reference the Numpy Array to create a AstroPy table for use in the rest of my code.
resultTable = Table(rows=result,names=('Column1','Column2','Column3'))