Reading in data from file using regex in python

Question

I have a data file with tons of data like:

{"Passenger Quarters",27.`,"Cardassian","not injured"},{"Passenger Quarters",9.`,"Cardassian","injured"},{"Passenger Quarters",32.`,"Romulan","not injured"},{"Bridge","Unknown","Romulan","not injured"}

I want to read in the data and save it in a list. I am having trouble getting the exact right code to exact the data between the { }. I don't want the quotes and the ` after the numbers. Also, data is not separated by line so how do I tell re.search where to begin looking for the next set of data?

Answer 1

At first glance, you can break this data into chunks by splitting it on the string },{ :

chunks = data.split('},{')
chunks[0] = chunks[0][1:]      # first chunk started with '{'
chunks[-1] = chunks[-1][:-1]   # last chunk ended with '}'

Now you have chunks like

"Passenger Quarters",27.`,"Cardassian","not injured"

and you can apply a regular expression to them.

Answer 2

The following will produce a list of lists, where each list is an individual record.

import re

data = '{"Passenger Quarters",27.`,"Cardassian","not injured"},{"Passenger Quarters",9.`,"Cardassian","injured"},{"Pssenger Quarters",32.`,"Romulan","not injured"},{"Bridge","Unknown","Romulan","not injured"}'

# remove characters we don't want and split into individual fields
badchars = ['{','}','`','.','"']
newdata = data.translate(None, ''.join(badchars))
fields = newdata.split(',')

# Assemble groups of 4 fields into separate lists and  append
# to the parent list.  Obvious weakness here is if there are
# records that contain something other than 4 fields
records = []
myrecord = []
recordcount = 1
for field in fields:
    myrecord.append(field)
    recordcount = recordcount + 1
    if (recordcount > 4):
        records.append(myrecord)
        myrecord = []
        recordcount = 1

for record in records:
    print record

Output:

['Passenger Quarters', '27', 'Cardassian', 'not injured']
['Passenger Quarters', '9', 'Cardassian', 'injured']
['Pssenger Quarters', '32', 'Romulan', 'not injured']
['Bridge', 'Unknown', 'Romulan', 'not injured']

Answer 3

You should do this in two passes. One to get the list of items and one to get the contents of each item:

import re
from pprint import pprint

data = '{"Passenger Quarters",27.`,"Cardassian","not injured"},{"Passenger Quarters",9.`,"Cardassian","injured"},{"Passenger Quarters",32.`,"Romulan","not injured"},{"Bridge","Unknown","Romulan","not injured"}'

# This splits up the data into items where each item is the
# contents inside a pair of braces
item_pattern = re.compile("{([^}]+)}")

# This plits up each item into it's parts. Either matching a string
# inside quotation marks or a number followed by some garbage
contents_pattern = re.compile('(?:"([^"]+)"|([0-9]+)[^,]+),?')

rows = []
for item in item_pattern.findall(data):
    row = []
    for content in contents_pattern.findall(item):
        if content[1]: # Number matched, treat it as one
            row.append(int(content[1]))
        else: # Number not matched, use the string (even if empty)
            row.append(content[0])
    rows.append(row)

pprint(rows)