Searching and splitting strings with special characters in python

Question

I'm scraping the timetable on Medmar website and would like to divide the route from:

"Pozzuoli » Ischia"
"Pozzuoli - Procida" 

to

"DEPARTURE PORT": 'Pozzuoli'
"ARRIVAL PORT": 'Ischia'
"DEPARTURE PORT": 'Pozzuoli'
"ARRIVAL PORT": 'Procida'

I've tried splitting the text from the list in two different ways depending if it has the "»" or "-" divider between the two ports. First I search for "»" or "-" and divide the string accordingly. For some reason, I'm getting a re error on search

re.error: unterminated character set at position 0

Code:

def port_name_regex(port_name, index):
     if re.search("[^\x00-\x7f",port_name):
        port_name = departure_port = re.split("[^\x00-\x7f]",port_name,1)[index].capitalize
        return port_name
     else:
        port_name = re.split("\w",port_name,1)[index].capitalize
        return port_name

medmar_live_departures_table = list(soup.select('li.tratta'))                
for li in medmar_live_departures_table:
    next_li = li.find_next_sibling("li")
    while next_li and next_li.get("data-toggle"):
        if next_li.get("class") == ["corsa-yes"]: 
            medmar_live_departures_data.append({  
            'DEPARTURE PORT': port_name_regex(li.text, 0),
            'ARRIVAL PORT': port_name_regex(li.text, -1),
            'DEPARTURE TIME': next_li.strong.text,
            'FERRY TYPE': "Traghetto",    
            'STATUS': "Active", 
            'OTHER INFO': "Next departure"  
           })
        elif next_li.get("class") == ["corsa-no"]:  
            medmar_live_departures_data.append({
                'DEPARTURE PORT': port_name_regex(li.text, 0),
                'ARRIVAL PORT': port_name_regex(li.text, -1),
                'DEPARTURE TIME' : next_li.strong.text,
                'FERRY TYPE': "Traghetto",  
                'STATUS': "Cancelled" 
            })
            next_li.find_next_sibling("li")
        else:    
            medmar_live_departures_data.append({
                'DEPARTURE PORT': port_name_regex(li.text, 0),
                'ARRIVAL PORT': port_name_regex(li.text, -1),
                'DEPARTURE TIME' : next_li.strong.text,
                'FERRY TYPE': "Traghetto",
                'STATUS': "Active"
            })
        next_li = next_li.find_next_sibling("li")

How do I solve this problem?

Answer 1

I ran into the same error and I solved it by replacing brackets and parentheses like so:

re.sub('\(|\)|\]|\[', '', word.lower())

The error hints at this (unterminated character set) - parens/brackets come in an open and close set, suggesting one is missing. Check your data for these characters.