将元组列表的列表转换为python中的列表列表的元组

Question

I am writing a program for NER tagging with nltk and Mallet. I have to convert between two formats of input data that I cannot change.

The data basically contains words with their associated tags for supervised learning, but there's a subdivision of the data into sentences, hence the list of lists.

The first format is

tuple(list(list(word)),list(list(tag))) 

and the second format is

list(list(tuple(word,tag))

Currently I am converting it like this (format 2 => format 1):

([[tup[0] for tup in sent] for sent in train_set],
 [[tup[1] for tup in sent] for sent in train_set])

Sample data:

 [[('Steve','PERSON'),('runs','NONE'),('Apple','ORGANIZATION')],[('Today','NONE'),('is','NONE'),('June','DATETIME'),('27th','DATETIME')]]

and expected output:

 ([['Steve', 'runs', 'Apple' ],['Today','is','June','27th']],
  [['PERSON','NONE','ORGANIZATION'],['NONE','NONE','DATETIME','DATETIME']])

I perform conversion in both directions

EDIT: I don't necessarily want it to be shorter - please just suggest a better (and more readable) way of doing it in python 2.7 (with code sample).

Answer 1

Converting list(list(tuple(word,tag)) to tuple(list(list(word)),list(list(tag))) is easy:

def convert(data_structure):
     sentences, tags = data_structure
     container = []
     for i in xrange(len(sentences)):
         container.append(zip(sentences[i], tags[i]))

     return container

The code for converting into the other direction is a bit longer but not very complicated if you simply use nested for loops:

def convert(data_structure):
    sentences = []
    tags = []

    for sentence in data_structure:
        sentence_words = []
        sentence_tags = []

        for word, tag in sentence:
            sentence_words.append(word)
            sentence_tags.append(tag)

        sentences.append(sentence_words)
        tags.append(sentence_tags)

    return (sentences, tags)

Perhaps the code can be shortened more but the general principle should be clear, hopefully.

Answer 2

You can convert the inner tuples to iterators(using iter ) and then call next on them in a nested list comprehension:

lis = [[('Steve','PERSON'),('runs','NONE'),('Apple','ORGANIZATION')],
       [('Today','NONE'),('is','NONE'),('June','DATETIME'),('27th','DATETIME')]]

it = [[iter(y) for y in x] for x in lis]
n = len(lis[0][0])  #Number of iterations required.
print [[[next(x) for x in i] for i in it] for _ in range(n)]

Output:

[[['Steve', 'runs', 'Apple'], ['Today', 'is', 'June', '27th']],
 [['PERSON', 'NONE', 'ORGANIZATION'], ['NONE', 'NONE', 'DATETIME', 'DATETIME']]]

Answer 3

I think the correct solution will be this one:

>>> data = [[('Steve','PERSON'),('runs','NONE'),('Apple','ORGANIZATION')],[('Today','NONE'),('is','NONE'),('June','DATETIME'),('27th','DATETIME')]]
>>> tuple([ map(list, (zip(*x))) for x in data ])
([['Steve', 'runs', 'Apple'], ['PERSON', 'NONE', 'ORGANIZATION']], [['Today', 'is', 'June', '27th'], ['NONE', 'NONE', 'DATETIME', 'DATETIME']])