这个 for 循环如何在 Spacy 的自定义 NER 训练代码中工作？

Question

I am writing a code to train custom entities in Spacy's NER engine. I am stuck in understanding a small part of the code from an online tutorial. Here's a link to the tutorial . The following is the code, I am stuck understanding the two for loops under the comment # add labels . I am new to python.

import spacy
################### Train Spacy NER.###########
def train_spacy():
    TRAIN_DATA = convert_dataturks_to_spacy("dataturks_downloaded.json");
    nlp = spacy.blank('en')  # create blank Language class
    # create the built-in pipeline components and add them to the pipeline
    # nlp.create_pipe works for built-ins that are registered with spaCy
    if 'ner' not in nlp.pipe_names:
        ner = nlp.create_pipe('ner')
        nlp.add_pipe(ner, last=True)

    # add labels
    for _, annotations in TRAIN_DATA:
        for ent in annotations.get('entities'):
            ner.add_label(ent[2])

Apparently, this for loop is adding custom labels to the NER. My questions are;

1. What is an 'annotations', what is its data type? (I googled for 'spacy annotation' but couldn't find the answer)
2. Why are there two variables to the left of 'in', ('_' and 'annotation') ?
3. What does ent[2] return? What's at pos 2?

Answer 1

Your questions can mostly be answered by understanding the function convert_dataturks_to_spacy . The code for this is in the same repo as the tutorial you are following.

1. The function returns a list of tuples where each tuple is made up of (text, {"entities": entities}) . annotations are the second element of each tuple.
2. Assigning multiple variables from an output is called tuple unpacking. Basically the for loop is saying for each tuple in training data assign the first element of the tuple to _ and the second element to annotations and then do some stuff. In python _ is often used as a throw-away variable ie something that isn't used elsewhere in the code but exists in your data.
3. ent[2] is the label of the entity being tagged. Looking at the code , an entity in dataturks is tuple with 3 elements - the start position in the string, the end position in the string and the label.