python正则表达式否定超前方法

Question

I'm now extracting firm's name from the text data(10-k statement data).

I first tried using nltk StanfordTagger and extracted all the word tagged as organization. However, it quiet often failed to recall all the names of firms, and as I'm applying tagger to every single related sentence, it took such a long time.

So, I'm trying to extract all the words starting with Capital letter(or the words characters are all comprised of Capital letters).

So I find out that the regex below helpful.

(?:[A-Z]+[a-zA-Z\-0-9]*\.?\s?)+

However, It cannot distinguish the name of segment from the name of firm.

For example,

sentence : The Company's customers include, among others, Conner Peripherals Inc.("Conner"), Maxtor Corporation ("Maxtor"). The largest proportion of Applieds consolidated net sales and profitability has been and continues to be derived from sales of manufacturing equipment in the Silicon Systems segment to the global semiconductor industry.

I want to extract Conner Peripherals Inc, Conner, Maxtor Corporation, Maxtor, Applieds, but not 'Silicon Systems' since it is the name of segment.

So, I tried using

(?:[A-Z]+[a-zA-Z\-0-9]*\.?\s?)+(?!segment|Segment)

However, it still extract 'Silicon Systems'.

Could you help me solving this problem?

(Or do you have any idea of how to extract only the firm's name from the text data?)

Thanks a lot!!!

Answer 1

You need to capture all consecutive texts! and then, mark individual words starting with caps as non-capturing( ?: ) so that you can capture consecutive words!

>>> re.findall("((?:[A-Z]+[a-zA-Z\-0-9']*\.?\s?)+)+?(?![Ss]egment)",sentence)
["The Company's ", 'Conner Peripherals Inc.', 'Conner', 'Maxtor Corporation ', 'Maxtor', 'The ', 'Applieds ', '']

Answer 2

The NLTK approach, or any machine learning, seems to be a better approach here. I can only explain what the difficulty and current issue with the regex approach are.

The problem is that the matches expected can contain space separated phrases, and you want to avoid matching a certain phrase ending with segment . Even if you correct the negative lookahead as (?!\\s*[Ss]egment) , and make the pattern linear with something like \\b[AZ][a-zA-Z0-9-]*(?:\\s+[AZ][a-zA-Z0-9-]*)*\\b\\.?(?!\\s+[sS]egment) , you will still match Silicon , a part of the unwanted match.

What you might try to do is to match all these entities and discard after matching, and only keep those entities in other contexts by capturing them into Group 1.

See the sample regex for this :

\b[A-Z][a-zA-Z0-9-]*(?:\s+[A-Z][a-zA-Z0-9-]*)*\b\.?\s+[sS]egment\b|(\b[A-Z][a-zA-Z0-9-]*(?:\s+[A-Z][a-zA-Z0-9-]*)*\b\.?)

Since it is unwieldy, you should think of building it from blocks, dynamically:

import re
entity_rx = r"\b[A-Z][a-zA-Z0-9-]*(?:\s+[A-Z][a-zA-Z0-9-]*)*\b\.?"
rx = r"{0}\s+[sS]egment\b|({0})".format(entity_rx)
s = "The Company's customers include, among others, Conner Peripherals Inc.(\"Conner\"), Maxtor Corporation (\"Maxtor\"). The largest proportion of Applieds consolidated net sales and profitability has been and continues to be derived from sales of manufacturing equipment in the Silicon Systems segment to the global semiconductor industry."
matches = filter(None, re.findall(rx, s))
print(matches) 
# => ['The Company', 'Conner Peripherals Inc.', 'Conner', 'Maxtor Corporation', 'Maxtor', 'The', 'Applieds']

So,

• \\b - matches a word boundary
• [AZ][a-zA-Z0-9-]* - an uppercase letter followed with letters/digits/ -
• (?:\\s+[AZ][a-zA-Z0-9-]*)* - zero or more sequences of
  • \\s+ - 1+ whitespaces
  • [AZ][a-zA-Z0-9-]* - an uppercase letter followed with letters/digits/ -
• \\b - trailing word boundary
• \\.? - an optional .

Then, this block is used to build

• {0}\\s+[sS]egment\\b - the block we defined before followed with
  • \\s+ - 1+ whitespaces
  • [sS]egment\\b - either segment or Segment whole words
• | - or
• ({0}) - Group 1 (what re.findall actually returns): the block we defined before.

filter(None, res) (in Python 2.x, in Python 3.x use list(filter(None, re.findall(rx, s))) ) will filter out empty items in the final list.