使用pyparsing查找关键字的前缀和后缀

Question

I'm trying to parse strings like this: aa bb first item ee ff

I need separate prefix ' aa bb ', keyword:' first item ' and suffix ' ee ff '

Prefix and suffix can be several words or even doesn't exist. Keyword is list of predefined values.

this is what I tried but it didn't work:

a = ZeroOrMore(Word(alphas)('prefix')) & oneOf(['first item', 'second item'])('word') & ZeroOrMore(Word(alphas)('suffix'))

Answer 1

First issue is your use of the '&' operator. In pyparsing, '&' produces Each expressions, which are like And s but accept the subexpressions in any order:

Word('a') & Word('b') & Word('c')

would match 'aaa bbb ccc', but also 'bbb aaa ccc', 'ccc bbb aaa', etc.

In your parser, you'll want to use the '+' operator, which produces And expressions. And s match several sub expressions, but only in the given order.

Secondly, one of the reasons for using pyparsing is to accept varying whitespace. Whitespace is an issue for parsers, especially when using str.find or regexes - in regexes, this usually manifests as lots of \\s+ fragments throughout your match expressions. In your pyparsing parser, if the input string contains 'first item' (two spaces between 'first' and 'item'), trying to match a literal string 'first item' will fail. Instead you should match the multiple words separately, probably using pyparsing's Keyword class, and let pyparsing skip over any whitespace between them. To simplify this, I wrote a short method wordphrase :

def wordphrase(s):
    return And(map(Keyword, s.split())).addParseAction(' '.join)
keywords = wordphrase('first item') | wordphrase('second item')
print(keywords)

prints:

{{"first" "item"} | {"second" "item"}}

indicating the each word will be parsed individually, with any number of spaces between the words.

Lastly, you have to write pyparsing parsers knowing that pyparsing does not do any lookahead. In your parser, the prefix expression ZeroOrMore(Word(alphas)) will match all the words in "aa bb first item ee ff" - then there is nothing left to match the keywords expression, so the parser fails. To code this in pyparsing, you have to write an expression in your ZeroOrMore for the prefix words that translates to "match every word of alphas, but first make sure we are not about to parse a keyword expression". In pyparsing, this kind of negative lookahead is implemented using NotAny , which you can create using the unary ~ operator. For readabiity we'll use keywords expression from above:

non_keyword = ~keywords + Word(alphas)
a = ZeroOrMore(non_keyword)('prefix') + keywords('word') + ZeroOrMore(Word(alphas))('suffix')

Here is a complete parser, and results using runTests against different sample strings:

def wordphrase(s):
    return And(map(Keyword, s.split())).addParseAction(' '.join)
keywords = wordphrase('first item') | wordphrase('second item')

non_keyword = ~keywords + Word(alphas)
a = ZeroOrMore(non_keyword)('prefix') + keywords('word') + ZeroOrMore(Word(alphas))('suffix')

text = """
    # prefix and suffix
    aa bb first item ee ff

    # suffix only
    first item ee ff

    # prefix only
    aa bb first item

    # no prefix or suffix
    first item

    # multiple spaces in item, replaced with single spaces by parse action
    first   item
    """

a.runTests(text)

Gives:

# prefix and suffix
aa bb first item ee ff
['aa', 'bb', 'first item', 'ee', 'ff']
- prefix: ['aa', 'bb']
- suffix: ['ee', 'ff']
- word: 'first item'

# suffix only
first item ee ff
['first item', 'ee', 'ff']
- suffix: ['ee', 'ff']
- word: 'first item'

# prefix only
aa bb first item
['aa', 'bb', 'first item']
- prefix: ['aa', 'bb']
- word: 'first item'

# no prefix or suffix
first item
['first item']
- word: 'first item'

# multiple spaces in item, replaced with single spaces by parse action
first   item
['first item']
- word: 'first item'

Answer 2

If I understood your question correctly this should do the trick:

toParse='aa bb first item ee ff'
keywords=['test 1','first item','test two']
for x in keywords:
    res=toParse.find(x)
    if res>=0:
        print('prefix='+toParse[0:res])
        print('keyword='+x)
        print('suffix='+toParse[res+len(x)+1:])
        break

Gives this result:

prefix=aa bb 
keyword=first item
suffix=ee ff