Python Context Free Grammar and PCFG generation benchmarks?

Question

I know there are various functions to use to general CFGs and PCFGs in Python; however they all seem to differ in speed.

Eg: NLTK, PyParsing.

Are there any recent benchmarks comparing various attributes related to speed and memory usage?

Answer 1

If you're looking at official benchmarking of NLP tools like other computing software, you may be deeply discouraged. Sadly the NLP researchers are trying to push the accuracy system above making it realtime. (It's surely nicer to say that I score X% more than the state-of-art rather than I save Y hrs / days training my model in research).

Often they have like 1 sentence in their research paper that says how long it takes to train their system eg In average, the sampling program run on the Wikipedia dump consumed 20G memory, and each round took about one week on a single AMD Dual-Core 1000MHZ processor. from www.aclweb.org/anthology/P10-1116.pdf

Anyway, since you want some benchmarks, so here's some of the homework that you can do with some googling =) www.aclweb.org/anthology/I11-1100 ‎. But once again you realized that they are benchmarking accuracy not speed =)