confidence level with crfsuite predictions

Question

I am using the CRFSuite package here http://www.chokkan.org/software/crfsuite/tutorial.html

and I have successfully used it to build a classifier and tag text. However, I'm wondering if I can get a confidence value for each prediction it makes?

It doesn't seem so. What I would really like is to get the probability of a word being each type of tag ('PER', 'LOC', 'MISC', etc), rather than just the prediction itself.

Answer 1

The API provides extracting conditional probabilities. I guess you mean the crfsuite binary does not have that as option. You could edit the source and add the option yourself

Answer 2

I hope this serves as an answer. Sklearn crfsuite provides probability for each label.

predict_marginals(X)
Make a prediction.

Parameters: X (list of lists of dicts) – feature dicts in python-crfsuite format
Returns:    y – predicted probabilities for each label at each position
Return type:    list of lists of dicts

Source: https://sklearn-crfsuite.readthedocs.io/en/latest/_modules/sklearn_crfsuite/estimator.html#CRF.predict_marginals