What is the marginal probabilities formula used in CRF++?

Question

CRF++ says it can:

"Can output marginal probabilities for all candidates" on its page: http://crfpp.sourceforge.net/

But what's the notation of the formula that's used to find these probabilities, in conditional random fields?

Someone told me it's not simply p(a|b) , because conditional random fields use context from adjacent observations.

What exactly are these marginal probabilities?

Answer 1

The conditional probability is just p(y|x) where y is a sequence of labels and x is the associated observed sequence.

The expression for this probability is just the softmax function \\exp( a_i ) / \\sum_{i'} \\exp ( a_{i'}) .

For a CRF, a_i is a function of the label sequence a_i = w \\cdot \\phi(x,y) , where \\phi(x,y) is a feature vector derived from a sequence and its labels.

This means that the sum in the denominator is over the exponential number of possible labels, \\mathcal{Y} :

\sum_{y' \in \mathcal{Y}} \exp ( w \cdot \phi(x,y) )