MultiLabel Classification using Conditional Random Field

Question

Is it possible to use Conditional Random Field for MultiLabel Classification? I saw a python CRF implementation at https://pystruct.github.io/user_guide.html , but couldn't figure a way to do multilabel classification.

Answer 1

The basic CRF doesn't support multilabel classification. However, some extensions have been explored, such as the Collective Multi-label (CML) and the Collective Multi-label with Features (CMLF). From (1):

A conditional random field (CRF) based model is presented in [21] where two multi-label graphical models has been proposed, both parameterizes label co-occurances. The Collective Multi-label (CML) classifier maintains feature accounting for label co-occurances and the Collective Multi-label with Features (CMLF) maintains parameters that correspond to features for each co-occuring label pair. Petterson et. al. recently presented another interesting generative modeling approach in a reverse manner, predicting a set of instances given the labels [39].

---

References:

• (1) Sorower, Mohammad S. "A literature survey on algorithms for multi-label learning." Oregon State University, Corvallis (2010). http://people.oregonstate.edu/~sorowerm/pdf/Qual-Multilabel-Shahed-CompleteVersion.pdf ; https://scholar.google.com/scholar?cluster=11211211207326445005&hl=en&as_sdt=0,22
• (21) N. Ghamrawi and A. Mccallum. Collective Multi-Label Classification. In Proceedings of the 3005 ACM Conference on Information and Knowledge Management (CIKM '05), pages 195–200, Bremen, Germany, 2005. http://www.dtic.mil/dtic/tr/fulltext/u2/a440081.pdf
• (39) James Petterson and Tiberio Caetano. Reverse multi-label learning. In J. Lafferty, CKI Williams, R. Zemel, J. Shawe-Taylor, and A. Culotta, editors, Advances in Neural Information Processing Systems 23, pages 1903–1911. 2010.

Answer 2

I encountered a modified CRF named fuzzy CRF as shown below.

Its mathematics is quite simple as we can see from equation 2 in the paper:

We just sum all the energies of the paths in the numerator, and the denominator remains the same. For inference, we can apply Viterbi or beam search.