Concept-Based Embeddingsfor Natural Language Processing
Table 2.1: Example BIO tagging for a sample of MUC-7 [11]
['Mr.(O) Dooner(B-PER) met(O) with(O) Martin(B-PER) Puris(I-PER) ,(O) president(O) and(O) chief(O) executive(O) officer(O) of(O) Ammirati(B-ORG) &(I-ORG) Puris(I-ORG) ,(O) about(O) McCann(B-ORG) ’s(O) acquiring(O) the(O) agency(O) with(O) billings(O) of(O) $400(B-MONEY) million(I-MONEY) ,(O) but(O) nothing(O) has(O) materialized(O) .(O)']
-Ch2S2SS5 -Chx2T1 Machine learning approaches typically model the two subtasks of NER as a sequence prediction task using machine learning based approach: given a word sequence W = w0 w1 w2 … wn, the objective of NER is to find the optimal tag sequence T = t0 t1 t2 … tn, where the tag set is pre-defined. One of the most popular tagging schema used by NER approaches is the BIO encoding. The set of BIO tags includes Beginning, Inside, and Outside tags. For example, the BIO scheme for person names includes BPers, IPers and O, which indicate the beginning of a person name, an internal word of a person name and the word belonging to none of the named entity types. Given the tagging schema and word sequence, the sequence labeling model searches for the best tag sequence with the parameters learned from training process. In the following, we review the popular sequence labeling models having been used for NER. Charniak et al. Given the word sequence W = w0 w1 w2 … wn, the best tagging sequence T = t0 t1 t2 … tn is defined as ^T1... n =argmaxT1..nP(T1... n,W1... n) HMM model assumes that the current word wi is generated from a multinomial distribution conditioned on current tag while current tag is only conditioned on previous tags. In a first order HMM, the above joint probability can be expanded to P(T1... n,W1... n)≈n∐i=1P(ti|ti−1)P(wi|ti) However, the assumption made by HMM is too strong for NER as the current tag depends on not only the previous tags but also the other types of characteristics in context. This makes less sense than assuming B-ORG depends on the previous word officer. Classic HMM does not allow such dependency between tags and words. Many variations of HMM model have been proposed for addressing this problem, most of which relax the Markov assumption to allow the dependency between previous tags and observations.