Data Mining of Causal Relations from Text: Analysing Maritime Accident Investigation Reports
TABLE VII: Term-document matrix for an example document collection.
['D/T', '[ITALIC] T1', '[ITALIC] T2', '[ITALIC] T3', '[ITALIC] T4', '[ITALIC] T5', '[ITALIC] T6', '[ITALIC] T7', '[ITALIC] T8', '[ITALIC] T9', '[ITALIC] T10', '[ITALIC] T11', '[ITALIC] T12', '[ITALIC] T13', '[ITALIC] T14', '[ITALIC] T15']
Let t be size of the terms set, and n be the size of the documents set. Then, all documents Di, i=1,⋯,n may be represented as t-dimensional vectors: Di=[ai1,ai2,⋯,ait] (18) where coefficients aik represent the values of term k in document Thus both documents and terms form a term-document matrix A(n×t). Rows of this matrix represent documents, and columns represent term vectors. Let us assume that position aik is set equal to 1, when term k appears in document i, and to 0 when it doesn’t appear in it. Documents set: D1: The King University College D2: King College Site Contents D3: University of King College D4: King County Bar Association D5: King County Government Seattle Washington D6: Martin Luther King The process of calculating weights of terms is called terms weighting. It addresses how important a term is with respect to a document (since not all terms are equally informative about the contents of the document). There are several main methods used to assign weights to terms. The simplest method is boolean terms weighting, which, as its name suggests, sets weights to 0 or 1 depending on the presence of term in document. Using this method causes loss of valuable information, as it differentiates only between two cases: presence or absence of term in document, and exact number of occurrences of word may indicate its importance in documents.