What is Shannons Information Theory?
Claude Shannon, a scientist working at Bells Labs, published his information Theory in 1948 and this had an immediate and lasting impact on data communication technology. Shannon demonstrated that the value, or entropy, of a piece of information is proportional to its probability and the entropy of a joint event is given by: conceptSearching interprets this in an IR context to compute the incremental value of a two-word term over its singleton components. Higher order compound terms are evaluated using their lower order compound components. It is no coincidence that the majority of compound terms are in fact proper nouns, noun phrases and verb phrases and it is these sentence fragments that convey the key concepts in most text. However, the concepts are identified without any linguistic analysis and so conceptSearching works with any vocabulary and is language dependent. The mathematical approach works because Shannon’s theory can be applied to any human language communication. The abi
