Message Redundancy

a concept in information theory. The presence of redundancy in a recording of messages from an information source is manifested in the possibility of writing the messages more briefly on the average using the same characters (that is, replacing the code with another using the same alphabet). For example, if the messages under consideration are sequences of characters 0 and 1 in which 1 is encountered, on the average, once per 10 characters, then the notation can be cut almost in half by using coding according to the rule

00 → 0,01 → 10,10 → 110,11 → 111

The maximum share of redundant characters is determined by the statistical characteristics of the message source under consideration and is also called its redundancy. In this meaning, the message redundancy R is determined according to the formula R = 1 – H/log₂m, where m is the number of letters in the alphabet, and H is the entropy of the source per letter of the message. In the example cited above, the message redundancy may be calculated as equal to 0.53. Minimum redundancy, R = 0, is found only for a sequence in which the characters are independent and can be equal to any of the m letters of the alphabet with a probability of 1/m.

The question of evaluating the redundancy of specific forms of messages (for example, written and oral speech, phototelegrams, and television pictures) is of practical importance. The magnitude of message redundancy in them is usually significant. For example, the message redundancy of written English speech is not less than 0.6. Great redundancy makes possible easier recognition of transmitted messages where there is interference in the channels of communication. From this point of view it is not always desirable to reduce message redundancy.