1.basic of information

Quantifying information:

• log2(N/M) bits of information
• information 由bits 测度
• encoding describes the process of assigning representations to information.

等可能情况

使用固定长度的编码：
ex. Decimal digits 10={0,1,2,3,4,5,6,7,8,9}
4-bits BCD（binary coded Decimal）
log2(10) = 3.322 <4 bits

ex. 86 English characters = {A-Z(26) , a-z(26) , punctuation(11), math(9), financial(4)}
7-bit ASCII
log2(86)=6.426 < 7 bits

编码数字

each bit is assigned a weight, ordered from right to left, these weights are increasing powers of 2.
和...
signed integers : 2's complement