UTF-16 Encoding • U+0000 to U+D7FF and U+E000 to U+FFFF – One code unit, i.e., 2 bytes • U+D800 to U+DFFF – Reserved • U+10000 to U+10FFFF – Two code units, i.e., 4 bytes 21
Encoding planes 1 to 16 • Code points: 10000 to 10FFFF 1. Subtract 10000 from code point => 0..FFFFF (20 bits) 2. Add 1 st 10 bits to D800 => 1 st code unit 3. Add 2 nd 10 bits to DC00 => 2 nd code unit 22
Examples • Encoding code points in BMP is easy 23 Subtract 0001 from 0010 => 0001
Decoding UTF-16 (big endian) • Divide file into 2-byte words • If the word starts with 1101 10xx – Must be the first code unit of a 2-unit encoding • Else if the word starts with 1101 11xx – Must be the second code unit of a 2-unit encoding • Else – Must be a 1-unit encoding 24
Big-Endian (BE) and Little-Endian (LE) • Two ways of storing a multi-byte word (code unit) – Not a problem in UTF-8 • UTF-16BE – BE: big endian where most significant bytes stored first – So no change to the order of bytes – So ABCD stored as “AB CD” • UTF-16LE – Reverse the order – ABCD stored as “CD AB” 25
Byte Order Mark (BOM) • Unicode recommends to add BOM to the beginning of a text – Tell which order the text follows – BOM: U+FEFF • “FE FF” => big endian • “FF FE” => small endian 26
Gulliver’s Travels Besides, our Histories of six thousand Moons make no mention of any other Regions, than the two great Empires of Lilliput and Blefuscu. Which two mighty Powers have, as I was going to tell you, been engaged in a most obstinate War for six and thirty Moons past. It began upon the following Occasion. It is allowed on all Hands, that the primitive way of breaking Eggs, before we eat them, was upon the larger End: But his present Majesty's Grand-father, while he was a Boy, going to eat an Egg, and breaking it according to the ancient Practice, happened to cut one of his Fingers. Whereupon the Emperor his Father published an Edict, commanding all his Subjects, upon great Penaltys, to break the smaller End of their Eggs. The People so highly resented this Law, that our Histories tell us there have been six Rebellions raised on that account; wherein one Emperor lost his Life, and another his Crown. These civil Commotions were constantly fomented by the Monarchs of Blefuscu; and when they were quelled, the Exiles always fled for Refuge to that Empire. It is computed, that eleven thousand Persons have, at several times, suffered Death, rather than submit to break their Eggs at the smaller End. Many hundred large Volumes have been published upon this Controversy: But the books of the Big-Endians have been long forbidden, and the whole Party rendered incapable by Law of holding Employments. During the Course of these Troubles, the Emperors of Blefuscu did frequently expostulate by their Ambassadors, accusing us of making a Schism in Religion, by offending against a fundamental Doctrine of our great Prophet Lustrog, in the fifty-fourth Chapter of the Brundrecal (which is their Alcoran.) This, however, is thought to be a meer Strain upon the Text: For the Words are these: That all true Believers shall break their Eggs at the convenient End: and which is the convenient End,
You've reached the end of your free preview.
Want to read all 57 pages?
- Fall '14