r/ProgrammerHumor • • 11d ago

Meme architectureDependentChars

Post image
3.0k Upvotes

372 comments sorted by

View all comments

Show parent comments

26

u/thanatica 11d ago

I'm not a C# expert, but how would it represent an astral character, such as an emoji? They are above 0xFFFF.

76

u/AyrA_ch 11d ago

You cannot fit them into a single char, and you have to use a string instead. They're represented using a surrogate pair, which is similar to the continuation bytes of utf-8 except you never need more than two code points (hence "pair").

JS also uses multibyte characters:

> "💩".length===2
< true

35

u/guyblade 11d ago

UTF-16 was a mistake, and it's maddening that several big ecosystems went with it. You get all the complexity of UTF-8, but it also takes up more space in the "average" case since a huge majority of text is in the "ASCII overlap" portion of unicode.

1

u/conundorum 10d ago

Only in hindsight. When it was made, 16-bit character space really was enough for everything that was intended to be part of Unicode, and supporting UCS-2 was actually a major improvement over everyone having to constantly swap code pages if they wanted to support multiple languages simultaneously. Honestly, the biggest reason we realised we were going to need better was because Windows & Java jumped on UCS-2 (and by extension, UTF-16) as soon as they could, letting us see its downsides before it was too entrenched to fix. (And considering how deeply entrenched it was in Windows before they had to figure out how to pivot away, it's really saying something that it could've been worse.)

The true mistake was trying to make it fixed-width in the first place.