ASCII and Unicode quotation marks (2007)

ASCII and Unicode quotation marks (2007)(cl.cam.ac.uk)

46 points by exvi 63 days ago | 29 comments

Findecanor 63 days ago |

In the '90s, MS-Windows by default used its own 8-bit character set CP-1252 that was a superset of ISO 8859-1 with a few additional characters, including left and right single and double quotation marks in 8859-1's unused code positions.

Microsoft Word used to "auto-correct" the ASCII codes 0x22 and 0x27 to those.

Also, ISO 8859-1 was the default character set for the web, and MS Word was in common use for making simple web pages... but without narrowing from CP-1252 to ISO 8859-1.

This had the effect that when you browsed one of those pages in a browser on another operating system, the quotation marks rendered as empty boxes ( = illegal character).

xigoi 63 days ago |

I was wondering why `this hideous quotation style' is used in so many places. Good historical window.

adornKey 63 days ago | |

The explanation seems to be that it looked good in some old fonts. But I think it was always some kind of abuse. On old Typewriters the accents were usually used for accents (é è). They didn't move the cursor, so using them for apostrophes wasn't that comfortable and interrupted writing flow. Accent + space looks a bit like a quotation mark, but the right place of an accent is usually on top of a letter.

1313ed01 63 days ago |

Special quotation marks sometimes end up in filenames (usually when I have saved a web page) and I hate trying to tab-complete or write globs for those. Of course it is equally annoying with other random unicode characters not present on my keyboard, but it mostly happens with quotation marks. (Yes, the solution is to copy-paste those characters in the terminal, but that is the annoying part, having to do that instead of just typing the next character).

black_knight 63 days ago | |

This problem was solved by Plan 9 (roughly 1990) where there was a compose key to turn sequences into Unicode characters. Say compose-f-a to get ∀. This was all configurable in /lib/keyboard.

On so-called modern X11 or Wayland based systems (Linux or *BSD), there is a similar feature called XCompose. Worse syntax, but still functional.

zahlman 63 days ago | | |

Being able to configure your system to type the characters really doesn't solve the problem. In particular, if you get data (including metadata such as filenames) from someone else, you need to recognize the characters, both to do the configuration and then actually type them. And characters are not glyphs. There are all kinds of cases where simply looking at something doesn't and can't tell you what characters are in it.

vincent-manis 63 days ago | | |

And just to point out that there is WinCompose, for Windows, and a somewhat janky but usable solution using Karabiner Elements and macos-compose for Mac.

trashb 63 days ago |

Some language settings (on windows) will auto replace the '' and "" set for ʻʼ and “” as that is the correct spelling in the set language. There is also the lower quotes that can be used but it seems usually a normal comma and double comma is used as codepoint (U+002C, U+201E) ,’ „”.

This really messed me up when I started programming since those quotes will not work when writing in a language that expects a set of the same character but they may use the same glyph. This is one of the many reasons I have my systems set to English.

I agree that for a normal writing environment it may be advantageous to have it auto replace since it is also just easier to hit the same key twice and have it auto open/close.

jeroenhd 63 days ago | |

I have never encountered that behaviour outside of Microsoft Word and its alternatives, I've always had this happen application-side. Is this an IME thing? Or a non-Unicode-compatible code page? Because I don't think there's any other Windows-side automatic replacement of that type.

Many blog engines online will also try to be helpful and replace quotes with smart quotes, which makes copy-pasting source code from tutorials quite a pain.

trashb 63 days ago | | |

From what I remember (it was a while back) it was both in notepad, notepad++ and geany. I was using, probably win xp or 7 at the time. I remember the only way I could fix it at the time was to change the language and keyboard settings to international English.

I can't seem to replicate the behavior now on win 11 even with the same language set and keyboard layout (system language set to English), so perhaps I'm misremembering? It seems that this keyboard layout does enable typing ¨ U+00A8 so perhaps I am confused with that and that some editors (word etc) do the opening/closing replacement.

IME is not used, though interestingly in Asian languages the do used even different quotes, example Japanese:「x」and『x』

fainpul 63 days ago |

But how can I enter ’ (U+2019) on macOS (US layout) without going through some magic incantation? It’s impossible!

jeroenhd 63 days ago | |

Option + ] will produce ‘, Option+Shift+] will produce ’. Similarly, “” can be produced with Option(+Shift)+[. Alternatively, Option+Shift+e will produce ´.

chrismorgan 63 days ago | | |

Ugh, that is such a bad arrangement of the four combos. It should obviously have been [ for left and ] for right (just as [ and ] are a pair), and Shift to turn single into double (just as Shift turns ' into ").

fainpul 63 days ago | | |

Thanks. This is better than the alternatives that I was aware of.

bitigchi 63 days ago | |

Just enable automatic substitutions from the Edit menu? You can do this on a per-app basis.

Terretta 63 days ago | |

MacOS, iPadOS, iOS on physical keyboards:

“ Option [

” Option {

‘ Option ]

’ Option }

… Option ;

- just -

– Option -

— Option _

• Option 8

fainpul 62 days ago | | |

> ’ Option }

But } needs shift. So this is Shift+Option+] which forces me to move my left hand off the base row. Not ideal.

trashb 63 days ago | |

you may be able to use the "ctrl (⌃) + cmd (⌘) + space" (character viewer) menu to select it? There may also be another shortcut for it.

shikaan 63 days ago |

Kinda OT: can anybody recognize the keyboard in the German keyboard shot? I have been looking for a similar ISO keyboard (with international layout) for a while

Findecanor 63 days ago | |

It looks like a Cherry MX 3000 (G80-3000) [1], of the older "winkeyless" variety in German layout. The later variant with windows keys and cheaper build is still being produced.

I bought several second-hand on German eBay, for scavenging parts for DIY mechanical keyboards back in the early '10s before the mechanical keyboard scene matured. The build is cheap and dated compared to modern mechanical keyboards, and newly produced keyboards are very overpriced. [2]

Most keyboard models are produced in ISO layouts, but you might have to look abroad to find one the right one or buy it from an online store selling internationally.

[1] https://wiki.themk.org/index.php/Cherry_G80-3000

[2] https://www.cherry.de/en-gb/product/g80-3000

jjgreen 63 days ago | | |

In the UK, "winkey" is a childish name for a penis, so careful now ...

allenu 63 days ago | |

Just a guess, but it looks like an IBM Model M but with a German layout, or at least something from that era.