I18n - Browsers and fonts

This page includes some personal experiences (almost entirely related to Win32 platforms) with browsers and fonts.

I'd definitely advise a visit to Alan Wood's Fonts page, which is excellent, and also follow some of his links to additional resources. The MS 'Font Properties Extension' is instructive, but for closer study I'd recommend a font utility such as ListFont or SIL?ViewGlyph.

User perceptions

There has been frequent discussion on usenet in which one or other participant asserts that the popular browser which they use is incapable of displaying a wide character repertoire. As a general statement, this is simply not true, and most scenarios turn out to be on the one hand, a misunderstanding of how characters are meant to work on the WWW, or on the other hand, a lack of appropriate fonts or of the browser configuration needed for using them. If you still don't believe they can, take a look at a screenshot made with Netscape 4.08 from around 1998, when properly set up: yes, NN4.08? showing Greek together with several Cyrillic alphabets.

Things have certainly got better in the last few years (this written in 2006), as OSes have come with a wider character repertoire, and browsers have come configured to take advantage. However, there are still plenty of folks running older systems, so I would suggest that authors who plan to use these features of HTML should probably put in a little effort to educate their readers in order to avoid disappointment. In particular, it should be noted that many versions of MS Windows OSs have installed by default with only a limited character repertoire, indeed multilingual support may be an installation option that you have to request specifically. The MS "web fonts pack", which was distributed chiefly to remedy this shortcoming in Win'95, is now past history (see further discussion below), but remaining Win98 users should make sure they haven't omitted multilanguage support from their OS installation.

Furthermore, there are specific problems (as detailed later in this page) with versions of MSIE up to including IE6; even if IE7 improves the behaviour (as recent reports suggest that it will), it appears that IE7 will not be available for OS versions before XP?SP2, so there will be plenty of folks stuck with older versions for a while yet.

Apparently, the effective character repertoire in Windows OSes can be dependent on which language options have been enabled; and the effects of that can be far from obvious. I found that installing the Japanese language option in Windows 2000 Pro resulted in a whole range of additional characters becoming evident, including characters with no obvious relevance to Japanese, for example additional "Misc Symbols" from the U+26xx range of Unicode.

These details differ between different versions of Windows, it seems; but, in general, users could be recommended to enable any multinational support and/or a selection of language options offered by their OS installation (see Control Panel> Regional Options in Win2000 for example), if they hope to see a wide repertoire of characters?- especially in MSIE, but also in third-party browsers such as the Mozilla family or Opera (since these browsers use the fonts that have been installed in the OS).

MS Font Properties Extension

The MS Font Properties Extension is available from the MS free font tools page and was last spotted as Version 2.1, which is now quite old and does not cover some of the newer Unicode character sub-ranges, but is worth getting anyway.

However, it's somewhat short on features, so I'd recommend also getting a feature-rich font browser such as SIL ViewGlyph utility, or one of those listed on Alan Wood's pages.

Font configuration per browser

Disclaimer: I'm not a browser developer myself, only a user of such systems: I don't understand the internals of font management in Win32 OSes (the main topic here), even less on X, and hardly at all on Mac. So these observations will be a wee bit superficial. But I believe they are nevertheless useful in a practical sense.

Let me stress that, in addition to having properly-composed HTML source with its correct character encoding (charset=) information, two conditions must be fulfilled for proper character presentation: the browser must have access to appropriate font(s), and must have the correct configuration to use them. It may be that, as installed, it's all ready to do that, and requires no further attention. But if it doesn't seem to be working properly, it could be that either of these preconditions is failing. Until you fix the right one, fiddling with the other one might only make matters worse.

References to "Netscape" browser in here are intended to apply to Netscape Communicator versions 4.* except where otherwise stated. References to "Mozilla" may generally be assumed to apply also to Netscape 6 and later, as well as to other Mozilla-family browsers such as FireFox. Netscape 4 can be considered practically obsolete by now (2005) and will probably be removed at the next major revision of this page, but it's still retained for now.

Netscape 4.* versions

This applies as long as you intend your documents to be compatible with the remaining users of Netscape 4.* versions. The key issue, in the context of this note, is that for getting coverage beyond a single 8-bit repertoire per document, Netscape4 needs to be coaxed into believing that it is working with Unicode, and to be given a comprehensive Unicode font for the purpose: it cannot piece together different writing systems from several fonts as the newer browsers can do. The usual ways of getting Netscape4 into its unicode mood (as discussed elsewhere in this area) are either to actually use utf-8 coding, or to use &-notations in a document that is coded in us-ascii, and then to pretend that the document is coded in utf-8 (which it is, in a degenerate sense, since us-ascii is a proper subset of utf-8). Just a routine warning that forms submission from such a page by NN4.* can be disastrous.

Suitable Unicode fonts are shown below.

Win MSIE (5, 5.5, 6)

MSIE uses fonts differently, and better, than Netscape4.*, although not so well as do Mozilla and Opera (to name but two). For different 'language scripts' (MSIE's terminology), IE can pick groups of characters out of different fonts, within the same web page. Given adequately populated fonts, this allows IE to support HTML4 correctly, including proper treatment of pages prepared in an 8-bit (or even 7-bit, i.e us-ascii) coding which contain characters (&-notations) from several different language groups in the same page. So MSIE supports this aspect of HTML4 correctly, in a way that Netscape4.* was architecturally incapable of doing.

Configuration of the default fonts in Win IE is via the dialog: Tools> Internet?Options> General> Fonts?- note however that the option for insisting on the browser's configured fonts rather than those specified by the incoming document is in the "Accessibility" dialog, not in the "Fonts" dialog.

For 8-bit character codings, unlike the defective Netscape4.*, IE sees no problem with rendering characters that are not in that 8-bit repertoire, when the HTML calls for them by means of &- notations.

The rest of this section deals predominantly with observations of IE's behaviour when the coding implies Unicode, i.e typically utf-8.

In its default configuration, MSIE can only display characters which are in Unicode "Plane 0" (the BMP or Basic Multilingual Plane), which is a hifalutin' way of saying those characters whose Unicode numbers do not exceed U+ffff. For NT-family OSes (NT4, Win/2000, Win/XP) there is optional support from MS, described by Tex Texin, i18nguy.com for those who wish it. He also describes a manual workaround which needs no extra fiddling with registry entries or loading extra software, but which only works for characters expressed in &#number; notation?- not for characters encoded in utf-8. See the cited URL for details.

When configuring default fonts for language groups, MSIE evidently knows which of your available fonts are available for each language group, and in this way restricts your selection in the font configuration dialog: however, as we will see, this isn't quite the whole story. The following details were first derived in detail with WinMSIE5.5 on Win/NT4, but other versions of IE and different MS-Windows OSes seem to be consistent with this too.

The actual selection of which font to use for displaying particular character ranges is more complex than this, and hasn't been understood in every detail. Following some observations made by James Kass and reported in somewhat worrying terms, I started to look at this area by at least studying how the default font selection works, relative to the configuration options provided. However, on closer study, I concluded that Kass's recommendation to use "User Defined" encoding for viewing the above-cited page is more in the nature of a diagnostic technique, and would not really be suitable for general web page authoring.

In fact, when I view the samples at the foot of his test page, in Win MSIE 5.5, or 6: when I set the View Encoding to utf-8, most of the texts display convincingly (albeit a few characters are missing here and there), whereas when I set it to User-defined, more than half of the characters are missing, due to the fact that it's trying to use fonts that I haven't got, and is being prevented by the user-defined coding from finding a substitute font. For any normal authored web page, the former behaviour seems to be the desirable one?- hence, utf-8; whereas for diagnosis, he understandably prefers the latter result?- hence, user-defined coding for diagnostics.

So, I should stress that I was working with the character encoding advertised as utf-8, the same as I would be using for other browsers?- especially Netscape-4.*.

The first curious observation was that if a distinctive font (e.g Arial Black) was configured for "Latin" scripts, then it was evident that this font was also being used as the actual default for some other language scripts, even though it had not been configured for them. A bit of detective work with the Registry Editor showed that at the registry path
HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\International\Scripts
there was a set of folders labelled with small numbers, and in these folders were found the font names which had been configured for the various "language scripts". In fact, there were just 37 such folders, and the font selection menu has a language scripts selection menu with exactly 37 entries: clearly this is no co-incidence. By changing the font selections and hunting around with Regedit, it was possible to identify which folder was which, and a selection of them are tabulated, in numerical order, alongside. Note that these numbers are in different ordering and groupings than the groupings in Unicode itself, as well as being unrelated to the (alphabetical) ordering in which the scripts appear in the configuration pull-down menu.

As I said: if one configures the browser to use a distinctive font, say Arial Black, for the "Latin-based" group, then the browser is found to be using this font also for Greek and Cyrillic, which are indeed the other writing systems which it covers?- irrespective of what is configured for Greek and Cyrillic. If, on the other hand, Latin is configured to use a font which does not cover any other language scripts (in Win/2K this might be for example "Albertus Extra Bold") then the font configuration selection(s) for Greek and/or Cyrillic come into play.

So: configuring the Latin font seemed to propagate to the font selection for all the language scripts which that chosen font covered. Such a procedure could indeed produce more-harmonious-looking results, since it attempts to minimise font changes; but on the other hand it could have the consequence of an incompletely-populated font (e.g Lucida Sans Unicode) being picked in preference for some particular language group, as a consequence of it having been selected for "Latin", even though a better-populated font for that language group was available (say, Arial Unicode MS or Code2000) and had actually been configured for that group. This results in IE displaying missing-character placeholders when one attempts to display characters that are not present in the browser-chosen font, even though the configured (but ignored) font for that language group does in fact contain them! There's an example of this behaviour reported for Urdu below.

On the other hand, if the text were Devanagari, which Lucida Sans Unicode doesn't cover, then IE would know better, and would look for another font. And indeed if I try to configure IE's font setting for Devanagari, the only option it gives me (on the particular machine tried) was Arial Unicode MS.

Similar considerations apply to author-specified font names, but I have only scratched the surface of how MSIE handles this in detail. Curiously, the discussion at the foot of Kass's page complains that font selection isn't working when utf-8 is used. I can't really speak for CJK scripts, as this is not my field, but that certainly wasn't the impression I got from the writing systems which I was trying. And it seems to me that trying to use user-defined coding in an attempt to circumvent IE's selection of the "wrong" font, is far more fraught with pitfalls in a general WWW context. More about this topic area under the "Monospaced fonts" subheading below.

The above observations all relate to a browser installed for the British locale?- so it's not surprising that the browser treats the Latin script as its first choice; I guess it may be that the font search is modified if the browser is installed for some other locale...(?) It's also intriguing that the folders 1, 2 and 4 are missing, and one naturally wonders what the gaps might be intended to hold...?

I don't think there is any great cause to get worried about what is reported above. For most non-specialist i18n practical purposes, it seems to me that the results will be at least viable. If you configure a Unicode font with an extensive coverage (such as Arial Unicode MS or Code2000) for all the language groups of interest to you, then IE is capable of displaying correctly (although those choices are admittedly less than ideal on cosmetic grounds). Don't forget that an inappropriate choice of font for the Latin-based selection can have knock-on effects on other language groups, as already noted. Complicated choices of font by the browser configuration, or by the author, could be rated, it seems, to make things worse, maybe even dramatically worse, than leaving well alone (there is an option on the browser's "Accessibility" menu for disabling document-specified font choices if found necessary).

Effect of character encodings

However, the behaviour described above does not in general apply when a document is sent out with other encodings. For example, a document sent out with iso-8859-5 would cause IE to use the "Cyrillic" font selection, even for display of Latin characters contained in the document. And similarly for other encodings associated with a specific writing system (IE does not appear to take account of HTML language attributes in making such a decision, unlike Mozilla).

Win IE installation

It's been noted that some Windows OSes installed by default with versions of the usual fonts (Arial, Verdana etc.) which contain only a restricted repertoire of characters. This was notoriously the case for the US version of Win98, for example: in order to get a better coverage, one should install the "Multilanguage support" option. The font file names are the same, but the difference is evident in their file sizes (for Win98, Arial should be above 300kByte, whereas the restricted-repertoire font was only around a third of that): however, later versions of MS Windows OSes seem to come with progressively more comprehensive fonts under the same names as before: normally their version number changes, but I haven't done a close study of that?- interested readers might want to review the history of older versions of their favourite fonts at the MS Typography Fonts and products page.

Note by the way (just in case it wasn't evident) that installing additional fonts, or enhanced versions of existing fonts, is an OS-wide operation, it makes them available not only to MSIE but also to any other browsers which you use.

Apparently, the effective character repertoire in Windows OSes can depend on which language options have been enabled; and the effects of that can be far from obvious. I found that installing the Japanese language option in Windows 2000 Pro resulted in a whole range of additional characters becoming evident which had no obvious relationship to Japanese: those characters included for example quite a number of additional "Misc Symbols" from the U+26xx range of Unicode. A similar effect was seen with Windows XP.

Win/IE: Tooltip Fonts (and window title too)

It's been noticed that IE's tooltips may exhibit only a limited character repertoire (possibly differing from one version of the OS to another). For example, a title= attribute containing ™ (the trademark character, ?) displayed only a rectangular placeholder on the tooltip.

A chance remark on a Usenet group brought me to a page by Jukka Korpela about setting the size of the tooltip font: the recipe given there (Windows Control-Panel> Display Properties> Appearance> Item: ToolTip) can also be used to change not only the size but also the selected font (which by default in earlier Windowses seemed to be "MS Sans Serif", a rather impoverished font even by Windows western-default standards!), and by selecting a richer-repertoire font I got much better results with the tooltips. In Windows/XP the configuration is found in ...Appearance> Advanced> Item: ToolTip. This is far from obvious, and few users will ever become aware of it I guess, so as web authors we would do well not to over-tax their browsers; but as web readers we might get some small advantage from it in configuring our own browsers, and dropping hints for other interested readers.

And I made the corresponding change for the window title bar, which has also had a positive effect when viewing pages which have i18n content in their