Generate glyphs for any font from google/fonts

[orangemug]

Generate glyphs for any google font mounting an index in the base directory

Where the index file would look like

// index.json
{
  fonts: [
    {
      name: "Noto Sans",
      specimen: "https://fonts.google.com/specimen/Noto+Sans",
      styles: [
        {name: "Normal / 400", path: "/glyphs/Noto Sans"},
        {name: "Normal / 400", path: "/glyphs/Noto Sans Bold"},
        {name: "Italic / 700", path: "/glyphs/Noto Sans Italic"},
        {name: "Italic / 700", path: "/glyphs/Noto Sans Bold Italic"}
      ]
    }
    /* ...  */
  ]
}

The idea being that app can load the index.json and have access to glyphs for all the typefaces.

[orangemug]

I have currently got the glyphs generating for all https://github.com/google/fonts typefaces locally

It'll generate all fonts in ~25mins which is ok for circle CI to automate.

However the size is about 2Gb which mean's we'll have to pick just a few typefaces to start off with until I get somewhere else to deploy to other that github pages (AWS probably)

$ du -h ./glyphs
2.0G glyphs/

[orangemug]

Working branch: https://github.com/orangemug/font-glyphs/tree/feature/all-google-fonts

[orangemug]

All fonts (or subset of) are now generating locally. We now need to provide fallback typeface generation.

Currently styles (osm-liberty for example) load a set of glyphs from the font store. These are stored by key including their fallback. So for example

Roboto
Roboto,Noto Sans CJK Regular

We currently only have Roboto in our rendered glyphs, which in turn doesn't support Chinese, Japanese or Korean (CJK). So osm-liberty doesn't currently render those glyphs.

To fix this we need to create Roboto,Noto Sans CJK Regular by merging the glyphs in the same manor as a browser would to provide a fallback font. We'll do this by checking if each glyph is available in the current font and falling back through the font stack until we find one

This can be done by simply merging the PBF files in the font-stack order.

[orangemug]

An example is up at orangemug/font-glyphs-v2 (temporary repo)

Demo: https://orangemug.github.io/font-glyphs-v2/example/#4.15/47.656/1.316
Font index: https://orangemug.github.io/font-glyphs-v2/glyphs/index.json

CC @lukasmartinelli you might be interested in using the font index (or something like this) for font selection in maputnik/editor

[lukasmartinelli]

An example is up at orangemug/font-glyphs-v2 (temporary repo)

Demo: https://orangemug.github.io/font-glyphs-v2/example/#4.15/47.656/1.316
Font index: https://orangemug.github.io/font-glyphs-v2/glyphs/index.json

CC @lukasmartinelli you might be interested in using the font index (or something like this) for font selection in maputnik/editor

Dude that is awesome!! Such a great idea with the repo and font index.

[orangemug]

awesome

Not yet but hopefully it will be, so the ultimate aim of this repo is to be able to use any google font falling back through the recommended stack from google fonts. Problem is at the moment is they tend to get kinda large with CJK, you can see that from the klokantech repo

$ du -h KlokanTech\ Noto\ Sans\ CJK\ Regular/
31M KlokanTech Noto Sans CJK Regular/ 

Additionally I plan to add what code point ranges it supports to the index.json. This will allow the consumer to know that this font can be used to these regions. Basically with the idea that you might want to create style for a smaller area other than the world so don't care about non ascii

[orangemug]

First attempt at a unicode range coverage report: https://orangemug.github.io/font-glyphs-v2/reports/char-table.html

Warning: 20mb it a HTML file

Updated: Font report https://orangemug.github.io/font-glyphs-v2/reports/char-table.html with noto-cjk and noto-emoji

[orangemug]

Update: orangemug/font-glyphs-v2 now allows for any font from google fonts to be generated. It also implements and index.json file as described above

orangemug/font-glyphs-v2 tracks the v2 branch in this repo. Due to some changes in font names to better match google-fonts, this would cause a breaking change to osm-liberty. We will raise a PR pre merge of v2 branch

[klokan]

Hi!

the https://orangemug.github.io/font-glyphs-v2/reports/char-table.html is damn cool!

I also love the /index.json - which is exactly something required for Maputnik editor to list the available fonts in the GUI.

While working on OpenMapTiles (the OSM2VectorTiles follower) - we made a basic catalog of fonts used in the open map styles and also downloadable individually from a mobile app or tileserver deploy.

See: https://github.com/openmaptiles/fonts

This allows us to run a preview of the GL Styles on Github with Travis like:
https://openmaptiles.github.io/positron-gl-style/

This is documented at:
https://github.com/klokantech/gl-style-package-spec

Any improvements or pull request towards this would be great. It would be also great to filter from KlokanTech Noto the glyphs not available in Arial Unicode.

[orangemug]

Hey @klokan thanks! I'll take a look more into this later in the week. I'll explain my aims of this project a little

• Hosting of fonts through an obvious process chain, I want a CI system to do the work
• Fallback fonts, the code in this repo should make it easy to add merging of fonts to produce stacks, for example Roboto,Noto
• Adding the coverage info into a font metadata file. Giving the option in Maputnik (or other editors) to tell you the coverage when you pick the font. My grand plan for this is that if you picked a 'script font' for an old style map, and used it for country names in the a style the editor could display This font is complete for 80% of labels the other 20% will use the fallback (only a partial idea in my head atm)

Basically I want to explore typography in maps, and I want the service to give all the information needed to make the right decision for a particular context in terms of typography.

If this repo can be useful for your (and @lukasmartinelli's) grand web mapping plan :) please let me know. I'm more than happy to add you as contributors, and even change ownership of the repo if thats helpful.

Anyway this ticket has gone way off topic (my fault... oops), hope this is helpful

[klokan]

These are great goals! I am really keen to sync, so we don't duplicate the effort - esp. the font catalog for Maputnik styling seems a shared goal.

You have really amazing results on the font content analyze here.

BTW your point two (fallback fonts / combined font stacks) - is already implemented inside of https://github.com/klokantech/tileserver-gl - feel free to check the source code.
My idea behind fonts.openmaptiles.org service was to set there also the publicly available merging (probably by redirects to tileserver-gl-light instance also serving the free vector tiles on CDN) - it is only now hosted on gh-pages.

@lukasmartinelli could you comment (or create a separate ticket on maputnik) about the requirements on index.json for fonts?

[orangemug]

BTW your point two (fallback fonts / combined font stacks) - is already implemented inside of https://github.com/klokantech/tileserver-gl - feel free to check the source code.

Well that's a lot better than what I had planned

These are great goals! I am really keen to sync, so we don't duplicate the effort - esp. the font catalog for Maputnik styling seems a shared goal.

This seems like a plan, looks like I need to do some reading to see where I can be useful. I don't have a lot of time, but if I can be helpful towards openmaptiles / maputnik I'll do what I can.

[klokan]

Just for reference - to have it in this ticket (which can be closed probably)

We have made a font catalog at:
https://github.com/openmaptiles/fonts/

List of the fonts is exposed to Maputnik via /fonts.json. See discussion at:
maptiler/tileserver-gl#104

A simple version of the font service (without combined fonts) can be served directly from GitHub gh-pages and can be static. For font concatenation, the TileServer GL or TileServer GL Light installation is the most straightforward.

[orangemug]

Hey @klokan unless I misunderstand openmaptiles doesn't really aim to cover what this ticket describes. This ticket aims to generate any google font via CI, something which currently works in v2. I'll aim to get back to this over the coming months. The idea is that rather than relying on a service to generate font stacks you can just do it yourself with a free github account and a circle CI service.

However I'll add https://github.com/openmaptiles/fonts/ to the README because it's currently a better alternative. On that note someone should probably create a PR for https://github.com/lukasmartinelli/osm-liberty/blob/gh-pages/style.json#L23

[orangemug]

Also @klokan to add, the initial problem for which this repo was created. When @lukasmartinelli originally generated the glyphs for osm-liberty there was an issue and the glyphs had gaps in the ranges. Using a SHA (submodule) of the font and a recipe (script) to generate the glyphs meant I now know where the glyphs are coming from and how they are generated. If there are issues it's easier to track them down. This was a design decision which may be a consideration for openmaptiles/fonts