Add OpenHistoricalMap layer to maps

[1ec5]

I’ve added a rudimentary OpenHistoricalMap layer to all the maps in Gramps Web. The maps still default to the OpenStreetMap layer, but the layer selection control is now available just about anywhere there’s a map.

For now, the OpenHistoricalMap layer just shows everything that’s in OHM, regardless of the date. This would obviously confuse the user. It would be straightforward to filter the map by date, either by installing the interactive Leaflet time slider plugin or by passing in a date from the surrounding context. I’m unsure of where to get a relevant date, but it seems like it’s within reach.

Here’s my hometown, Loveland, Ohio, and the late owner of a former hardware store there:

Fixes #437, working towards Mantis #13271.

[1ec5]

The MapLibre GL Leaflet plugin hooks into the L namespace that Leaflet exposes. This required refactoring many references to Leaflet in order to still build and run while satisfying ESLint. I’m not wedded to this approach; let me know if you can think of a less invasive way to build with these dependencies.

[1ec5]

OHM offers multiple official stylesheets. For now, I’ve gone with the default “Historical” style. If desired, Gramps can define its own custom style, following the MapLibre GL Style Specification. Tools like Fresco and Maputnik allow you to design custom styles visually.

[1ec5]

Some of these maps come with search boxes powered by OpenStreetMap’s Nominatim instance. OHM has its own Nominatim instance too. It’s a bit rough around the edges, but in the future, maybe we could switch to it whenever the map is showing the OHM layer, for consistency.

[1ec5]

This compatibility shim works, but it preserves the Leaflet user experience, which is much less fluid than MapLibre GL JS. Migrating from Leaflet to MapLibre GL JS would enable smooth zooming, rotation, and better rendering performance. Something to consider for the future.

[jeffreyameyer]

I’m unsure of where to get a relevant date,

Does this refer to "where to get the date from Gramps" or "where to get a date from OHM?". I believe it's Gramps, but I wasn't 100% sure.

[1ec5]

I’m referring to getting dates within the current Gramps view that can be fed into the map component. For example, in one of the screenshots above, I’ve hovered over an event in 1895 – the map could automatically filter to 1895 while hovering over that item.

[DavidMStraub]

Thanks so much, that looks awesome! 😍 I will review it as soon as I can, and for sure will be able to support on the dates question.

[DavidMStraub]

Rebased to main.

[DavidMStraub]

I finally had time to test it and it works & looks great! Also the changes to the code like the use of L make sense.

Concerning the time slider, I feel a more minimal slider would fit better - and we could use it regardless of map layer to only show pins of places with events in a given period.

Do I understand correctly the OHM map can either use a fixed date or a range?

[1ec5]

Yes, the data is all available on the client side; it’s up to the client code to decide what to show or filter out. Unfortunately, the existing time slider library’s UI might be a bit much for some of the map views, but there would be nothing wrong with writing a homegrown UI just for Gramps.

With a slider, we’d need to decide the earliest and latest date the user should be allowed to choose. Alternatively, we could present the user with a date control. A range could simply entail showing two date controls instead of just one. Unfortunately, the HTML slider control only supports one knob, not two knobs, so we’d need to create and style a custom control if we need a double-knob slider to represent a date range.

[DavidMStraub]

Got it! We are using Material Web and they have range sliders (https://material-web.dev/components/slider/stories/), so that would be fairly easy to implement; I want to play around with some options in the next days. I suspect a slider is not so convenient on mobile.

As for the earliest date (the latest date would be today), we could select it based on the earliest event in the tree, but I feel like hard coding it to 1500 or 1600 when using a slider is probably ok because I imagine there are few genealogies that extend beyond it (and we could define the leftmost position of the slider as minus infinity).

[1ec5]

Here’s what the map looks like with the new time slider from 1ec5#1:

[1ec5]

This uses the legacy filter syntax from the MapLibre Style Specification. Eventually, once the Historical style is migrated to the newer expression syntax in OpenHistoricalMap/issues#775, we’ll need to update this code accordingly. It’ll be quite similar, something like this:

  const dateFilter = [
    'all',
    ['any', ['!', ['get', 'start_decdate']], ['<', ['get', 'start_decdate'], decimalYear + 1]],
    ['any', ['!', ['get', 'end_decdate']], ['>=', ['get', 'end_decdate'], decimalYear]],
  ]

But we’ll probably want to take advantage of the let expression to simplify this method overall:

  const dateFilter = [
    'let',
    'decimalYear', decimalYear,
    [
      'all',
      ['any', ['!', ['get', 'start_decdate']], ['<', ['get', 'start_decdate'], ['+', ['var', 'decimalYear'], 1]]],
      ['any', ['!', ['get', 'end_decdate']], ['>=', ['get', 'end_decdate'], ['var', 'decimalYear']]],
    ],
  ]

A bit more verbose, but then on subsequent applications, we’ll be able to manipulate the filter less invasively and probably more performantly:

  if (filter && filter[0] == 'let' && filter[1] == 'decimalYear') {
    newFilter[2] = decimalYear
    return newFilter
  }

[DavidMStraub]

I now added filtering the map pins by the time slider.

I also added two additional buttons:

• the cogwheel allows to change the "tolerance" (central year ± x) for place filtering
• the switch allows to disable/enable place filtering

In the OSM view, the slider is disabled when the switch is off, while in the OHM view the slider is always enabled as it not only filters places but also the map.

I have three open points:

• Calendars other than Gregorian are not handled properly yet
• The layer control should be on top of the time slider, not below
• The OHM layer is initially not filtered

I don't know how to fix the last point. Applying the filter after switching layers does not work as the map is not fully loaded yet. And I haven't managed to find an appropriate event to listen to. 🫤

[1ec5]

Calendars other than Gregorian are not handled properly yet

How does Gramps handle non-Gregorian dates in general? Do they get normalized to Gregorian? Is there a conversion utility for situations where there isn’t a risk of dataloss?

On the OHM side, recording non-Gregorian dates is still an open question, but I suspect the tiles will ultimately normalize them to the proleptic Gregorian calendar for a consistent display.

[DavidMStraub]

How does Gramps handle non-Gregorian dates in general? Do they get normalized to Gregorian? Is there a conversion utility for situations where there isn’t a risk of dataloss?

Dates are stored as [year, month, day] and calendar type (Gregorian, Julian, Hebrew, Islamic, ...). Conversions are done by implementing functions [year, month, day] ↔ numeric Julian day number for each calendar.

I could change something in the backend to use the existing Python functions for the conversion; the alternative would be to implement the functions in this module in Javascript.

For the time being I wouldn't even feel too bad treating Julian dates as Gregorian and ignoring all other dates.

[DavidMStraub]

To solve the overlap issue I now simply moved the slider below the map. That makes it wider which is helpful on mobile (especially in portrait mode).

[1ec5]

For the time being I wouldn't even feel too bad treating Julian dates as Gregorian and ignoring all other dates.

That sounds pretty reasonable to me for the time being, since the default tolerance is already much more generous than the Julian offset for a given date. Since countries adopted the Gregorian calendar in different years, we’d be showing pins from different calendars simultaneously. That probably calls for the time slider to have its own calendar option, eventually.

[DavidMStraub]

Handling all calendars turned to be much simpler than I thought. If we accept ±1 year inaccuracy Swedish, Julian, and Gregorian are all the same; French, Hebrew and Persian are just a constant offset; and Islamic is a linear function.

I also made the slider's maximum extent set automatically to the earliest event in the tree rather than hard-coding it to 1500.

Now I want to add tooltips to the 2 new buttons and we're done I think 🙂

[1ec5]

The calendar conversions here are good motivation for OHM to add formal support for non-Gregorian dates. I think we’ll try to do it in a way that’s backwards-compatible with the existing tile schema, but I’ll keep you posted if anything needs tweaking later.

[1ec5]

By the way, the map data and filtering code support BCE dates. Quite unlikely, of course, but I mention this in case we do decide to allow it in the future.

The only gotcha is that the year is offset by one because there’s no year 0, per ISO 8601. Currently this component is using −1 as a magic number for an unset year; we’d need to use something else like undefined.

[DavidMStraub]

Yes you're right. I think it's pretty ok for Gramps Web, but of course not ok for OHM in general. So would be more elegant to handle in full generality in the future here as well.