Urban/rural or density-based rendering differences

[pnorman]

Cross-ref #1705 #1956, and others.

This is distinct from separate styles. An example is @systemed having different default surface assumptions for footpaths in rural areas than in urban areas. He is using country-specific urban area data that we don't have for this style.

I don't think it's an option for a few reasons

• We don't have a clear datasource for urban areas. Place polygon coverage is limited, and anything else is indirect and would require processing
• It would stretch our resources thinner. I don't think we'd be able to adequately test all combinations of rural/urban on new PRs
• It'd add significant layer complexity. We'd need to either
  ** Add another preprocessed file and load it into the database, increasing install difficulty and deployment difficulty;
  ** Compute areas in DB at query time, involving reasonably complex SQL; or
  ** Pre-compute areas from the DB, requiring triggers, background jobs, or other tools which would increase difficulty
• It'd increase MSS complexity. The scale would depend on how many features would have different renderings in each, but as I tweeted our MSS total is larger than HOT OSM, OSM Bright, Stamen Toner, and CartoDB basemaps combined. We need to be reducing this, not increasing it.

[systemed]

It doesn't significantly increase MSS complexity. My experience with cycle.travel is that it actually makes maintenance easier, because on the few rules where it matters, you just need two sensible, easy-to-implement defaults for rural and urban areas, rather than having to spend hours experimenting with placements and spacing to find the sweet spot that works (= isn't completely intolerable) for both. Most of the time it's just a matter of changing the zoom level in the selector - "show this at z14 in rural areas, z16 in cities". So a comment like this by @matkoniecz simply isn't borne out by my experience.

However, for the other reasons you cite, I'd agree that it isn't suitable for openstreetmap-carto. I'd also expect that mappers would come to rely on whatever set of urban boundaries or density criterion were chosen for osm-carto, making assumptions that wouldn't translate well to third-party styles.

Finally, it's pretty difficult to get decent built-up area boundaries for most countries. (The UK, USA, and Canada are honourable exceptions.)

[imagico]

Regarding preprocessing possibilities - there is some info and sample data on http://www.imagico.de/map/osm_builtup_en.php

Adding something like that to openstreetmapdata.com to be used in this and other map styles would be possible although there would of course be some development work required to set this up to work reliably on a regular basis. And the various problems of using such data in a map would still be there. And showing/not showing or differently rendering something depending on data that is not directly in the OSM db is probably also confusing for mappers.

[kocio-pl]

Thanks @pnorman for opening separate ticket for this issue!

From my perspective (map user, not the technical infrastructure maintainer) this would be very useful:

• While this is not the same as country specific rendering, it could ease the most severe pains with some country-related problems (I mean differences between developed and undeveloped countries, including such cases as Australia, which has a lot of rural areas) with just a few general cases instead of ~150 sets of rules for each country.
• We already try to use some context (like considering area or population as a hint how important is this object), but it's a random process (shop/amenity icons hiding when too dense) and causes a lot of friction, because we're trying to find the middle ground where the context can be much different. See the problems with drinking water points (city square vs outdoor - both not served well currently IMO: too visible vs not visible enough), path visibility or restaurant (inn)/fuel station (near rural road they are very important POIs, while in the city they are no more important to see than typical shop) and so on. The danger is that we will move toward hiding a lot of things in the places where they would be good to see just because there exist some places where they can be annoying.

[Ircama]

Please, apologize for this trivial code (done in minutes, might be wrong). If I put the following in project.yaml:

CASE WHEN ST_Intersects(way, (select ST_Union(way) from planet_osm_polygon where landuse IN ('commercial',
  'construction', 'garages', 'industrial', 'military', 'railway', 'residential', 'retail',
  'village_green', 'recreation_ground', 'village_green', 'greenhouse_horticulture', 'brownfield')))
  THEN 'yes' ELSE 'no'
END AS is_settlement,

And then I manage it in MSS files, e.g.:

  [feature = 'amenity_place_of_worship'][is_settlement = 'no'][zoom >= 13],
  [feature = 'amenity_place_of_worship'][zoom >= 16]

It looks working (but very slow).

In the consideration that the landuse feature might not be universally adopted so far, we could start with a filter, where is_settlement is only applied within some areas like natural=mountain_range, natural=massif, place=region & region:type=mountain_area: those features look to be already in place for the most important mountain areas and would limit the test to a very restricted set of features.

Would it be acceptable?

[gravitystorm]

It looks working (but very slow).

Your SQL query is trying to st_union all those polygons for the entire world (or whatever extract you have loaded). If you're going to union things then you want to add "and where st_intersects(way, !bbox!)" to your inner query, so that only features that are within the current bounding box of the map are unioned.

Also, watch out for polygons that are not st_valid, since osm2pgsql creates such polygons and st_union will error out when it finds one (causing the tile to not render).

[Ircama]

The following code looks working better:

CASE WHEN !pixel_width!::real*!pixel_height!::real > 1400::double precision THEN 'outzoom'
  WHEN EXISTS (SELECT 1 FROM mountain m WHERE ST_Intersects(a.way, m.way) LIMIT 1) THEN 'yes' ELSE 'no' END AS is_mountain,
CASE WHEN !pixel_width!::real*!pixel_height!::real > 1400::double precision THEN 'outzoom'
  WHEN EXISTS (SELECT 1 FROM settlement s WHERE ST_Intersects(a.way, s.way) LIMIT 1) THEN 'yes' ELSE 'no' END AS is_settlement,

I tested different queries including different constructs, usage of !bbox! (check note below), left outer joins, different types of Postgis spatial joins like ST_DWithin or others, unions, etc. and this to me appears the most efficient among the ones I checked; I verified it on a pretty dense region finding decent performance and executing flawlessly. I used the following layers with the same project: landcover, buildings-major, amenity-points-poly, amenity-points, text-poly, text-point.

The code relies on the following:

CREATE OR REPLACE VIEW settlement AS SELECT way FROM planet_osm_polygon b WHERE ST_IsValid(b.way)
AND b.way_area > 59750::double precision AND b.landuse IN ('commercial',
'construction', 'garages', 'industrial', 'military', 'railway', 'residential', 'retail','village_green',
'recreation_ground', 'greenhouse_horticulture', 'brownfield') OR
b.place IN ('city', 'town', 'village', 'square', 'city_block', 'neighbourhood', 'quarter', 'suburb', 'borough');

CREATE OR REPLACE VIEW mountain AS SELECT way FROM planet_osm_polygon b
WHERE ST_IsValid(b.way) AND b.way_area > 59750::double precision
AND "natural" IN ('mountain_range', 'massif');

CREATE INDEX settlement_gix ON planet_osm_polygon USING GIST (way)
WHERE ST_IsValid(way) AND way_area > 59750::double precision
AND landuse IN ('commercial', 'construction', 'garages', 'industrial',
'military', 'railway', 'residential', 'retail', 'village_green', 'recreation_ground',
'greenhouse_horticulture', 'brownfield')
OR place IN ('city', 'town', 'village', 'square', 'city_block', 'neighbourhood', 'quarter', 'suburb', 'borough');

CREATE INDEX mountain_gix ON planet_osm_polygon USING GIST (way) WHERE ST_IsValid(way)
AND way_area > 59750::double precision AND
"natural" IN ('mountain_range', 'massif');

Views are not necessary; they permit a cleaner code but need to be created before running the renderer. The views also avoid to carelessly change queries without dropping and recreating indexes (anyway, if views are not wanted, a comment in the code might address this).

Indexes are strongly suggested to significantly improve performance.

The code allows introducing the following two topology selectors within CartoCSS:

• is_settlement: valued to 'yes' when the feature is within a built-up zone (and set to 'no' when outside)
• is_mountain: valued to 'yes' when the feature is within a mountain area (and set to 'no' for default zones outside the related mountain polygons).

Both parameters are valued to 'outzoom' when outside the minimum allowed zooming that enables the topology queries. The zoom filter allows including the code within layers configured for different minzoom, as well as ensures performance at low zooms (e.g., < 13), where the rendering differentiation is not needed.

The appropriate condition to differentiate zones outside built-up places would be [is_settlement='no'].

The appropriate condition to differentiate zones within mountains would be [is_mountain='yes'] (typically associated to [is_settlement='no']).

As soon as hstore is available, is_mountain selector could also benefit from place=region & region:type=mountain_area, that is rather used in some zones.

In the consideration that mountain polygons are currently almost unused in OSM, is_mountain=yes and is_settlement=no allows a smooth introduction within a startup phase only dedicated to these very specific areas, without affecting the standard rendering. Subsequently, the general differentiation of the rendering outside built-up areas could be evaluated.

For instance, to render at lower zoom a little church in the mountains preserving the default style outside, the following can be used:

[feature = 'amenity_place_of_worship'] [is_mountain = 'yes'][is_settlement = 'no'][zoom >= 13],
[feature = 'amenity_place_of_worship'][zoom >= 16]

IMHO the introduction of these rendering differentiators will improve the quality of the maps.

Would it be worthwhile to have the code within a new PR?

Is there a possibility for further code optimizations?

Notice that it has to be verified where to put the new indexes (and views).

Additional note:

I noticed that !bbox! does not work within the above query, like in following:

CASE WHEN !pixel_width!::real*!pixel_height!::real > 1400::double precision THEN 'outzoom'
WHEN EXISTS (SELECT 1 FROM mountain m WHERE m.way && !bbox! AND ST_Intersects(a.way, m.way) LIMIT 1) THEN 'yes' ELSE 'no' END AS is_mountain,

And this is because !bbox! it is wrongly expanded to the maxint values:

ST_SetSRID('BOX3D(-3.402823466385289e+038 -3.402823466385289e+038,3.402823466385289e+038 3.402823466385289e+038)'::box3d, 900913)

Why does this happen and which is the correct syntax to get appropriate bounding?

[This post is re-editing my previous one, which did not provide a step forward to @gravitystorm hints]
[Edited to include the "Additional note"]

[Ircama]

Is there a way to test the above code with a wide area (or maybe with the world) to verify whether its performance is acceptable?

[imagico]

Regarding the !bbox! problem - this has been discussed previously on #1853 - solution in #1853 (comment)

[Ircama]

Thanks a lot for this hint! (Indeed I searched but...)

I guess you would mean I should need to convert:

CASE WHEN !pixel_width!::real*!pixel_height!::real > 1400::double precision THEN 'outzoom'
WHEN EXISTS (SELECT 1 FROM mountain m
WHERE m.way && !bbox! AND
ST_Intersects(a.way, m.way) LIMIT 1) THEN 'yes' ELSE 'no' END AS is_mountain,

into:

CASE WHEN !pixel_width!::real*!pixel_height!::real > 1400::double precision THEN 'outzoom'
WHEN EXISTS (SELECT 1 FROM mountain m
WHERE m.way && CASE WHEN ST_Perimeter(!bbox!) > 3e+38 THEN m.way ELSE !bbox! END AND
ST_Intersects(a.way, m.way) LIMIT 1) THEN 'yes' ELSE 'no' END AS is_mountain,

I will perform some benchmark, above all to verify whether there will be some performance improvement vs. this:

CASE WHEN !pixel_width!::real*!pixel_height!::real > 1400::double precision THEN 'outzoom'
WHEN EXISTS (SELECT 1 FROM mountain m
WHERE ST_Intersects(a.way, m.way) LIMIT 1) THEN 'yes' ELSE 'no' END AS is_mountain,

[kocio-pl]

Just a general idea: one of the most important thing about urban/rural distinction is to not clutter the urban areas - see for example problem with wayside shrine (#131 (comment)). I guess we could use simple heuristic to find out if we're safe enough measuring distance from the nearest city. There's probably no such danger in the village, so I would treat it as a rural area anyway, ignoring landuse=residential, since it tells us less about density than a type of the settlement. We could also check if we're inside city area for additional safety.

It's just a simplification, but it relies on pretty common feature (cities tagging) and would be quite simple math operation. I'm not sure however if it's possible using our current toolset.

[kocio-pl]

Follow up to #2945 (comment).

Looks like ua2 algorithm is not suited for motels/inns (see #1705), because they are usually out of town, but near the road.

[kocio-pl]

@SK53: What do you think about my simple idea with circles around cities (#1957 (comment))? Could you compare them?