Slow rendering on low zoom #3534
Comments
|
Hi, thanks for the report. Low zoom rendering slowdown is expected side effect of #3458 probably. I'm not sure if we can do anything about it. Do you have any idea? |
|
Which version does not have this performance regression? I'll try to downgrade and test. |
|
Another slow query detected: SELECT ST_AsBinary("way") AS geom,"int_intermittent","landuse","natural","waterway","way_pixels" FROM (SELECT
way,
"natural",
waterway,
landuse,
name,
way_area/NULLIF(2445.98::real*2445.98::real,0) AS way_pixels,
CASE WHEN tags->'intermittent' IN ('yes')
OR tags->'seasonal' IN ('yes', 'spring', 'summer', 'autumn', 'winter', 'wet_season', 'dry_season')
THEN 'yes' ELSE 'no' END AS int_intermittent
FROM planet_osm_polygon
WHERE
(waterway IN ('dock', 'riverbank')
OR landuse IN ('reservoir', 'basin')
OR "natural" IN ('water', 'glacier'))
AND building IS NULL
AND way_area > 1*2445.98::real*2445.98::real
ORDER BY COALESCE(layer,0), way_area DESC
) AS water_areas WHERE "way" && ST_SetSRID('BOX3D(-626172.1357124997 -626172.1357124997,10644926.3071125 10644926.3071125)'::box3d, 3857) |
|
It was merged in the latest version, so v4.16.0 should be much better in this respect. Could you also check if there is any difference in speed without |
|
Same Subquery Scan on landcover_low_zoom (cost=9138117.92..9216093.64 rows=514098 width=68) (actual time=111080.940..160803.317 rows=6726331 loops=1)
-> Gather Merge (cost=9138117.92..9198100.21 rows=514098 width=322) (actual time=111079.982..120744.601 rows=6726331 loops=1)
Workers Planned: 2
Workers Launched: 2
-> Sort (cost=9137117.90..9137760.52 rows=257049 width=322) (actual time=111000.217..111808.478 rows=2242110 loops=3)
Sort Key: planet_osm_polygon.way_area DESC, (COALESCE(('wetland_'::text || CASE WHEN (planet_osm_polygon."natural" = ANY ('{wetland,mud}'::text[])) THEN CASE WHEN (planet_osm_polygon."natural" = 'mud'::text) THEN planet_osm_polygon."natural" ELSE (planet_osm_polygon.tags -> 'wetland'::text) END ELSE NULL::text END), ('landuse_'::text || CASE WHEN (planet_osm_polygon.landuse = ANY ('{forest,farmland,residential,commercial,retail,industrial,meadow,vineyard,orchard}'::text[])) THEN planet_osm_polygon.landuse ELSE NULL::text END), ('natural_'::text || CASE WHEN (planet_osm_polygon."natural" = ANY ('{wood,sand,scree,shingle,bare_rock,heath,grassland,scrub}'::text[])) THEN planet_osm_polygon."natural" ELSE NULL::text END)))
Sort Method: quicksort Memory: 2977536kB
-> Parallel Bitmap Heap Scan on planet_osm_polygon (cost=268310.16..9114019.88 rows=257049 width=322) (actual time=15107.299..106185.400 rows=2242110 loops=3)
Recheck Cond: ((way && '01030000A0110F00000100000005000000D81B7C45F81B23C1D81B7C45F81B23C10000000000000000D81B7C45F81B23C198DDD3C9B74D6441000000000000000098DDD3C9B74D644198DDD3C9B74D6441000000000000000098DDD3C9B74D6441D81B7C45F81B23C10000000000000000D81B7C45F81B23C1D81B7C45F81B23C10000000000000000'::geometry) AND (way_area > '23300'::double precision))
Filter: ((way_area > '59828.1806485391'::double precision) AND ((landuse = ANY ('{forest,farmland,residential,commercial,retail,industrial,meadow,vineyard,orchard}'::text[])) OR ("natural" = ANY ('{wood,wetland,mud,sand,scree,shingle,bare_rock,heath,grassland,scrub}'::text[]))))
Rows Removed by Filter: 2178079
Heap Blocks: exact=1291624
-> Bitmap Index Scan on planet_osm_polygon_way_area_z10 (cost=0.00..268155.93 rows=13014469 width=0) (actual time=12102.114..12102.114 rows=13260568 loops=1)
Index Cond: (way && '01030000A0110F00000100000005000000D81B7C45F81B23C1D81B7C45F81B23C10000000000000000D81B7C45F81B23C198DDD3C9B74D6441000000000000000098DDD3C9B74D644198DDD3C9B74D6441000000000000000098DDD3C9B74D6441D81B7C45F81B23C10000000000000000D81B7C45F81B23C1D81B7C45F81B23C10000000000000000'::geometry)
Planning time: 5.482 ms
Execution time: 162487.065 ms
(16 rows) |
|
In water_areas the slowdown might be the result of rendering glaciers earlier (z5+ instead of z8+). |
What changes you were testing? Have you ever tried rendering this style sheet before, especially low zooms? What is the deployment you are testing? |
|
I mean the same result without |
|
Primary deployment of this style is default map layer on OSM.org. They render zoom levels z0-z12 only about every 2 weeks on the weekend because it's a big task performance wise. Rendering natural areas from z5 makes things worse, but with their rendering schedule it's still acceptable, because it takes only few hours per month, the main feedback loop is about fast updates on z13+. Rendering the whole planet on low zoom is demanding job for the current hardware, no matter how powerful it is, see for example: https://munin.openstreetmap.org/openstreetmap.org/render.openstreetmap.org/index.html |
|
witch indexes are used and how there are built? |
|
Indexes which were created by osm2pgsql and indexes from the output of indexes.py |
|
Since there is no easy way to help that, I will close this ticket. However feel free to discuss issues if you have some other problems. |
I compared the performance of OSM-Carto on zoom levels 5 to 10 of a 1 pixel and a 0.01 pixel limit for the landcover-low-zoom layer as part of an style upgrade from 4.13.0 to 4.19.0 this week. Tests were rendering a 2048x2048 px metatile using Nik4, PostgreSQL 10 and a whole planet database on a production-ready machine (not in use currently and therefore idle). The presence of landcover on lower zoom levels increases the rendering time of a metatile in central/eastern Europe as following (times in seconds): version 4.13.0: version 4.19.0: I tried to build a special index for landcover-low-zoom. Although the database queries were using that index (I tested them with landcover-low-zoom currently uses an area size limit of 0.01 pixel (see #3458 (comment) for rendering examples). If I set this limit to 1.0 pixel, the slow zoom levels are 50% faster to render. Using However, landuser-area-symbols is slow as well. I tried following changes:
CREATE INDEX planet_osm_polygon_landuse_area_symbols_med
ON planet_osm_polygon
USING GIST (way)
WHERE
("natural" = ANY (ARRAY['marsh'::text, 'mud'::text, 'wetland'::text, 'beach'::text, 'shoal'::text, 'reef'::text, 'scrub'::text, 'sand'::text]))
AND way_area > 1300::real;Splitting the layer and creating a special index does not any significant effect. The landover-area-symbols layer renders patterns for forests, deserts and wetlands above their polygons. This makes not a lot of sense if the polygons are very small. If I set the minimum limit to 4 square pixel instead of 1 square pixel, rendering does not become faster either. @rearden-steel If you want to boost low zoom rendering, consider using shape files with simplified geometries (and don't forget to build an index on these shape files using |
|
Thanks for testing that! We currently lack performance analysis tools (#1287) and persons involved in this specific aspect. If you would like to make something about it, that would be really great. |
|
The numbers are more or less how i would expect them.
The question is at which of the zoom levels the way_area based partial indices currently used are actually most efficient. It is likely that all low zoom landcover and water queries are using one of these indices. The real question is if creating an additional one and moving the existing ones can create enough improvement to justify the additional bloat. Just creating one additional index, possibly very similar in size to the existing ones, will not answer this question.
landover-area-symbols before #2746 started at z10 and this was for a good reason - pattern rendering does not make much sense below. A high way_area threshold is problematic there for the same reason as for the base color - and not really that useful ultimately for z10 and above, landover-area-symbols in contrast to the base landcover layer does not render tags that are particularly common in use for small polygons. At the risk of sounding like a broken record - at the low zoom levels rendering each resolution separately is increasingly wasteful in terms of computing resources. And if you have times of six to ten minutes per metatile for z<=8 that is quite a lot even if you can skim this through some optimization or by sacrificing quality. If like on the OSMF servers you only update these scales once per month that is not a big practical problem but still it should be clear that this comes from the fundamental structure of the rendering framework and is not just a matter of insufficient optimization. This is what i tried to show with the low zoom demo: http://blog.imagico.de/on-basic-small-scale-landcover-rendering/. Reopening because this is a serious issue that cannot just be ignored while successively extending rendering of existing layers to lower zoom levels (water from z6 to z0, landcover from z8 to z5, landover-area-symbols from z10 to z5). This also strongly relates to the question what the vision for the low zoom levels actually is here (see #1975 (comment)). |
|
While slow rendering on low zoom is not a huge problem for the
production servers, it makes me quite miserable when trying to test
any improvements to the rendering. Even rendering a "small" area at z8
like Wales can take an unreasonable amount of time. Therefore, I agree
that this issue needs to be addressed, even if it requires using more
preprocessed data.
…On 2/15/19, Christoph Hormann ***@***.***> wrote:
Reopened #3534.
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#3534 (comment)
|
Expected behavior
Actual behavior
Slow rendering on the latest version on full planet.
Slow query:
Links and screenshots illustrating the problem
The text was updated successfully, but these errors were encountered: