File size

[zelaomauricio]

I downloaded the file (Brazil) and tried to open the file in Qgis, I tried to split the file and it was not possible because of the size (19gb). Does anyone have a solution?

[andreiste]

You could use ogr2ogr, FME or ArcGIS Data Interoperability to transform the geojson file in another format that uses spatial indexes. I used ArcGIS Data Interoperability to export the data into a file geodatabase and it worked.

[rbrundritt]

Similar to #2 the file is massive (19GB when compressed), no desktop app is going to easily load that. Loading this into a database and querying a subset or converting that data into an optimized format for visualization (i.e. map tiles) are the common approach in this situation. The file uses geojsonl format note the "l". This is an optimized geojson file format for large datasets. Each feature is on a single line and there is only one feature per line. As such, you can read the file as a stream, line by line, a common approach for reading large text files.

[gtdca98]

Use linux split comand

split --verbose -l50000 country.geojsonl country_segment.

reff: https://www.networkworld.com/article/3489256/breaking-linux-files-into-pieces-with-the-split-command.html#:~:text=If%20you%20want%20your%20file,which%20may%20have%20fewer%20lines.

[giswqs]

@gtdca98 Thanks for sharing the split command. I modified it a little to split the file into equal size chunks and keep the file extension.

split -l 100000 country.geojsonl country_ --additional-suffix=.geojsonl

[andwoi]

We've also added a python script to show how this can be accomplished: https://github.com/microsoft/GlobalMLBuildingFootprints/blob/main/scripts/read-large-files.py

with some additional logic, one can produce spatial partitions.