feature: improve diff cmd with copy opts (#1183)

* add most of the `mbtiles copy` options to the `mbtiles diff` command
* reworked the book

docs/src/SUMMARY.md

@@ -26,8 +26,9 @@
   - [Recipes](recipes.md)
 - [Tools](tools.md)
   - [martin-cp bulk tile generation](martin-cp.md)
-  - [MBTiles Info and Metadata](mbtiles-meta.md)
-  - [MBTiles Copying / Diffing](mbtiles-copy.md)
-  - [MBTiles Validation](mbtiles-validation.md)
+  - [MBTiles Metadata](mbtiles-meta.md)
   - [MBTiles Schemas](mbtiles-schema.md)
+  - [Copying MBTiles](mbtiles-copy.md)
+  - [Diffing/Patching MBTiles](mbtiles-diff.md)
+  - [Validating MBTiles](mbtiles-validation.md)
 - [Development](development.md)

docs/src/mbtiles-copy.md

@@ -18,13 +18,13 @@ mbtiles copy normalized.mbtiles dst.mbtiles \
 
 ## `mbtiles copy --diff-with-file`
 
-Copy command can also be used to compare two mbtiles files and generate a delta (diff) file. The diff file can be applied to the `src_file.mbtiles` elsewhere, to avoid copying/transmitting the entire modified dataset.  The delta file will contain all tiles that are different between the two files (modifications, insertions, and deletions as `NULL` values), for both the tile and metadata tables.
-
-There is one exception: `agg_tiles_hash` metadata value will be renamed to `agg_tiles_hash_in_diff`, and a new `agg_tiles_hash` will be generated for the diff file itself. This is done to avoid confusion when applying the diff file to the original file, as the `agg_tiles_hash` value will be different after the diff is applied. The `apply-diff` command will automatically rename the `agg_tiles_hash_in_diff` value back to `agg_tiles_hash` when applying the diff.
+This option is identical to using [`mbtiles diff ...`](mbtiles-diff.md). The following commands two are equivalent:
 
 ```shell
-mbtiles copy src_file.mbtiles diff_file.mbtiles \
-         --diff-with-file modified_file.mbtiles
+mbtiles diff file1.mbtiles file2.mbtiles diff.mbtiles
+
+mbtiles copy file1.mbtiles diff.mbtiles \
+        --diff-with-file file2.mbtiles
 ```
 
 ## `mbtiles copy --apply-patch`
@@ -33,28 +33,5 @@ Copy a source file to destination while also applying the diff file generated by
 
 ```shell
 mbtiles copy src_file.mbtiles dst_file.mbtiles \
-        --apply-patch diff_file.mbtiles
-```
-
-## `mbtiles apply-patch`
-
-Apply the diff file generated from `copy` command above to an mbtiles file. The diff file can be applied to the `src_file.mbtiles` elsewhere, to avoid copying/transmitting the entire modified dataset.
-
-Note that the `agg_tiles_hash_in_diff` metadata value will be renamed to `agg_tiles_hash` when applying the diff. This is done to avoid confusion when applying the diff file to the original file, as the `agg_tiles_hash` value will be different after the diff is applied.
-
-```shell
-mbtiles apply-patch src_file.mbtiles diff_file.mbtiles
-```
-
-#### Applying diff with SQLite
-
-Another way to apply the diff is to use the `sqlite3` command line tool directly. This SQL will delete all tiles from `src_file.mbtiles` that are set to `NULL` in `diff_file.mbtiles`, and then insert or update all new tiles from `diff_file.mbtiles` into `src_file.mbtiles`, where both files are of `flat` type. The name of the diff file is passed as a query parameter to the sqlite3 command line tool, and then used in the SQL statements.  Note that this does not update the `agg_tiles_hash` metadata value, so it will be incorrect after the diff is applied.
-
-```shell
-sqlite3 src_file.mbtiles \
-  -bail \
-  -cmd ".parameter set @diffDbFilename diff_file.mbtiles" \
-  "ATTACH DATABASE @diffDbFilename AS diffDb;" \
-  "DELETE FROM tiles WHERE (zoom_level, tile_column, tile_row) IN (SELECT zoom_level, tile_column, tile_row FROM diffDb.tiles WHERE tile_data ISNULL);" \
-  "INSERT OR REPLACE INTO tiles (zoom_level, tile_column, tile_row, tile_data) SELECT * FROM diffDb.tiles WHERE tile_data NOTNULL;"
+        --apply-patch diff.mbtiles
 ```

docs/src/mbtiles-diff.md

@@ -2,25 +2,45 @@
 
 ## `mbtiles diff`
 
-Diff command compares two mbtiles files `A` and `B`, and generates a diff (delta) file.
-If the diff file is [applied](mbtiles-copy.md#mbtiles-apply-patch) to `A`, it will produce `B`.  
-The diff file will contain all tiles that are different between the two files
-(modifications, insertions, and deletions as `NULL` values), for both the tile and metadata tables.  
-The only exception is `agg_tiles_has` metadata value. It will be renamed to `agg_tiles_hash_in_diff` and a
-new `agg_tiles_hash` will be generated for the diff file itself.
+Copy command can also be used to compare two mbtiles files and generate a delta (diff) file. The diff file can be [applied](#mbtiles-apply-patch) to the `src_file.mbtiles` elsewhere, to avoid copying/transmitting the entire modified dataset.  The delta file will contain all tiles that are different between the two files (modifications, insertions, and deletions as `NULL` values), for both the tile and metadata tables.
+
+There is one exception: `agg_tiles_hash` metadata value will be renamed to `agg_tiles_hash_in_diff`, and a new `agg_tiles_hash` will be generated for the diff file itself. This is done to avoid confusion when applying the diff file to the original file, as the `agg_tiles_hash` value will be different after the diff is applied. The `apply-diff` command will automatically rename the `agg_tiles_hash_in_diff` value back to `agg_tiles_hash` when applying the diff.
 
 ```shell
-# This command will comapre `a.mbtiles` and `b.mbtiles`, and generate a new diff file `diff.mbtiles`.
-mbtiles diff a.mbtiles b.mbtiles diff.mbtiles
+# This command will compare `file1.mbtiles` and `file2.mbtiles`, and generate a new diff file `diff.mbtiles`.
+mbtiles diff file1.mbtiles file2.mbtiles diff.mbtiles
 
-# If diff.mbtiles is applied to a.mbtiles, it will produce b.mbtiles 
-mbtiles apply-diff a.mbtiles diff.mbtiles b2.mbtiles
+# If diff.mbtiles is applied to file1.mbtiles, it will produce file2.mbtiles 
+mbtiles apply-diff file1.mbtiles diff.mbtiles file2a.mbtiles
 
-# b.mbtiles and b2.mbtiles should now be the same
+# file2.mbtiles and file2a.mbtiles should now be the same
 # Validate both files and see that their hash values are identical
-mbtiles validate b.mbtiles
-[INFO ] The agg_tiles_hashes=E95C1081447FB25674DCC1EB97F60C26 has been verified for b.mbtiles
+mbtiles validate file2.mbtiles
+[INFO ] The agg_tiles_hashes=E95C1081447FB25674DCC1EB97F60C26 has been verified for file2.mbtiles
+
+mbtiles validate file2a.mbtiles
+[INFO ] The agg_tiles_hashes=E95C1081447FB25674DCC1EB97F60C26 has been verified for file2a.mbtiles
+```
+
+## `mbtiles apply-patch`
+
+Apply the diff file generated with the `mbtiles diff` command above to an MBTiles file. The diff file can be applied to the `src_file.mbtiles` that has been previously downloaded to avoid copying/transmitting the entire modified dataset again. The `src_file.mbtiles` will modified in-place. It is also possible to apply the diff file while copying the source file to a new destination file, by using the [`mbtiles copy --apply-patch`](mbtiles-copy.md#mbtiles-copy---apply-patch) command.
+
+Note that the `agg_tiles_hash_in_diff` metadata value will be renamed to `agg_tiles_hash` when applying the diff. This is done to avoid confusion when applying the diff file to the original file, as the `agg_tiles_hash` value will be different after the diff is applied.
 
-mbtiles validate b2.mbtiles
-[INFO ] The agg_tiles_hashes=E95C1081447FB25674DCC1EB97F60C26 has been verified for b2.mbtiles
+```shell
+mbtiles apply-patch src_file.mbtiles diff_file.mbtiles
+```
+
+#### Applying diff with SQLite
+
+Another way to apply the diff is to use the `sqlite3` command line tool directly. This SQL will delete all tiles from `src_file.mbtiles` that are set to `NULL` in `diff_file.mbtiles`, and then insert or update all new tiles from `diff_file.mbtiles` into `src_file.mbtiles`, where both files are of `flat` type. The name of the diff file is passed as a query parameter to the sqlite3 command line tool, and then used in the SQL statements.  Note that this does not update the `agg_tiles_hash` metadata value, so it will be incorrect after the diff is applied.
+
+```shell
+sqlite3 src_file.mbtiles \
+  -bail \
+  -cmd ".parameter set @diffDbFilename diff_file.mbtiles" \
+  "ATTACH DATABASE @diffDbFilename AS diffDb;" \
+  "DELETE FROM tiles WHERE (zoom_level, tile_column, tile_row) IN (SELECT zoom_level, tile_column, tile_row FROM diffDb.tiles WHERE tile_data ISNULL);" \
+  "INSERT OR REPLACE INTO tiles (zoom_level, tile_column, tile_row, tile_data) SELECT * FROM diffDb.tiles WHERE tile_data NOTNULL;"
 ```

martin/src/srv/server.rs

@@ -150,6 +150,7 @@ pub fn new_server(config: SrvConfig, state: ServerState) -> MartinResult<(Server
         .workers(worker_processes)
         .run()
         .err_into();
+
     Ok((Box::pin(server), listen_addresses))
 }