Generate an automatic summary of an upload

documentation/docs/help/en/Custom imagery.md

@@ -6,7 +6,7 @@
 * imagery layers in [MBTiles](https://github.com/mapbox/mbtiles-spec) and [PMtiles V3](https://github.com/protomaps/PMTiles/blob/main/spec/v3/spec.md) format on device
 * WMS servers supporting images in EPSG:3857 (including alternative names like EPSG:900913 etc) and EPSG:4326 projections
 
-Besides manually adding layers you can add WMS layers by querying a WMS server for a list of supported layers see [layer control](Main%20map%20display.md#Layer%20control), or by querying the Open Aerial Map (OAM) service.
+Besides manually adding layers you can add WMS layers by querying a WMS server for a list of supported layers see [layer control](Main%20map%20display.md#layer_control), or by querying the Open Aerial Map (OAM) service.
 
 __NOTE__ even though the URLs may look similar to those for a WMS layer, we do not support requesting data from ESRI MapServer with their native protocol.
 

documentation/docs/help/en/Introduction.md

@@ -165,9 +165,9 @@ When the red lock is displayed all non-editing actions are available. Additional
 
 Select the same button or menu item you did for the download and now select "Upload data to OSM server".
 
-Vespucci supports OAuth authorization and the classical username and password method. OAuth is preferable since it avoids sending passwords in the clear.
+Vespucci supports OAuth 2, OAuth 1.0a authorization and the classical username and password method. Since July 1st 2024 the standard OpenStreetMap API only supports OAuth 2 and other methods are only available on private installations of the API or other projects that have repurposed OSM software.  
 
-New Vespucci installs will have OAuth enabled by default. On your first attempt to upload modified data, a page from the OSM website loads. After you have logged on (over an encrypted connection) you will be asked to authorize Vespucci to edit using your account. If you want to or need to authorize the OAuth access to your account before editing there is a corresponding item in the "Tools" menu.
+Authorizing Vespucci to access your account on your behalf requires you to one time login with your display name and password. If your Vespucci install isn't authorized when you attempt to upload modified data you will be asked to login to the OSM website (over an encrypted connection). After you have logged on you will be asked to authorize Vespucci to edit using your account. If you want to or need to authorize the OAuth access to your account before editing there is a corresponding item in the "Tools" menu.
 
 If you want to save your work and do not have Internet access, you can save to a JOSM compatible .osm file and either upload later with Vespucci or with JOSM. 
 

documentation/docs/help/en/Main map display.md

@@ -37,6 +37,8 @@ _shop_, _amenity_, _leisure_, _tourism_, _craft_, _office_ or _emergency_. If an
 
 Tapping an entry in the display will center the map on the object and select it, tapping it a second time (just as on the map display) will start the property editor, in _Tag only_ mode the property editor will start directly as expected. The POI entries are highlighted with the same validation indication as map icons, see [validation styling](https://github.com/MarcusWolschon/osmeditor4android/blob/master/src/main/assets/styles/Color-round.xml#L39).
 
+<a id="layer_control"></a>
+
 ### Layer control
 
 Vespucci currently supports multiple tiled background imagery layers, multiple tiled overlay layers (both raster and Mapbox vector tiles), a grid/scale layer, a task layer, a photo layer, multiple GeoJSON layers, multiple GPX/GPS layers and, naturally, an OSM data layer. Tapping the layer control (upper right corner) will display the layer dialog).
@@ -85,7 +87,7 @@ The layer dialog supports the following actions on the layer entries:
         * __Configure...__ Change layer settings
         * __Discard__ Turn this layer off. For the task layer this will free resources if the app is exited and re-started.
     * Data layer:
-        * __Configure...__ Select the API instance, configure the URLs including read-only sources and authentication method. Basic Authentication, OAuth 1.0a and OAuth 2 are supported, however the API instance on openstreetmap.org only supports OAuth 2 since June 2024.
+        * __Configure...__ Select the API instance, configure the URLs including read-only sources and authentication method. Basic Authentication, OAuth 1.0a and OAuth 2 are supported, however the API instance on openstreetmap.org only supports OAuth 2 since July 2024.
     * Data and Tasks layers:
         * __Info__ Display some information on the contents.
         * __Prune__ remove downloaded data from storage that is outside of the current screen and unmodified.
@@ -169,6 +171,8 @@ To reposition or remove the "on-map" GPS button use the "Follow position button
  * **Pause GPX track** - pause recording the current GPX track
  * **Clear GPX track** - clear the current GPX track
 
+<a id="transfer"></a>
+
 ### ![Transfer](../images/menu_transfer.png) Transfer
 
 Select either the transfer icon ![Transfer](../images/menu_transfer.png) or the "Transfer" menu item. This will display seven or eight options:
@@ -183,7 +187,7 @@ Select either the transfer icon ![Transfer](../images/menu_transfer.png) or the
  * **Pan and zoom auto download** - download the area shown in the current screen automatically *(requires network connectivity)*
  * **Update data** - re-download data for all areas and update what is in memory *(requires network connectivity)*
  * **Clear data** - remove any OSM data in memory
- * **File...** - saving and loading OSM data to/from on device files.
+ * **File...** - saving and loading OSM data to/from on device files. <a id="file"></a>
     * **Export changes to OSC file** - write a ".osc" format file containing the current edits
     * **Apply changes from OSC file** - read a ".osc" format file and apply its contents
     * **Save to JOSM file...** - save as a JOSM compatible XML format file
@@ -206,6 +210,8 @@ Select either the transfer icon ![Transfer](../images/menu_transfer.png) or the
 
 Show the user preference screens. The settings are split into two sets: the first screen contains the more commonly used preferences, the "Advanced preferences" contains the less used ones. 
 
+<a id="tools"></a>
+
 ### ![Tools](../images/menu_tools.png) Tools…
 
  * **Apply stored offset to imagery** - apply stored offset, if it exists, for the current background layer 

documentation/docs/help/en/Uploading your changes.md

@@ -0,0 +1,33 @@
+# Uploading your changes
+
+While Vespucci allows you to save your edits to local files on your device, most use cases center around uploading and publishing your data 
+to a server providing the [OpenStreetMap editing API](https://wiki.openstreetmap.org/wiki/API_v0.6) and that will be the instance available on [openstreetmap.org](https://openstreetmap.org) if you are contributing to regular OpenStreetMap. The following information only concerns itself with this standard configuration, other APIs and non-standard authorizations methods can be configured in the [data layer configuration](Main%20map%20display.md#layer_control). 
+
+## Authorization
+
+To successfully publish your edits you need to authorize your Vespucci on your device to access your account on openstreetmap.prg on your behalf. You can start the authorization process by either selecting the corresponding option in the configuration dialog display on initial install of the app, by starting an upload process or by using the _Authorize OAuth_ option from the [Tools menu](Main%20map%20display.md#tools).
+
+_Note_ that you will need to have your OSM display name and password available and be connected to the Internet to successfully complete authorization, if you don't already have an account you will have to create one outside of Vespucci prior to starting authorization. While it technically would be possible to create the account from inside the app, doing so invokes additional requirements from google that we cannot fulfill for legal reasons.
+
+During the authorization process you will be connected to openstreetmap.org and asked to login to your account, you then need to confirm the authorization and will be returned back to Vespuccis map map display.
+
+If you have pending edits and for whatever reason cannot authorize, Vespucci reliably saves your edits on your device until you upload them or explicitly delete them. If you want to be extra safe you can [save them to a local file](Main%20map%20display.md#file) in OCS or (J)OSM format that you can read with Vespucci or import in to JOSM. 
+
+## Uploading
+
+To upload your edits you can either select _Upload data to OSM server_ from the [Transfer menu](Main%20map%20display.md#trasfer), or select _Upload element_ from the overflow menu in any of the element selection modes or multi-select. The later will only upload the selected elements.
+
+The _Upload changes_ dialog display two tabs _Changes_ and _Properties_. The _Changes_ tab displays a list of all the OSM elements that will be affected by the upload, the type of change and an information button that allows you to view further information on the element and jump to it to make corrections. The _Properties_ tab allows you to set a _comment_
+for this upload and document that _source_ that was used to create the changes (for example _survey_ if you were personally present at the location in question). Prior entries in both
+fields are retained and can be used when appropriate.
+
+It is considered best practice to provide meaningful information for other mappers in these fields. However Vespucci will automatically include a summary of your changes (starting with version 20.2) and a list of all imagery layers used for the current set of changes. If you leave the comment field empty it will include text pointing to the automatically generated summary.
+
+## Upload options
+
+- _Close changeset_ close the changeset after the upload. If this is unchecked, changesets will be left open and further uploads will be appended. Note that the changeset may be automatically closed by the API, this is detected on upload and a new changeset will be created. The default setting for this is enabled.
+- _Close open changeset_ only displayed if an open changeset has been detected (that is _Close changeset_ must be disabled). Close the current changeset after upload. This setting only applies to the current upload. The default setting for this is disabled.
+- _Request review_ add a tag asking for a review of the changes. Note this does not stop your changes from being immediately available after you have completed the upload, it is 
+simply an indication to other contributors and they might or might not actually review your edits. The default setting for this is disabled.
+- _Warn if comment is empty_ As described above it is best practice to add a meaningful comment to your upload, if you don't want to do this, unchecking this option
+will disable warnings that the comment is empty and disable the automatic switch to the _Properties_ tab prior to the upload. This is available from version 20.2 on, the default setting for this is enabled.

documentation/mkdocs.yml

@@ -63,6 +63,7 @@ nav:
     - 'Tag only mode': 'help/en/Tag only mode.md'
     - 'Indoor mode': 'help/en/Indoor mode.md'
     - 'Address mode': 'help/en/Address mode.md'
+    - 'Uploading your changes': 'help/en/Uploading your changes.md'
     - 'Conflict resolution': 'help/en/Conflict resolution.md'
     - 'Presets': 'help/en/Presets.md' 
     - 'Aligning background imagery': 'help/en/Aligning background imagery.md'

src/androidTest/java/de/blau/android/osm/TransferMenuTest.java

@@ -20,6 +20,8 @@
 import org.junit.Test;
 import org.junit.runner.RunWith;
 import org.xml.sax.SAXException;
+import org.xmlpull.v1.XmlPullParserException;
+import org.xmlpull.v1.XmlPullParserFactory;
 
 import com.orhanobut.mockwebserverplus.MockWebServerPlus;
 
@@ -38,10 +40,10 @@
 import de.blau.android.Logic;
 import de.blau.android.Main;
 import de.blau.android.R;
-import de.blau.android.R.string;
 import de.blau.android.SignalHandler;
 import de.blau.android.SignalUtils;
 import de.blau.android.TestUtils;
+import de.blau.android.listener.UploadListener;
 import de.blau.android.prefs.API;
 import de.blau.android.prefs.AdvancedPrefDatabase;
 import de.blau.android.prefs.Preferences;
@@ -133,6 +135,8 @@ public void dataUpload() {
             fail(e1.getMessage());
         }
         UploadConflictTest.fillCommentAndSource(instrumentation, device);
+        uiSelector = new UiSelector().className("android.widget.Button").instance(1); // dialog upload button
+        button = device.findObject(uiSelector);
         try {
             button.clickAndWaitForNewWindow();
         } catch (UiObjectNotFoundException e1) {
@@ -152,10 +156,12 @@ public void dataUpload() {
             mockServer.takeRequest(); // capabilities query
             RecordedRequest recordedRequest = mockServer.takeRequest(); // changeset
             // this currently doesn't work
-            // Changeset changeset = Changeset.parse(XmlPullParserFactory.newInstance().newPullParser(),
-            // new ByteArrayInputStream(recordedRequest.getBody().readByteArray()));
-            // assertEquals(COMMENT_1, changeset.tags.get("comment"));
-            // assertEquals(SOURCE_1, changeset.tags.get("source"));
+            byte[] request = recordedRequest.getBody().readByteArray();
+            Changeset changeset = Changeset.parse(XmlPullParserFactory.newInstance().newPullParser(), new ByteArrayInputStream(request));
+            assertEquals(UploadConflictTest.COMMENT_1, changeset.getTags().get(Tags.KEY_COMMENT));
+            assertEquals("1 Tertiary", changeset.getTags().get(UploadListener.V_CREATED));
+            assertEquals("1 untagged way", changeset.getTags().get(UploadListener.V_DELETED));
+            assertTrue(changeset.getTags().get(UploadListener.V_MODIFIED).contains("2 unknown object to Public transport route (Legacy)"));
             recordedRequest = mockServer.takeRequest(); // diff upload
             OsmChangeParser ocParser = new OsmChangeParser();
             ocParser.start(new ByteArrayInputStream(recordedRequest.getBody().readByteArray()));
@@ -171,7 +177,7 @@ public void dataUpload() {
             Way w = (Way) storage.getOsmElement(Way.NAME, 210461100);
             assertNotNull(w);
             assertEquals(OsmElement.STATE_DELETED, w.getState());
-        } catch (InterruptedException | SAXException | IOException | ParserConfigurationException e) {
+        } catch (InterruptedException | SAXException | IOException | ParserConfigurationException | XmlPullParserException e) {
             fail(e.getMessage());
         }
     }

src/androidTest/java/de/blau/android/osm/UploadConflictTest.java

@@ -55,8 +55,8 @@ public class UploadConflictTest {
 
     public static final int TIMEOUT = 90;
 
-    private static final String SOURCE_1  = "source 1";
-    private static final String COMMENT_1 = "comment 1";
+    static final String SOURCE_1  = "source 1";
+    static final String COMMENT_1 = "comment 1";
 
     MockWebServerPlus       mockServer = null;
     Context                 context    = null;

src/main/assets/help/en/Custom imagery.html

@@ -12,7 +12,7 @@ <h2>Supported layer types</h2>
 <li>imagery layers in <a href="https://github.com/mapbox/mbtiles-spec">MBTiles</a> and <a href="https://github.com/protomaps/PMTiles/blob/main/spec/v3/spec.md">PMtiles V3</a> format on device</li>
 <li>WMS servers supporting images in EPSG:3857 (including alternative names like EPSG:900913 etc) and EPSG:4326 projections</li>
 </ul>
-<p>Besides manually adding layers you can add WMS layers by querying a WMS server for a list of supported layers see <a href="Main%20map%20display.md#Layer%20control">layer control</a>, or by querying the Open Aerial Map (OAM) service.</p>
+<p>Besides manually adding layers you can add WMS layers by querying a WMS server for a list of supported layers see <a href="Main%20map%20display.md#layer_control">layer control</a>, or by querying the Open Aerial Map (OAM) service.</p>
 <p><strong>NOTE</strong> even though the URLs may look similar to those for a WMS layer, we do not support requesting data from ESRI MapServer with their native protocol.</p>
 <h2>Adding a custom imagery source</h2>
 <p>To add a custom layer goto the <em>Preferences</em> screen and select <em>Custom imagery</em>, press the <em>+</em> button to add a new layer, or you can select the same from the layer control, &quot;+&quot; menu. In the form you can set</p>

src/main/assets/help/en/Introduction.html

@@ -123,8 +123,8 @@ <h3>Vespucci in &quot;locked&quot; mode</h3>
 <h3>Saving Your Changes</h3>
 <p><em>(requires network connectivity)</em></p>
 <p>Select the same button or menu item you did for the download and now select &quot;Upload data to OSM server&quot;.</p>
-<p>Vespucci supports OAuth authorization and the classical username and password method. OAuth is preferable since it avoids sending passwords in the clear.</p>
-<p>New Vespucci installs will have OAuth enabled by default. On your first attempt to upload modified data, a page from the OSM website loads. After you have logged on (over an encrypted connection) you will be asked to authorize Vespucci to edit using your account. If you want to or need to authorize the OAuth access to your account before editing there is a corresponding item in the &quot;Tools&quot; menu.</p>
+<p>Vespucci supports OAuth 2, OAuth 1.0a authorization and the classical username and password method. Since July 1st 2024 the standard OpenStreetMap API only supports OAuth 2 and other methods are only available on private installations of the API or other projects that have repurposed OSM software.</p>
+<p>Authorizing Vespucci to access your account on your behalf requires you to one time login with your display name and password. If your Vespucci install isn't authorized when you attempt to upload modified data you will be asked to login to the OSM website (over an encrypted connection). After you have logged on you will be asked to authorize Vespucci to edit using your account. If you want to or need to authorize the OAuth access to your account before editing there is a corresponding item in the &quot;Tools&quot; menu.</p>
 <p>If you want to save your work and do not have Internet access, you can save to a JOSM compatible .osm file and either upload later with Vespucci or with JOSM.</p>
 <h4>Resolving conflicts on uploads</h4>
 <p>Vespucci has a simple conflict resolver. However if you suspect that there are major issues with your edits, export your changes to a .osc file (&quot;Export&quot; menu item in the &quot;Transfer&quot; menu) and fix and upload them with JOSM. See the detailed help on <a href="Conflict%20resolution.md">conflict resolution</a>.</p>