Mirror mojo fails due cache files being too long

[rbioteau]

When mirroring a target platform containing the following repository

<plugin>
            <groupId>org.eclipse.tycho.extras</groupId>
            <artifactId>tycho-p2-extras-plugin</artifactId>
            <version>4.0.10</version>
            <executions>
              <execution>
                <phase>prepare-package</phase>
                <goals>
                  <goal>mirror</goal>
                </goals>
              </execution>
            </executions>
            <configuration>
              <source>
                <repository>
                  <url>https://groovy.jfrog.io/artifactory/plugins-release/e4.33</url>
                  <layout>p2</layout>
                </repository>
              </source>
              <ius>
                <iu>
                  <id>org.codehaus.groovy.compilerless.feature.feature.group</id>
                  <version>5.5.0.v202409302028-e2409</version>
                </iu>
                <iu>
                  <id>org.codehaus.groovy.m2eclipse.feature.feature.group</id>
                  <version>5.5.0.v202409302028-e2409</version>
                </iu>
                <iu>
                  <id>org.codehaus.groovy30.feature.feature.group</id>
                  <version>5.5.0.v202409302028-e2409</version>
                </iu>
                <iu>
                  <id>org.codehaus.groovy.eclipse.feature.feature.group</id>
                  <version>5.5.0.v202409302028-e2409</version>
                </iu>
              </ius>
            </configuration>
          </plugin>

The mojo fails with the following error:

download from https://groovy.jfrog.io/artifactory/plugins-release-local/org/codehaus/groovy/groovy-eclipse-integration/5.5.0/e4.33/plugins/org.codehaus.groovy_4.0.23.v202409302028-e2409.jar failed]: /home/romain/.m2/repository/.cache/tycho/https/storage.googleapis.com/art-groovy-filestore-1478119448/filestore/fd/fd701dc97c13a0248cdd59f861529c6f40f84670/X-Artifactory-artifactPath=org%2Fcodehaus%2Fgroovy%2Fgroovy-eclipse-integration%2F5.5.0%2Fe4.33%2Fplugins%2Forg.codehaus.groovy_4.0.23.v202409302028-e2409.jar/X-Artifactory-originPackageType=maven/X-Artifactory-originProjectKey=default/X-Artifactory-originRepoType=local/X-Artifactory-originRepositoryKey=plugins-release-local/X-Artifactory-packageType=maven/X-Artifactory-projectKey=default/X-Artifactory-repoType=local/X-Artifactory-repositoryKey=plugins-release-local/X-Artifactory-username=anonymous/X-Goog-Algorithm=GOOG4-RSA-SHA256/X-Goog-Credential=gro7976160665383240%40aol-prod.iam.gserviceaccount.com%2F20241129%2Fauto%2Fstorage%2Fgoog4_request/X-Goog-Date=20241129T100207Z/X-Goog-Expires=60/X-Goog-SignedHeaders=host/response-content-disposition=attachment%3Bfilename%3D%22org.codehaus.groovy_4.0.23.v202409302028-e2409.jar%22/response-content-type=application%2Fjava-archive/x-jf-traceId=3b1a4ee0e941c91a/X-Goog-Signature=1f19412cfab85789a5a9df9dc1890de7536a2116657a9d9087d4517ec891b9ef0b62a43feea9e547c1eaeae448a268b14929ed7de30cd6fe565cdecafc36e5bef3469bc5b889ad30c8e5f4794f041b51a687f2e9899e311fd98134af6bb41098ffd8d061b0bab11eb87b67185bc02289051f9c187784ac07f5f87c89a7478663892f91815213ac7914b95a5276c54ef6cb3b16c240d6e9560e76a9dd86fb3ea07139d398e308fcc684d31d650e8e4111465722c8694e920224f76d00ebaf1bbe7068b603836567df3b95d39f629001657aaa1212ede9e9606f0c8e6451c7b2222e7d74454a68ef8314dbb65399c766a9400e7164b1b171445d36ea464a3bc5ca.headers (Name is too long) -> [Help 1]

[rbioteau]

I manage to workaround the issue patching the SharedHttpCacheStorage.

diff --git a/p2-maven-plugin/src/main/java/org/eclipse/tycho/p2maven/transport/SharedHttpCacheStorage.java b/p2-maven-plugin/src/main/java/org/eclipse/tycho/p2maven/transport/SharedHttpCacheStorage.java
index b39b3c5..70ffa9b 100644
--- a/p2-maven-plugin/src/main/java/org/eclipse/tycho/p2maven/transport/SharedHttpCacheStorage.java
+++ b/p2-maven-plugin/src/main/java/org/eclipse/tycho/p2maven/transport/SharedHttpCacheStorage.java
@@ -251,17 +251,30 @@
 				}
 				updateHeader(response, code);
 				if (isRedirected(code)) {
-					File cachedFile = SharedHttpCacheStorage.this.getCacheEntry(getRedirect(uri), logger)
-							.getCacheFile(transportFactory);
-					// https://github.com/eclipse-tycho/tycho/issues/2938
-					// Redirect may change extension. P2's SimpleMetadataRepositoryFactory relies on
-					// accurate file extension to be cached.
-					// Copying file to accommodate original request and its file extension.
-					// Once https://github.com/eclipse-equinox/p2/issues/355 is fixed, cachedFile
-					// may be returned directly without copying.
-					response.close(); // early close before doing unrelated file I/O
-					FileUtils.copyFile(cachedFile, file);
-					return file;
+					try {
+						File cachedFile = SharedHttpCacheStorage.this.getCacheEntry(getRedirect(uri), logger)
+								.getCacheFile(transportFactory);
+						// https://github.com/eclipse-tycho/tycho/issues/2938
+						// Redirect may change extension. P2's SimpleMetadataRepositoryFactory relies on
+						// accurate file extension to be cached.
+						// Copying file to accommodate original request and its file extension.
+						// Once https://github.com/eclipse-equinox/p2/issues/355 is fixed, cachedFile
+						// may be returned directly without copying.
+						response.close(); // early close before doing unrelated file I/O
+						FileUtils.copyFile(cachedFile, file);
+						return file;
+					}catch (IOException e) {
+						logger.warn("Failed to cache file: " + e.getMessage());
+
+						HttpTransport redirectTransport = transportFactory.createTransport(getRedirect(uri));
+						redirectTransport.get(r -> {
+							try (OutputStream os = new BufferedOutputStream(new FileOutputStream(file))) {
+								r.transferTo(os);
+							}
+							return null;
+						});
+						return file;
+					}
 				}
 				if (exits) {
 					FileUtils.forceDelete(file);

[kysmith-csg]

@rbioteau could you submit this as a pull request? We are facing the same issue (also with jfrog).

[TIBCOrkrajews]

Caching using redirected URLs to cloud/CDN providers as the key is unlikely to function as a cache because the URL is likely to contain a token with a very short lifespan. Repeated requests to the same original URL will yield different redirected URLs and there will never be a cache hit for more than a few minutes.