Limited path rewrites.

[LexManos]

Request details

I know I'm late to the party, however looking at upgrading form 2.x to 3.x and saw that path rewriting was removed. As this makes sense from a complexity viewpoint. Is there any possibility of a limited scope of this returning. My main issue is that the maven I run is old and used by tons of people/old software and it's a real hassle/impossible to get them to update anything. However the maven is really simple, it's just one repo with a couple of proxied repos behind it. I don't have to worry about people uploading to re-written addresses because everyone who uploads is easily contactable and can update their paths. I'm concerned with the plethora of existing links out in the internet and in old versions of the software.

So what I am requesting is a single configuration value on GET/HEAD requests.
As a proof of concept I tried this code, and it works exactly how I want it.

--- a/reposilite-backend/src/main/kotlin/com/reposilite/maven/infrastructure/MavenEndpoints.kt
+++ b/reposilite-backend/src/main/kotlin/com/reposilite/maven/infrastructure/MavenEndpoints.kt
@@ -22,6 +22,7 @@ import com.reposilite.maven.api.DeleteRequest
 import com.reposilite.maven.api.DeployRequest
 import com.reposilite.maven.api.LookupRequest
 import com.reposilite.shared.extensions.resultAttachment
+import com.reposilite.storage.api.Location
 import com.reposilite.web.api.ReposiliteRoute
 import io.javalin.community.routing.Route.DELETE
 import io.javalin.community.routing.Route.GET
@@ -60,8 +61,15 @@ internal class MavenEndpoints(
     )
     private val findFile = ReposiliteRoute<Unit>("/{repository}/<gav>", HEAD, GET) {
         accessed {
-            requireGav { gav ->
-                LookupRequest(this?.identifier, requireParameter("repository"), gav)
+            requireGav { gavRaw ->
+                var repo = requireParameter("repository")
+                var gav = gavRaw
+                if (mavenFacade.getRepository(repo) == null) {
+                    gav = Location.of(repo + '/' + gav)
+                    repo = "releases"
+                }
+
+                LookupRequest(this?.identifier, repo, gav)
                     .let { request -> mavenFacade.findFile(request) }
                     .peek {
                         ctx.resultAttachment(

Note: this is just a quick POC as I have no idea how you'd want to store this config value or expose it in the settings menu. But I think it may be simple enough in this limited scope to be worth considering.

Heck, if this is possible in some other way let me know, your update guide says this can be imitated with custom repos, I'm assuming you mean by having a repo for each of the root folders and redirecting it using a loopback url to 'release' repo? Which is a real pain to setup. If there is a simpler option please let me know.

[dzikoysk]

Hey, did you consider using a plugin for this? There's a plugin that already does that:

• https://github.com/blakesmods/reposilite-plugin-path-redirect

You could also build something similar, probably even as a script in Groovy.

I know I'm late to the party, however looking at upgrading form 2.x to 3.x

Just when I started planning for 4.x 😅

[LexManos]

I'm more then happy to use a plugin for this, just trying to figure out one that works. The linked one does not for my use cause because it requires the client to support 30x redirects. Which unfortunately a lot of implementations do not. (yes I know it's a standard that has existed forever, but thus is legacy clients). Currently going down the path of trying to get javlin to be able to reprocess a quest with a modified path. Not having much luck.

Also, is there some plugin repo? I only see the ones listed on your site made by you. Or is it something to just search github for?

[UpcraftLP]

How about running reposilite behind a reverse proxy instead? (nginx, traefik, caddy.. to name a few)

That way you could simply apply some URL rewrite rules at the proxy level and have them be opaque to the connecting client.

[LexManos]

That is a possibility, everything is already passing through traefik for one of the servers I manage. But it would be nice to have a simpler/light weight solution for the other servers where a reverse proxy isn't already setup. It's a useful feature of 2.x, that was nuked. Just seems weird to setup a whole other docker/server for what in effect ends up being just editing two strings in the handler.

Plus doing it on Reposilite's end allows for one place integration with adding/removing repositories.
Meaning, to add a Repo using a reverse proxy I would need to add the repo in reposilite, then edit the proxy config to add the repo name to the whitelist of non-rewrites and reboot the proxy.

[dzikoysk]

I think that the proper solution should be covered on a plugin api level. I also don't want to make this feature a part of Reposilite's config, because it'd kinda open this whole topic once again.

Probably the easiest way to expose this function could be achieved by moving current endpoint implementation to some sort of a factory, so you'd just simply do something like registerRoute("/*", factory.createMavenEndpoint("releases")).

I'm quite tired right now, so I'll take a look at it a little bit later - hopefully tomorrow.

[LexManos]

Alright, I was able to find a solution that is functional. I'm not the biggest fan of it because it has hardcoded a few magic strings {to filter out frontend files}, and i cant get config to work correct, so if you have any ideas for a better solution let me know.

[dzikoysk]

Touching underlying servlet api and frontend specific files might not be the best way to go, I'm not sure if it'd work properly with further request processing 🤔 Instead, I'd try to iterate over root directories in redirected repo and just register a new route for each one of them:

reposilite/reposilite-backend/src/main/kotlin/com/reposilite/redirect/RedirectPlugin.kt

Lines 18 to 66 in a5b206b

	@Plugin(name = "redirect", dependencies = ["local-configuration", "frontend", "maven"])
	class RedirectPlugin : ReposilitePlugin() {
	private val redirectTo: String? = System.getProperty("reposilite.redirect.default-repository", "")
	override fun initialize(): Facade? {
	if (redirectTo.isNullOrEmpty()) {
	return null
	}
	val mavenFacade = facade<MavenFacade>()
	val mavenEndpoints = MavenEndpoints(
	mavenFacade = mavenFacade,
	frontendFacade = facade<FrontendFacade>(),
	compressionStrategy = facade<LocalConfiguration>().compressionStrategy.get()
	)
	logger.info("")
	logger.info("--- Redirect")
	val redirectedRoutes = mavenFacade.getRepository(redirectTo)
	?.storageProvider
	?.getFiles(Location.of("/"))
	?.orNull()
	?.map {
	logger.info("Redirecting /${it.getSimpleName()}/<gav> to /$redirectTo/${it.getSimpleName()}/<gav>")
	ReposiliteRoute<Unit>("/${it.getSimpleName()}/<gav>", HEAD, GET) {
	accessed {
	mavenEndpoints.findFile(
	ctx = ctx,
	identifier = this?.identifier,
	repository = redirectTo,
	gav = it.resolve(requireParameter("gav"))
	)
	}
	}
	}
	?: emptyList()
	event { event: RoutingSetupEvent ->
	event.register(redirectedRoutes)
	}
	return null
	}
	}

~ #2077

We could hide this behind reposilite.redirect.default-repository property, like we do for some other hidden functionalities, so we don't have to expose this in the public config. Thanks to that, such implementation could be probably just merged to the core sources.

[dzikoysk]

Released with 3.5.9:

• https://github.com/dzikoysk/reposilite/releases/tag/3.5.9

I didn't have time to test it tho, so let me know in case of any issues.

[dzikoysk]

I noticed that we're missing docs for the redirect plugin, so let's keep it open until I find time to cover that in the guide.

[LexManos]

Saw this popup in my github notifications again.
Does this provide a solution for the following situation:

Single releases repo which proxies maven central.
Single file in this repo:
/releases/group/name/version/name-version.jar

Hosted at: https://maven.my.site/
Request https://maven.my.site/group/name/version/name-version.jar serves /releases/group/name/version/name-version.jar
Request https://maven.my.site/ant/ant-jai/1.6.5/ant-jai-1.6.5.jar serves https://repo1.maven.org/maven2/ant/ant-jai/1.6.5/ant-jai-1.6.5.jar

Note the lack of 'ant' directory in the /releases/ root folder. I don't think your solution would work in this case as it explicitly enumerates all top level directories.

This is my main concern/smallest use case.

---

Below is a overarching "I would like a system like this" explination.
Ideally, for my use case (12 years of minecraft modding) I would have a setup like this.
Note: Not in your config setup just a json representation of the basic concept.

{  
  "base": "https://maven.my.site/",
  "mainRepo": "releases",
  "repos": [
    {
      "name": "releases",
      "quota": "0b",
      "proxies": [{
          "target": "team",
          "filter": "net.team"
        }, {
          "target": "mark",
          "filter": "org.mark"
        }, {
          "target": "sally",
          "filter": "org.sally"
        }, {
          "target": "wayde",
          "filter": "com.wayde"
        }, {
          "target": "central"
        }
      ]
    }, {
      "name": "team"
      "quota": "10GB"
    }, {
      "name": "mark"
      "quota": "1GB"
    }, {
      "name": "sally",
      "quota": "50MB"
    }, {
      "name": "wayde",
      "quota": "5GB",
      "proxies": [
        {
          "target": "https://maven.wayde.com/",
          "filter": "com.wayde",
          "cache": true
        }
      ]
    }, {
      "name": "central",
      "quota": "500GB",
      "proxies": [
        {
          "target": "https://repo1.maven.org/maven2/",
          "cache": true
        }
      ]
    }
  ],
  "users": [
    {
      "name": "mark",
      "permissions": {
        "/mark/org/mark/": "rw",
        "/team/net/team/": "rw"
      }
    }, {
      "name": "sally",
      "permissions": {
        "/sally/org/sally/": "rw",
        "/team/net/team/": "rw"
      }
    }
  ]
}

the following files on disc:

/team/net/team/team-project/1.0/team-project-1.0.jar
/mark/org/mark/mark-project/1.0/mark-project-1.0.jar
/sally/org/sally/sally-project/1.0/sally-project-1.0.jar

The following files exist remotely:

https://maven.wayde.com/com/wayde/cool-tool/1.0/cool-tool-1.0.jar
https://repo1.maven.org/maven2/com/google/guava/guava/10.0/guava-10.0.jar

All of this data should be accessible via the https://maven.my.site/address

Success Default:
  https://maven.my.site/net/team/team-project/1.0/team-project-1.0.jar
  https://maven.my.site/org/mark/mark-project/1.0/mark-project-1.0.jar
  https://maven.my.site/org/sally/sally-project/1.0/sally-project-1.0.jar
  https://maven.my.site/com/wayde/cool-tool/1.0/cool-tool-1.0.jar
  https://maven.my.site/com/google/guava/guava/10.0/guava-10.0.jar
  
Success Explicit:
  https://maven.my.site/team/net/team/team-project/1.0/team-project-1.0.jar
  https://maven.my.site/mark/org/mark/mark-project/1.0/mark-project-1.0.jar
  
Failure Explicit:
  https://maven.my.site/mark/org/sally/sally-project/1.0/sally-project-1.0.jar

Ideally users would be able to publish to the root directory as well, but this is far less important as configuring dev's is far easier then users.

PUT --user mark https://maven.my.site/org/mark/mark-project/2.0/mark-project-2.0.jar
  -> /mark/org/mark/mark-project/2.0/mark-project-2.0.jar
PUT --user mark https://maven.my.site/net/team/team-project/2.0/team-project-2.0.jar
  -> /team/net/team/team-project/2.0/team-project-2.0.jar
PUT --user mark https://maven.my.site/team/net/team/team-project/2.0/team-project-2.0.jar
  -> /team/net/team/team-project/2.0/team-project-2.0.jar
PUT --user mark https://maven.my/site/org/sally/sally-project/2.0/sally-project-2.0.jar
  -> UNAUTHORIZED

Overall this boils down to somthing like this:

  name, path = request.path.split('/', 1)
  repo = repos.get(name)
  if (repo == null) {
    repo = defaultRepo
    path = name + '/' + path
  }
  
  for (proxy : proxies) {
    if (proxy.filterMatches(path)) {
      repo = proxy.repo
      break
    }
  }
  
  repo.process(path, request)

Yes there are issues if there is a folders in a proxied repo that matches a repo name. Making us not know if they mean the artifact or the repo. But honestly I don't think that's a concern worth caring about.

But with this layout you can see some of the features that I would like to be able to do:
Keeping proxied files from polluting existing repos on disc
Allow users to have access to multiple repos, already doable I think
Allow for quotas for teams or specific users, done by repo based quotas and manual management of access.
Single external access point that does not expose any internal configuration/information of the server.

Anyways, I don't actually expect you to implement this especially not any time soon. But this is what I would like to have possible. If it is possible to do this like this I would like know how and preferably have a useable example.

[dzikoysk]

Note the lack of 'ant' directory in the /releases/ root folder. I don't think your solution would work in this case as it explicitly enumerates all top level directories.

Yup, it was not supposed to support mirrored content - just a simple redirect for a local content. I don't think it makes sense to it provide a more advanced support at this point, especially that I have a feeling that it could be tackled at the proxy level 🤔

I'll try to think about it during 4.x development phase, maybe we can sneak in something helpful for such use-case.

But with this layout you can see some of the features that I would like to be able to do: [...]

Yeah, I think I get the idea. My main concern is that it introduces complexity at both levels - implementation & UX. By design, Reposilite also tries to provide the simplest possible solution that is flexible enough for most of the people - that's why we don't even have user accounts in the first place. Anyway, some significant stuff will change in 4.x, so thanks for the extra input - I'll try to think about that too.

[LexManos]

Yup, it was not supposed to support mirrored content - just a simple redirect for a local content. I don't think it makes sense to it provide a more advanced support at this point, especially that I have a feeling that it could be tackled at the proxy level 🤔

How would it be addressed at the proxy level? Or do you mean at the reverse proxy on the host side?
Ideally it'd be nice to not have a layer on top of reposilite.

Anyways, fundamentally to fit my currently active usecase all that would be needed is the "make everything point at releases" code I posted in my first post. So that I can update without having to redirect all old users.

I'll try to think about it during 4.x development phase, maybe we can sneak in something helpful for such use-case.

Looking forward to it, currently using the 2.x branch as it's functional and i've modified it to my needs. But would be nice to not use a fork.

Yeah, I think I get the idea. My main concern is that it introduces complexity at both levels - implementation & UX. By design, Reposilite also tries to provide the simplest possible solution that is flexible enough for most of the people - that's why we don't even have user accounts in the first place. Anyway, some significant stuff will change in 4.x, so thanks for the extra input - I'll try to think about that too.

Ya, that whole description was a wish list style, and addressed multiple different feature wants.
I understand that you want Reposilite to be simple, but the current solution for hosting maven repositories is either this, or super complex things like sonatype. So having a little extra features would be nice. If it could be designed in a way that plugins could address it that would be good enough for me.