build.mill files compiled by Scala 3

[bishabosha]

This is work in progress to fix #3152

Numerous fixes were also needed to com-lihaoyi/mainargs, com-lihaoyi/sourcecode, and com-lihaoyi/mill-moduledefs

With the current state, only 1 example/integration tests is still failing:

• integration.feature[plugin-classpath].local

known TODOs:

• Discover macro
• Applicative macro
• Caller macro
• Cross.Factory macro
• EnclosingClass macro
• Task macros
• Cacher macro
• Moduledefs compiler plugin (override inferrence)
• All core Mill modules compile with Scala 3.5.0
• Fix Zinc reporter patch linenumbers of build scripts
• Check that bytecode analyzers work with Scala 3
• cleanup library dependency conflicts
• Support new Scala 3 syntax in build.sc files
• Fix BSP reporter linenumbers for build scripts (Zinc reporter forwards to bsp)
• Cleanup compiler warnings for outdated syntax

known incompatibilities:

• can't use ExplicitResultTypes scalafix rule - need to upgrade mill-scalafix
• Cross.scala uses the new quoted type syntax which scalafmt crashes on, (and version is frozen) so skip the file upgraded Scalafmt so not skipped anymore
• skipping Mima currently due to 1000s of errors (perhaps we should generate filters?)
• filtered one flaky test from example.thirdparty[mockito]
• filtered out integration.feature[plugin-classpath] due to third party plugin dep

[lihaoyi]

Thanks @bishabosha ! Lets get your mainargs changes landed, that way we can cut a release and you can test your WIP in CI

[lefou]

+1 for switching over to using directives!

• Support an alternative to $-imports #3062

[bishabosha]

@lihaoyi see com-lihaoyi/mainargs#148

[bishabosha]

[Edit: after switching to depending on mainargs 0.7.2] in https://github.com/com-lihaoyi/mill/actions/runs/10418081276/job/28853542122?pr=3369 you can see integration tests like example/tasks/6-workers/local passing

[bishabosha]

Rebased and included the new mill-moduledefs 0.11.0-M1

[bishabosha]

@lihaoyi just recording here that the mockito third party test can be flaky (specifically StubbingLookupListenerCallbackTest.add_listeners_concurrently_sanity_check):

https://github.com/mockito/mockito/blob/1e337ed703ce2b78ce9ea40ff8b8c645f8219cf2/src/test/java/org/mockitousage/debugging/StubbingLookupListenerCallbackTest.java#L199-L202

as seen in the CI run https://github.com/com-lihaoyi/mill/actions/runs/10909315276/job/30277476457?pr=3369#step:9:4941

[lihaoyi]

Seems like some of the new ./mill -w 'integration.invalidation[codesig-subfolder].fork.test' mill.integration.CodeSigSubfolderTests.simple tests are failing on Scala 3. Probably just need to adjust some of the codesig heuristics we added in #4116 to be compatible with Scala 3

[lihaoyi]

Seems like some lazy val initialization code in the <clinit> static initializer changes when you rename lazy vals in Scala 3

[lihaoyi]

Seems like the Scala 3.6.2 upgrade is causing new issues, may be worth punting on that for now and just getting onto 3.5.x first

[lihaoyi]

Somehow it is this line that is causing the bootstrap compile to hang

--- a/main/codesig/src/ReachabilityAnalysis.scala
+++ b/main/codesig/src/ReachabilityAnalysis.scala
@@ -91,7 +91,7 @@ class CallGraphAnalysis(
     case None => ujson.Obj()
   }
 
-  logger.mandatoryLog(spanningInvalidationTree)
+  //logger.mandatoryLog(spanningInvalidationTree)
 }
 

[lihaoyi]

Looks like bootstrapping on Scala 3.6.2 works again after optimizing def breadthFirst a bit. Not sure why it wasn't causing problems before, and why it started causing problems now, but optimizing the queue management in the BFS seems to unblock most of the CI jobs