Make runtime multistage programming and macros play together better

[markehammons]

Right now, using runtime multistage programming in macros is a nightmare because the 'Type' typeclass from macros is very hard to hide. Take for example this prototype Vector that uses macros and MSP:

trait V2[T]:
  type X = T
  def add(v1: ArraySeq[T], v2: ArraySeq[T], idx: Int): T
  def genArray(len: Int): Array[T]
  val arr: ArraySeq[T]
  def sum(oVec: V2[T]): T

object V2: 
  inline def apply[T: Numeric](array: ArraySeq[T]) = ${applyImpl[T]('{array}, '{summon[Numeric[T]]})}

  def applyImpl[T](array: Expr[ArraySeq[T]], num: Expr[Numeric[T]])(using qt: Quotes)(using t: Type[T]) = '{
    new V2[T]{
      def genArray(len: Int) = ${
        array match 
          case '{$x: ArraySeq[Int]} => '{new Array[Int](len)}.asExprOf[Array[T]]
          case _ => ???
      }
    

      def add(v: ArraySeq[T], v2: ArraySeq[T], idx: Int): T = ${ 
        ('{v(idx)}, '{v2(idx)}) match
          case ('{$x: Int}, '{$y: Int}) => '{$x + $y}.asExprOf[T]
          case (x,y) => '{$num.plus($x, $y)}
      }
    

      given Compiler = Compiler.make(this.getClass.getClassLoader)
      val arr = $array

      val adder = run(V2RT.genSumX[V2RT.Hider[T]]($array.size))



      def sum(oVec: V2[T]) = V2RT.Hider.unhide(adder(V2RT.Hider(this),V2RT.Hider(oVec)))
    }
  }


object V2RT: 
    def genSumX[T](arity: Int)(using qt: Quotes, t: Type[T]) = 
        '{

            (v1: V2[T], v2: V2[T]) => v1.add(v1.arr, v2.arr, 0)
        }

    opaque type Hider[T] = T

    object Hider:
        def apply[T](v: V2[T]): V2[Hider[T]] = v
        def unhide[T](h: Hider[T]): T = h

The opaque type Hider here is used to hide the compile time Type[T] from genSumX, which causes a compile time error complaining about the compile time Type[T] being used in too low of a scope.

@smarter @nicolasstucki

[smarter]

Can you describe the problem in more details? Is it that there are two classes called Type? If so, can't you rely on their fully-qualified name to distinguish them?

[markehammons]

The phase consistency principle prevents us from using certain values at certain levels of quotes. Type[T] is created at level 0 at compile time in this code. genSumX tries to summon a Type[T] for the T it's given too. The issue is that the compiler sees this, and tries to use the level 0 Type[T] for genSumX, which violates the phase consistency principle. However, since genSumX is meant to generate and compile an expression at compile time, this should not be the case, as it can (and should) summon a Type[T] of its own at runtime.

This is entirely a problem of the compiler seeing genSumX needs a Type[T] and trying to use the compile time version instead of stepping back and letting the runtime quotes generate the necessary information. I show this by making genSumX ask for Type[Hider[T]]. The compiler stops trying to use the compile time Type[T], and the runtime still knows how to summon the right type.

[markehammons]

The compiler error may make the issue clearer.

the code println(withQuotes(V2RT.genSumX[T]($array.size).show))
causes the following error:

[error] 41 |      println(withQuotes(V2RT.genSumX[T]($array.size).show))
[error]    |                                                     ^
[error]    |                         access to parameter t from wrong staging level:
[error]    |                          - the definition is at level 0,
[error]    |                          - but the access is at level 1.

Using Hider[T] fixes the issue.

[nicolasstucki]

Minimized to

import scala.quoted.*

def applyImpl[T](using qt: Quotes)(using t: Type[T]) = '{
  given Quotes = ???
  getType[T](using Type.of[T])
}

def getType[T](using Type[T])  = ???

The fix is

-  getType[T]
+  getType[T](using Type.of[T])

Not sure if we can automatically transform a reference to t in level 1 or more to a Type.of[T].