Spyglass parsers as library

[theogiraudet]

Hi!

I would like to use the different parsers provided by Spyglass in a TypeScript program.
To be precise, I am looking for a way to provide a data pack format and to get parsers to transform the different data pack artifacts into ASTs.
I saw that Spyglass is modular, and I can import parts of it, such as @spyglassmc/java-edition or @spyglassmc/malfunction, but I have difficulties identifying their main endpoints.
If you have a few pointers to give me, it would be very nice!

Thanks!

[misode]

It depends a bit on what exactly you need. If you also want to validate the JSON and NBT then the setup will be a bit more complicated. It will also be a bit different depending on if you're working in nodejs or a browser environment. We don't have docs right now, but this should give you a lead:

The central part is either Service or Project (service is technically a thin wrapper around project, so you could do without it).

import * as core from '@spyglassmc/core'
import { NodeJsExternals } from '@spyglassmc/core/lib/nodejs.js'
import * as mcdoc from '@spyglassmc/mcdoc'
import * as je from '@spyglassmc/java-edition'

const service = new core.Service({
	logger: console,
	profilers: new core.ProfilerFactory(console, [
		'cache#load',
		'cache#save',
		'project#init',
		'project#ready',
		'project#ready#bind',
	]),
	project: {
		cacheRoot: 'file:///cache/',
		projectRoots: ['file:///project/'],
		externals: NodeJsExternals,
		initializers: [mcdoc.initialize, je.initialize],
	},
})
await service.project.ready()

You will likely have to adjust the cache root and project root depending on your needs, also you may not be able to use the je.initialize because it is fairly tightly coupled to the language server right now.

If you're looking on how to do this in the browser, you could take some inspiration from how I am currently building it for my website: https://github.com/misode/misode.github.io/blob/mcdoc/src/app/services/Spyglass.ts#L223

[theogiraudet]

Hi Misode, and thanks for your answer.
I will be more precise about what I want.
In an "ideal world", I would like to parse a whole datapack, get the reference to other files (function call, predicate evaluation, etc), and remove some commands/files according to specific rules.
So having a "giant" AST of the whole datapack.
I think using Spyglass as a library is a good way to realize that since you already have a parser and a formatter/pretty printer.
So yes, I also need to parse JSON/NBT and even Minecraft JSON data such as tags and predicates.

I know about LSP and all this stuff, but I'm usually more familiar with language workbenches such as Langium than handmade implementations, so I'm a bit lost in the code…

So I tried a bit with your starting point and here is the code I currently have, to read one file:

import * as core from '@spyglassmc/core'
import * as mcf from '@spyglassmc/mcfunction'
import * as je from '@spyglassmc/java-edition'
import { NodeJsExternals } from '@spyglassmc/core/lib/nodejs.js'
import path from "node:path"
import process from "node:process"
import { promises as fs } from 'node:fs'

interface Document {
  uri: string
  languageId: string
  version: number
  getText(): string
}

function createDocument(uri: string, content: string): Document {
  return {
    uri,
    languageId: 'mcfunction',
    version: 0,
    getText: () => content
  }
}

async function parseMcfunction(filePath: string) {
  
  const cacheRoot = path.resolve(process.cwd(), '.cache').replaceAll(path.sep, path.posix.sep);
  const projectRoot = process.cwd().replaceAll(path.sep, path.posix.sep);

  const service = new core.Service({
    logger: console,
    project: {
      cacheRoot: `file://${cacheRoot}/`,
      defaultConfig: core.ConfigService.merge(core.VanillaConfig, { env: { dependencies: [] } }),
      externals: NodeJsExternals,
      initializers: [mcf.initialize, je.initialize],
      projectRoots: [`file://${projectRoot}/`],
    },
  })


  await service.project.ready()

  const fileUri = `file://${filePath.replace(/\\/g, '/')}`
  const content = await fs.readFile(filePath, 'utf8')
  
  const document = createDocument(fileUri, content)
  await service.project.onDidOpen(document.uri, document.languageId, document.version, document.getText())
  const result = await service.project.ensureClientManagedChecked(document.uri)

  if (result) {
    const { node } = result
    console.log('Parsed AST:', node, null, 2)
    
    if (node.parserErrors?.length) {
      console.log('Parser errors:', node.parserErrors)
    }
    if (node.checkerErrors?.length) {
      console.log('Checker errors:', node.checkerErrors) 
    }
  }

  await service.project.cacheService.save()
}

const filePath = path.resolve(process.cwd(), process.argv[2] || './test.mcfunction')
parseMcfunction(filePath).catch(console.error)

Is it a "good way" to do that or do you have more "high-level" API/functions to recommend (especially to also manage JSON files)?
I don't need the other services and I only need to parse once.

Thanks!