Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unsupported json feature in xgrammar #192

Open
gryffindor-rr opened this issue Feb 12, 2025 · 3 comments
Open

Unsupported json feature in xgrammar #192

gryffindor-rr opened this issue Feb 12, 2025 · 3 comments

Comments

@gryffindor-rr
Copy link

I'm also trying to use vLLM to support guided decoding of my request. currently it's default guided backend set to xgrammar (which I thought having better performance). however, in my case, it falls back to 'outlines' because it detected some json feature are not supported by xgrammar. see https://github.com/vllm-project/vllm/blob/e92694b6fe264a85371317295bca6643508034ef/vllm/model_executor/guided_decoding/utils.py#L6

I'm wondering if this limitation still exists? my target json schema is like
{
"type": "object",
"additionalProperties": false,
"properties": {
"POINT": {
"oneOf": [
{
"type": "integer",
"minimum": 0,
"maximum": 500
},
{
"$ref": "#/$defs/Location",
}
]
}
}
"$defs": {
"Location": {
"type": "array",
"items": {
"type": "integer",
"minimum": 0,
"maximum": 1000
},
"minItems": 2,
"maxItems": 2
}
}
}

I'm wondering why xgrammar did not support array and integer min and max range?

@Ubospica
Copy link
Collaborator

Hi @gryffindor-rr, thanks for asking about that. Currently, the min and max integer range is supported (we are enhancing it in #182 to better support negative integer range). The num of items is still ongoing. Please stay tuned!

@Ubospica
Copy link
Collaborator

We will sync with vLLM team to enable more features of json schema with XGrammar backend.

@russellb
Copy link

We will sync with vLLM team to enable more features of json schema with XGrammar backend.

I work on vLLM and try to watch the activity on this repo so I catch updates. Let me know if something is working that we haven't enabled! I'm about to turn regex on since it seems like we should be good in 0.1.12.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants