Support for Apple MLX backend

[notnotrishi]

Would be nice to have support for language models on Apple MLX

[Harsha-Nori]

Hi @notnotrishi, I'm a bit new to the MLX world. Do most people use the mlx_lm subpackage to interface with MLX based transformer models? https://github.com/ml-explore/mlx-examples/blob/main/llms/README.md

[notnotrishi]

@Harsha-Nori yes, that's my understanding.

also, awni from mlx team is very helpful should your team has any questions

[painpoung]

Hi,
I don't know if someone pick up this but if not I would like to work on it (assuming you have time before it got implemented 😄 ).

The plan if I understand well is to propose something such as

from guidance import models, gen

# load a model (could be Transformers, LlamaCpp, VertexAI, OpenAI...)
mlx_model = models.MlxLM(path) 

# append text or generations to the model
mlx_model + f'Do you want a joke or a poem? ' + gen(stop='.')

And to do so we "just" need to add a _mlx.py in the models sub package (or like transformers with a sub package) using the mlx_lm library to handle the LLM (or the https://github.com/ToluClassics/mlx-transformers mlx-transformers cause it would be easier since I would only have to "copy" the _transformers.py or even just modify the _transformers.py via good check at the right place).

Am I totally off the track or Is it the path to follow?