Error: "Failed: keyNotFound(base: "Linear", key: "weight")" for Qwen LLMs with mlx-swift 0.21.3

[adrgrondin]

I updated the project to use mlx-swift 0.21.3 and it broke the Qwen LLMs. It seems that is started with this commit on mlx-swift with the changes to MLXNN/Module.swift.

Here is the error I'm getting:
Failed: keyNotFound(base: "Linear", key: "weight")

And the model tested:
mlx-community/Qwen1.5-0.5B-Chat-4bit
mlx-community/Qwen2.5-7B-Instruct-4bit
mlx-community/Qwen2.5-1.5B-Instruct-4bit

Not sure if it can be fixed by updating the implementation in this repo or an update to mlx-swift is needed but thought I would create this issue if someone with more knowledge knows what's going on.

Thanks to all the maintainer of this project. It's amazing!

[rudrankriyam]

Mentioned here: ml-explore/mlx-swift#195

For now, I downgraded to 0.21.2 in a fork of mlx-swift-example