Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Loading a model is going to HuggingFace #1

Open
ShakunBaichoo opened this issue Nov 6, 2024 · 0 comments
Open

Loading a model is going to HuggingFace #1

ShakunBaichoo opened this issue Nov 6, 2024 · 0 comments

Comments

@ShakunBaichoo
Copy link

Hi!
The code to load the GREAT model is going to HuggingFace to download the model. I have set the path as prescribed. I would like to ensure that it loads it locally as our cluster env does not have access to Internet.

Here is the code I am using:
import os
import sys
sys.path.append('tabularfm')

from tabularfm.utils.cli import get_config, create_model, create_model_config
from tabularfm.utils.processing import load_model_weights
from transformers import AutoTokenizer

configs

model_type = 'great'
config_path = 'configs/great.yaml'

data_path = # kaggle path

data_path = 'datasets/gittables_v4' # gittables
finetune_path = 'models/great_gittables'

load and create config

configs = get_config(config_path)
model_config_finetune = create_model_config(data_path, configs, model_type, config_type = "finetune")
model_config_finetune['pretrained_llm'] = os.path.join(finetune_path, 'weights')

load model

great_model = create_model(model_type, model_config_finetune)

Kindly advise.

Thanks,
Shkauntala

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant