Can Not Load a Saved Model Within the Same Environment

[shahiryar]

Hi,
I fine-tuned the following hugging face distilbert model for text classification.

from transformers import TFAutoModelForSequenceClassification

model = TFAutoModelForSequenceClassification.from_pretrained(
    "distilbert-base-uncased", num_labels=2, id2label=id2label, label2id=label2id
)

After fine-tuning the model, it was saved successfully, however, when I tried loading it, it gave me this error as shown in the image below: ValueError: Layer 'pre_classifier' expected 0 variables, but received 2 variables during loading. Expected: []

The Summary of the model looks like this:

Model: "tf_distil_bert_for_sequence_classification"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 distilbert (TFDistilBertMa  multiple                  66362880  
 inLayer)                                                        
                                                                 
 pre_classifier (Dense)      multiple                  590592    
                                                                 
 classifier (Dense)          multiple                  1538      
                                                                 
 dropout_19 (Dropout)        multiple                  0         
                                                                 
=================================================================
Total params: 66955010 (255.41 MB)
Trainable params: 66955010 (255.41 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________

What could have done wrong and how can I fix this issue so can load my model again?

[dugujiujian1999]

Try: #18830
What versions of Keras and TensorFlow were used for saving?

[sachinprasadhs]

@shahiryar , Could you please use Keras 3 and import Keras directly and try it.
Refer this Migration guide for any reference https://keras.io/guides/migrating_to_keras_3/

[github-actions]

This issue is stale because it has been open for 14 days with no activity. It will be closed if no further activity occurs. Thank you.

[shahiryar]

I upgraded my environment to use Keras 3, but that didn't work. It is probably because I am using pretrained model from hugging face which uses keras 2 on the backend.
However, I found a way around it by saving the model in a .h5 file and then using the .load_weights function to load the model.
That works for me.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No