Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to resume training #30

Open
hitbuyi opened this issue Feb 28, 2024 · 1 comment
Open

Failed to resume training #30

hitbuyi opened this issue Feb 28, 2024 · 1 comment

Comments

@hitbuyi
Copy link

hitbuyi commented Feb 28, 2024

When I trained the model at the 25th epoch, it halted for mannual stopping, I rewite the code in train_openlane.py

# TODO template config file.
if __name__ == '__main__':
    import warnings
    warnings.filterwarnings("ignore")
    #openlane_cfg_path= os.getcwd()+'/openlane_config.py'
    openlane_cfg_path= os.path.abspath(os.path.dirname(__file__)) + '/openlane_config.py'
    ckpt_path = '/media/hitbuyi/E8E4A23EE4A20F3E/dataset/openlane1.2/model/BEV_LaneDet/ep025.pth'
    #worker_function('./openlane_config.py', gpu_id=[4, 5, 6, 7])
    #worker_function(openlane_cfg_path, gpu_id=[0])
    worker_function(openlane_cfg_path, gpu_id=[0],checkpoint_path = ckpt_path)

thus

    if checkpoint_path:
        if getattr(configs, "load_optimizer", True):
            resume_training(checkpoint_path, model.module, optimizer, scheduler)
        else:
            load_checkpoint(checkpoint_path, model.module, None)

was called, but the code failed, it prompted the error in load_model.py

def load_checkpoint(checkpoint, net, optimizer=None, map_loc="cuda"):
    sd = torch.load(checkpoint, map_location=map_loc)
    net.load_state_dict(sd['models'])
    if optimizer and sd['optimizer']:
        optimizer.load_state_dict(sd['optimizer'])
    return sd

How to solve this issue?

@hitbuyi hitbuyi changed the title Each epoch loads all the dataset Failed to resume training Mar 3, 2024
@shupinghu
Copy link

Hello, I am a new researcher of 3D-BEV-LaneDet, could you please share the source code with me?
My Email [email protected]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants