Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

About the long text input in Scholar-XL dataset #1

Open
DavidLeexxxx opened this issue May 27, 2023 · 0 comments
Open

About the long text input in Scholar-XL dataset #1

DavidLeexxxx opened this issue May 27, 2023 · 0 comments

Comments

@DavidLeexxxx
Copy link

Thanks for this excellent work in this specific domain. When I ran the CNN-base method on the Scholar-XL dataset which is a dataset for NER tasks, I came across with the token limitation errors, as we all know that bert can only accept an input token length within 512. I recognize that this error occurred pobaby because some inputs' lengths are larger than 512. This "out of token length" problem has also been mentioned in https://www.aminer.cn/scholar-profiling. However I counldn't find an solution or hyper parameters settings such as "max-length" about this problem in the source code(train_CNN.py). So could you please update your code about this or give an explanation about how to treat this problem in order to get the baseline results mentioned in this repo.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant