BUG: Weird behavior on complex Nvidia docs

[KastanDay]

1. Created non-existent links
2. Took a very long time (10s of minutes)

https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html

[jkmin3]

I believe this is happening because there is just so much content on a single page that it gets stuck on the split and upload part of the process.

[KastanDay]

@jkmin3 Why would it get stuck on the split and upload? We've uploaded entire books no problem... has to be something more??

[KastanDay]

Related to #68

[KastanDay]

Josh says "I believe this is happening because there is just so much content on a single page that it gets stuck on the split and upload part of the process."

[jkmin3]

So in line 737:

asyncio.run(oai.process_api_requests_from_file())

This line is what's causing the long process, not too sure what this line does exactly

[KastanDay]

Thanks for noticing this!! i’ll investigate tomorrow