[ci] fix linkchecker job

[jameslamb]

Fixes #6756

The relative path we pass to its config file needs to be updated (I think I broke this in #6497).

Notes for Reviewers

How I tested this

Manually triggered the link check workflow from this branch:

• first build: https://github.com/microsoft/LightGBM/actions/runs/12335479058
• after config changes from [ci] fix linkchecker job #6757 (review): https://github.com/microsoft/LightGBM/actions/runs/12335867947
• setting threads=10: https://github.com/microsoft/LightGBM/actions/runs/12336243590
• setting threads=4: https://github.com/microsoft/LightGBM/actions/runs/12336463177

[StrikerRUS]

It works now, thanks a lot for quick fix!

As for the

Warning    [http-rate-limited] Rate limited (Retry-After: None)
Result     Valid: 429 Too Many Requests

errors, could you please try decreasing the maxrequestspersecond for one host to 0.1?

LightGBM/docs/.linkcheckerrc

Line 2 in b33a12e

maxrequestspersecond=1

Refer to

• [ci][docs] set limit for max requests per second to one host in linkchecker #3785 (comment)
• https://linkchecker.github.io/linkchecker/man/linkcheckerrc.html#checking

Values less than 1 and at least 0.001 can be used.

It's now possible to use values less than 1.

[jameslamb]

Just pushed 5265786 with that change. I also put a floor on linkchecker pointing at the latest version... to be sure we always get a version that respects values less than 1 and to make installation a little faster.

Thanks for the references and the suggestion! That saved me some effort figuring out how to get around those errors 😊

new test build: https://github.com/microsoft/LightGBM/actions/runs/12335867947

[StrikerRUS]

new test build

In case of successful run, we can try to increase threads up to default 10, I guess.

LightGBM/docs/.linkcheckerrc

Line 6 in a924516

threads=1

I hope that linkchecker ensures that requests to one host are done from a single thread.
linkchecker/linkchecker#193

[jameslamb]

Oh interesting, yeah hopefully that tool divides the hosts across threads instead of just dividing links. Pushed 4061211 setting threads=10, let's see how it goes: https://github.com/microsoft/LightGBM/actions/runs/12336243590

[jameslamb]

It's still taking a very long time with nthreads=10... maybe even slower than with nthreads=1: https://github.com/microsoft/LightGBM/actions/runs/12336243590/job/34428554660

I guess because the GitHub Actions Linux runners only have 4 vCPUS: https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories

nthreads=10 might be oversubscribing them (just like we see with LightGBM and recommend against:

LightGBM/R-package/R/lightgbm.R

Lines 128 to 130 in 1090a93

	#' @param num_threads Number of parallel threads to use. For best speed, this should be set to the number of
	#' physical cores in the CPU - in a typical x86-64 machine, this corresponds to half the
	#' number of maximum threads.

Trying a build limiting to 4 threads: https://github.com/microsoft/LightGBM/actions/runs/12336463177

[StrikerRUS]

Heh, still 48 minutes with 4 threads...

Seems that maxrequestspersecond has higher priority over threads and they don't play well with each other 😞

Anyway, we have successful cron job with less than 1 hour execution time that is run once a day. I believe it's totally fine and we shouldn't spend more time for the speedup!

[jameslamb]

we have successful cron job with less than 1 hour execution time that is run once a day. I believe it's totally fine and we shouldn't spend more time for the speedup!

Yep I agree! I'll merge this.