improve placing algorithm

[trinity-1686a]

Description

fix #5010
improve the job placing algorithm, to increase the chance of hitting the searcher with the highest affinity.
Also add a primitive cost evaluation of splits, based only on the count of documentation.

How was this PR tested?

updated tests
added tests with many splits
tested manually with #4998 applied to verify the number of jobs run with affinity=0 improved (it did)

i had to modify some things to keep some root_search tests working, as now it is possible that searching 2 splits with 2 searchers ends up querying only one searcher.
(with N the job cost, assuming all jobs equals,target_load = ⌈2 * N * (1+ ε) / 2⌉ = N+1, after the 1st split, load is only N so the 2nd split can use the same searcher).
I've documented multiple fixes, and implemented the one that seemed most sensible to me

[fulmicoton]

I think .iter_mut().find(||) would be clearer than this loop with break?

          let chosen_node: &mut CandidateNodes =
                if let Some(candidate) = candidate_nodes.iter_mut()
                    .find(|candidate| candidate.load < target_load) {
                   candidate
                } else {
                    &mut candidate_nodes[0]
                };

[fulmicoton]

see comments

[github-actions]

On SSD:

Average search latency is 1.0x that of the reference (lower is better).
Ref run id: 1988, ref commit: 2bf586e
Link

On GCS:

Average search latency is 0.923x that of the reference (lower is better).
Ref run id: 1989, ref commit: 2bf586e
Link