[API server] hornor cgroup cpu limit when set worker numbers

[aylei]

close #4847

Tested (run the relevant ones):

On my laptop:

$ sky api stop && sky api start && grep "Starting SkyPilot API serve" ~/.sky/api_server/server.log
...
INFO:__main__:Starting SkyPilot API server, workers=1

$ sky api stop && sky api start --deploy && grep "Starting SkyPilot API serve" ~/.sky/api_server/server.log
SkyPilot API server stopped.
...
INFO:__main__:Starting SkyPilot API server, workers=12

On K8S (cpu limit to 4 cores):

INFO:__main__:Starting SkyPilot API server, workers=4

• Code formatting: bash format.sh
• Manual test, see above
• Unit test coverage
• All smoke tests: pytest tests/test_smoke.py
• Relevant individual smoke tests: pytest tests/test_smoke.py::test_fill_in_the_name
• Backward compatibility tests: conda deactivate; bash -i tests/backward_compatibility_tests.sh

[Michaelvll]

Thanks @aylei! LGTM with some minor nits.

[Michaelvll]

This might be a bit too detailed for a user to know. Can we hide the num workers in logger.debug or some other logs, not directly printing to the terminal?

[aylei]

IMHO, users of remote API server and sky CLI do not see this log in normal use cases unless they check it explicitly in ~/.sky/api_server/server.log or by kubectl logs (usually for diagnosis). In the meantime, infra operators of a centralized API server do care about the detailed setup. This info would help like the number of Long and Short workers we printed previously for them. Wdyt?

[Michaelvll]

Ahh, that makes sense, as long as it is not shown to the user in console, it is good to me. In that case, let's make the workers here more explicit being uvicorn workers: {num_workers}

[Michaelvll]

Nice! This is great. Let's move this unit_tests to the specific subfolder, i.e. unit_tests/sky/server/test_server.py?

[concretevitamin]

Can we keep directories flat?

[Michaelvll]

It can be confusing with a flat structure, as we have multiple server.py in different directories, e.g. jobs.server.server.py and serve.server.server.py. It seems a best practice to mirror the structure of tests: https://pytest-with-eric.com/pytest-best-practices/pytest-organize-tests/#Best-Practice-2-Test-Structure-Should-Mirror-Application-Code
and some other sources.