Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify autotune.php WEB_CONCURRENCY output a bit more #688

Merged
merged 7 commits into from
Feb 7, 2024

Conversation

dzuelke
Copy link
Contributor

@dzuelke dzuelke commented Feb 6, 2024

For consistency with previous versions of the buildpack, and to reduce potential for confusion (think e.g. number of cores on shared dynos), the CPU core count is now only printed in verbose mode or when there is no ${DYNO} env var.

However, should the log_2 CPU core count based WEB_CONCURRENCY limit kick in, a notice is now always printed.

Also adds -h/--help switches, and different verbosity levels.

Related to #685 and #684.

Fixes #689.

GUS-W-14634717

Do not output CPU core info unless in verbose mode, but do print a notice when the number of processes gets limited to below RAM_AVAIL/memory_limit.

This reduces the potential for confusion (think number of cores on shared dynos), and keeps the printed info consistent with prior versions of the buildpack.
@dzuelke dzuelke requested a review from a team as a code owner February 6, 2024 11:55
@dzuelke
Copy link
Contributor Author

dzuelke commented Feb 6, 2024

Output examples (all with memory_limit = 128M).

1X dyno:

Available RAM is 512M Bytes
PHP memory_limit is 128M Bytes

Container with 512M Bytes RAM and 8 CPU cores:

Available RAM is 512M Bytes
Number of CPU cores is 8
PHP memory_limit is 128M Bytes
Calculated number of workers based on RAM and CPU cores is 32
Maximum number of workers that fit available RAM at memory_limit is 4
Limiting number of workers to 4

Performance-M dyno with --verbose option:

Available RAM is 2560M Bytes
Number of CPU cores is 2
Determining scaling factor based on a memory_limit of 128M
Scaling factor is 5
PHP memory_limit is 128M Bytes
Calculated number of workers based on RAM and CPU cores is 20

Container with 4G Bytes RAM and 2 CPU cores:

Available RAM is 4G Bytes
Number of CPU cores is 2
PHP memory_limit is 128M Bytes
Calculated number of workers based on RAM and CPU cores is 20
Maximum number of workers that fit available RAM at memory_limit is 32
Limiting number of workers to 20

Now prints number of CPU cores and, if applied, a core-based worker limit, if there is no $DYNO env var.
@dzuelke dzuelke force-pushed the log2-web-concurrency branch 2 times, most recently from 6280460 to 332001f Compare February 6, 2024 15:13
For memory limits that do not result in an integer quotient when divided by CALC_BASE
Print calculation result in simple verbose mode, and add RAM/memory_limit result when printing limit notice
Copy link
Member

@edmorley edmorley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI is failing

@dzuelke dzuelke requested a review from edmorley February 7, 2024 20:38
@dzuelke dzuelke merged commit a61d2a9 into main Feb 7, 2024
4 checks passed
@dzuelke dzuelke deleted the log2-web-concurrency branch February 7, 2024 21:25
@heroku-linguist heroku-linguist bot mentioned this pull request Feb 9, 2024
@jwage
Copy link

jwage commented Feb 10, 2024

Suddenly I am getting this:

2024-02-10T02:08:34.086250+00:00 app[web.2]: DOCUMENT_ROOT changed to 'public/'
2024-02-10T02:08:34.144353+00:00 app[web.2]: Using Nginx server-level configuration include 'nginx.conf'
2024-02-10T02:08:34.239393+00:00 app[web.2]: $WEB_CONCURRENCY env var is set, skipping automatic calculation
2024-02-10T02:08:34.254244+00:00 app[web.2]: Starting php-fpm with 4 workers...
2024-02-10T02:08:34.367235+00:00 app[web.2]: Starting nginx...

It is on a private-s dyno type with 1024MB of memory.

My public/.user.ini looks like this:

memory_limit = 32M
max_execution_time = 20

I used to get 32 workers.

I don't have a $WEB_CONCURRENCY environment variable set anywhere in my settings or application.

@edmorley
Copy link
Member

edmorley commented Feb 10, 2024

@jwage Hi! Trying a new app now (which doesn't have WEB_CONCURRENCY set), I don't get that output, but instead:

2024-02-10T10:14:36.346843+00:00 app[web.1]: DOCUMENT_ROOT changed to 'web/'
2024-02-10T10:14:36.521889+00:00 app[web.1]: Available RAM is 512M Bytes
2024-02-10T10:14:36.521911+00:00 app[web.1]: PHP memory_limit is 128M Bytes
2024-02-10T10:14:36.545437+00:00 app[web.1]: Starting php-fpm with 4 workers...

Is it possible you are using another buildpack alongside the PHP buildpack, which is running after the PHP buildpack, and so setting it's own WEB_CONCURRENCY value?

If so, then you need to make sure the PHP buildpack runs as the last buildpack (presuming it's the primary language for your app), as mentioned on:
https://devcenter.heroku.com/articles/using-multiple-buildpacks-for-an-app#adding-a-buildpack

The buildpack for the primary language of your app should always be the last buildpack in the list. This ensures that defaults for that primary language are applied instead of those for another language, and allows Heroku to correctly detect the primary language of your app.

@jwage
Copy link

jwage commented Feb 10, 2024

That is it. I need the PHP buildpack to be first for another reason. I suppose I will set the env variable manually to work around this.

I reordered node to be after PHP because the composer install for PHP makes something available that is necessary for the node build to succeed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

PHP-FPM Worker Process Configuration Error, Fractional Worker Count Issue
3 participants