WEB_CONCURRENCY auto-tuning: PHP memory_limit improvements, streamlined output, verbose mode

[dzuelke]

• some smaller deprecation warning and invalid data handling fixes
• improve detection of memory_limit - now correctly applies it exactly as PHP-FPM does, even with nested config includes and globbing, and following PHP-FPM's priorities (also see FPM: php_value and php_admin_value entries are applied in reverse order [.phpt test included] php/php-src#13249)
• streamlined, more consistent output
• verbose mode support for additional info

GUS-W-14901875
GUS-W-14901894

[dzuelke]

For easier review, here is the output of diff -u <(git diff main..3866053e bin/heroku-php-apache2) <(git diff main..3866053e bin/heroku-php-nginx) ("diff of diffs" for the two files), to show that they contain identical changes except some comments and variable names:

--- /dev/fd/11	2024-01-31 09:45:11.557373911 +0100
+++ /dev/fd/13	2024-01-31 09:45:11.557404452 +0100
@@ -1,13 +1,13 @@
-diff --git a/bin/heroku-php-apache2 b/bin/heroku-php-apache2
-index 98b9c5ac..70bd4fde 100755
---- a/bin/heroku-php-apache2
-+++ b/bin/heroku-php-apache2
-@@ -366,13 +366,34 @@ if [[ -n ${httpd_config:-} || ( ${httpd_config:=$(findconfig "$httpd_version" "$
- fi
- httpd_config=$(php_passthrough "$httpd_config")
+diff --git a/bin/heroku-php-nginx b/bin/heroku-php-nginx
+index a9113307..8423fa04 100755
+--- a/bin/heroku-php-nginx
++++ b/bin/heroku-php-nginx
+@@ -366,13 +366,34 @@ nginx_config=$(php_passthrough "$nginx_config")
+ 
+ nginx_main=$(findconfig "$nginx_version" "$bp_dir/conf/nginx/main.conf")
  
 +fpm_pidfile=$(mktemp -t "heroku.php-fpm.pid-$PORT.XXXXXX" -u)
-+httpd_pidfile=$(httpd -t -D DUMP_RUN_CFG 2> /dev/null | sed -n -E 's/PidFile: "(.+)"/\1/p') # get PidFile location
++nginx_pidfile=$(mktemp -t "heroku.nginx.pid-$PORT.XXXXXX" -u)
 +
 +# we need to dump the FPM config to fetch possible values for memory_limit in php_value and php_admin_value pool declarations
 +# parsing this by hand is cumbersome - there might be recursive includes in the config, glob patterns, etc
@@ -21,10 +21,10 @@
 +trap 'trap - EXIT; rm -f "$fpm_config_tmp"' EXIT
 +echo -e "\n[global]\nlog_level = notice" >> "$fpm_config_tmp"
 +
-+# build command string arrays for PHP-FPM and HTTPD
++# build command string arrays for PHP-FPM and Nginx
 +# we're using an array, because we need to correctly preserve quoting, spaces, etc
 +fpm_command=( php-fpm --pid "$fpm_pidfile" --nodaemonize -y "$fpm_config_tmp" ${php_config:+-c "$php_config"} )
-+httpd_command=( httpd -D NO_DETACH -c "Include $httpd_config" )
++nginx_command=( nginx -c "$nginx_main" -g "pid $nginx_pidfile; include $nginx_config;" )
 +
  mlib="/sys/fs/cgroup/memory/memory.limit_in_bytes"
  if [[ -f "$mlib" ]]; then
@@ -76,7 +76,7 @@
  fi
  
 -fpm_conftest=
--httpd_conftest=
+-nginx_conftest=
 -fpm_config_tmp=$fpm_config
 -case $conftest in
 -	[24])
@@ -88,58 +88,58 @@
 -		;;& # resume
 -	1)
 -		fpm_conftest="-t"
--		httpd_conftest="-t"
+-		nginx_conftest="-t"
 -		;;
 -	2)
 -		fpm_conftest="-tt"
--		httpd_conftest="-t"
+-		nginx_conftest="-t"
 -		;;
 -	3)
 -		fpm_conftest="-t"
--		httpd_conftest="-S"
+-		nginx_conftest="-T"
 -		;;
 -	4)
 -		fpm_conftest="-tt"
--		httpd_conftest="-S"
+-		nginx_conftest="-T"
 -		;;
 -esac
 -
 -fpm_pidfile=$(mktemp -t "heroku.php-fpm.pid-$PORT.XXXXXX" -u)
--httpd_pidfile=$(httpd -t -D DUMP_RUN_CFG 2> /dev/null | sed -n -E 's/PidFile: "(.+)"/\1/p') # get PidFile location
+-nginx_pidfile=$(mktemp -t "heroku.nginx.pid-$PORT.XXXXXX" -u)
 -
--# build command string arrays for PHP-FPM and HTTPD
+-# build command string arrays for PHP-FPM and Nginx
 -# we're using an array, because we need to correctly preserve quoting, spaces, etc
 -fpm_command=( php-fpm ${fpm_conftest} --pid "$fpm_pidfile" --nodaemonize -y "$fpm_config_tmp" ${php_config:+-c "$php_config"} )
--httpd_command=( httpd ${httpd_conftest} -D NO_DETACH -c "Include $httpd_config" )
+-nginx_command=( nginx ${nginx_conftest} -c "$nginx_main" -g "pid $nginx_pidfile; include $nginx_config;" )
 -
  if [[ $conftest ]]; then
 -	echo -e "\n$(basename "$0"): Config testing php-fpm using ${fpm_command[@]}:" >&2
 -	"${fpm_command[@]}"
--	echo -e "\n$(basename "$0"): Config testing httpd using ${httpd_command[@]}:" >&2
--	"${httpd_command[@]}"
+-	echo -e "\n$(basename "$0"): Config testing nginx using ${nginx_command[@]}:" >&2
+-	"${nginx_command[@]}"
 +	case $conftest in
 +		1)
 +			fpm_conftest="-t"
-+			httpd_conftest="-t"
++			nginx_conftest="-t"
 +			;;
 +		2)
 +			fpm_conftest="-tt"
-+			httpd_conftest="-t"
++			nginx_conftest="-t"
 +			;;
 +		3)
 +			fpm_conftest="-t"
-+			httpd_conftest="-S"
++			nginx_conftest="-T"
 +			;;
 +		*)
 +			fpm_conftest="-tt"
-+			httpd_conftest="-S"
++			nginx_conftest="-T"
 +			;;
 +	esac
 +
 +	echo -e "\n$(basename "$0"): Config testing php-fpm using ${fpm_command[@]} ${fpm_conftest}:" >&2
 +	"${fpm_command[@]}" ${fpm_conftest}
-+	echo -e "\n$(basename "$0"): Config testing httpd using ${httpd_command[@]} ${httpd_conftest}:" >&2
-+	"${httpd_command[@]}" ${httpd_conftest}
++	echo -e "\n$(basename "$0"): Config testing nginx using ${nginx_command[@]} ${nginx_conftest}:" >&2
++	"${nginx_command[@]}" ${nginx_conftest}
  	echo -e "\n$(basename "$0"): All configs okay." >&2
  	exit 0
  fi

[dzuelke]

For reference, outputs now:

Regular:

Using PHP-FPM configuration include 'fpm_custom.conf'
Available RAM is 512M Bytes
PHP memory_limit is 24M Bytes
Starting php-fpm with 21 workers...
Starting httpd...

Verbose:

Using PHP-FPM configuration include 'fpm_custom.conf'
Using PHP-FPM configuration file 'vendor/heroku/heroku-buildpack-php/conf/php/8/php-fpm.conf'
Using PHP configuration (php.ini) file '.heroku/php/etc/php/php.ini'
Using Apache2 VirtualHost-level configuration include 'vendor/heroku/heroku-buildpack-php/conf/apache2/default_include.conf'
Using Apache2 configuration file 'vendor/heroku/heroku-buildpack-php/conf/apache2/heroku.conf'
Reading available RAM from '/sys/fs/cgroup/memory/memory.limit_in_bytes'
Calculating WEB_CONCURRENCY...
Available RAM is 512M Bytes
memory_limit changed by /app/.user.ini
memory_limit overridden by php_admin_value in PHP-FPM configuration
PHP memory_limit is 24M Bytes
WEB_CONCURRENCY=21 (RAM / memory_limit)
Starting log redirection...
Starting php-fpm with 21 workers...
Waiting for php-fpm to finish initializing...
Starting httpd...
Application ready for connections on port 40162.