php-spx support

[ihor-sviziev]

It would be nice to see the support of https://github.com/NoiseByNorthwest/php-spx in Warden for profiling web requests.
It might be some built-in feature or just an instruction.
Maybe someone is already using it?

[ihor-sviziev]

Here are the instructions that I did:

Installing php-spx extension

1. Install the php-spx extension:

# opening warden shell in our project
warden shell
# logging in as a root user
sudo su -
# installing all needed dependencies to build php-spx
yum install php-devel
# going to temp diretory
cd /tmp/
# installing php-spx extensino
git clone https://github.com/NoiseByNorthwest/php-spx.git
cd php-spx/
phpize
./configure
make
sudo make install
# enabling php-spx php extension & configuring it
cat - <<EOF > /etc/php.d/99-spx.ini
extension=spx.so
spx.debug=1
spx.http_enabled=1
spx.http_ip_whitelist="*"
spx.http_key="dev"
spx.http_trusted_proxies=REMOTE_ADDR
EOF

After running all these commands, we should log out from the root user and from the warden shell (2 times type exit).

2. Restart the php-fpm container:

warden env restart php-fpm

Great! We did it! Let's check if it works fine:

3. Verity it the php-spx is installed
   Open our warden shell and run php -m | grep SPX. You should see in the output "SPX". If you don't - pls check if you did everything according to instructions.

Opening control panel

https://mydomain.test/?SPX_KEY=dev&SPX_UI_URI=/ (replace mydomain.test to your project domain)

Usage

Basic usage is shown there:
https://github.com/NoiseByNorthwest/php-spx#basic-usage

CLI usage example

As an example, you can run the following command inside your warden shell:

SPX_ENABLED=1 php bin/magento cache:flush full_page

The result will be like this:

If you want to generate a full report (that will be available through web UI), you should also add SPX_REPORT=full, e.g.

SPX_ENABLED=1 SPX_REPORT=full php bin/magento cache:flush full_page

Browser usage example

For generating report from the browser, you should:

1. go to "Control Pane" (see the "Opening control panel" block above)
2. Turn on "Enabled" and "Automatic start" checkboxes (this will add to this browser needed cookies for turning on profiling)
3. Go to your website and do needed processes

Opening a full report

Since we have a Varnish, which caches all requests to SPX (for retrieving list of profiles), we have 2 options:

Flush varnish cache & view the list in the SPX control panel

1. Flush varnish cache using this command:

warden env exec -T varnish varnishadm 'ban req.url ~ .' 

2. Open the control panel and see the list of available profiles at the bottom in the grid:
   https://mydomain.test/?SPX_KEY=dev&SPX_UI_URI=/ (replace mydomain.test to your project domain)

Manually retrieve the list of profiles

1. in the warden shell you can find the list of generated reports by this command:

ls -l /tmp/spx/

Example output:

www-data@myproject-php-fpm:/var/www/html$ ls -l /tmp/spx/
total 51552
-rw-rw-r-- 1 www-data www-data      630 Nov 21 08:21 spx-full-20231121_082153-myproject-php-fpm-92-497582204.json
-rw-r--r-- 1 www-data www-data   563250 Nov 21 08:21 spx-full-20231121_082153-myproject-php-fpm-92-497582204.txt.gz
-rw-rw-r-- 1 www-data www-data      672 Nov 21 08:21 spx-full-20231121_082153-myproject-php-fpm-93-61385113.json
-rw-r--r-- 1 www-data www-data   520714 Nov 21 08:21 spx-full-20231121_082153-myproject-php-fpm-93-61385113.txt.gz

Each profile will have a .json file and a .txt.gz file.

2. To open one of these reports, you should open in browser:
   https://mydomain.test/?SPX_UI_URI=/report.html&key=spx-full-20231121_082153-myproject-php-fpm-92-497582204 (replace mydomain.test to your project domain and spx-full-20231121_082153-myproject-php-fpm-92-497582204 to your profile name w/o
   file extension)

[ihor-sviziev]

@navarr what do you think about providing an image with php-spx similar to backfire?
Also, would be great if someone can convert it to a separate page on https://warden.dev/ website

[ihor-sviziev]

@navarr what do you think about this idea?

[navarr]

I'm not opposed! I don't have the capacity to implement myself right now, however.

Regarding VARNISH, we don't have to worry about that. We can easily configure NGINX to bypass varnish if an SPX parameter is specified.

[dudzio12]

Great job, nice tool 👍 I would love to see this as part of the base warden :) For now you can always add your own images in project's Dockerfile, which should be better option, since every time you rebuild your image, you will loose extension.

This is example of installing SPX on 8.2 fpm-debug and fpm images:

.warden/php-fpm/Dockerfile

ARG IMG_PREFIX="magento2"
ARG PHP_VERSION="8.2"

FROM docker.io/wardenenv/php-fpm:${PHP_VERSION}-${IMG_PREFIX}
USER www-data

RUN sudo yum install -y php-devel php-pear

RUN cd /tmp/ \
    && git clone https://github.com/NoiseByNorthwest/php-spx.git \
    && cd php-spx/ \
    && phpize \
    && ./configure \
    && make \
    && sudo make install

RUN echo -e 'extension=spx.so\n\
spx.debug=1\n\
spx.http_enabled=1\n\
spx.http_ip_whitelist="*"\n\
spx.http_key="dev"\n\
spx.http_trusted_proxies=REMOTE_ADDR' >> /tmp/99-spx.ini


RUN sudo mv /tmp/99-spx.ini /etc/php.d/99-spx.ini

.warden/php-debug/Dockerfile

ARG IMG_PREFIX="magento2-xdebug3"
ARG PHP_VERSION="8.2"

FROM docker.io/wardenenv/php-fpm:${PHP_VERSION}-${IMG_PREFIX}
USER www-data

RUN sudo yum install -y php-devel php-pear

RUN cd /tmp/ \
    && git clone https://github.com/NoiseByNorthwest/php-spx.git \
    && cd php-spx/ \
    && phpize \
    && ./configure \
    && make \
    && sudo make install

RUN echo -e 'extension=spx.so\n\
spx.debug=1\n\
spx.http_enabled=1\n\
spx.http_ip_whitelist="*"\n\
spx.http_key="dev"\n\
spx.http_trusted_proxies=REMOTE_ADDR' >> /tmp/99-spx.ini


RUN sudo mv /tmp/99-spx.ini /etc/php.d/99-spx.ini

.warden/warden-env.yml

version: "3.5"

services:
  php-fpm:
    build:
      context: ./.warden/php-fpm
  php-debug:
    build:
      context: ./.warden/php-debug

Downside of this approach is that after pulling new image version you have to run warden env build, but it seems more friendly than installing it manually on OS.

[bap14]

I've created a PR to build the images and give NGINX and Varnish support for SPX. There's still leg-work to be done on the warden side of things, but this will at least start building the base SPX image.

wardenenv/images#17

[ihor-sviziev]

Thank you so much for that! I just wonder, how can we use it now? :)

[ihor-sviziev]

Here is the answer: wardenenv/images#17 (comment)
PS: there is a pull request that makes this even way simpler #820

[MagePsycho]

Where are we at regarding php-spx support?
I am planning to give it a try

[SamJUK]

+1 for this, been using the following shell script to bootstrap SPX into the current warden environment, as and when needed. Be nice to have a feature flag in the ENV

Essentially Ihor's steps with a added Varnish cache bypass
https://gist.github.com/SamJUK/b3becaf6723acf4208eb5b8d92ef24f4

[bap14]

@navarr Given the current state of PHP images being built, you think this is good to look at merging now?

[monteshot]

@ihor-sviziev @navarr @SamJUK @bap14
Please review this comment #820 (comment)
Looks like it is a huge problem(for now at least for me)