Brief Traffic Spikes to Fishpig WP API Multiple Times Per Day

[SamJUK]

Hi apologies if this has been fixed in a recent version (we are a few behind in production).

On higher traffic production sites, we have been seeing spikes of high traffic directed at the Fishpig Wordpress endpoints briefly for under < 5 minutes at a time, multiple times per day. To note: We run the Wordpress instances on the sites isolated with low resources, since they are not public facing and only serve the fishpig blog.

The current theory for this is Fishpig cache is being lost due to cache lifetime, purge etc. Then a bunch of uncached requests are made consecutively, all triggering separate API requests due to the cache being empty at the time they came in. Once the first request completes and populates cache, the traffic spike subside. Although we still have a bunch of queued WP Api requests that was initiated before the cache was populated.

Potentially introducing some form of locking for the API requests, so that we don't issue duplicate requests.
Then delay the response of additional requests until the first request finishes and use that cached content. (Potential to speed up the following requests as they do not need to make their own HTTP requests now)
Or soft reject any new requests with a exception log message and disable the Fishpig functionality.

[bentideswell]

Hi Sam, do you have some examples of the endpoints being hit?

Also what FishPig add-on modules do you have installed? This will affect what API requests are triggered.

[SamJUK]

This New Relic Screenshot represents the past 7 days on a 1 hour timeseries for all /wp/index.php requests.

Reason for the increase in just the Hello requests is going to be due to either the request timing out or returning a 499/500 series error due to system load. If I am correct that would cause a exception to be throw skipping over the data request?

As for the Addons the list is:

    "fishpig/magento2-wordpress-integration": "^3.0",
    "fishpig/magento2-wordpress-integration-multisite": "^3.0",
    "fishpig/magento2-wordpress-integration-yoastseo": "^3.0",

[bentideswell]

Hi Sam, thanks for providing the details. I've released verison 3.30.0 and this includes a lock on the API requests and should solve the problem. Give it a test and let me know how you get on.

[SamJUK]

Hi Ben,

Thanks for the quick turn around on this. Looking over the release, that looks like it should resolve the issues!
I'll try to schedule the update after the Bank Holiday Weekend.

Thanks
Sam