Add direct database insertion option for fast visit generation

[bx80]

Description:

This PR adds a new console command to the visitor generator which will directly insert fake visits into the database without using the tracker API.

This is intended for quickly creating large datasets where speed of data generation is more important than a perfect simulation of tracking requests.

Usage:

./console visitorgenerator:generate-visits-db [--idsite IDSITE] [--days DAYS] 
[--start-date START-DATE] 
[--limit-visits LIMIT-VISITS] 
[--limit-random-percent LIMIT-RANDOM-PERCENT] 
[--threads THREAD-COUNT]
[--conversion-percent CONVERSION-PERCENT]
[--min-actions MIN-ACTIONS]
[--max-actions MAX-ACTIONS]
[--actions-pool-size ACTIONS-POOL-SIZE]
[--v] [--vv] [--vvv]

Unoptimized performance on basic hardware is currently 350 - 700 visits per second (~2,500 insert/s) compared to ~7 visits per second for visits generated using the normal tracker API method.

Fixes #138

Review

• Functional review done
• Potential edge cases thought about (behavior of the code with strange input, with strange internal state or possible interactions with other Matomo subsystems)
• Usability review done (is anything maybe unclear or think about anything that would cause people to reach out to support)
• Security review done
• Wording review done
• Code review done
• Tests were added if useful/possible
• Reviewed for breaking changes
• Developer changelog updated if needed
• Documentation added if needed
• Existing documentation updated if needed

[bx80]

Added support for multiple threads using the Symphony\Process component, which this PR adds to the plugin composer dependencies. This is only used when the --threads=x option is used and will divide the daily limit by the number of specified threads, for example, if generating data for 3 days with a 1,000 visit limit per day and 4 threads, then each thread would simultaneously run for all three days but with a 250 limit per day. All multi-process input will be captured and displayed by the parent process along with a total summary of generated data.

[michalkleiner]

Nice work! There's somehow more to it than I envisaged 🙈
Had an initial look over the code and left a few questions, will try to run it locally as well.

[AltamashShaikh]

@bx80 Since we are not prefixing the tables, getting below errors

In GenerateVisitsDb.php line 495:
                                                                               
  SQLSTATE[42S02]: Base table or view not found: 1146 Table 'matomo_local.log  
  _action' doesn't exist   

[AltamashShaikh]

@bx80 Left comments on slack thread regarding warnings being received, when there is no goal also when there is 1 goal.

[AltamashShaikh]

@bx80 functional testing now works as expected

./console visitorgenerator:generate-visits-db --idsite 4 --limit-visits=200 --days=3 --threads=8
8 threads requested
Starting threads........ [8 threads were started]
........................
Site Id                             4
Time taken                      3.03s
Visits generated                  192
Visits actions generated          842
Actions generated                 842
Conversions generated               6
Visits per second                  63 / sec
Queries per second                621 / sec

[bx80]

Thanks for the review @AltamashShaikh 👍

If there is no more review feedback, can someone in @matomo-org/plugin-reviewers merge this? (I don't have access).