[O11y][MySQL] Update query for performance data stream

[harnish-elastic]

• Enhancement

Proposed commit message

• Modify the metricbeat code to support schemaname.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

• Relates [MySQL] [Bug] Record the schema name additionally to prevent data loss and digest_text identification. integrations#9239

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @harnish-elastic? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b mysql-update-query upstream/mysql-update-query
git merge upstream/main
git push upstream mysql-update-query

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 0e87886

History

• 💚 Build #947 succeeded 465e2c4
• 💚 Build #945 succeeded 444ea6c

cc @harnish-elastic

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 0e87886

History

• 💚 Build #1385 succeeded 465e2c4
• 💚 Build #1383 succeeded 444ea6c

cc @harnish-elastic

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 0e87886

History

• 💚 Build #2222 succeeded 465e2c4
• 💚 Build #2220 succeeded 444ea6c

cc @harnish-elastic

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 0e87886

History

• 💚 Build #2229 succeeded 465e2c4
• 💚 Build #2227 succeeded 444ea6c

cc @harnish-elastic

[elasticmachine]

💔 Build Failed

• Buildkite Build
• Commit: 0e87886

Failed CI Steps

• :linux: Load dynamic x-pack libbeat pipeline

History

• 💔 Build #992 failed 465e2c4
• 💔 Build #990 failed 444ea6c

cc @harnish-elastic

[elasticmachine]

💔 Build Failed

• Buildkite Build
• Commit: 0e87886

Failed CI Steps

• :linux: Load dynamic x-pack packetbeat pipeline

History

• 💔 Build #950 failed 465e2c4
• 💔 Build #948 failed 444ea6c

cc @harnish-elastic

[elasticmachine]

💔 Build Failed

• Buildkite Build
• Commit: 0e87886

Failed CI Steps

• :linux: Load dynamic x-pack metricbeat pipeline

History

• 💔 Build #960 failed 465e2c4
• 💔 Build #958 failed 444ea6c

cc @harnish-elastic

[elasticmachine]

💔 Build Failed

• Buildkite Build
• Commit: 0e87886

Failed CI Steps

• :linux: Unit Tests / family/core-rhel-9
• :windows: Unit Tests / family/core-windows-2016
• :windows: Unit Tests / family/core-windows-2022

History

• 💔 Build #2519 failed 465e2c4
• 💔 Build #2517 failed 444ea6c

cc @harnish-elastic

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2024-05-09T08:54:54.691+0000

• Duration: 106 min 15 sec

Test stats 🧪

Test	Results
Failed	0
Passed	4630
Skipped	905
Total	5535

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[elasticmachine]

💔 Build Failed

• Buildkite Build
• Commit: 0e87886

Failed CI Steps

• :windows:-family/core-windows-2016 Unit Tests
• :windows:-family/core-windows-2022 Unit Tests

History

• 💔 Build #3870 failed 465e2c4
• 💔 Build #3868 failed 444ea6c

cc @harnish-elastic

[elasticmachine]

💔 Build Failed

• Buildkite Build
• Commit: 0e87886

Failed CI Steps

• :python: Python Integration Tests

History

cc @harnish-elastic

[kush-elastic]

@harnish-elastic,
Can you use mage update and then push again?
Some checks are failing.

[harnish-elastic]

@harnish-elastic You can remove the changes in *.bat because I think the issue that we were facing is already addressed by this commit: f87a592

Updated, thanks!

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b mysql-update-query upstream/mysql-update-query
git merge upstream/main
git push upstream mysql-update-query

[muthu-mps]

Do you think that the schema would be more relevant field name here?

[harnish-elastic]

Yes, we can do that. Thanks!

[harnish-elastic]

While doing schemaname to schema we are getting the following error. We can keep schemaname to avoid this error.

{"log.level":"error","@timestamp":"2024-05-31T11:46:34.810+0530","log.logger":"mysql.performance","log.origin":{"function":"[github.com/elastic/beats/v7/metricbeat/module/mysql/query.(*MetricSet).Fetch](http://github.com/elastic/beats/v7/metricbeat/module/mysql/query.%28*MetricSet%29.Fetch)","[file.name](http://file.name/)":"query/query.go","file.line":112},"message":"error doing query {events_statements SELECT schema_name AS schema, digest_text, count_star, avg_timer_wait, max_timer_wait, last_seen /*!80001 ,quantile_95 / \n  FROM performance_schema.events_statements_summary_by_digest \n  ORDER BY avg_timer_wait DESC \n  LIMIT 10;\n table %!s(bool=true)}%!(EXTRA *mysql.MySQLError=Error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'schema, digest_text, count_star, avg_timer_wait, max_timer_wait, last_seen /!80' at line 1)","[service.name](http://service.name/)":"metricbeat","ecs.version":"1.6.0"}

[shmsr]

Which error? I think you missed pasting it?

[harnish-elastic]

Mb, Updated

[dliappis]

Can we address the failure for check-metricbeat by running make update, as suggested in the output?

When I run it locally I get:

$ make update

$ git diff
<snipped output with changes to two files, see below>

$ git status
On branch pr/38363
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   metricbeat/docs/fields.asciidoc
        modified:   metricbeat/module/mysql/fields.go

[dliappis]

Thanks for addressing #38363 (review)

LGTM from my side/CI PoV as well.

[harnish-elastic]

Thanks for addressing #38363 (review)

LGTM from my side/CI PoV as well.

Thank you! 🙌

[niraj-elastic]

LGTM

[milan-elastic]

Looks good!

[muthu-mps]

Looks good!