Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implement regex scan and data generation #11

Merged

Conversation

tender-barbarian
Copy link
Member

@tender-barbarian tender-barbarian commented Nov 20, 2024

Implemented regex scanning and data generation for common values such as: IP, MAC, Email, URL. More common values will be added at a later time.

Also slightly reworked how values are replaced:

  1. Previously replacement happened whenever value which should be anonymized was discovered in a single log line.
  2. Currently values to be replaced are discovered first, added to replacementMap (oldValue:newValue) and only then replacements are executed on log line (according to replacementMap).

Such change allows to keep values unique, rather than replacing iterations of same original value to different new value each time. Uniqueness is kept for single log line only - we can later expand it to whole input data, but we need to consider how keeping potentially massive maps will affect memory.

@tender-barbarian tender-barbarian force-pushed the implement_data_generation_and_regexp_scanning branch 2 times, most recently from 3360f15 to 00764cd Compare November 20, 2024 14:08
@tender-barbarian tender-barbarian force-pushed the implement_data_generation_and_regexp_scanning branch 2 times, most recently from 653af55 to 482c529 Compare November 21, 2024 08:57
@tender-barbarian tender-barbarian force-pushed the implement_data_generation_and_regexp_scanning branch from 482c529 to 3641ebd Compare November 21, 2024 09:59
@tender-barbarian tender-barbarian merged commit 24cf4e5 into main Nov 21, 2024
3 checks passed
@tender-barbarian tender-barbarian deleted the implement_data_generation_and_regexp_scanning branch November 21, 2024 10:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants