Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: convert FileMetaData to ParquetMetaData #2980

Merged
merged 7 commits into from
Dec 26, 2023

Conversation

QuenKar
Copy link
Contributor

@QuenKar QuenKar commented Dec 22, 2023

I hereby agree to the terms of the GreptimeDB CLA

What's changed and what's your intention?

part of #2963 , this pr is pre-work.
support to convert from FileMetaData to ParquetMetaData.

Checklist

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.

Refer to a related PR or issue link (optional)

#2963

@QuenKar QuenKar added the docs-not-required This change does not impact docs. label Dec 22, 2023
Copy link

codecov bot commented Dec 22, 2023

Codecov Report

Attention: 18 lines in your changes are missing coverage. Please review.

Comparison is base (6c1c7d8) 85.85% compared to head (dfc76e4) 85.24%.
Report is 25 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #2980      +/-   ##
===========================================
- Coverage    85.85%   85.24%   -0.62%     
===========================================
  Files          775      784       +9     
  Lines       124707   126760    +2053     
===========================================
+ Hits        107070   108053     +983     
- Misses       17637    18707    +1070     

@fengys1996 fengys1996 self-requested a review December 25, 2023 07:59
src/mito2/src/sst/parquet/helper.rs Outdated Show resolved Hide resolved
src/mito2/src/sst/parquet/writer.rs Outdated Show resolved Hide resolved
@QuenKar QuenKar requested a review from evenyag December 26, 2023 03:04
Copy link
Contributor

@evenyag evenyag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly LGTM.

src/mito2/src/sst/parquet/helper.rs Outdated Show resolved Hide resolved
src/mito2/src/sst/parquet/helper.rs Outdated Show resolved Hide resolved
src/mito2/src/sst/parquet/helper.rs Outdated Show resolved Hide resolved
Co-authored-by: Yingwen <[email protected]>
@evenyag evenyag requested a review from fengys1996 December 26, 2023 03:26
Copy link
Contributor

@fengys1996 fengys1996 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@fengys1996 fengys1996 added this pull request to the merge queue Dec 26, 2023
Merged via the queue into GreptimeTeam:develop with commit 95f172e Dec 26, 2023
13 checks passed
MichaelScofield pushed a commit to MichaelScofield/greptimedb that referenced this pull request Dec 26, 2023
* feat: convert `FileMetaData` to `ParquetMetaData` (GreptimeTeam#2980)

* feat: can convert Format FileMetaData to ParquetMetaData

* test: parquet metadata equal

* chore: test

* chore: cr comment

Co-authored-by: fys <[email protected]>

* chore: cr comment

* refactor: type name

* chore: cr comment

Co-authored-by: Yingwen <[email protected]>

---------

Co-authored-by: fys <[email protected]>
Co-authored-by: Yingwen <[email protected]>

* refactor(remote_wal): add StandaloneWalConfig (GreptimeTeam#3002)

* feat: integrate remote wal to standalone

* fix: test

* refactor: refactor standalone wal config

* chore: change default kafka port to 9092

* chore: apply suggestions from CR

---------

Co-authored-by: niebayes <[email protected]>

---------

Co-authored-by: Wei <[email protected]>
Co-authored-by: fys <[email protected]>
Co-authored-by: Yingwen <[email protected]>
Co-authored-by: Weny Xu <[email protected]>
Co-authored-by: niebayes <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs-not-required This change does not impact docs.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants