Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(map): cast null or map() to explicit map type #3847

Merged
merged 1 commit into from
Apr 17, 2024

Conversation

aceforeverd
Copy link
Collaborator

@aceforeverd aceforeverd commented Apr 3, 2024

Enhancements to new insert row builder:

Auto casting to target type when necessary, which usually happens with insert statement while a default schema defined by target table.

With the patch (whenever InsertRowBuilder used), it can

  1. automatically construct safe value for NULL place holder, so insert statement like insert to t1 values (1, null, null, map()) simply work for table t1 (int, timestamp, map<int, string>, map<timestamp, int>)
  2. supports constructing null or empty values from offline engine.

@github-actions github-actions bot added batch-engine openmldb batch(offline) engine execute-engine hybridse sql engine storage-engine openmldb storage engine. nameserver & tablet labels Apr 3, 2024
Copy link
Contributor

github-actions bot commented Apr 3, 2024

SDK Test Report

101 files  ±0  101 suites  ±0   2m 13s ⏱️ ±0s
349 tests ±0  335 ✅ ±0  14 💤 ±0  0 ❌ ±0 
475 runs  ±0  461 ✅ ±0  14 💤 ±0  0 ❌ ±0 

Results for commit 38978b9. ± Comparison against base commit 7f758af.

This pull request removes 30 and adds 9 tests. Note that renamed tests count towards both.
  PARTITION BY db1.t1.col2 ORDER BY db1.t1.col1
  PARTITION BY t1.col2 ORDER BY t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](1)
 ) limit 10;](2)
 ) limit 10;](3)
 FROM db1.t1
 FROM t1
 WINDOW w1 AS (
 last join db2.t2 order by db2.t2.col1
…
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlLastJoinWithMultipleDB[,  SELECT sum(db1.t1.col1) over w1 as sum_t1_col1, db2.t2.str1 as t2_str1
 FROM db1.t1
 last join db2.t2 order by db2.t2.col1
 on db1.t1.col1 = db2.t2.col1 and db1.t1.col2 = db2.t2.col0
 WINDOW w1 AS (
  PARTITION BY db1.t1.col2 ORDER BY db1.t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](2)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlLastJoinWithMultipleDB[db1,  SELECT sum(t1.col1) over w1 as sum_t1_col1, db2.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0
 WINDOW w1 AS (
  PARTITION BY t1.col2 ORDER BY t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](1)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlLastJoinWithMultipleDB[null,  SELECT sum(db1.t1.col1) over w1 as sum_t1_col1, db2.t2.str1 as t2_str1
 FROM db1.t1
 last join db2.t2 order by db2.t2.col1
 on db1.t1.col1 = db2.t2.col1 and db1.t1.col2 = db2.t2.col0
 WINDOW w1 AS (
  PARTITION BY db1.t1.col2 ORDER BY db1.t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](3)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[, SELECT db2.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Fail to transform data provider op: table t1 not exists in database []](4)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[db1, SELECT db1.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Column Not found: db1.t2.str1](2)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[db1, SELECT db2.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Column Not found: .t2.col1](3)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[db1, SELECT t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Column Not found: .t2.str1](1)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[null, SELECT db2.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Fail to transform data provider op: table t1 not exists in database []](5)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlWindowLastJoin[ SELECT sum(t1.col1) over w1 as sum_t1_col1, t2.str1 as t2_str1
 FROM t1
 last join t2 order by t2.col1
 on t1.col1 = t2.col1 and t1.col2 = t2.col0
 WINDOW w1 AS (
  PARTITION BY t1.col2 ORDER BY t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](1)

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented Apr 3, 2024

Linux Test Report

    57 files     244 suites   1h 39m 51s ⏱️
12 642 tests 12 635 ✅ 7 💤 0 ❌
17 924 runs  17 917 ✅ 7 💤 0 ❌

Results for commit 38978b9.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented Apr 3, 2024

HybridSE Mac Test Report

20 132 tests  +8   20 130 ✅ +8   8m 27s ⏱️ -12s
   256 suites ±0        2 💤 ±0 
    68 files   ±0        0 ❌ ±0 

Results for commit 38978b9. ± Comparison against base commit 7f758af.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented Apr 3, 2024

HybridSE Linux Test Report

20 132 tests  +8   20 130 ✅ +8   6m 19s ⏱️ -2s
   256 suites ±0        2 💤 ±0 
    68 files   ±0        0 ❌ ±0 

Results for commit 38978b9. ± Comparison against base commit 7f758af.

♻️ This comment has been updated with latest results.

@aceforeverd aceforeverd force-pushed the feat-map-type-casting branch from 2991987 to c14fa9f Compare April 3, 2024 16:00
Copy link

codecov bot commented Apr 3, 2024

Codecov Report

Attention: Patch coverage is 74.00000% with 52 lines in your changes are missing coverage. Please review.

Project coverage is 74.88%. Comparing base (86d640a) to head (c14fa9f).
Report is 2 commits behind head on main.

Files Patch % Lines
hybridse/src/codec/fe_row_codec.cc 62.26% 20 Missing ⚠️
hybridse/src/codegen/ir_base_builder.cc 16.66% 10 Missing ⚠️
hybridse/src/vm/jit.cc 33.33% 8 Missing ⚠️
hybridse/src/case/sql_case.cc 37.50% 5 Missing ⚠️
hybridse/src/udf/udf.cc 0.00% 4 Missing ⚠️
hybridse/src/codegen/struct_ir_builder.cc 86.36% 3 Missing ⚠️
hybridse/src/codegen/cast_expr_ir_builder.cc 94.73% 1 Missing ⚠️
hybridse/src/node/expr_node.cc 85.71% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #3847      +/-   ##
============================================
+ Coverage     74.87%   74.88%   +0.01%     
  Complexity      658      658              
============================================
  Files           742      742              
  Lines        133925   134056     +131     
  Branches       1387     1387              
============================================
+ Hits         100277   100391     +114     
- Misses        33344    33361      +17     
  Partials        304      304              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@tobegit3hub tobegit3hub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Auto casting to target type when necessary, which usually happens with
insert statement while a default schema defined by target table. This
supports constructing null or empty values from offline engine.
@aceforeverd aceforeverd force-pushed the feat-map-type-casting branch from c14fa9f to 38978b9 Compare April 9, 2024 13:59
@github-actions github-actions bot removed the storage-engine openmldb storage engine. nameserver & tablet label Apr 9, 2024
@aceforeverd aceforeverd merged commit 8934906 into 4paradigm:main Apr 17, 2024
26 of 28 checks passed
@aceforeverd aceforeverd deleted the feat-map-type-casting branch April 17, 2024 10:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
batch-engine openmldb batch(offline) engine execute-engine hybridse sql engine
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants