Code

Integration Tests and Demo rework#101

Code

Merged

ravi-databricks merged 35 commits intodatabrickslabs:issue_97databrickslabs/dlt-meta:issue_97from dvanderwood:issue_97_98dvanderwood/dlt-meta:issue_97_98Copy head branch name to clipboardOct 14, 2024

+1,641-1,183

Commits

Commits on Oct 1, 2024

• added integration test outputs to gitinore

  dvanderwood

  committed
  7a6b28f

  Copy full SHA for 7a6b28f

• uc catalog name consistency in sample code

  dvanderwood

  committed
  8fc401d

  Copy full SHA for 8fc401d

Commits on Oct 2, 2024

• updated runner notebook upload to .py files from .dbc so updates can be made directly in editor and don't need to be modified in Databricks and downloaded as .dbc files

  dvanderwood

  committed
  04adc91

  Copy full SHA for 04adc91

• lower the source during the arg parse instead of at each time it is used

  dvanderwood

  committed
  4b9bffc

  Copy full SHA for 4b9bffc

• Complete rewrite of the arg parser for the intgegration tests

  dvanderwood

  committed
  5a38c85

  Copy full SHA for 5a38c85

• Update args parsing

  dvanderwood

  committed
  72fe9dc

  Copy full SHA for 72fe9dc

• begin movign to uc volumes only and getting rid of dbfs as well as simplifying configs

  dvanderwood

  committed
  c337f59

  Copy full SHA for c337f59

• uc resource generation update

  dvanderwood

  committed
  3e1e486

  Copy full SHA for 3e1e486

Commits on Oct 3, 2024

• continued unused file, and dbfs code removal along with code clean-up and simplification

  dvanderwood

  committed
  6a8eb47

  Copy full SHA for 6a8eb47

Commits on Oct 4, 2024

• update upload to databricks

  dvanderwood

  committed
  f360160

  Copy full SHA for f360160

• only upload wheel to workspace location

  dvanderwood

  committed
  c678ac8

  Copy full SHA for c678ac8

• added uc volume upload of the wheel if provided

  dvanderwood

  committed
  d6020b5

  Copy full SHA for d6020b5

• wheel upload added, all data uploads reworked

  dvanderwood

  committed
  acee79b

  Copy full SHA for acee79b

• initial upload and setup for integration testing for cloud files is done

  dvanderwood

  committed
  d316554

  Copy full SHA for d316554

Commits on Oct 7, 2024

• job workflow clean-up

  dvanderwood

  committed
  7b71fc9

  Copy full SHA for 7b71fc9

• cloud files testing, added back accidently removed json

  dvanderwood

  committed
  5ee1843

  Copy full SHA for 5ee1843

• cloud files integration test works

  dvanderwood

  committed
  e70f85d

  Copy full SHA for e70f85d

• formatting

  dvanderwood

  committed
  9e77b1f

  Copy full SHA for 9e77b1f

Commits on Oct 8, 2024

• continuing code simplication between 3 integration run types

  dvanderwood

  committed
  e78c751

  Copy full SHA for e78c751

• redundant doc strings removal

  dvanderwood

  committed
  1cdb503

  Copy full SHA for 1cdb503

• remove early exit

  dvanderwood

  committed
  37652f9

  Copy full SHA for 37652f9

• eventhub_accesskey_name updates

  dvanderwood

  committed
  a6249fe

  Copy full SHA for a6249fe

• fixed job create

  dvanderwood

  committed
  c740093

  Copy full SHA for c740093

• fixed clean up

  dvanderwood

  committed
  b769973

  Copy full SHA for b769973

• redundant message removed

  dvanderwood

  committed
  f6bb5fe

  Copy full SHA for f6bb5fe

• formatting

  dvanderwood

  committed
  381531c

  Copy full SHA for 381531c

• non cloud files testing, niether can be confirmed right now

  dvanderwood

  committed
  eb44fa5

  Copy full SHA for eb44fa5

Commits on Oct 9, 2024

• af cloud demo update

  dvanderwood

  committed
  f6954fb

  Copy full SHA for f6954fb

• formatting

  dvanderwood

  committed
  ec9a317

  Copy full SHA for ec9a317

• formatting

  dvanderwood

  committed
  ae27548

  Copy full SHA for ae27548

Commits on Oct 10, 2024

• linting fixes

  dvanderwood

  committed
  a60d0e2

  Copy full SHA for a60d0e2

Commits on Oct 14, 2024

• formatting and linting

  dvanderwood

  committed
  7b17edf

  Copy full SHA for 7b17edf

• removed match syntax since testing on python 3.9

  dvanderwood

  committed
  0d9975a

  Copy full SHA for 0d9975a

• removed other match statement

  dvanderwood

  committed
  386db9d

  Copy full SHA for 386db9d

• uc volume path for onboarding file test

  dvanderwood

  committed
  05e8b3a

  Copy full SHA for 05e8b3a