Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

switch disable default config to enable default #40

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .buildkite/scripts/run_models.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ dbt seed --target "$db" --full-refresh
dbt run --target "$db" --full-refresh
dbt test --target "$db"
## UPDATE FOR VARS HERE, IF NO VARS, PLEASE REMOVE
dbt run --vars '{marketo__enable_campaigns: true, marketo__enable_programs: true, marketo__activity_delete_lead_enabled: false}' --target "$db" --full-refresh
dbt run --vars '{marketo__enable_campaigns: false, marketo__enable_programs: false, marketo__activity_delete_lead_enabled: false}' --target "$db" --full-refresh
dbt test --target "$db"
### END VARS CHUNK, REMOVE IF NOT USING
dbt run-operation fivetran_utils.drop_schemas_automation --target "$db"
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
# dbt_marketo_source v0.13.0
[PR #40](https://github.com/fivetran/dbt_marketo_source/pull/40) includes the following updates:

## Breaking Change
- Updates the default configuration for the `marketo__enable_campaigns` and `marketo__enable_programs` variables from disabled to enabled, activating the `stg_marketo__campaigns` and `stg_marketo__program` models by default.
- Quickstart dynamically manages these settings, but non-Quickstart users or those not syncing campaign or program tables should adjust the variables accordingly. Refer to the [README](https://github.com/fivetran/dbt_marketo_source?tab=readme-ov-file#step-4-enablingdisabling-models) for details.

# dbt_marketo_source v0.12.0
[PR #38](https://github.com/fivetran/dbt_marketo_source/pull/38) includes the following updates:

Expand Down
15 changes: 5 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ If you are **not** using the [Marketo transformation package](https://github.co
```yml
packages:
- package: fivetran/marketo_source
version: [">=0.12.0", "<0.13.0"]
version: [">=0.13.0", "<0.14.0"]
```

### Step 3: Define database and schema variables
Expand All @@ -56,17 +56,12 @@ vars:
```

### Step 4: Enabling/Disabling Models
This package takes into consideration tables that may not be synced due to slowness caused by the Marketo API. By default the `campaign` and `program` models are disabled. If you sync these tables, enable the modeling done by adding the following to your `dbt_project.yml` file:
This package takes into consideration tables that may not be synced due to slowness caused by the Marketo API. By default the `campaign`, `program`, and `activity_delete_lead` tables are enabled. If you do not sync these tables, disable the related models by adding the following to your `dbt_project.yml` file:
```yml
vars:
marketo__enable_campaigns: true # Enable if Fivetran is syncing the campaign table
marketo__enable_programs: true # Enable if Fivetran is syncing the program table
```

Alternatively, you may need to disable certain models. The below models can be disabled by adding them to your `dbt_project.yml` file:
```yml
vars:
marketo__activity_delete_lead_enabled: false # Disable if you do not have the activity_delete_lead table
marketo__enable_campaigns: False # Disable if Fivetran is not syncing the campaign table. Will disable the stg_marketo__program and stg_marketo__campaigns models.
marketo__enable_programs: False # Disable if Fivetran is not syncing the program table. Will disable the stg_marketo__program model.
marketo__activity_delete_lead_enabled: False # Disable if Fivetran is not syncing the activity_delete_lead table
```

### (Optional) Step 5: Additional configurations
Expand Down
2 changes: 1 addition & 1 deletion dbt_project.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: 'marketo_source'
version: '0.12.0'
version: '0.13.0'
config-version: 2
require-dbt-version: [">=1.3.0", "<2.0.0"]
models:
Expand Down
2 changes: 1 addition & 1 deletion docs/catalog.json

Large diffs are not rendered by default.

47 changes: 37 additions & 10 deletions docs/index.html

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/manifest.json

Large diffs are not rendered by default.

1 change: 0 additions & 1 deletion docs/run_results.json

This file was deleted.

2 changes: 1 addition & 1 deletion integration_tests/dbt_project.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: 'marketo_source_integration_tests'
version: '0.12.0'
version: '0.13.0'
profile: 'integration_tests'
config-version: 2
vars:
Expand Down
2 changes: 1 addition & 1 deletion models/stg_marketo__campaigns.sql
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=var('marketo__enable_campaigns', False)) }}
{{ config(enabled=var('marketo__enable_campaigns', True)) }}

with base as (

Expand Down
2 changes: 1 addition & 1 deletion models/stg_marketo__program.sql
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=var('marketo__enable_campaigns', False) and var('marketo__enable_programs', False)) }}
{{ config(enabled=var('marketo__enable_campaigns', True) and (var('marketo__enable_programs', True))) }}

with base as (

Expand Down
2 changes: 1 addition & 1 deletion models/tmp/stg_marketo__campaign_tmp.sql
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=var('marketo__enable_campaigns', False)) }}
{{ config(enabled=var('marketo__enable_campaigns', True)) }}

select *
from {{ var('campaign') }}
2 changes: 1 addition & 1 deletion models/tmp/stg_marketo__program_tmp.sql
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=var('marketo__enable_campaigns', False) and var('marketo__enable_programs', False)) }}
{{ config(enabled=var('marketo__enable_campaigns', True) and var('marketo__enable_programs', True)) }}

select *
from {{ var('program') }}