Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

switch disable default config to enable default #43

Merged
merged 13 commits into from
Jan 7, 2025
4 changes: 2 additions & 2 deletions .buildkite/scripts/run_models.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ dbt seed --target "$db" --full-refresh
dbt run --target "$db" --full-refresh
dbt test --target "$db"
## UPDATE FOR VARS HERE, IF NO VARS, PLEASE REMOVE
dbt run --vars '{lead_history_columns: ['first_name', 'lead_status', 'urgency', 'priority', 'relative_score', 'relative_urgency', 'demographic_score_marketing', 'behavior_score_marketing'], marketo__enable_campaigns: true, marketo__enable_programs: true, marketo__activity_delete_lead_enabled: false}' --target "$db" --full-refresh
dbt test --target "$db" --vars '{lead_history_columns: ['first_name', 'lead_status', 'urgency', 'priority', 'relative_score', 'relative_urgency', 'demographic_score_marketing', 'behavior_score_marketing'], marketo__enable_campaigns: true, marketo__enable_programs: true, marketo__activity_delete_lead_enabled: false}' --target "$db"
dbt run --vars '{lead_history_columns: ['first_name', 'lead_status', 'urgency', 'priority', 'relative_score', 'relative_urgency', 'demographic_score_marketing', 'behavior_score_marketing'], marketo__enable_campaigns: false, marketo__enable_programs: false, marketo__activity_delete_lead_enabled: false}' --target "$db" --full-refresh
dbt test --target "$db" --vars '{lead_history_columns: ['first_name', 'lead_status', 'urgency', 'priority', 'relative_score', 'relative_urgency', 'demographic_score_marketing', 'behavior_score_marketing'], marketo__enable_campaigns: false, marketo__enable_programs: false, marketo__activity_delete_lead_enabled: false}' --target "$db"
### END VARS CHUNK, REMOVE IF NOT USING
dbt run-operation fivetran_utils.drop_schemas_automation --target "$db"
14 changes: 14 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,17 @@
# dbt_marketo v0.13.0
[PR #43](https://github.com/fivetran/dbt_marketo/pull/43) includes the following updates:

## Breaking Change
fivetran-joemarkiewicz marked this conversation as resolved.
Show resolved Hide resolved
- Changes the default enable/disable config from `disable` to `enable` for the `marketo__enable_campaigns` and `marketo__enable_programs` variables since the Quickstart product can dynamically enable/disable such components. If you are not a Quickstart user nor are syncing the respective `campaign` or `program` tables, you must configure the variables accordingly. Disable the related models or fields by adding the following to your `dbt_project.yml` file:

```yml
vars:
marketo__enable_campaigns: False # Disable if Fivetran is not syncing the campaign table. This will disable the marketo__campaigns, marketo__programs, marketo__email_stats__by_campaign, marketo__email_stats__by_program models.
marketo__enable_programs: False # Disable if Fivetran is not syncing the program table. This will disable the marketo__programs and marketo__email_stats__by_program models.
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like how we're calling out here what will be disabled if these are turned off. However, would you be able to replace this section with a bit more clear direction as to what is being turned on by default, since that's a more relevant change from this update. It would be great as well to have those be included in maybe there own bullets and not hidden as comments within a code block.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update-- let me know what you think


Refer to the [README](https://github.com/fivetran/dbt_marketo?tab=readme-ov-file#step-4-enablingdisabling-models) for more details.

# dbt_marketo v0.12.1
[PR #42](https://github.com/fivetran/dbt_marketo/pull/42) includes the following updates:

Expand Down
13 changes: 4 additions & 9 deletions README.md
fivetran-joemarkiewicz marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -76,19 +76,14 @@ vars:
For additional configurations for the source models, please visit the [Marketo source package](https://github.com/fivetran/dbt_marketo_source).

### Step 4: Enabling/Disabling Models
This package takes into consideration tables that may not be synced due to slowness caused by the Marketo API. By default the `campaign` and `program` related-models are disabled. If you sync these tables, enable the modeling done by adding the following to your `dbt_project.yml` file:

This package takes into consideration tables that may not be synced due to slowness caused by the Marketo API. By default the `campaign`, `program`, and `activity_delete_lead` tables are enabled. If you do not sync these tables, disable the related models or fields by adding the following to your `dbt_project.yml` file:
```yml
vars:
marketo__enable_campaigns: true # Enable if Fivetran is syncing the campaign table
marketo__enable_programs: true # Enable if Fivetran is syncing the program table
marketo__enable_campaigns: False # Disable if Fivetran is not syncing the campaign table. This will disable the marketo__campaigns, marketo__programs, marketo__email_stats__by_campaign, marketo__email_stats__by_program models.
marketo__enable_programs: False # Disable if Fivetran is not syncing the program table. This will disable the marketo__programs and marketo__email_stats__by_program models.
marketo__activity_delete_lead_enabled: false # Disable if Fivetran is not syncing the activity_delete_lead table
```

Alternatively, you may need to disable certain models. The below models can be disabled by adding them to your `dbt_project.yml` file:
```yml
vars:
marketo__activity_delete_lead_enabled: false # Disable if you do not have the activity_delete_lead table
```
### (Optional) Step 5: Additional configurations
<details open><summary>Expand/Collapse details</summary>
<br>
Expand Down
2 changes: 1 addition & 1 deletion dbt_project.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: 'marketo'
version: '0.12.1'
version: '0.13.0'
config-version: 2
require-dbt-version: [">=1.3.0", "<2.0.0"]
on-run-start: "{{ lead_history_columns_warning() }}"
Expand Down
2 changes: 1 addition & 1 deletion docs/catalog.json

Large diffs are not rendered by default.

47 changes: 37 additions & 10 deletions docs/index.html

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/manifest.json

Large diffs are not rendered by default.

6 changes: 1 addition & 5 deletions integration_tests/dbt_project.yml
Original file line number Diff line number Diff line change
@@ -1,16 +1,12 @@
name: 'marketo_integration_tests'
version: '0.12.1'
version: '0.13.0'
profile: 'integration_tests'
config-version: 2

models:
+schema: "marketo_{{ var('directed_schema','dev') }}"

vars:
# enable when regenerating docs
# marketo__enable_campaigns: true
# marketo__enable_programs: true

marketo_schema: marketo_integration_tests_6
marketo_source:
marketo_lead_identifier: "marketo_lead_data"
Expand Down
2 changes: 1 addition & 1 deletion models/intermediate/marketo__email_stats__by_campaign.sql
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=var('marketo__enable_campaigns', False)) }}
{{ config(enabled=var('marketo__enable_campaigns', True)) }}

with email_sends as (

Expand Down
2 changes: 1 addition & 1 deletion models/intermediate/marketo__email_stats__by_program.sql
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=var('marketo__enable_campaigns', False) and var('marketo__enable_programs', False)) }}
{{ config(enabled=var('marketo__enable_campaigns', True) and var('marketo__enable_programs', True)) }}

with email_sends as (

Expand Down
2 changes: 1 addition & 1 deletion models/marketo__campaigns.sql
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=var('marketo__enable_campaigns', False)) }}
{{ config(enabled=var('marketo__enable_campaigns', True)) }}

with campaigns as (

Expand Down
6 changes: 3 additions & 3 deletions models/marketo__email_sends.sql
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ with sends as (
select *
from {{ ref('marketo__unsubscribes__by_sent_email') }}

{% if var('marketo__enable_campaigns', False) %}
{% if var('marketo__enable_campaigns', True) %}

), campaigns as (

Expand Down Expand Up @@ -73,13 +73,13 @@ with sends as (

select
booleans.*,
{% if var('marketo__enable_campaigns', False) %}
{% if var('marketo__enable_campaigns', True) %}
campaigns.campaign_type,
campaigns.program_id,
fivetran-joemarkiewicz marked this conversation as resolved.
Show resolved Hide resolved
{% endif %}
email_templates.is_operational
from booleans
{% if var('marketo__enable_campaigns', False) %}
{% if var('marketo__enable_campaigns', True) %}
left join campaigns using (campaign_id)
{% endif %}
left join email_templates
Expand Down
2 changes: 1 addition & 1 deletion models/marketo__programs.sql
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{ config(enabled=var('marketo__enable_campaigns', False) and var('marketo__enable_programs', False)) }}
{{ config(enabled=var('marketo__enable_campaigns', True) and var('marketo__enable_programs', True)) }}

with programs as (

Expand Down
8 changes: 6 additions & 2 deletions packages.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
packages:
- package: fivetran/marketo_source
version: [">=0.12.0", "<0.13.0"]
# - package: fivetran/marketo_source
# version: [">=0.13.0", "<0.14.0"]

- git: https://github.com/fivetran/dbt_marketo_source.git
revision: default_enable
warn-unpinned: false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reminder to swap before merge