diff --git a/.buildkite/scripts/run_models.sh b/.buildkite/scripts/run_models.sh index a876cda8..3819cb80 100644 --- a/.buildkite/scripts/run_models.sh +++ b/.buildkite/scripts/run_models.sh @@ -19,7 +19,7 @@ dbt deps dbt seed --target "$db" --full-refresh dbt run --target "$db" --full-refresh dbt test --target "$db" -dbt run --vars '{stripe__using_invoices: false, stripe__using_payment_method: false, stripe__using_subscriptions: false, stripe_timezone: "America/New_York", stripe__using_subscription_history: false, stripe__using_price: false}' --target "$db" +dbt run --vars '{stripe__using_invoices: false, stripe__using_payment_method: false, stripe__using_subscriptions: false, stripe_timezone: "America/New_York", stripe__using_subscription_history: false, stripe__using_price: false, stripe__convert_values: true}' --target "$db" dbt test --target "$db" dbt run-operation fivetran_utils.drop_schemas_automation --target "$db" diff --git a/CHANGELOG.md b/CHANGELOG.md index 7dff3cac..ff87e417 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,9 +1,41 @@ -# dbt_stripe version.version +# dbt_stripe v0.16.0 +## Breaking Change +- The aggregated amount-based fields in `stripe__daily_overview` and `stripe__customer_overview` now reflect the raw smallest units (e.g., cents), following Stripe's raw data, instead of converted dollar amounts. This change standardizes values across all models. + + **Impact**: Customers using these models should note that the values will appear inflated compared to the previous dollar-based representation. + +- Important to note: + - The cent-to-dollar conversion has been moved upstream and is now controlled by a new variable: `stripe__convert_values` which is disabled by default. + - To enable the conversion and divide all amount-based fields by 100, set `stripe__convert_values` to `True` in your `project.yml`. + - See below for more context behind these changes and for detailed setup instructions, see the [README](https://github.com/fivetran/dbt_stripe?tab=readme-ov-file#enabling-cent-to-dollar-conversion). + +## Feature Update: Optional Conversion to Major Units +- **Background**: Stripe provides amount-based fields (e.g., `amount`, `net`, `fee`) in the smallest denomination. For currencies with minor and major units (e.g., USD), 100 represents 100 cents (1 USD). For currencies without minor units (e.g., JPY), 100 represents 100 JPY. + +- This PR introduces a variable `stripe__convert_values` (disabled by default) upstream in the staging models which allows users the option to divide all amount-based fields by 100. + +- If your data is using a currency with major and minor units, it may be useful to enable this variable if you wish to divide amounts by 100 and therefore convert values into major units. For information on how to do so, refer to the [README]((https://github.com/fivetran/dbt_stripe?tab=readme-ov-file#enabling-cent-to-dollar-conversion)) on enabling the `stripe__convert_values` variable. + - Examples of currencies using minor units (in which enabling `stripe__convert_values` may be useful) include USD (100 = 1 USD), EUR (100 = 1 Euro), and CAD (100 = 1 CAD). + +- If your Stripe data is *not* using a currency involving minor units, it may make more sense to retain the amount-based fields as raw smallest units by leaving `stripe__convert_values` disabled. + - Examples of currencies NOT using minor units include JPY (100 = 100 JPY), IDR (100 = 100 IDR), KRW (100 = 100 KRW). + +## Notes +- Currently this package does not support multiple currencies, but we have created a [feature request to support multiple currencies](https://github.com/fivetran/dbt_stripe/issues/102) where you are welcome to provide feedback or contribute to the discussion. + +## Under the Hood +- Updated the `run_models.sh` script to test for when `stripe__convert_values` is set to True. +- Added a consistency test for `stripe__customer_overview`. ## Documentation +- Updated the descriptions for all amount-based fields to specify the grain of the values and add information about the `stripe__convert_values` variable. - Added Quickstart model counts to README. ([#103](https://github.com/fivetran/dbt_stripe/pull/103)) - Corrected references to connectors and connections in the README. ([#103](https://github.com/fivetran/dbt_stripe/pull/103)) +## Under the Hood +- Updated the `run_models.sh` script to test for when `stripe__convert_values` is set to True. +- Added a consistency test for `stripe__customer_overview`. + # dbt_stripe v0.15.1 ## Bug Fixes diff --git a/README.md b/README.md index b645dbd7..3ed9a553 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,6 @@ -

+# Stripe Transformation dbt Package ([Docs](https://fivetran.github.io/dbt_stripe/)) + +

@@ -13,7 +15,6 @@

-# Stripe Transformation dbt Package ([Docs](https://fivetran.github.io/dbt_stripe/)) ## What does this dbt package do? - Produces modeled tables that leverage Stripe data from [Fivetran's connector](https://fivetran.com/docs/applications/stripe) in the format described by [this ERD](https://fivetran.com/docs/applications/stripe#schemainformation) and build off the output of our [stripe source package](https://github.com/fivetran/dbt_stripe_source). - Enables you to better understand your Stripe transactions. The package achieves this by performing the following: @@ -74,7 +75,7 @@ Include the following stripe package version in your `packages.yml` file: ```yaml packages: - package: fivetran/stripe - version: [">=0.15.0", "<0.16.0"] + version: [">=0.16.0", "<0.17.0"] ``` Do **NOT** include the `stripe_source` package in this file. The transformation package itself has a dependency on it and will install the source package as well. @@ -222,6 +223,17 @@ vars: stripe__subscription_metadata: ['the', 'list', 'of', 'property', 'fields'] # Note: this is case-SENSITIVE and must match the casing of the property as it appears in the JSON ``` +#### Enabling Cent to Dollar Conversion + +Amount-based fields, such as `amount` and `net`, are typically displayed in the smallest denomination (e.g., cents for USD). By default, amount-based fields will be in this raw form. However, some currencies use major and minor units (for example, cents and dollars when using USD). In these cases, it may be useful to divide the amounts by 100, converting amounts to major units (dollars for USD). To enable the division, configure the `stripe__convert_values` to `true` in your project.yml: + +```yml +vars: + stripe__convert_values: true # default is false +``` + +If you are working in a currency that does not differentiate between minor and major units, such as JPY or KRW, it may make more sense to keep the amount-based fields in raw form and therefore the package can be ran without configuration. As `stripe__convert_values` is disabled by default, these fields will not be impacted. + #### Change the build schema By default, this package builds the stripe staging models within a schema titled (`` + `_stg_stripe`) in your destination. If this is not where you would like your stripe staging data to be written to, add the following configuration to your root `dbt_project.yml` file: diff --git a/dbt_project.yml b/dbt_project.yml index 0ef93407..4b005ce0 100644 --- a/dbt_project.yml +++ b/dbt_project.yml @@ -1,7 +1,7 @@ config-version: 2 name: 'stripe' -version: '0.15.1' +version: '0.16.0' require-dbt-version: [">=1.3.0", "<2.0.0"] models: stripe: diff --git a/integration_tests/ci/sample.profiles.yml b/integration_tests/ci/sample.profiles.yml index 7b435f07..0dc6fc75 100644 --- a/integration_tests/ci/sample.profiles.yml +++ b/integration_tests/ci/sample.profiles.yml @@ -16,13 +16,13 @@ integration_tests: pass: "{{ env_var('CI_REDSHIFT_DBT_PASS') }}" dbname: "{{ env_var('CI_REDSHIFT_DBT_DBNAME') }}" port: 5439 - schema: stripe_integrations_tests_15 + schema: stripe_integrations_tests_18 threads: 8 bigquery: type: bigquery method: service-account-json project: 'dbt-package-testing' - schema: stripe_integrations_tests_15 + schema: stripe_integrations_tests_18 threads: 8 keyfile_json: "{{ env_var('GCLOUD_SERVICE_KEY') | as_native }}" snowflake: @@ -33,7 +33,7 @@ integration_tests: role: "{{ env_var('CI_SNOWFLAKE_DBT_ROLE') }}" database: "{{ env_var('CI_SNOWFLAKE_DBT_DATABASE') }}" warehouse: "{{ env_var('CI_SNOWFLAKE_DBT_WAREHOUSE') }}" - schema: stripe_integrations_tests_15 + schema: stripe_integrations_tests_18 threads: 8 postgres: type: postgres @@ -42,13 +42,13 @@ integration_tests: pass: "{{ env_var('CI_POSTGRES_DBT_PASS') }}" dbname: "{{ env_var('CI_POSTGRES_DBT_DBNAME') }}" port: 5432 - schema: stripe_integrations_tests_15 + schema: stripe_integrations_tests_18 threads: 8 databricks: catalog: "{{ env_var('CI_DATABRICKS_DBT_CATALOG') }}" host: "{{ env_var('CI_DATABRICKS_DBT_HOST') }}" http_path: "{{ env_var('CI_DATABRICKS_DBT_HTTP_PATH') }}" - schema: stripe_integrations_tests_15 + schema: stripe_integrations_tests_18 threads: 8 token: "{{ env_var('CI_DATABRICKS_DBT_TOKEN') }}" type: databricks \ No newline at end of file diff --git a/integration_tests/dbt_project.yml b/integration_tests/dbt_project.yml index 727fd7ac..debf3425 100644 --- a/integration_tests/dbt_project.yml +++ b/integration_tests/dbt_project.yml @@ -1,7 +1,7 @@ config-version: 2 name: 'stripe_integration_tests' -version: '0.15.1' +version: '0.16.0' profile: 'integration_tests' @@ -9,8 +9,8 @@ profile: 'integration_tests' models: +schema: "stripe_{{ var('directed_schema','dev') }}" -vars: - stripe_schema: stripe_integrations_tests_15 +vars: + stripe_schema: stripe_integrations_tests_18 stripe__standardized_billing_model_enabled: true stripe__using_credit_notes: true stripe_source: diff --git a/integration_tests/tests/consistency/consistency_customer_overview.sql b/integration_tests/tests/consistency/consistency_customer_overview.sql new file mode 100644 index 00000000..bbefe42e --- /dev/null +++ b/integration_tests/tests/consistency/consistency_customer_overview.sql @@ -0,0 +1,70 @@ +{{ config( + tags="fivetran_validations", + enabled=var('fivetran_validation_tests_enabled', false) +) }} + +-- this test ensures the customer_overview end model matches the prior version by comparing the aggregated fields +-- the below iterates through the prod and dev names to reduce redundancy of logic +with +{% for prod_or_dev in ('prod', 'dev') %} + {% set cols = [ + 'total_sales', + 'total_refunds', + 'total_gross_transaction_amount', + 'total_fees', + 'total_net_transaction_amount', + 'total_sales_count', + 'total_refund_count', + 'sales_this_month', + 'refunds_this_month', + 'gross_transaction_amount_this_month', + 'fees_this_month', + 'net_transaction_amount_this_month', + 'sales_count_this_month', + 'refund_count_this_month', + 'total_failed_charge_count', + 'total_failed_charge_amount', + 'failed_charge_count_this_month', + 'failed_charge_amount_this_month' + ] %} + + {{ prod_or_dev }} as ( + select + {% for col in cols %} + {% if not loop.first %}, {% endif %} + floor(sum({{ col }})) as summed_{{ col }} -- floor and sum is to keep consistency between dev and prod aggs + {% endfor %} + from {{ target.schema }}_stripe_{{ prod_or_dev }}.stripe__customer_overview + ), +{% endfor %} + +prod_not_in_dev as ( + -- rows from prod not found in dev + select * from prod + except distinct + select * from dev +), + +dev_not_in_prod as ( + -- rows from dev not found in prod + select * from dev + except distinct + select * from prod +), + +final as ( + select + *, + 'from prod' as source + from prod_not_in_dev + + union all -- union since we only care if rows are produced + + select + *, + 'from dev' as source + from dev_not_in_prod +) + +select * +from final \ No newline at end of file diff --git a/models/docs.md b/models/docs.md index 5474e7df..8c603159 100644 --- a/models/docs.md +++ b/models/docs.md @@ -1,2 +1,2 @@ {% docs source_relation -%} The source where this data was pulled from. If you are making use of the `union_schemas` variable, this will be the source schema. If you are making use of the `union_databases` variable, this will be the source database. If you are not unioning together multiple sources, this will be an empty string. -{%- enddocs %} +{%- enddocs %} \ No newline at end of file diff --git a/models/intermediate/int_stripe__account_daily.sql b/models/intermediate/int_stripe__account_daily.sql index aaeb67a4..65c54b39 100644 --- a/models/intermediate/int_stripe__account_daily.sql +++ b/models/intermediate/int_stripe__account_daily.sql @@ -81,21 +81,21 @@ select daily_account_balance_transactions.date_day, daily_account_balance_transactions.account_id, daily_account_balance_transactions.source_relation, - coalesce(daily_account_balance_transactions.total_daily_sales_amount/100.0,0) as total_daily_sales_amount, - coalesce(daily_account_balance_transactions.total_daily_refunds_amount/100.0,0) as total_daily_refunds_amount, - coalesce(daily_account_balance_transactions.total_daily_adjustments_amount/100.0,0) as total_daily_adjustments_amount, - coalesce(daily_account_balance_transactions.total_daily_other_transactions_amount/100.0,0) as total_daily_other_transactions_amount, - coalesce(daily_account_balance_transactions.total_daily_gross_transaction_amount/100.0,0) as total_daily_gross_transaction_amount, - coalesce(daily_account_balance_transactions.total_daily_net_transactions_amount/100.0,0) as total_daily_net_transactions_amount, - coalesce(daily_account_balance_transactions.total_daily_payout_fee_amount/100.0,0) as total_daily_payout_fee_amount, - coalesce(daily_account_balance_transactions.total_daily_gross_payout_amount/100.0,0) as total_daily_gross_payout_amount, - coalesce(daily_account_balance_transactions.daily_net_activity_amount/100.0,0) as daily_net_activity_amount, - coalesce((daily_account_balance_transactions.daily_net_activity_amount + daily_account_balance_transactions.total_daily_gross_payout_amount)/100.0, 0) as daily_end_balance_amount, + coalesce(daily_account_balance_transactions.total_daily_sales_amount,0) as total_daily_sales_amount, + coalesce(daily_account_balance_transactions.total_daily_refunds_amount,0) as total_daily_refunds_amount, + coalesce(daily_account_balance_transactions.total_daily_adjustments_amount,0) as total_daily_adjustments_amount, + coalesce(daily_account_balance_transactions.total_daily_other_transactions_amount,0) as total_daily_other_transactions_amount, + coalesce(daily_account_balance_transactions.total_daily_gross_transaction_amount,0) as total_daily_gross_transaction_amount, + coalesce(daily_account_balance_transactions.total_daily_net_transactions_amount,0) as total_daily_net_transactions_amount, + coalesce(daily_account_balance_transactions.total_daily_payout_fee_amount,0) as total_daily_payout_fee_amount, + coalesce(daily_account_balance_transactions.total_daily_gross_payout_amount,0) as total_daily_gross_payout_amount, + coalesce(daily_account_balance_transactions.daily_net_activity_amount,0) as daily_net_activity_amount, + coalesce((daily_account_balance_transactions.daily_net_activity_amount + daily_account_balance_transactions.total_daily_gross_payout_amount), 0) as daily_end_balance_amount, coalesce(daily_account_balance_transactions.total_daily_sales_count, 0) as total_daily_sales_count, coalesce(daily_account_balance_transactions.total_daily_payouts_count, 0) as total_daily_payouts_count, coalesce(daily_account_balance_transactions.total_daily_adjustments_count, 0) as total_daily_adjustments_count, coalesce(daily_failed_charges.total_daily_failed_charge_count, 0) as total_daily_failed_charge_count, - coalesce(daily_failed_charges.total_daily_failed_charge_amount/100, 0) as total_daily_failed_charge_amount + coalesce(daily_failed_charges.total_daily_failed_charge_amount, 0) as total_daily_failed_charge_amount from daily_account_balance_transactions left join daily_failed_charges diff --git a/models/stripe.yml b/models/stripe.yml index 7ca85a8a..6e3e93a7 100644 --- a/models/stripe.yml +++ b/models/stripe.yml @@ -33,11 +33,11 @@ models: - name: balance_transaction_currency description: Three-letter ISO currency code, in lowercase. - name: balance_transaction_amount - description: Gross amount of the transaction, in cents. + description: Gross amount of the transaction. "{{ doc('convert_values') }}" - name: balance_transaction_fee - description: Fees (in cents) paid for this transaction. + description: Fees (in currency units) paid for this transaction. - name: balance_transaction_net - description: Net amount of the transaction, in cents. Gross amount minus fees, refunds, and disputes + description: Net amount of the transaction. Gross amount minus fees, refunds, and disputes. "{{ doc('convert_values') }}" - name: balance_transaction_type description: The type of transaction. Possible values are adjustment, advance, advance_funding, application_fee, application_fee_refund, charge, connect_collection_transfer, issuing_authorization_hold, issuing_authorization_release, issuing_dispute, issuing_transaction, payment, payment_failure_refund, payment_refund, payout, payout_cancel, payout_failure, refund, refund_failure, reserve_transaction, reserved_funds, stripe_fee, stripe_fx_fee, tax_fee, topup, topup_reversal, transfer, transfer_cancel, transfer_failure, or transfer_refund. - name: balance_transaction_reporting_category @@ -47,7 +47,7 @@ models: - name: balance_transaction_description description: An arbitrary string attached to the object. Often useful for displaying to users. - name: customer_facing_amount - description: The transaction amount that the customer sees. + description: The transaction amount that the customer sees. "{{ doc('convert_values') }}" - name: customer_facing_currency description: Three-letter ISO currency code that the customer sees. - name: effective_at @@ -179,19 +179,19 @@ models: - name: source_relation description: "{{ doc('source_relation') }}" - name: latest_dispute_amount_won - description: Latest disputed amount that was won in favor of the merchant. Usually the amount of the charge, but can differ (usually because of currency fluctuation or because only part of the order is disputed). + description: Latest disputed amount that was won in favor of the merchant. Usually the amount of the charge, but can differ (usually because of currency fluctuation or because only part of the order is disputed). "{{ doc('convert_values') }}" - name: latest_dispute_amount_lost - description: Latest disputed amount that was lost and returned to the customer. Usually the amount of the charge, but can differ (usually because of currency fluctuation or because only part of the order is disputed). + description: Latest disputed amount that was lost and returned to the customer. Usually the amount of the charge, but can differ (usually because of currency fluctuation or because only part of the order is disputed). "{{ doc('convert_values') }}" - name: latest_dispute_amount_under_review - description: Latest disputed amount that is currently under review by the bank. Usually the amount of the charge, but can differ (usually because of currency fluctuation or because only part of the order is disputed). + description: Latest disputed amount that is currently under review by the bank. Usually the amount of the charge, but can differ (usually because of currency fluctuation or because only part of the order is disputed). "{{ doc('convert_values') }}" - name: latest_dispute_amount_needs_response - description: Latest disputed amount that currently needs a response (the dispute has been filed but the merchant has not yet responded with evidence). Usually the amount of the charge, but can differ (usually because of currency fluctuation or because only part of the order is disputed). + description: Latest disputed amount that currently needs a response (the dispute has been filed but the merchant has not yet responded with evidence). Usually the amount of the charge, but can differ (usually because of currency fluctuation or because only part of the order is disputed). "{{ doc('convert_values') }}" - name: latest_dispute_amount_warning_closed - description: Latest disputed amount that is currently of status `warning_under_closed` (early fraud warning being closed due to no formal dispute). Usually the amount of the charge, but can differ (usually because of currency fluctuation or because only part of the order is disputed). + description: Latest disputed amount that is currently of status `warning_under_closed` (early fraud warning being closed due to no formal dispute). Usually the amount of the charge, but can differ (usually because of currency fluctuation or because only part of the order is disputed). "{{ doc('convert_values') }}" - name: latest_dispute_amount_warning_under_review - description: Latest disputed amount that is currently of status `warning_under_review` (card issuer suspects possible fraud but hasn't yet escalated the situation to a full dispute). Usually the amount of the charge, but can differ (usually because of currency fluctuation or because only part of the order is disputed). + description: Latest disputed amount that is currently of status `warning_under_review` (card issuer suspects possible fraud but hasn't yet escalated the situation to a full dispute). Usually the amount of the charge, but can differ (usually because of currency fluctuation or because only part of the order is disputed). "{{ doc('convert_values') }}" - name: latest_dispute_amount_warning_needs_response - description: Latest disputed amount that is currently of status `warning_needs_response` (early fraud warning has been escalated into formal dispute or card issuer has requested more information). Usually the amount of the charge, but can differ (usually because of currency fluctuation or because only part of the order is disputed). + description: Latest disputed amount that is currently of status `warning_needs_response` (early fraud warning has been escalated into formal dispute or card issuer has requested more information). Usually the amount of the charge, but can differ (usually because of currency fluctuation or because only part of the order is disputed). "{{ doc('convert_values') }}" - name: dispute_count description: Count of disputes raised against this transaction. If > 1, join in `dispute` data for additional information on each dispute. @@ -213,11 +213,11 @@ models: - name: total_refunds description: Sum of all refunds made to this customer. - name: total_gross_transaction_amount - description: Gross amount of all transactions made with this customer. + description: Gross amount of all transactions made with this customer. "{{ doc('convert_values') }}" - name: total_fees - description: Total fees charged on all transactions made with this customer. + description: Total fees charged on all transactions made with this customer. "{{ doc('convert_values') }}" - name: total_net_transaction_amount - description: Net amount of all transactions made with this customer. This is the gross amount excluding fees, refunds, and disputes. + description: Net amount of all transactions made with this customer. This is the gross amount excluding fees, refunds, and disputes. "{{ doc('convert_values') }}" - name: total_sales_count description: The total number of sales made to this customer. - name: total_refund_count @@ -227,11 +227,11 @@ models: - name: refunds_this_month description: Sum of refunds made to this customer this month. - name: gross_transaction_amount_this_month - description: Gross amount of transactions made with this customer this month. + description: Gross amount of transactions made with this customer this month. "{{ doc('convert_values') }}" - name: fees_this_month - description: Total fees charged on transactions made with this customer this month. + description: Total fees charged on transactions made with this customer this month. "{{ doc('convert_values') }}" - name: net_transaction_amount_this_month - description: Net amount of transactions made with this customer this month. + description: Net amount of transactions made with this customer this month. "{{ doc('convert_values') }}" - name: sales_count_this_month description: The number of sales made to this customer this month. - name: refund_count_this_month @@ -243,11 +243,11 @@ models: - name: total_failed_charge_count description: The total number of incomplete charges this customer has had. - name: total_failed_charge_amount - description: The total amount from incomplete charges this customer has had. + description: The total amount from incomplete charges this customer has had. "{{ doc('convert_values') }}" - name: failed_charge_count_this_month description: The number of incomplete charges this customer has had this month. - name: failed_charge_amount_this_month - description: The amount from incomplete charges this customer has had this month. + description: The amount from incomplete charges this customer has had this month. "{{ doc('convert_values') }}" - name: customer_currency description: Currency used by the customer. - name: default_card_id @@ -300,19 +300,19 @@ models: - name: currency description: The currency that the invoice is in. Three-letter ISO currency code, in lowercase. - name: amount_due - description: The amount, if any, that the customer has paid on the invoice + description: The amount, if any, that the customer has paid on the invoice. "{{ doc('convert_values') }}" - name: amount_paid - description: The amount, if any, that the customer has paid on the invoice + description: The amount, if any, that the customer has paid on the invoice. "{{ doc('convert_values') }}" - name: amount_due - description: Final amount due at this time for this invoice. If the invoice’s total is smaller than the minimum charge amount, for example, or if there is account credit that can be applied to the invoice, the amount_due may be 0. If there is a positive starting_balance for the invoice (the customer owes money), the amount_due will also take that into account. The charge that gets generated for the invoice will be for the amount specified in amount_due. + description: Final amount due at this time for this invoice. If the invoice’s total is smaller than the minimum charge amount, for example, or if there is account credit that can be applied to the invoice, the amount_due may be 0. If there is a positive starting_balance for the invoice (the customer owes money), the amount_due will also take that into account. The charge that gets generated for the invoice will be for the amount specified in amount_due. "{{ doc('convert_values') }}" - name: subtotal - description: The amount of the invoice before discounts and taxes. + description: The amount of the invoice before discounts and taxes. "{{ doc('convert_values') }}" - name: tax - description: The amount of tax being charged in the invoice. + description: The amount of tax being charged in the invoice. "{{ doc('convert_values') }}" - name: total description: The Total after discounts and taxes. - name: amount_remaining - description: The amount of the invoice remaining to be paid. + description: The amount of the invoice remaining to be paid. "{{ doc('convert_values') }}" - name: attempt_count description: Number of payment attempts made for this invoice, from the perspective of the payment retry schedule. - name: invoice_memo @@ -324,7 +324,7 @@ models: - name: balance_transaction_id description: The ID of the balance transaction object representing payment - name: charge_amount - description: The amount charged to the customer. + description: The amount charged to the customer. "{{ doc('convert_values') }}" - name: charge_status description: The status of the charge for the invoice. - name: charge_created_at @@ -374,7 +374,7 @@ models: - name: invoice_item_id description: The ID of the invoice item this item is a part of - name: invoice_line_item_amount - description: Amount for this line item. + description: Amount for this line item. "{{ doc('convert_values') }}" - name: currency description: Currency of this line item. - name: invoice_line_item_memo @@ -408,13 +408,13 @@ models: - name: invoice_due_date description: Date when payment for the invoice is due. - name: invoice_amount_due - description: Final amount due at this time for this invoice. If the invoice’s total is smaller than the minimum charge amount, for example, or if there is account credit that can be applied to the invoice, the amount_due may be 0. If there is a positive starting_balance for the invoice (the customer owes money), the amount_due will also take that into account. The charge that gets generated for the invoice will be for the amount specified in amount_due. + description: Final amount due at this time for this invoice. If the invoice’s total is smaller than the minimum charge amount, for example, or if there is account credit that can be applied to the invoice, the amount_due may be 0. If there is a positive starting_balance for the invoice (the customer owes money), the amount_due will also take that into account. The charge that gets generated for the invoice will be for the amount specified in amount_due. "{{ doc('convert_values') }}" - name: invoice_amount_paid - description: The amount, if any, that the customer has paid on the invoice + description: The amount, if any, that the customer has paid on the invoice. "{{ doc('convert_values') }}" - name: invoice_subtotal - description: The amount of the invoice before discounts and taxes. + description: The amount of the invoice before discounts and taxes. "{{ doc('convert_values') }}" - name: invoice_tax - description: The amount of tax being charged in the invoice. + description: The amount of tax being charged in the invoice. "{{ doc('convert_values') }}" - name: invoice_total description: The total of the invoice after discounts and taxes. - name: connected_account_id @@ -430,7 +430,7 @@ models: - name: price_plan_is_active description: Boolean indicating if the plan or price is active (true) or in-active (false). - name: price_plan_amount - description: The unit amount in cents to be charged, represented as a whole integer if possible. + description: The unit amount in currency units to be charged, represented as a whole integer if possible. "{{ doc('convert_values') }}" - name: price_plan_interval description: The frequency at which a subscription is billed. One of day, week, month or year. - name: price_plan_interval_count @@ -485,17 +485,17 @@ models: - name: number_invoices_generated description: Number of invoices that have been generated for this subscription. - name: total_amount_billed - description: The total amount that has been billed to the customer for this subscription. + description: The total amount that has been billed to the customer for this subscription. "{{ doc('convert_values') }}" - name: total_amount_paid - description: The total amount paid by the customer for this subscription. + description: The total amount paid by the customer for this subscription. "{{ doc('convert_values') }}" - name: total_amount_remaining - description: Any outstanding balance for invoices related to this subscription. + description: Any outstanding balance for invoices related to this subscription. "{{ doc('convert_values') }}" - name: most_recent_invoice_created_at description: The timestamp of the most recent invoice for this subscription created. - name: average_invoice_amount - description: The average amount of invoices generated for this subscription + description: The average amount of invoices generated for this subscription. "{{ doc('convert_values') }}" - name: average_line_item_amount - description: The average sum of the line items for invoices generated for this subscription + description: The average sum of the line items for invoices generated for this subscription. "{{ doc('convert_values') }}" - name: avg_num_line_items description: The average number of line items for invoices generated for this subscription - name: source_relation @@ -517,25 +517,25 @@ models: - name: source_relation description: "{{ doc('source_relation') }}" - name: total_daily_sales_amount - description: Total amount for when transaction type is a charge or payment. + description: Total amount for when transaction type is a charge or payment. "{{ doc('convert_values') }}" - name: total_daily_refunds_amount - description: Total amount for when transaction type is payment_refund or refund. + description: Total amount for when transaction type is payment_refund or refund. "{{ doc('convert_values') }}" - name: total_daily_adjustments_amount - description: Total amount for when transaction type is adjustment + description: Total amount for when transaction type is adjustment. "{{ doc('convert_values') }}" - name: total_daily_other_transactions_amount - description: Total amount for when transaction type is not of the prior types. + description: Total amount for when transaction type is not of the prior types. "{{ doc('convert_values') }}" - name: total_daily_gross_transaction_amount - description: Total amount for when transaction type is not payout or transfer + description: Total amount for when transaction type is not payout or transfer "{{ doc('convert_values') }}" - name: total_daily_net_transactions_amount - description: Total net for when transaction type is not payout or transfer. + description: Total net for when transaction type is not payout or transfer. "{{ doc('convert_values') }}" - name: total_daily_payout_fee_amount - description: Total fee for when transaction type is payout or transfer. + description: Total fee for when transaction type is payout or transfer. "{{ doc('convert_values') }}" - name: total_daily_gross_payout_amount - description: Total amount for when transaction type is payout or transfer. + description: Total amount for when transaction type is payout or transfer. "{{ doc('convert_values') }}" - name: daily_net_activity_amount - description: Net amount minus fees from payouts and transfers. + description: Net amount minus fees from payouts and transfers. "{{ doc('convert_values') }}" - name: daily_end_balance_amount - description: daily_net_activity_amount + total_daily_gross_payout_amount + description: daily_net_activity_amount + total_daily_gross_payout_amount. "{{ doc('convert_values') }}" - name: total_daily_sales_count description: Count of total balance transactions where type is payment or charge. - name: total_daily_payouts_count @@ -545,27 +545,27 @@ models: - name: total_daily_failed_charge_count description: Count of incomplete charges. - name: total_daily_failed_charge_amount - description: Total amount of incomplete charges. + description: Total amount of incomplete charges. "{{ doc('convert_values') }}" - name: rolling_total_daily_sales_amount - description: Rolling total of total_daily_sales_amount + description: Rolling total of total_daily_sales_amount. "{{ doc('convert_values') }}" - name: rolling_total_daily_refunds_amount - description: Rolling total of total_daily_refunds_amount + description: Rolling total of total_daily_refunds_amount. "{{ doc('convert_values') }}" - name: rolling_total_daily_adjustments_amount - description: Rolling total of total_daily_adjustments_amount + description: Rolling total of total_daily_adjustments_amount. "{{ doc('convert_values') }}" - name: rolling_total_daily_other_transactions_amount - description: Rolling total of total_daily_other_transactions_amount + description: Rolling total of total_daily_other_transactions_amount. "{{ doc('convert_values') }}" - name: rolling_total_daily_gross_transaction_amount - description: Rolling total of total_daily_gross_transaction_amount + description: Rolling total of total_daily_gross_transaction_amount. "{{ doc('convert_values') }}" - name: rolling_total_daily_net_transactions_amount - description: Rolling total of total_daily_net_transactions_amount + description: Rolling total of total_daily_net_transactions_amount. "{{ doc('convert_values') }}" - name: rolling_total_daily_payout_fee_amount - description: Rolling total of total_daily_payout_fee_amount + description: Rolling total of total_daily_payout_fee_amount. "{{ doc('convert_values') }}" - name: rolling_total_daily_gross_payout_amount - description: Rolling total of total_daily_gross_payout_amount + description: Rolling total of total_daily_gross_payout_amount. "{{ doc('convert_values') }}" - name: rolling_daily_net_activity_amount - description: Rolling total of daily_net_activity_amount + description: Rolling total of daily_net_activity_amount. "{{ doc('convert_values') }}" - name: rolling_daily_end_balance_amount - description: Rolling total of daily_end_balance_amount + description: Rolling total of daily_end_balance_amount. "{{ doc('convert_values') }}" - name: rolling_total_daily_sales_count description: Rolling total of total_daily_sales_count - name: rolling_total_daily_payouts_count @@ -575,7 +575,7 @@ models: - name: rolling_total_daily_failed_charge_count description: Rolling total of total_daily_failed_charge_count - name: rolling_total_daily_failed_charge_amount - description: Rolling total of total_daily_failed_charge_amount + description: Rolling total of total_daily_failed_charge_amount. "{{ doc('convert_values') }}" - name: stripe__activity_itemized_2 description: Modeled after the [titular report](https://stripe.com/docs/reports/activity#downloading-data) from Stripe, each record represents a balance transaction and additional details such as associated customer, charge, refund, fee, and invoice information. This class of Activity reports allows you to see your payments activity alongside the related fees for the same period of time. Designed specifically for users with Interchange Plus (IC+) pricing, it presents users with fees on the date they incur them. This makes it useful for calculating fee accrual entries and understanding your net payments activity. This itemized report contains a full list of individual transactions. @@ -594,7 +594,7 @@ models: - name: currency description: Three-letter ISO code for the currency in which the amount is defined. - name: amount - description: Amount of this activity, expressed in major units of the currency (e.g., dollars for USD, or yen for JPY). + description: Amount of this activity. - name: charge_id description: The unique ID of the related charge, if any. For charges, this will be the charge itself; for refunds or disputes, this will be the original charge being refunded or disputed. - name: payment_intent_id @@ -682,7 +682,7 @@ models: - name: statement_descriptor description: The dynamic statement descriptor or suffix specified when the related charge was created. - name: customer_facing_amount - description: For transactions associated with charges, refunds, or disputes, the amount of the original charge, the refund, or the latest dispute. If the customer was charged in a different currency than your account’s default, this field will reflect the amount as seen by the customer. + description: For transactions associated with charges, refunds, or disputes, the amount of the original charge, the refund, or the latest dispute. If the customer was charged in a different currency than your account’s default, this field will reflect the amount as seen by the customer. - name: balance_transaction_description description: An arbitrary string attached to the balance transaction. - name: connected_account_id diff --git a/models/stripe__customer_overview.sql b/models/stripe__customer_overview.sql index 5c60fc03..85dfe6f2 100644 --- a/models/stripe__customer_overview.sql +++ b/models/stripe__customer_overview.sql @@ -148,18 +148,18 @@ with balance_transaction_joined as ( customer.created_at as customer_created_at, customer.currency as customer_currency, {{ dbt_utils.star(from=ref('stg_stripe__customer'), relation_alias='customer', except=['customer_id','description','created_at','currency','metadata','source_relation']) }}, - coalesce(transactions_grouped.total_sales/100.0, 0) as total_sales, - coalesce(transactions_grouped.total_refunds/100.0, 0) as total_refunds, - coalesce(transactions_grouped.total_gross_transaction_amount/100.0, 0) as total_gross_transaction_amount, - coalesce(transactions_grouped.total_fees/100.0, 0) as total_fees, - coalesce(transactions_grouped.total_net_transaction_amount/100.0, 0) as total_net_transaction_amount, + {{ 'coalesce(transactions_grouped.total_sales, 0)' if not var('stripe__convert_values', false) else 'coalesce(transactions_grouped.total_sales, 0)' }} as total_sales, + {{ 'coalesce(transactions_grouped.total_refunds, 0)' if not var('stripe__convert_values', false) else 'coalesce(transactions_grouped.total_refunds, 0)' }} as total_refunds, + {{ 'coalesce(transactions_grouped.total_gross_transaction_amount, 0)' if not var('stripe__convert_values', false) else 'coalesce(transactions_grouped.total_gross_transaction_amount, 0)' }} as total_gross_transaction_amount, + {{ 'coalesce(transactions_grouped.total_fees, 0)' if not var('stripe__convert_values', false) else 'coalesce(transactions_grouped.total_fees, 0)' }} as total_fees, + {{ 'coalesce(transactions_grouped.total_net_transaction_amount, 0)' if not var('stripe__convert_values', false) else 'coalesce(transactions_grouped.total_net_transaction_amount, 0)' }} as total_net_transaction_amount, coalesce(transactions_grouped.total_sales_count, 0) as total_sales_count, - coalesce(transactions_grouped.total_refund_count, 0) as total_refund_count, - coalesce(transactions_grouped.sales_this_month/100.0, 0) as sales_this_month, - coalesce(transactions_grouped.refunds_this_month/100.0, 0) as refunds_this_month, - coalesce(transactions_grouped.gross_transaction_amount_this_month/100.0, 0) as gross_transaction_amount_this_month, - coalesce(transactions_grouped.fees_this_month/100.0, 0) as fees_this_month, - coalesce(transactions_grouped.net_transaction_amount_this_month/100.0, 0) as net_transaction_amount_this_month, + coalesce(transactions_grouped.total_refund_count, 0) as total_refund_count, + {{ 'coalesce(transactions_grouped.sales_this_month, 0)' if not var('stripe__convert_values', false) else 'coalesce(transactions_grouped.sales_this_month, 0)' }} as sales_this_month, + {{ 'coalesce(transactions_grouped.refunds_this_month, 0)' if not var('stripe__convert_values', false) else 'coalesce(transactions_grouped.refunds_this_month, 0)' }} as refunds_this_month, + {{ 'coalesce(transactions_grouped.gross_transaction_amount_this_month, 0)' if not var('stripe__convert_values', false) else 'coalesce(transactions_grouped.gross_transaction_amount_this_month, 0)' }} as gross_transaction_amount_this_month, + {{ 'coalesce(transactions_grouped.fees_this_month, 0)' if not var('stripe__convert_values', false) else 'coalesce(transactions_grouped.fees_this_month, 0)' }} as fees_this_month, + {{ 'coalesce(transactions_grouped.net_transaction_amount_this_month, 0)' if not var('stripe__convert_values', false) else 'coalesce(transactions_grouped.net_transaction_amount_this_month, 0)' }} as net_transaction_amount_this_month, coalesce(transactions_grouped.sales_count_this_month, 0) as sales_count_this_month, coalesce(transactions_grouped.refund_count_this_month, 0) as refund_count_this_month, transactions_grouped.first_sale_date, @@ -185,27 +185,28 @@ with balance_transaction_joined as ( customer.created_at as customer_created_at, customer.currency as customer_currency, {{ dbt_utils.star(from=ref('stg_stripe__customer'), relation_alias='customer', except=['customer_id','description','created_at','currency','metadata','source_relation']) }}, - coalesce(transactions_grouped.total_sales/100.0, 0) as total_sales, - coalesce(transactions_grouped.total_refunds/100.0, 0) as total_refunds, - coalesce(transactions_grouped.total_gross_transaction_amount/100.0, 0) as total_gross_transaction_amount, - coalesce(transactions_grouped.total_fees/100.0, 0) as total_fees, - coalesce(transactions_grouped.total_net_transaction_amount/100.0, 0) as total_net_transaction_amount, + {{ 'coalesce(transactions_grouped.total_sales, 0)' if not var('stripe__convert_values', false) else 'coalesce(transactions_grouped.total_sales, 0)' }} as total_sales, + {{ 'coalesce(transactions_grouped.total_refunds, 0)' if not var('stripe__convert_values', false) else 'coalesce(transactions_grouped.total_refunds, 0)' }} as total_refunds, + {{ 'coalesce(transactions_grouped.total_gross_transaction_amount, 0)' if not var('stripe__convert_values', false) else 'coalesce(transactions_grouped.total_gross_transaction_amount, 0)' }} as total_gross_transaction_amount, + {{ 'coalesce(transactions_grouped.total_fees, 0)' if not var('stripe__convert_values', false) else 'coalesce(transactions_grouped.total_fees, 0)' }} as total_fees, + {{ 'coalesce(transactions_grouped.total_net_transaction_amount, 0)' if not var('stripe__convert_values', false) else 'coalesce(transactions_grouped.total_net_transaction_amount, 0)' }} as total_net_transaction_amount, coalesce(transactions_grouped.total_sales_count, 0) as total_sales_count, - coalesce(transactions_grouped.total_refund_count, 0) as total_refund_count, - coalesce(transactions_grouped.sales_this_month/100.0, 0) as sales_this_month, - coalesce(transactions_grouped.refunds_this_month/100.0, 0) as refunds_this_month, - coalesce(transactions_grouped.gross_transaction_amount_this_month/100.0, 0) as gross_transaction_amount_this_month, - coalesce(transactions_grouped.fees_this_month/100.0, 0) as fees_this_month, - coalesce(transactions_grouped.net_transaction_amount_this_month/100.0, 0) as net_transaction_amount_this_month, + coalesce(transactions_grouped.total_refund_count, 0) as total_refund_count, + {{ 'coalesce(transactions_grouped.sales_this_month, 0)' if not var('stripe__convert_values', false) else 'coalesce(transactions_grouped.sales_this_month, 0)' }} as sales_this_month, + {{ 'coalesce(transactions_grouped.refunds_this_month, 0)' if not var('stripe__convert_values', false) else 'coalesce(transactions_grouped.refunds_this_month, 0)' }} as refunds_this_month, + {{ 'coalesce(transactions_grouped.gross_transaction_amount_this_month, 0)' if not var('stripe__convert_values', false) else 'coalesce(transactions_grouped.gross_transaction_amount_this_month, 0)' }} as gross_transaction_amount_this_month, + {{ 'coalesce(transactions_grouped.fees_this_month, 0)' if not var('stripe__convert_values', false) else 'coalesce(transactions_grouped.fees_this_month, 0)' }} as fees_this_month, + {{ 'coalesce(transactions_grouped.net_transaction_amount_this_month, 0)' if not var('stripe__convert_values', false) else 'coalesce(transactions_grouped.net_transaction_amount_this_month, 0)' }} as net_transaction_amount_this_month, coalesce(transactions_grouped.sales_count_this_month, 0) as sales_count_this_month, coalesce(transactions_grouped.refund_count_this_month, 0) as refund_count_this_month, transactions_grouped.first_sale_date, transactions_grouped.most_recent_sale_date, coalesce(failed_charges_by_customer.total_failed_charge_count, 0) as total_failed_charge_count, - coalesce(failed_charges_by_customer.total_failed_charge_amount/100, 0) as total_failed_charge_amount, + {{ 'coalesce(failed_charges_by_customer.total_failed_charge_amount, 0)' if not var('stripe__convert_values', false) else 'coalesce(failed_charges_by_customer.total_failed_charge_amount, 0)' }} as total_failed_charge_amount, coalesce(failed_charges_by_customer.failed_charge_count_this_month, 0) as failed_charge_count_this_month, - coalesce(failed_charges_by_customer.failed_charge_amount_this_month/100, 0) as failed_charge_amount_this_month, + {{ 'coalesce(failed_charges_by_customer.failed_charge_amount_this_month, 0)' if not var('stripe__convert_values', false) else 'coalesce(failed_charges_by_customer.failed_charge_amount_this_month, 0)' }} as failed_charge_amount_this_month, customer.source_relation + from customer left join transactions_grouped diff --git a/packages.yml b/packages.yml index 33040fb8..84107994 100644 --- a/packages.yml +++ b/packages.yml @@ -1,3 +1,7 @@ packages: -- package: fivetran/stripe_source - version: [">=0.12.0", "<0.13.0"] \ No newline at end of file +# - package: fivetran/stripe_source +# version: [">=0.12.0", "<0.13.0"] + + - git: https://github.com/fivetran/dbt_stripe_source.git + revision: feature/standardize_cent_conversions + warn-unpinned: false \ No newline at end of file