Add var to persist former-agents' roles for SLA metrics #43

fivetran-jamie · 2023-12-05T23:49:03Z

PR Overview

This PR will address the following Issue/Feature:
fivetran/dbt_zendesk#120 and #40

This PR will result in the following new package version:

v0.10.1 -- by default nothing will change for folks. they'll need to explicitly set the internal user criteria var

Please provide the finalized CHANGELOG entry which details the relevant changes included in this PR:

dbt_zendesk_source v0.10.1

PR #43 introduces the following updates:

Feature Updates

Added the internal_user_criteria variable, which can be used to mark internal users whose USER.role may have changed from agent to end-user after they left your organization. This variable accepts SQL that may reference any non-custom field in USER, and it will be wrapped in a case when statement in the stg_zendesk__user model.
- Example usage:

# dbt_project.yml
vars:
  zendesk_source:
    internal_user_criteria: "lower(email) like '%@fivetran.com' or external_id = '12345' or name in ('Garrett', 'Alfredo')" # can reference any non-custom field in USER

Output: In stg_zendesk__user, users who match your criteria and have a role of end-user will have their role switched to agent. This will ensure that downstream SLA metrics are appropriately calculated.

Under the Hood

Updated the way we dynamically disable sources. Previously, we used a custom meta.is_enabled flag, but, since we added this, dbt-core introduced a native config.enabled attribute. We have opted to use the dbt-native config instead.
Updated the pull request templates.
Included auto-releaser GitHub Actions workflow to automate future releases.

PR Checklist

Basic Validation

Please acknowledge that you have successfully performed the following commands locally:

dbt run –full-refresh && dbt test
dbt run (if incremental models are present)

Before marking this PR as "ready for review" the following have been applied:

The appropriate issue has been linked, tagged, and properly assigned
All necessary documentation and version upgrades have been applied
docs were regenerated (unless this PR does not include any code or yml updates)
BuildKite integration tests are passing
Detailed validation steps have been provided below

Detailed Validation

Please share any and all of your validation steps:

Will validate internally with @jackiexsun

fivetran-joemarkiewicz · 2023-12-06T15:10:36Z

CHANGELOG.md

+```yml
+# dbt_project.yml
+vars:
+  zendesk__internal_user_criteria: "lower(email) like '%@fivetran.com' or external_id = 12345 or name in ('Garrett', 'Alfredo')" # can reference any non-custom field in USER


Omg love this example!

fivetran-jamie · 2023-12-07T18:53:33Z

README.md

 ### Add passthrough columns
 This package includes all source columns defined in the staging models. However, the `stg_zendesk__ticket` model allows for additional columns to be added using a pass-through column variable. This is extremely useful if you'd like to include custom fields to the package.
 ```yml
 vars:
  zendesk__ticket_passthrough_columns: [account_custom_field_1, account_custom_field_2]
 ```

+### Mark Former Internal Users as Agents


reminder to myself to add this to the transform README (no release needed there tho) after approval

fivetran-joemarkiewicz

@fivetran-jamie just a few comments around the changes. Once you reply to them and make any updates then let me know for re-review and I can check again and approve.

fivetran-joemarkiewicz · 2024-01-08T21:54:39Z

CHANGELOG.md

+[PR #43](https://github.com/fivetran/dbt_zendesk_source/pull/43) introduces the following updates:
+
+## Feature Updates
+- Added the `internal_user_criteria` variable, which can be used to mark internal users whose `USER.role` may have changed from `agent` to `end-user` after they left your organization.


Maybe it would be relevant to include here that this is used as a case when clause in the staging model. Just so users know the context of the variable. The example shows this well, but I feel a bit of an explanation to this would be helpful to someone new to the concept.

added a brief note!

fivetran-joemarkiewicz · 2024-01-08T21:56:46Z

README.md

+### Mark Former Internal Users as Agents
+If a team member leaves your organization and their internal account is deactivated, their `USER.role` will switch from `agent` or `admin` to `end-user`. This will skew historical ticket SLA metrics, as we calculate reply times and other metrics based on `agent` or `admin` activity only.
+
+To persist the integrity of historical ticket SLAs and mark these former team members as agents, provide the `internal_user_criteria` variable with a SQL clause to identify them, based on fields in the `USER` table:


Can we include a similar update to one I suggested in the CHANGELOG here as well?

added a little something

fivetran-joemarkiewicz · 2024-01-08T21:58:42Z

models/stg_zendesk__user.sql

        email,
        name,
        organization_id,
+        {% if var('internal_user_criteria', false) -%}


I didn't know that false would work in this context. I thought it would need to be [] (or something like that). Are we sure this works on all dbt versions supported by the package? It may be worthwhile to test this on dbt 1.3.0 just to make sure.

just ran it on dbt-bigquery==1.3.0 with and without providing internal_user_criteria: "lower(email) like '%@fivetran.com'" and it works as expected 😄

yeah i think it works similarly to how we provide false (or true i suppose) as the default value for using_<possibly missing source table> variables

fivetran-joemarkiewicz

@fivetran-jamie these changes look good to me. I just have one small request to update the example snippet before moving the a release review as I ran into a small issue when trying to copy/paste the snippet provided.

Thanks!

CHANGELOG.md

README.md

Co-authored-by: Joe Markiewicz <[email protected]>

fivetran-jamie added 3 commits December 5, 2023 11:03

docs

9e9000e

give defaualt value to new var

7279234

fix formatting

56a68df

fivetran-jamie self-assigned this Dec 5, 2023

more docs

ba778b9

fivetran-joemarkiewicz reviewed Dec 6, 2023

View reviewed changes

fivetran-jamie marked this pull request as ready for review December 7, 2023 18:51

fivetran-jamie commented Dec 7, 2023

View reviewed changes

fivetran-jamie changed the title ~~docs~~ Add var to persist former-agents' roles for SLA metrics Dec 7, 2023

fivetran-jamie added 2 commits December 7, 2023 15:10

add docs

b5db00f

remove prefix from new var

0ddf561

fivetran-jamie requested a review from fivetran-joemarkiewicz January 8, 2024 16:55

fivetran-joemarkiewicz requested changes Jan 8, 2024

View reviewed changes

joe feedback

b6d3096

fivetran-joemarkiewicz approved these changes Jan 9, 2024

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

README.md Outdated Show resolved Hide resolved

fivetran-jamie and others added 3 commits January 9, 2024 12:04

Update CHANGELOG.md

c245c12

Co-authored-by: Joe Markiewicz <[email protected]>

Update README.md

7ea2f4c

Co-authored-by: Joe Markiewicz <[email protected]>

finalizing

bafb884

fivetran-jamie merged commit 75d9621 into main Jan 10, 2024
7 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add var to persist former-agents' roles for SLA metrics #43

Add var to persist former-agents' roles for SLA metrics #43

fivetran-jamie commented Dec 5, 2023 •

edited

Loading

fivetran-joemarkiewicz Dec 6, 2023

fivetran-jamie Dec 7, 2023

fivetran-joemarkiewicz left a comment

fivetran-joemarkiewicz Jan 8, 2024

fivetran-jamie Jan 9, 2024

fivetran-joemarkiewicz Jan 8, 2024

fivetran-jamie Jan 9, 2024

fivetran-joemarkiewicz Jan 8, 2024

fivetran-jamie Jan 9, 2024

fivetran-joemarkiewicz left a comment

Add var to persist former-agents' roles for SLA metrics #43

Add var to persist former-agents' roles for SLA metrics #43

Conversation

fivetran-jamie commented Dec 5, 2023 • edited Loading

PR Overview

dbt_zendesk_source v0.10.1

Feature Updates

Under the Hood

PR Checklist

Basic Validation

Detailed Validation

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fivetran-joemarkiewicz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fivetran-joemarkiewicz left a comment

Choose a reason for hiding this comment

fivetran-jamie commented Dec 5, 2023 •

edited

Loading