Skip to content

Commit

Permalink
initial build clean up
Browse files Browse the repository at this point in the history
  • Loading branch information
fivetran-kristin committed May 16, 2020
1 parent c9d8be2 commit 830ba43
Show file tree
Hide file tree
Showing 5 changed files with 26 additions and 17 deletions.
20 changes: 15 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Github
# GitHub

This package models Github data from [Fivetran's connector](https://fivetran.com/docs/applications/github). It uses data in the format described by [this ERD](https://docs.google.com/presentation/d/1lx6ez7-x-s-n2JCnCi3SjG4XMmx9ysNUvaNCaWc3I_I/edit).
This package models GitHub data from [Fivetran's connector](https://fivetran.com/docs/applications/GitHub). It uses data in the format described by [this ERD](https://docs.google.com/presentation/d/1lx6ez7-x-s-n2JCnCi3SjG4XMmx9ysNUvaNCaWc3I_I/edit).

This package is designed to do the following:
* Add descriptions to tables and columns that are synced using Fivetran
Expand All @@ -10,7 +10,7 @@ This package is designed to do the following:

### Models

This package contains staging models, designed to work simultaneously with our [github modeling package](https://github.com/fivetran/dbt_github). The staging models are designed to:
This package contains staging models, designed to work simultaneously with our [GitHub modeling package](https://github.com/fivetran/dbt_github). The staging models are designed to:
* Remove any rows that are soft-deleted
* Name columns consistently across all packages:
* Boolean fields are prefixed with `is_` or `has_`
Expand All @@ -22,9 +22,19 @@ This package contains staging models, designed to work simultaneously with our [
Check [dbt Hub](https://hub.getdbt.com/) for the latest installation instructions, or [read the docs](https://docs.getdbt.com/docs/package-management) for more information on installing packages.

## Configuration
The [variables](https://docs.getdbt.com/docs/using-variables) needed to configure this package are as follows:
By default this package will run using your target database, and the `github` schema. If this is not where your GitHub data is (perhaps your gitHub schema is `Github_fivetran`), please add the following configuration to your `dbt_project.yml` file:

TBD
```yml
# dbt_project.yml

...
config-version: 2

vars:
github_source:
github_database: your_database_name
github_schema: your_schema_name
```
### Contributions ###
Expand Down
2 changes: 2 additions & 0 deletions dbt_project.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ config-version: 2
name: 'github_source'
version: '0.0.1'

require-dbt-version: [">=0.17.0", "<0.18.0"]

models:
github_source:
materialized: ephemeral
14 changes: 5 additions & 9 deletions models/src_github.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@ version: 2

sources:
- name: github

database: "{{var ('github_database', target.database)}}"
schema: "{{var ('github_schema', 'github')}}"

loader: fivetran
loaded_at_field: _fivetran_synced

Expand Down Expand Up @@ -43,14 +45,6 @@ sources:
- name: closed
description: Boolean variable for if the issue was closed (true) or re-opened (false)

- name: issue_comment
description: Comments made to an issue
columns:
- name: issue_id
description: Foreign key that references the issue table
- name: uer_id
description: Foreign key that references the user table, representing the user that made the comment.

- name: issue_label
description: The current labels attached to an issue
loaded_at_field: _fivetran_synced
Expand Down Expand Up @@ -171,6 +165,8 @@ sources:
description: Timestamp of when the review was submitted.
- name: requested_id
description: Foreign key that references the user table, representing the user that was requested to review a PR.
- name: removed
description: Boolean variable indicating if the requester was removed from the PR (true) or added to the PR (false).

- name: user
description: Table representing contributors to a git project
Expand Down
3 changes: 2 additions & 1 deletion models/stg_github_requested_reviewer_history.sql
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ with requested_reviewer_history as (
select
pull_request_id,
created_at,
requested_id
requested_id,
removed
from requested_reviewer_history
)

Expand Down
4 changes: 2 additions & 2 deletions models/stg_github_user.sql
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
with user as (
with github_user as (

select *
from {{ source('github', 'user') }}
Expand All @@ -11,7 +11,7 @@ with user as (
name,
bio,
company
from user
from github_user
)

select *
Expand Down

0 comments on commit 830ba43

Please sign in to comment.