forked from edanalytics/edu_edfi_source
-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Base models #11
Open
maxalmazov
wants to merge
4
commits into
main
Choose a base branch
from
models-from-json-files
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Base models #11
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
37 changes: 37 additions & 0 deletions
37
models/staging/edfi_3/base/base_ef3__open_staff_positions.sql
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,37 @@ | ||
with open_staff_positions as ( | ||
{{ source_edfi3('open_staff_positions') }} | ||
), | ||
renamed as ( | ||
select | ||
tenant_code, | ||
api_year, | ||
pull_timestamp, | ||
last_modified_timestamp, | ||
file_row_number, | ||
filename, | ||
is_deleted, | ||
|
||
v:id::string as record_guid, | ||
v:educationOrganizationReference:educationOrganizationId::int as ed_org_id, | ||
v:positionTitle::string as position_title, | ||
v:requisitionNumber::string as requisition_number, | ||
v:datePosted::date as date_posted, | ||
v:datePostingRemoved::date as date_posting_removed, | ||
case | ||
when array_size(v:academicSubjects) > 1 | ||
then False | ||
else True | ||
end as is_single_subject_identifier, | ||
-- descriptors | ||
{{ extract_descriptor('v:employmentStatusDescriptor::string') }} as employment_status, | ||
{{ extract_descriptor('v:staffClassificationDescriptor::string') }} as staff_classification, | ||
{{ extract_descriptor('v:programAssignmentDescriptor::string') }} as program_assignment, | ||
{{ extract_descriptor('v:postingResultDescriptor::string') }} as posting_result, | ||
-- references | ||
v:educationOrganizationReference as education_organization, | ||
-- unflattened lists | ||
v:academicSubjects as v_academic_subjects, | ||
v:instructionalGradeLevels as v_instructional_grade_levels | ||
from open_staff_positions | ||
) | ||
select * from renamed |
34 changes: 34 additions & 0 deletions
34
models/staging/edfi_3/base/base_ef3__organization_departments.sql
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
with organization_departments as ( | ||
{{ source_edfi3('organization_departments') }} | ||
), | ||
renamed as ( | ||
select | ||
tenant_code, | ||
api_year, | ||
pull_timestamp, | ||
last_modified_timestamp, | ||
file_row_number, | ||
filename, | ||
is_deleted, | ||
|
||
v:id::string as record_guid, | ||
v:parentEducationOrganizationReference:educationOrganizationId::int as ed_org_id, | ||
v:organizationDepartmentId::int as ed_org_department_id, | ||
v:nameOfInstitution::int as name_of_institution, | ||
v:shortNameOfInstitution::int as short_name_of_institution, | ||
v:webSite::int as web_site, | ||
-- descriptors | ||
{{ extract_descriptor('v:academicSubjectDescriptor::string') }} as academic_subject, | ||
{{ extract_descriptor('v:operationalStatusDescriptor::string') }} as operational_status, | ||
--references | ||
v:parentEducationOrganizationReference as education_organization, | ||
-- unflattened lists | ||
v:categories as v_categories, | ||
v:addresses as v_addresses, | ||
v:identificationCodes as v_identification_codes, | ||
v:indicators as v_indicators, | ||
v:institutionTelephones as v_institution_telephones, | ||
v:internationalAddresses as v_international_addresses | ||
from organization_departments | ||
) | ||
select * from renamed |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
with payrolls as ( | ||
{{ source_edfi3('payrolls') }} | ||
), | ||
renamed as ( | ||
select | ||
tenant_code, | ||
api_year, | ||
pull_timestamp, | ||
last_modified_timestamp, | ||
file_row_number, | ||
filename, | ||
is_deleted, | ||
|
||
v:id::string as record_guid, | ||
v:accountReference:educationOrganizationId::int as ed_org_id, | ||
v:accountReference:accountIdentifier::string as account_id, | ||
v:staffReference:staffUniqueId::string as staff_unique_id, | ||
v:asOfDate::date as as_of_date, | ||
v:amountToDate::int as amount_to_date, | ||
-- references | ||
v:accountReference as account, | ||
v:staffReference as staff | ||
from payrolls | ||
) | ||
select * from renamed |
25 changes: 25 additions & 0 deletions
25
models/staging/edfi_3/base/base_ef3__post_secondary_events.sql
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
with post_secondary_events as ( | ||
{{ source_edfi3('post_secondary_events') }} | ||
), | ||
renamed as ( | ||
select | ||
tenant_code, | ||
api_year, | ||
pull_timestamp, | ||
last_modified_timestamp, | ||
file_row_number, | ||
filename, | ||
is_deleted, | ||
|
||
v:id::string as record_guid, | ||
v:postSecondaryInstitutionReference:postSecondaryInstitutionId::int as post_secondary_institution_id, | ||
v:studentReference:studentUniqueId::int as student_unique_id, | ||
v:eventDate::date as event_date, | ||
-- descriptors | ||
{{ extract_descriptor('v:postSecondaryEventCategoryDescriptor::string') }} as post_secondary_event_category, | ||
-- references | ||
v:postSecondaryInstitutionReference as post_secondary_institution, | ||
v:studentReference as student | ||
from post_secondary_events | ||
) | ||
select * from renamed |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
with report_cards as ( | ||
{{ source_edfi3('report_cards') }} | ||
), | ||
renamed as ( | ||
select | ||
tenant_code, | ||
api_year, | ||
pull_timestamp, | ||
last_modified_timestamp, | ||
file_row_number, | ||
filename, | ||
is_deleted, | ||
|
||
v:id::string as record_guid, | ||
v:educationOrganizationReference:educationOrganizationId::int as ed_org_id, | ||
v:studentReference:studentUniqueId::int as student_unique_id, | ||
v:gpaCumulative::int as gpa_cumulative, | ||
v:gpaGivenGradingPeriod::int as gpa_given_grading_period, | ||
v:numberOfDaysAbsent::int as number_of_days_absent, | ||
v:numberOfDaysInAttendance::int as number_of_days_in_attendance, | ||
v:numberOfDaysTardy::int as number_of_days_tardy, | ||
-- references | ||
v:gradingPeriodReference as grading_period | ||
v:studentReference as student | ||
v:educationOrganizationReference as education_organization_reference | ||
-- unflattened lists | ||
v:gradePointAverages as v_grade_point_averages, | ||
v:grades as v_grades, | ||
v:studentCompetencyObjectives as v_student_competency_objectives, | ||
v:studentLearningObjectives as v_student_learning_objectives | ||
from report_cards | ||
) | ||
select * from renamed |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
with restraint_events as ( | ||
{{ source_edfi3('restraint_events') }} | ||
), | ||
renamed as ( | ||
select | ||
tenant_code, | ||
api_year, | ||
pull_timestamp, | ||
last_modified_timestamp, | ||
file_row_number, | ||
filename, | ||
is_deleted, | ||
|
||
v:id::string as record_guid, | ||
v:restraintEventIdentifier::string as record_guid, | ||
v:schoolReference:schoolId::int as school_id, | ||
v:studentReference:studentUniqueId::string as student_unique_id, | ||
v:eventDate::date as event_date, | ||
-- descriptors | ||
{{ extract_descriptor('v:educationalEnvironmentDescriptor::string')}} as educational_environment, | ||
-- references | ||
v:schoolReference as school | ||
v:studentReference as student | ||
-- unflattened lists | ||
v:programs as v_programs | ||
v:reasons as v_reasons | ||
from restraint_events | ||
) | ||
select * from renamed |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@holdorph said we don't need to create models for empty JSON files...but maybe i missed something
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
but there is such API in docs