Update Data Uses for 1.4 (#107)

* Update Data uses for 1.4 * feat: refactor how the default taxonomy is set up to make modification easier * Update Data Uses * feat: add a suite of "count" tests for the default taxonomy * feat: set proper Defaults for Optional fields * feat: rework top-level and second-level uses * feat: advertising restructure * fix: pytest * feat: add a new export script for the default taxonomy that includes YAML, JSON and CSV * feat: fix the CSV export and add a simple sorting to the default data types * feat: add a CI check for the taxonomy export * fix: uppercase fides key * fix: pylint * fix: exported taxonomy * feat: remove accidental files * fix: remove alignment spaces from data_uses csv * Add missing data use * Update export_default_taxonomy to generate root nodes and copy to docs CSVs * Run export_default_taxonomy locally * Fixes for taxonomy visualization * feat: make all of the data use updates based on the latest spreadsheet * docs: changelog * feat: update descriptions based on new spreadsheet * Apply suggestions from code review * fix: update exported resources * feat: add the authentication use --------- Co-authored-by: Neville Samuell <[email protected]>
IABTechLab · May 31, 2023 · 1362767 · 1362767
1 parent af67da5
commit 1362767
Show file tree

Hide file tree

Showing 29 changed files with 3,039 additions and 1,368 deletions.
diff --git a/.github/workflows/pr_checks.yml b/.github/workflows/pr_checks.yml
@@ -56,6 +56,24 @@ jobs:
       - name: Run formatter
         run: make black
 
+  Export:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v3
+
+      - name: Set Up Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: "3.10.11"
+          cache: "pip"
+
+      - name: Install Package
+        run: pip install .
+
+      - name: Run Export
+        run: python scripts/export_default_taxonomy.py
+
   Pylint:
     needs: Build
     runs-on: ubuntu-latest

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -19,6 +19,7 @@ The types of changes are:
 ### Changed
 
 - Make `PrivacyDeclaration` use pydantic `orm_mode` [#101](https://github.com/ethyca/fideslang/pull/101)
+- Updated the default data uses [#107](https://github.com/ethyca/fideslang/pull/107)
 
 ### Remove
 

diff --git a/data_files/data_categories.csv b/data_files/data_categories.csv
@@ -1,66 +1,58 @@
-privacy_key,name,parent_key,description
-data_category,Data Category,,
-system,System Data,data_category,"Data unique to, and under control of the system."
-system.authentication,Authentication Data,system,Data used to manage access to the system.
-system.operations,Operations Data,system,Data used for system operations.
-user,User Data,data_category,"Data related to the user of the system, either provided directly or derived based on their usage."
-user.account,Account Data,user,Data related to a user's system account.
-user.account.contact,Account Contact Data,user.account,Contact data related to a system account.
-user.account.contact.city,Account City,user.account.contact,Account's city level address data.
-user.account.contact.country,Account Country,user.account.contact,Account's country level address data.
-user.account.contact.email,Account Email,user.account.contact,Account's email address.
-user.account.contact.phone_number,Account Phone Number,user.account.contact,Account's phone number.
-user.account.contact.postal_code,Account Postal Code,user.account.contact,Account's postal code.
-user.account.contact.state,Account State,user.account.contact,Account's state level address data.
-user.account.contact.street,Account Street,user.account.contact,Account's street level address.
-user.account.payment,Payment Data,user.account,Payment data related to system account.
-user.account.payment.financial_account_number,Account Payment Financial Account Number,user.account.payment,"Financial account number for an account's payment card, bank account, or other financial system."
-user.device,Device Data,user,"Data related to a user's device, configuration, and settings."
-user.device.cookie_id,Cookie ID,user.device,Cookie unique identification number.
-user.device.device_id,Device ID,user.device,Device unique identification number.
-user.device.ip_address,IP Address,user.device,Unique identifier related to device connection.
-user.contact,Contact Data,user,User contact data for purposes other than account management.
-user.contact.city,User Contact City,user.contact,"User's city level address data."
-user.contact.country,User Contact Country,user.contact,"User's country level address data."
-user.contact.email,User Contact Email,user.contact,"User's email address."
-user.contact.phone_number,User Contact Phone Number,user.contact,"User's phone number."
-user.contact.postal_code,User Contact Postal Code,user.contact,"User's postal code."
-user.contact.state,User Contact State,user.contact,"User's state level address data."
-user.contact.street,User Contact Street,user.contact,"User's street level address data."
-user.credentials,Credentials,user,User authentication data.
-user.credentials.biometric_credentials,Biometric Credentials,user.credentials,Credentials for system authentication.
-user.credentials.password,Password,user.credentials,Password for system authentication.
-user.financial,Financial Data,user,Payment data and financial history.
-user.financial.account_number,Financial Account Number,user.financial,"User's account number for a payment card, bank account, or other financial system."
-user.government_id,Government ID,user,State provided identification data.
-user.government_id.drivers_license_number,"Driver's License Number",user.government_id,State issued driving identification number.
-user.government_id.national_identification_number,National Identification Number,user.government_id,State issued personal identification number.
-user.government_id.passport_number,Passport Number,user.government_id,State issued passport data.
-user.biometric,Biometric Data,user,Encoded characteristics of a user.
-user.biometric_health,Biometric Health Data,user,"Encoded characteristics of a user's health."
-user.browsing_history,Browsing History,user,Content browsing history of a user.
-user.childrens,Children's Data,user,Data relating to children.
-user.date_of_birth,Date of Birth,user,"User's date of birth."
-user.demographic,Demographic Data,user,Demographic data about a user.
-user.gender,Gender,user,Gender of an individual.
-user.genetic,Genetic Data,user,Data about the genetic makeup provided by a user.
-user.health_and_medical,Health and Medical Data,user,"Health records or individual's personal medical information."
-user.job_title,Job Title,user,Professional data.
-user.location,Location Data,user,Records of the location of a user.
-user.name,Name,user,"User's real name."
-user.non_specific_age,Non-Specific Age,user,Age range data.
-user.media_consumption,Media Consumption Data,user,Media type consumption data of a user.
-user.observed,Observed Data,user,Data collected through observation of use of the system.
-user.organization,Organization Data,user,Data that is linked to, or identifies an organization.
-user.political_opinion,Political Opinion,user,"Data related to the individual's political opinions."
-user.profiling,Profiling Data,user,Preference and interest data about a user.
-user.race,Race,user,Racial or ethnic origin data.
-user.religious_belief,Religious Belief,user,Religion or religious belief.
-user.search_history,Search History,user,Records of search history and queries of a user.
-user.sensor,Sensor Data,user,Non-user identifiable measurement data derived from sensors and monitoring systems.
-user.sexual_orientation,Sexual Orientation,user,Personal sex life or sexual data.
-user.social,Social Data,user,Social activity and interaction data.
-user.telemetry,Telemetry Data,user,User measurement data from system sensors and monitoring.
-user.unique_id,Unique ID,user,Unique identifier for a user assigned through system use.
-user.user_sensor,User Sensor Data,user,Measurement data derived about a user's environment through system use.
-user.workplace,Workplace,user,Organization of employment.
+fides_key,is_default,name,organization_fides_key,parent_key,tags,description
+data_category,,Data Category,,,,
+system,True,System Data,default_organization,data_category,,"Data unique to, and under control of the system."
+system.authentication,True,Authentication Data,default_organization,system,,Data used to manage access to the system.
+system.operations,True,Operations Data,default_organization,system,,Data used for system operations.
+user,True,User Data,default_organization,data_category,,"Data related to the user of the system, either provided directly or derived based on their usage."
+user.payment,True,Payment Data,default_organization,user,,Payment data related to user.
+user.biometric,True,Biometric Data,default_organization,user,,Encoded characteristics provided by a user.
+user.biometric_health,True,Biometric Health Data,default_organization,user,,Encoded characteristic collected about a user.
+user.browsing_history,True,Browsing History,default_organization,user,,Content browsing history of a user.
+user.demographic,True,Demographic Data,default_organization,user,,Demographic data about a user.
+user.contact,True,Contact Data,default_organization,user,,Contact data collected about a user.
+user.device,True,Device Data,default_organization,user,,"Data related to a user's device, configuration and setting."
+user.gender,True,Gender,default_organization,user,,Gender of an individual.
+user.location,True,Location Data,default_organization,user,,Records of the location of a user.
+user.media_consumption,True,Media Consumption Data,default_organization,user,,Media type consumption data of a user.
+user.non_specific_age,True,Non-Specific Age,default_organization,user,,Age range data.
+user.observed,True,Observed Data,default_organization,user,,Data collected through observation of use of the system.
+user.profiling,True,Profiling Data,default_organization,user,,Preference and interest data about a user.
+user.race,True,Race,default_organization,user,,Racial or ethnic origin data.
+user.religious_belief,True,Religious Belief,default_organization,user,,Religion or religious belief.
+user.search_history,True,Search History,default_organization,user,,Records of search history and queries of a user.
+user.sexual_orientation,True,Sexual Orientation,default_organization,user,,Personal sex life or sexual data.
+user.social,True,Social Data,default_organization,user,,Social activity and interaction data.
+user.telemetry,True,Telemetry Data,default_organization,user,,User identifiable measurement data from system sensors and monitoring.
+user.unique_id,True,Unique ID,default_organization,user,,Unique identifier for a user assigned through system use.
+user.user_sensor,True,User Sensor Data,default_organization,user,,Measurement data about a user's environment through system use.
+user.organization,True,Organization Identifiable Data,default_organization,user,,"Data that is linked to, or identifies an organization."
+user.workplace,True,Workplace,default_organization,user,,Organization of employment.
+user.sensor,True,Sensor Data,default_organization,user,,Measurement data from sensors and monitoring systems.
+user.childrens,True,Children's Data,default_organization,user,,Data relating to children.
+user.credentials,True,Credentials,default_organization,user,,User authentication data.
+user.date_of_birth,True,Date of Birth,default_organization,user,,User's date of birth.
+user.financial,True,Financial Data,default_organization,user,,Payment data and financial history.
+user.genetic,True,Genetic Data,default_organization,user,,Data about the genetic makeup provided by a user.
+user.government_id,True,Government ID,default_organization,user,,State provided identification data.
+user.health_and_medical,True,Health and Medical Data,default_organization,user,,Health records or individual's personal medical information.
+user.job_title,True,Job Title,default_organization,user,,Professional data.
+user.name,True,Name,default_organization,user,,User's real name.
+user.political_opinion,True,Political Opinion,default_organization,user,,Data related to the individual's political opinions.
+user.contact.address,True,Contact Data,default_organization,user.contact,,Contact address data collected about a user.
+user.contact.email,True,User Contact Email,default_organization,user.contact,,User's contact email address.
+user.contact.phone_number,True,User Contact Phone Number,default_organization,user.contact,,User's phone number.
+user.contact.address.city,True,User Contact City,default_organization,user.contact.address,,User's city level address data.
+user.contact.address.country,True,User Contact Country,default_organization,user.contact.address,,User's country level address data.
+user.contact.address.postal_code,True,User Contact Postal Code,default_organization,user.contact.address,,User's postal code.
+user.contact.address.state,True,User Contact State,default_organization,user.contact.address,,User's state level address data.
+user.contact.address.street,True,User Contact Street,default_organization,user.contact.address,,User's street level address data.
+user.credentials.biometric_credentials,True,Biometric Credentials,default_organization,user.credentials,,Credentials for system authentication.
+user.credentials.password,True,Password,default_organization,user.credentials,,Password for system authentication.
+user.device.cookie_id,True,Cookie ID,default_organization,user.device,,Cookie unique identification number.
+user.device.device_id,True,Device ID,default_organization,user.device,,Device unique identification number.
+user.device.ip_address,True,IP Address,default_organization,user.device,,Unique identifier related to device connection.
+user.financial.account_number,True,User Financial Account Number,default_organization,user.financial,,"User's account number for a payment card, bank account, or other financial system."
+user.government_id.drivers_license_number,True,Driver's License Number,default_organization,user.government_id,,State issued driving identification number.
+user.government_id.national_identification_number,True,National Identification Number,default_organization,user.government_id,,State issued personal identification number.
+user.government_id.passport_number,True,Passport Number,default_organization,user.government_id,,State issued passport data.
+user.payment.financial_account_number,True,Account Payment Financial Account Number,default_organization,user.payment,,"Financial account number for an account's payment card, bank account, or other financial system."