Development (#51)

* SDPT v2 removal factor refactoring.

* adding calc spreadsheet, R script to reformat, and reformated v2 RF spreadsheet

* Working on rasterizing SDPT v2.

* Working on SDPT postgis preprocessing.

* Docker-compose build wasn't working; issue with postgres part. Trying to fix it.

* Working through plantation preparation script updates.

* Rasterizing 1x1 tiles from postgis essentially works-- minimal test.

* Cleaning up script. 1x1 tile creation part still seems to work.

* Cleaned up 1x1 tile creation function and it works. Haven't tested 10x10 tile creation function yet.

* 1x1 and 10x10 planted forest tile creation works on one test tile locally.

* Updated documentation and preprocessing code chunks.

* Testing rasterization on SDPT v2 with dummy RF and stdev on ec2.

* Testing rasterization on SDPT v2 with dummy RF and stdev on ec2.

* Testing rasterization on SDPT v2 with dummy RF and stdev on ec2.

* Testing rasterization on SDPT v2 with dummy RF and stdev on ec2.

* Testing rasterization on SDPT v2 with dummy RF and stdev on ec2.

* Testing rasterization on SDPT v2 with dummy RF and stdev on ec2.

* Testing rasterization on SDPT v2 with dummy RF and stdev on ec2.

* Testing rasterization on SDPT v2 with dummy RF and stdev on ec2.

* Testing rasterization on SDPT v2 with dummy RF and stdev on ec2.

* Testing rasterization on SDPT v2 with dummy RF and stdev on ec2.

* Testing rasterization on SDPT v2 with dummy RF and stdev on ec2.

* Testing rasterization on SDPT v2 with dummy RF and stdev on ec2.

* Testing rasterization on SDPT v2 with dummy RF and stdev on ec2.

* Testing rasterization on SDPT v2 with dummy RF and stdev on ec2.

* Testing rasterization on SDPT v2 with dummy RF and stdev on ec2.

* Various SDPT v2 changes, including paths.

* updated:
constants_and_names.py - added information for 6 updated SDPTv2 tile datasets found on gfw-data-lake
universal_util.py - modifed s3_file_download function to download SDPTv2 tile datasets from gfw-data-lake

* updated:
refactored all instances of pattern_gain_data_lake to pattern_data_lake
universal_util.py - modified s3_folder_download function for gfw-data-lake tile sets
mp_prep_other_inputs_one_off - updated code to download SDPTv2 tile datasets

* updated:
universal_util.py - modified rewindow and mp_warp_to_hansen functions for gfw-data-lake tile sets
mp_prep_other_inputs_one_off - updated code to rewindow, gdalwarp, and upload all SDPTv2 datasets to s3 storage. Also, modified to require a process number to run one specific one-off process at a time.
Successfully run for one test tile set (AGC removal factor). Currently running for the remaining datasets.

* updated:
mp_prep_other_inputs_one_off - added TODOs from David's comments and included planted forest establishment year for future runs.
Successfully ran (locally) for all tiles in each dataset.

* updated:
cleaned up the planted_forest_prep folder.

* Tier 1 RF Updates for Temperate Forests
updated constants_and_names.py:
- version changed to 1.2.4
- gain_spreadsheet changed to updated s3 file ending with the date 20230821
the following variables have been changed to updated s3 directories with the date 20230815: model_extent_dir, age_cat_IPCC_dir, annual_gain_AGB_IPCC_defaults_dir, annual_gain_BGB_IPCC_defaults_dir, annual_gain_AGC_all_types_dir, annual_gain_BGC_all_types_dir, annual_gain_AGC_BGC_all_types_dir, gain_year_count_dir, cumul_gain_AGCO2_all_types_dir, cumul_gain_BGCO2_all_types_dir, cumul_gain_AGCO2_BGCO2_all_types_dir, cumul_gain_AGCO2_BGCO2_all_types_forest_extent_dir, emis_pool_run_date, emis_run_date_biomass_soil, emis_run_date_soil_only, net_flux_dir, net_flux_forest_extent_dir, cumul_gain_AGCO2_BGCO2_all_types_per_pixel_full_extent_dir, cumul_gain_AGCO2_BGCO2_all_types_per_pixel_forest_extent_dir, gross_emis_all_gases_all_drivers_biomass_soil_per_pixel_full_extent_dir, gross_emis_all_gases_all_drivers_biomass_soil_per_pixel_forest_extent_dir, net_flux_per_pixel_full_extent_dir, net_flux_per_pixel_forest_extent_dir, output_aggreg_dir, stdev_annual_gain_AGB_IPCC_defaults_dir, stdev_annual_gain_AGC_all_types_dir

(cherry picked from commit b7277db)

* Tier 1 RF Updates for Temperate Forests
updated constants_and_names.py:
- removal_forest_type_dir changed to updated s3 directory with the date 20230815 (accidentally missed during previous commit)

(cherry picked from commit a392f43)

* Need to recreate soil C tiles because I accidentally deleted them.

* Need to recreate soil C tiles because I accidentally deleted them.

* Going to make new soil carbon folders.

* Going to make new soil carbon folders.

* Creating mangrove soil C tiles first.

* Creating mangrove soil C tiles first.

* Creating mangrove soil C tiles first.

* Creating mangrove soil C tiles first.

* Creating combined soil C now.

* Creating combined soil C now.

* Creating combined soil C now.

* Creating combined soil C now.

* Creating combined soil C now.

* Creating combined soil C now.

* Creating combined soil C now.

* Creating new total C in 2000 tiles.

* Total C 2000 tiles are too big without bigtiff.

* New Develop Branch:
- Merged SDPTv2 and IPCC Tier 1 feature branches together to create a new develop branch and successfully ran 4 tiles locally
- updated plantation_type in constants.h
- updated drivers_processed_dir with terminal /, pattern_annual_gain_AGC_BGC_planted_forest, pattern_stdev_annual_gain_AGC_BGC_planted_forest, and soil_C_full_extent_2000_dir in constants_and_names.py
- Updated cn.count threshold from 2 to 12 for local processing in mp_annual_gain_rate_AGC_BGC_all_forest_types.py and mp_gross_removals_all_forest_types.py
- Updated out_pattern from '_Mt_per_year' to '_Mt_per_year_' in universal_util.py

* the following variables have been changed to updated s3 directories with the date 20231109: model_extent_dir, age_cat_IPCC_dir, annual_gain_AGB_IPCC_defaults_dir, annual_gain_BGB_IPCC_defaults_dir, annual_gain_AGC_all_types_dir, annual_gain_BGC_all_types_dir, annual_gain_AGC_BGC_all_types_dir, removal_forest_type_dir, gain_year_count_dir, cumul_gain_AGCO2_all_types_dir, cumul_gain_BGCO2_all_types_dir, cumul_gain_AGCO2_BGCO2_all_types_dir, cumul_gain_AGCO2_BGCO2_all_types_forest_extent_dir, emis_pool_run_date, emis_run_date_biomass_soil, emis_run_date_soil_only, net_flux_dir, net_flux_forest_extent_dir, cumul_gain_AGCO2_BGCO2_all_types_per_pixel_full_extent_dir, cumul_gain_AGCO2_BGCO2_all_types_per_pixel_forest_extent_dir, gross_emis_all_gases_all_drivers_biomass_soil_per_pixel_full_extent_dir, gross_emis_all_gases_all_drivers_biomass_soil_per_pixel_forest_extent_dir, net_flux_per_pixel_full_extent_dir, net_flux_per_pixel_forest_extent_dir, output_aggreg_dir, stdev_annual_gain_AGB_IPCC_defaults_dir, stdev_annual_gain_AGC_all_types_dir

* Changed branch name and version number in constants_and_names.py

* Making new soil C uncertainty tiles.

* EC2 Select Tile Run Update:
- Ran 4 tiles on an EC2 instance and fixed the following issues:
- Updated the path for planted forest removal factors from annual_gain_AGC_BGC_planted_forest_dir = os.path.join(s3_base_dir, f'gfw2-data/climate/carbon_model/annual_removal_factor_planted_forest/{planted_forest_version}_AGC_BGC/{planted_forest_output_date}/')
to annual_gain_AGC_BGC_planted_forest_dir = os.path.join(s3_base_dir, f'annual_removal_factor_planted_forest/{planted_forest_version}_AGC_BGC/{planted_forest_output_date}/')
- Updated soil tile patterns from pattern_soil_C_full_extent_2000_non_mang = 'soil_C_ha_full_extent_2000_non_mangrove_Mg_ha' to pattern_soil_C_full_extent_2000_non_mang = 'soil_C_full_extent_2000_non_mangrove_Mg_C_ha' and pattern_soil_C_full_extent_2000 = 't_soil_C_ha_full_extent_2000' to pattern_soil_C_full_extent_2000 = 'soil_C_full_extent_2000_Mg_C_ha'

* EC2 Select Tile Run Update:
- Ran 4 tiles on an EC2 instance and fixed the following issues for the create_derivative_outputs stage:
- changed dates for input tile sets from 20231109 to 20238888: cumul_gain_AGCO2_all_types_dir, cumul_gain_BGCO2_all_types_dir, cumul_gain_AGCO2_BGCO2_all_types_dir, cumul_gain_AGCO2_BGCO2_all_types_forest_extent_dir, emis_run_date_soil_only, and net_flux_dir.

* EC2 Select Tile Run Update:
- Ran 4 tiles on an EC2 instance and fixed the following issues for the create_derivative_outputs stage:
- changed dates for input tile sets from 20231109 to 20238888: emis_run_date_biomass_soil

* EC2 Full Model Run Update:
- Ran 4 tiles on an EC2 instance and successfully completed the create_derivative_outputs stage
- Updated output dates in constants_and_names.py from 20238888 to 20231114 and from 20231109 to 20231114

* Updates:
- Added CarbonFluxQA code in analyses folder to calculate ArcGIS zonal stats for model output
- Fixed a comment typo in model_extent.py

* Making new soil C uncertainty tiles.

* Adding updated CarbonFluxQA folder in analyses folder and removing hidden .git file (copied over from CarbonFluxQA repo)

---------

Co-authored-by: dagibbs22 <[email protected]>

Dockerfile

@@ -30,6 +30,7 @@ RUN apt-get update -y && apt-get install -y \
     tmux \
     git \
     vim \
+    zip \
     && apt-get clean all
 
 

analyses/CarbonFluxQA/Components/01_DownloadFiles.py

@@ -0,0 +1,4 @@
+from funcs import download_files
+
+# Create folder structure, download files, and clip TCL rasters to GADM extent
+download_files()

analyses/CarbonFluxQA/Components/02_CreateMasks.py

@@ -0,0 +1,9 @@
+import constants_and_names as cn
+from funcs import create_masks
+
+# Set input folders = Mask, Inputs folders and select tcd_threshold/ gain/ save intermediate values
+create_masks(cn.tcd_threshold, cn.gain, cn.save_intermediates)
+
+# Other options:
+# create_masks([0, 75], cn.gain, False)
+# create_masks([30], cn.gain, True)

analyses/CarbonFluxQA/Components/03_ZonalStats_Masked.py

@@ -0,0 +1,5 @@
+import constants_and_names as cn
+from funcs import zonal_stats_masked
+
+# Calculate zonal stats for all input rasters at each tcd threshold value
+zonal_stats_masked(cn.aois_folder, cn.input_folder, cn.mask_output_folder, cn.outputs_folder)

analyses/CarbonFluxQA/Components/04_ZonalStats_Annualized.py

@@ -0,0 +1,6 @@
+import constants_and_names as cn
+from funcs import zonal_stats_annualized
+
+# Calculate emissions for each year of tree cover loss using TCL rasters
+zonal_stats_annualized(cn.tcl_clip_folder, cn.input_folder, cn.mask_output_folder, cn.annual_folder)
+

analyses/CarbonFluxQA/Components/05_ZonalStats_Clean.py

@@ -0,0 +1,4 @@
+from funcs import zonal_stats_clean
+
+#Combine and clean masked and annual zonal stats output into final csv
+zonal_stats_clean()

analyses/CarbonFluxQA/README.md

@@ -0,0 +1,157 @@
+### Purpose
+
+    The purpose of this tool is to automate the zonal statistics portion of the QAQC process for the
+    Annual Carbon Flux Model (Harris, 2021) Update. Each year, the Carbon Flux Model is run with updated activity data
+    for tree cover loss (Hansen, 2013) and auxiliary inputs. Before the data is launched on Global Forest Watch, the
+    outputs are compared across platforms and methods including the GeoTrellis Tool, GFW Dashboard download spreadsheets,
+    the GFW API, and ArcGIS Zonal Statistics calculations. 
+
+### Overview
+
+    This code has been automated to create the working directory folder structure, download all of the required data 
+    from s3, and produce a summary csv with the resulting zonal statisitcs. 
+
+### User Inputs
+
+#### Area(s) of Interest:
+
+    The only file(s) required by the user are the shapefile(s) for the area(s) of interest. The shapefile(s) need to be 
+    located in a subfolder in the woking directory named "AOIS".
+
+    This tool is set up to run statistics for as many areas of interest as the user provides. We used country 
+    boundaries for Indonesia and Gambia from the GADM 3.6 Dataset, available for download here:
+    [](https://gadm.org/download_country_v3.html).
+
+    The Indonesia boundary is IND.14.13 and the Gambia boundary is GMB.2. 
+
+    These inputs will need to be updated if and when GFW switches to a newer version of GADM. 
+
+| Dataset             | Directory                 | Description                             |
+|---------------------|---------------------------|-----------------------------------------|
+| Area(s) of Interest | {working_directory}/AOIS/ | Shapefiles for the area(s) of interest. |
+
+
+#### User-Specified Parameters: 
+    You must update the constants_and_names.py file with the path to your working_directory. This is the folder which 
+    contains your areas of interest and where all of the data and results will be saved. There are a number of other 
+    arguments in the constants_and_names.py file that users have the option to update. A description of each argument is 
+    detailed below:  
+
+| Argument                | Description                                                                  | Type       |
+|-------------------------|------------------------------------------------------------------------------|------------|
+| working_directory       | Directory which contains the AOIS subfolder.                                 | String     |
+| overwrite_arcgis_output | Whether or not you want to overwrite previous arcpy outputs.                 | Boolean    |
+| loss_years              | Number of years of tree cover loss in the TCL dataset.                       | Integer    |
+| model_run_date          | s3 folder where per-pixel outputs from most recent model run are located.    | String     |
+| tile_list               | List of 10 x 10 degree tiles that overlap with all aoi(s).                   | List       |
+| tile_dictionary         | Dictionary that matches each country to their overlapping tiles.             | Dictionary |
+| extent                  | Which tile set(s) to download for zonal stats (options: forest, full, both). | String     |
+| tcd_threshold           | List of tree cover density thresholds to mask by.                            | List       |
+| gain                    | Whether to include tree cover gain pixels in masks.                          | Boolean    |
+| save_intermediates      | Whether to save intermediate masks (useful for troubleshooting).             | Boolean    |
+
+
+### Datasets
+
+#### Carbon Flux Model Data:
+
+    Three separate outputs from the Carbon Flux Model, each with two different extents, are used as inputs in 
+    this tool. This is a total of six different possible inputs. Inputs include gross emissions (all gasses), 
+    gross removals (CO2), and net flux (CO2e). All are in inputs Mg / pixel. You have the option to calculate 
+    zonal statistics according to tile extent: forest extent only, full extent only, or both extents.
+
+| AOI | Extent | Type            | Units         | Tile     |
+|-----|--------|-----------------|---------------|----------|
+| IDN | Forest | Gross Emissions | Mg CO2e/pixel | 00N_110E |
+| IDN | Forest | Gross Removals  | Mg CO2/pixel  | 00N_110E |
+| IDN | Forest | Net Flux        | Mg CO2e/pixel | 00N_110E |
+| IDN | Full   | Gross Emissions | Mg CO2e/pixel | 00N_110E |
+| IDN | Full   | Gross Removals  | Mg CO2/pixel  | 00N_110E |
+| IDN | Full   | Net Flux        | Mg CO2e/pixel | 00N_110E |
+| GMB | Forest | Gross Emissions | Mg CO2/pixel  | 20N_020W |
+| GMB | Forest | Gross Removals  | Mg CO2e/pixel | 20N_020W |
+| GMB | Forest | Net Flux        | Mg CO2/pixel  | 20N_020W |
+| GMB | Full   | Gross Emissions | Mg CO2e/pixel | 20N_020W |
+| GMB | Full   | Gross Removals  | Mg CO2/pixel  | 20N_020W |
+| GMB | Full   | Net Flux        | Mg CO2e/pixel | 20N_020W |
+
+
+#### Auxiliary Datasets:
+
+    Other auxiliary inputs for this tool include:
+
+| Dataset              | Use Description                                                                          |
+|----------------------|------------------------------------------------------------------------------------------|
+| Tree Cover Gain      | Used to create tree cover gain mask.                                                     |
+| Above Ground Biomass | Used to filter tree cover gain mask to only pixels that contain biomass.                 |
+| Tree Cover Density   | Used to create density threshold mask.                                                   |
+| Mangrove Extent      | Used to create Mangrove mask. Areas of mangrove included in mask.                        |
+| Pre-2000 Plantations | Used to create Pre-2000 plantations mask. Pre-2000 plantations masked from calculations. |
+| Tree Cover Loss      | Used to calculate annual emissions.                                                      |
+
+
+### Outputs:
+
+    The final outputs include one csv file summarizing results for each entry described in the "Carbon Flux Model 
+    Inputs" table. Additionally, separate csv files for annual emissions are produced. 
+
+### Code Summary
+
+#### calculate_zonal_statistics
+    This file is for running the code in its entirety. This script will execute all functions in the repository 
+    consecutively and produce output csvs.
+
+#### constants_and_names
+    This file stores all of the input arguments provided to the functions. Any changes to the arguments in this file 
+    will be applies to all scripts in the repository. 
+
+#### funcs
+    This file stores all of the functions used in the tool. Any edits to functions would be made in this file.
+
+#### components
+
+    This folder houses individual scripts for running separate functions. These can be useful for running particular 
+    functions separately and testing edits/ troubleshootins. Each function is described below. 
+
+##### 01 Download Files
+    This script creates the folder structure (other than the AOI folder) and downloads all of the required datasets from 
+    s3 using the paths provided in the the constant_and_names file. You will need to set your AWS_ACCESS_KEY and 
+    AWS_SECRET_ACCESS_KEY in your environment variables for this step to work (assuming you have s3 copy permissions).
+
+##### 02 Create Masks
+    This script uses data on tree cover density, tree cover gain, mangrove extent, WHRC biomass, and pre-2000 plantations
+    to replicate the masks that are used in GFW data processing. This facilitates direct comparison with results from the GFW 
+    dashboard, geotrellis client, and GFW API. The script creates masks based on criteria for each input dataset and saves these
+    masks in a sub directory. These masks are used later as extent inputs in the Zonal Statistics Masked script.
+
+##### 03 Zonal Stats Masked
+    This script calculates zonal statistics for each area of interest and carbon dataset combination and applies each  
+    mask saved in the Mask/ Mask directory. The number of masks depend on the number of tcd_threshold values you indicated 
+    and wheter or not you set the save_intermediate flag to True. At minimum, this will include final masks for each 
+    tcd_threshold value and at maximum this will be the number of tcd_threshold values multiplied by the number of 
+    intermediate masks (this varies depending on whether or not the area of interest includes mangroves and/ or 
+    pre-2000 plantations). 
+
+##### 04 Zonal Stats Annualized
+    This script calculates annual emissions in each area of interest using the tree cover loss dataset. 
+
+##### 05 Zonal Stats Cleaned
+    This script utilizes pandas to compile the results of all analyses and export them into a user-friendly csv file.
+
+### Running the Code
+    To run the code, you will need to set up a workspace with the AOI inputs organized into the correct directory, 
+    update the user inputs section of the constants_and_names.py file, and provide your AWS_ACCESS_KEY and 
+    AWS_SECRET_ACCESS_KEY in your environment variables. 
+
+    This code is built on arcpy, which will require a valid ArcGIS license to run.
+
+### Other Notes
+    Updates in progress include...
+
+    - Currently, the annual zonal stats do not sum to the total emissions using the TCL dataset from s3, 
+    but they do when using the previous TCL clipped rasters. For now, reach out to Erin Glen or Melissa Rose for 
+    the previous TCL clipped rasters. 
+
+#### Contact Info
+    Erin Glen - [email protected]
+    Melissa Rose - [email protected]

analyses/CarbonFluxQA/calculcate_zonal_stats.py

@@ -0,0 +1,22 @@
+import constants_and_names as cn
+from funcs import download_files, create_masks, zonal_stats_masked, zonal_stats_annualized, zonal_stats_clean
+
+#Execute Download File...
+print("Step 1: Downloading Files... \n")
+download_files()
+
+#Execute Create Masks...
+print("Step 2: Creating Masks... \n")
+create_masks(cn.tcd_threshold, cn.gain, cn.save_intermediates)
+
+#Execute Calculate Zonal Stats Masked...
+print("Step 3: Calculating Zonal Stats with Masks... \n")
+zonal_stats_masked(cn.aois_folder, cn.input_folder, cn.mask_output_folder, cn.outputs_folder)
+
+#Execute Calculcate Zonal Stats Annualized...
+print("Step 4: Calculating Zonal Stats Annualized... \n")
+zonal_stats_annualized(cn.tcl_clip_folder, cn.input_folder, cn.mask_output_folder, cn.annual_folder)
+
+#Execute Zonal Stats Clean...
+print("Step 5: Cleaning Zonal Stats... \n")
+zonal_stats_clean()

analyses/CarbonFluxQA/constants_and_names.py

@@ -0,0 +1,115 @@
+import arcpy
+import os
+
+#####################################################################################
+# USER INPUTS
+#####################################################################################
+# Set the working directory to the folder which contains the AOIS subfolder
+working_directory = r"C:\GIS\carbon_model\CarbonFlux_QA_v1.3.1"
+
+# Whether you want to overwrite previous arcpy outputs
+overwrite_arcgis_output = True
+
+# With each model update, change loss years and model_run_date
+    # loss_years = number of years of tree cover loss (if input loss raster is changed, this must be changed, too)
+    # model_run_date = s3 folder where per-pixel outputs from most recent model run are located
+loss_years = 22
+model_run_date = '20231114'
+
+# List of tile_ids to process (change according to which tiles overlap with your AOIS shapefiles)
+tile_list = ['00N_110E', '20N_020W']
+
+# Dictionary to cross-reference countries to their tile_id and GADM boundaries
+tile_dictionary = {"IDN": "00N_110E",
+                   "GMB": "20N_020W"}
+
+# Choose which extent to use for emission, removal, and net flux zonal stats
+    # options = 'forest', 'full', or 'both'
+extent = 'full'
+
+# List of tree cover density thresholds to mask by
+tcd_threshold = [0, 30, 75]
+gain = True
+
+# Flag to save intermediate masks during create_masks()
+save_intermediates = False
+
+#####################################################################################
+# DEFAULT INPUTS
+#####################################################################################
+
+# Setting the arcpy environ workspace
+arcpy.env.workspace = working_directory
+arcpy.env.overwriteOutput = overwrite_arcgis_output
+
+# Directories to be created/ checked
+aois_folder = os.path.join(arcpy.env.workspace,"AOIS")
+input_folder = os.path.join(arcpy.env.workspace,"Input")
+mask_folder = os.path.join(arcpy.env.workspace,"Mask")
+mask_input_folder = os.path.join(arcpy.env.workspace,"Mask", "Inputs")
+mask_output_folder = os.path.join(arcpy.env.workspace,"Mask", "Mask")
+gain_folder = os.path.join(mask_input_folder, "Gain")
+mangrove_folder = os.path.join(mask_input_folder, "Mangrove")
+plantations_folder = os.path.join(mask_input_folder, "Pre_2000_Plantations")
+tcd_folder = os.path.join(mask_input_folder, "TCD")
+whrc_folder = os.path.join(mask_input_folder, "WHRC")
+outputs_folder = os.path.join(arcpy.env.workspace, "Outputs")
+csv_folder = os.path.join(outputs_folder, "CSV")
+annual_folder = os.path.join(outputs_folder, "Annual")
+tcl_folder = os.path.join(arcpy.env.workspace, "TCL")
+tcl_input_folder = os.path.join(tcl_folder, "Inputs")
+tcl_clip_folder = os.path.join(tcl_folder, "Clip")
+
+# Filepath prefix for tile download step
+s3_base_dir = 's3://gfw2-data/climate/carbon_model/'
+
+## Input folder s3 filepath informaiton
+# Gross emissions per pixel in forest extent
+gross_emis_forest_extent_s3_path = os.path.join(s3_base_dir, f'gross_emissions/all_drivers/all_gases/biomass_soil/standard/forest_extent/per_pixel/{model_run_date}/')
+gross_emis_forest_extent_s3_pattern = f'gross_emis_all_gases_all_drivers_Mg_CO2e_pixel_biomass_soil_forest_extent_2001_{loss_years}'
+
+# Gross emissions per pixel in all pixels
+gross_emis_full_extent_s3_path = os.path.join(s3_base_dir, f'gross_emissions/all_drivers/all_gases/biomass_soil/standard/full_extent/per_pixel/{model_run_date}/')
+gross_emis_full_extent_s3_pattern = f'gross_emis_all_gases_all_drivers_Mg_CO2e_pixel_biomass_soil_full_extent_2001_{loss_years}'
+
+# Gross removals per pixel in forest extent
+gross_removals_forest_extent_s3_path = os.path.join(s3_base_dir, f'gross_removals_AGCO2_BGCO2_all_forest_types/standard/forest_extent/per_pixel/{model_run_date}/')
+gross_removals_forest_extent_s3_pattern = f'gross_removals_AGCO2_BGCO2_Mg_pixel_all_forest_types_forest_extent_2001_{loss_years}'
+
+# Gross removals per pixel in all pixels
+gross_removals_full_extent_s3_path = os.path.join(s3_base_dir, f'gross_removals_AGCO2_BGCO2_all_forest_types/standard/full_extent/per_pixel/{model_run_date}/')
+gross_removals_full_extent_s3_pattern = f'gross_removals_AGCO2_BGCO2_Mg_pixel_all_forest_types_full_extent_2001_{loss_years}'
+
+# Net flux per pixel in forest extent
+netflux_forest_extent_s3_path = os.path.join(s3_base_dir, f'net_flux_all_forest_types_all_drivers/biomass_soil/standard/forest_extent/per_pixel/{model_run_date}/')
+netflux_forest_extent_s3_pattern = f'net_flux_Mg_CO2e_pixel_biomass_soil_forest_extent_2001_{loss_years}'
+
+# Net flux per pixel in all pixels
+netflux_full_extent_s3_path = os.path.join(s3_base_dir, f'net_flux_all_forest_types_all_drivers/biomass_soil/standard/full_extent/per_pixel/{model_run_date}/')
+netflux_full_extent_s3_pattern = f'net_flux_Mg_CO2e_pixel_biomass_soil_full_extent_2001_{loss_years}'
+
+## Mask, Inputs folder s3 filepath informaiton
+# Hansen removals tiles based on canopy height (2000-2020)
+gain_s3_path = 's3://gfw-data-lake/umd_tree_cover_gain_from_height/v202206/raster/epsg-4326/10/40000/gain/geotiff/'
+gain_s3_pattern = ''
+gain_local_pattern = 'tree_cover_gain_2000_2020'
+
+# Woods Hole aboveground biomass 2000 version 4 tiles
+whrc_s3_path = 's3://gfw2-data/climate/WHRC_biomass/WHRC_V4/Processed/'
+whrc_s3_pattern = "t_aboveground_biomass_ha_2000"
+
+# Tree cover density 2000 tiles
+tcd_s3_path = 's3://gfw2-data/forest_cover/2000_treecover/'
+tcd_s3_pattern = 'Hansen_GFC2014_treecover2000'
+
+# Processed mangrove aboveground biomass in the year 2000
+mangrove_s3_path = os.path.join(s3_base_dir, 'mangrove_biomass/processed/standard/20190220/')
+mangrove_s3_pattern = 'mangrove_agb_t_ha_2000'
+
+# Pre-2000 plantations
+plantation_s3_path = os.path.join(s3_base_dir, 'other_emissions_inputs/IDN_MYS_plantation_pre_2000/processed/20200724/')
+plantation_s3_pattern = 'plantation_2000_or_earlier_processed'
+
+# Annual Hansen loss tiles (2001-2022)
+loss_s3_path = 's3://gfw2-data/forest_change/hansen_2022/'
+loss_s3_pattern = 'GFW2022'