Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add capability to handle tmd input #2740

Merged
merged 34 commits into from
May 7, 2024
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
6dc7b7a
Add capability to handle tmd.csv input file
martinholmer Apr 18, 2024
c1ecae6
Add nocover codecov pragmas to tmd statements in two modules
martinholmer Apr 18, 2024
81472e8
Add tmd_weights.csv.gz to MANIFEST.in
martinholmer May 2, 2024
bb0132c
Add taxcalc/tmd_weights.csv.gz file
martinholmer May 2, 2024
87aecb7
Fix use of Records.tmd_constructor static function
martinholmer May 2, 2024
1c6ec0d
Revise tmd_weights.csv.gz to include integer scaled weight values
martinholmer May 2, 2024
975b5a4
Install more-precise weights
martinholmer May 3, 2024
7ffa774
Simplify Records.tmd_construction static function logic
martinholmer May 3, 2024
b3ccad7
Install new tmd weights and growfactors files
martinholmer May 4, 2024
f84f9fa
Add using_tmd arguments to grow-related constructors
martinholmer May 4, 2024
fab6bd6
Simplify GrowFactors contructor
martinholmer May 4, 2024
0e4f8e5
Add using_tmd argument to GrowDiff constructor
martinholmer May 4, 2024
6b60ef7
More revisions to growfactors.py and growdiff.py code
martinholmer May 4, 2024
1b027d8
Revise test_growfactors.py to reflect new constructor argument
martinholmer May 4, 2024
536142f
Rollback recent growfactors/growdiff changes on this branch
martinholmer May 5, 2024
012ee76
Install new thru-2034 tmd_weights.csv.gz and growfactors.csv files
martinholmer May 5, 2024
da99dde
Fix typos in growfactors.py
martinholmer May 5, 2024
01a8d71
Install thru-2074 growfactors and tmd_weights; make params go thru 2074
martinholmer May 5, 2024
37304b8
Fix two test failures
martinholmer May 5, 2024
8c0eed3
Revise some policy tests to make them execute faster
martinholmer May 5, 2024
3076408
Disable pytest_sessionfinish in tests/conftest.py
martinholmer May 5, 2024
9dce222
Fix typo in test_puf_var_stats.py
martinholmer May 5, 2024
6053d49
Fix typo in test_policy.py
martinholmer May 6, 2024
9088a1e
Add custom marks to pytest.ini
martinholmer May 6, 2024
b583be6
Make environment.yml requirements consistent with conda.recipe/meta.yaml
martinholmer May 6, 2024
1cd7e3d
Merge in recent changes on master branch
martinholmer May 6, 2024
d53fd3e
Streamline pytest test timing logic
martinholmer May 6, 2024
b1cd909
Reorder markers in pytest.ini file
martinholmer May 6, 2024
7f64089
Remove obsolete .github/workflows/build_new_test_benchmark.yml
martinholmer May 6, 2024
75d7b81
Add test_stats_benchmark.csv from master branch
martinholmer May 6, 2024
8519a6c
Update .github/workflows/build_and_test.yml file
martinholmer May 6, 2024
7906d3d
Change 2074 to 2034 in *.json and *.py files
martinholmer May 7, 2024
f215b09
Remove obsolete taxcalc/tests/test_stats_benchmark.csv file
martinholmer May 7, 2024
6a869c6
Add back the test_stats_benchmark.csv file
martinholmer May 7, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ read-the-docs/_build/

# IRS-SOI PUF data file
puf.csv
tmd.csv

# Jupyter notebook checkpoints
*.ipynb_checkpoints*
Expand Down
2 changes: 1 addition & 1 deletion MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@ include taxcalc/policy_current_law.json
include taxcalc/puf_weights.csv.gz
include taxcalc/puf_ratios.csv
include taxcalc/records_variables.json

include taxcalc/tmd_weights.csv.gz
8 changes: 4 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -51,19 +51,19 @@ endef
.PHONY=pytest-cps
pytest-cps:
@$(pytest-setup)
@cd taxcalc ; pytest -n4 --disable-warnings -m "not requires_pufcsv and not pre_release"
@cd taxcalc ; pytest -n4 --disable-warnings -m "not requires_pufcsv and not requires_tmdcsv and not pre_release"
@$(pytest-cleanup)

.PHONY=pytest
pytest:
@$(pytest-setup)
@cd taxcalc ; pytest -n4 -m "not pre_release"
@cd taxcalc ; pytest -n4 --disable-warnings -m "not pre_release"
@$(pytest-cleanup)

.PHONY=pytest-all
pytest-all:
@$(pytest-setup)
@cd taxcalc ; pytest -n4 -m ""
@cd taxcalc ; pytest -n4 --disable-warnings -m ""
@$(pytest-cleanup)

define tctest-cleanup
Expand Down Expand Up @@ -103,7 +103,7 @@ define coverage-cleanup
rm -f .coverage htmlcov/*
endef

COVMARK = "not requires_pufcsv and not pre_release"
COVMARK = "not requires_pufcsv and not requires_tmdcsv and not pre_release"

OS := $(shell uname -s)

Expand Down
16 changes: 8 additions & 8 deletions conda.recipe/meta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@ build:

requirements:
build:
- "python<3.12"
- "numpy>=1.14"
- "pandas>=1.2.0"
- "bokeh>=1.4.0, <3.0.0"
- "python>=3.9, <3.12"
- "numpy>=1.20, <2.0"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it important to provide an upper bound on the versions for certain packages, like numpy and pandas? Historically, Tax-Calculator has tried to remain compatible with the most recent versions of the dependencies.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are several deprecation warnings in the code (mostly related to pandas) that say the current code will cause errrors when pandas 3.0 is used. I check the pandas web site and they said 3.0 would be released any day now.

So, the limits are temporary. They will be removed when the deprecated code gets fixed.

- "pandas>=2.0, <3.0"
- "bokeh>=1.4, <3.0"
- requests
- numba
- "paramtools>=0.18.0"
Expand All @@ -21,10 +21,10 @@ requirements:
- openpyxl

run:
- "python<3.12"
- "numpy>=1.14"
- "pandas>=1.2.0"
- "bokeh>=1.4.0, <3.0.0"
- "python>=3.9, <3.12"
- "numpy>=1.20, <2.0"
- "pandas>=2.0, <3.0"
- "bokeh>=1.4, <3.0"
- requests
- numba
- "paramtools>=0.18.0"
Expand Down
2 changes: 1 addition & 1 deletion docs/usage/tcja_after_2025.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ expire). Compatible input datasets include:
[taxdata](https://github.com/PSLmodels/taxdata) repository and
available only to those with access to the 2011 IRS/SOI PUF

* several newer CSV-formatted input files created in the PSLmodels
* the newer `tmd.csv` file generated in the PSLmodels
[tax-microdata](https://github.com/PSLmodels/tax-microdata-benchmarking)
repository that are based on the 2015 IRS/SOI PUF and on more recent CPS
data, and are available only to those with access to the 2015
Expand Down
14 changes: 7 additions & 7 deletions environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,22 @@ name: taxcalc-dev
channels:
- conda-forge
dependencies:
- "python<3.12"
- curl
- "numpy>=1.14"
- "pandas>=1.2.0"
- "bokeh>=1.4.0, <3.0.0"
- "python>=3.9, <3.12"
- "numpy>=1.20, <2.0"
- "pandas>=2.0, <3.0"
- "bokeh>=1.4, <3.0"
- requests
- aiohttp
- numba
- "paramtools>=0.18.0"
- "fsspec<=0.8.7"
- aiohttp
- curl
- pytest
- pytest-pep8
- pytest-xdist
- pycodestyle
- pylint
- coverage
- "paramtools>=0.18.0"
- behresp
- openpyxl
- pip
Expand Down
6 changes: 6 additions & 0 deletions pytest.ini
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,15 @@ testpaths =
taxcalc
markers =
requires_pufcsv
requires_tmdcsv
pre_release
local
compatible_data
benefits
itmded_vars
pep8
cpscsv_agg
pufcsv_agg
rtr
reforms
extend_tcja
1 change: 1 addition & 0 deletions taxcalc.egg-info/SOURCES.txt
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ taxcalc/puf_weights.csv.gz
taxcalc/records.py
taxcalc/records_variables.json
taxcalc/taxcalcio.py
taxcalc/tmd_weights.csv.gz
taxcalc/utils.py
taxcalc/utilsprvt.py
taxcalc.egg-info/PKG-INFO
Expand Down
6 changes: 5 additions & 1 deletion taxcalc/cli/tc.py
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,11 @@ def cli_tc_main():
sys.stderr.write(tcio.errmsg)
sys.stderr.write('USAGE: tc --help\n')
return 1
aging = inputfn.endswith('puf.csv') or inputfn.endswith('cps.csv')
aging = (
inputfn.endswith('puf.csv') or
inputfn.endswith('cps.csv') or
inputfn.endswith('tmd.csv')
)
tcio.init(input_data=inputfn, tax_year=taxyear,
baseline=args.baseline,
reform=args.reform, assump=args.assump,
Expand Down
4 changes: 2 additions & 2 deletions taxcalc/consumption.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
"validators": {
"range": {
"min": 2013,
"max": 2034
"max": 2074
}
}
},
Expand Down Expand Up @@ -282,4 +282,4 @@
}
}
}
}
}
2 changes: 1 addition & 1 deletion taxcalc/data.py
Original file line number Diff line number Diff line change
Expand Up @@ -276,6 +276,6 @@ def _read_weights(self, weights):

def _extrapolate(self, year):
"""
Apply to dats variables the growth factor values for specified year.
Apply to data variables the growth factor values for specified year.
"""
# Override this empty method in subclass
4 changes: 2 additions & 2 deletions taxcalc/growdiff.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
"validators": {
"range": {
"min": 2013,
"max": 2034
"max": 2074
}
}
},
Expand Down Expand Up @@ -500,4 +500,4 @@
}
}
}
}
}
12 changes: 9 additions & 3 deletions taxcalc/growdiff.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
import os
import numpy as np
from taxcalc.parameters import Parameters
from taxcalc.policy import Policy
from taxcalc.growfactors import GrowFactors


Expand All @@ -26,8 +27,8 @@ class GrowDiff(Parameters):
class instance: GrowDiff
"""

JSON_START_YEAR = 2013 # must be same as Policy.JSON_START_YEAR
DEFAULT_NUM_YEARS = 22 # must be same as Policy.DEFAULT_NUM_YEARS
JSON_START_YEAR = Policy.JSON_START_YEAR
DEFAULT_NUM_YEARS = Policy.DEFAULT_NUM_YEARS
DEFAULTS_FILE_NAME = 'growdiff.json'
DEFAULTS_FILE_PATH = os.path.abspath(os.path.dirname(__file__))

Expand Down Expand Up @@ -75,8 +76,13 @@ def apply_to(self, growfactors):
"""
assert isinstance(growfactors, GrowFactors)
for gfvn in GrowFactors.VALID_NAMES:
_gfvn = '_{}'.format(gfvn)
_gfvn = f'_{gfvn}'
for i in range(0, self.num_years):
cyr = i + self.start_year
diff_array = getattr(self, _gfvn)
growfactors.update(gfvn, cyr, diff_array[i])

def set_rates(self):
"""
Unimplemented base class method that is not used here.
"""
Loading
Loading