Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add data source tests to CI for MSN and Yahoo (closes #59) #84

Merged
merged 6 commits into from
Jul 28, 2024
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .github/workflows/python-app.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,5 +32,4 @@ jobs:
pip install pytest==8.2.1
pip install -e .
- name: Test with pytest
run: |
python run_all_tests.py && pytest
run: pytest
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this run all the tests still? Does the rename of the tests fix this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With run_all_tests.py I see 18 tests being run:

python3 run_all_tests.py
test_parse_pe_ratios_should_properly_calculate_pe_ratios (test_MSNMoney.MSNMoneyTest.test_parse_pe_ratios_should_properly_calculate_pe_ratios) ... ok
test_parse_pe_ratios_should_return_false_if_too_few_pe_ratios (test_MSNMoney.MSNMoneyTest.test_parse_pe_ratios_should_return_false_if_too_few_pe_ratios) ... ok
test_parse_pe_ratios_should_return_false_when_no_data (test_MSNMoney.MSNMoneyTest.test_parse_pe_ratios_should_return_false_when_no_data) ... ok
test_calculate_estimated_future_price (test_RuleOneInvestingCalculations.RuleOneInvestingCalculationsTest.test_calculate_estimated_future_price) ... ok
test_calculate_future_eps (test_RuleOneInvestingCalculations.RuleOneInvestingCalculationsTest.test_calculate_future_eps) ... ok
test_calculate_future_pe (test_RuleOneInvestingCalculations.RuleOneInvestingCalculationsTest.test_calculate_future_pe) ... ok
test_calculate_margin_of_safety (test_RuleOneInvestingCalculations.RuleOneInvestingCalculationsTest.test_calculate_margin_of_safety) ... ok
test_calculate_roic (test_RuleOneInvestingCalculations.RuleOneInvestingCalculationsTest.test_calculate_roic) ... ok
test_calculate_sticker_price (test_RuleOneInvestingCalculations.RuleOneInvestingCalculationsTest.test_calculate_sticker_price) ... ok
test_compound_annual_growth_rate_both_negative_decrease (test_RuleOneInvestingCalculations.RuleOneInvestingCalculationsTest.test_compound_annual_growth_rate_both_negative_decrease) ... ok
test_compound_annual_growth_rate_both_negative_increase (test_RuleOneInvestingCalculations.RuleOneInvestingCalculationsTest.test_compound_annual_growth_rate_both_negative_increase) ... ok
test_compound_annual_growth_rate_decrease (test_RuleOneInvestingCalculations.RuleOneInvestingCalculationsTest.test_compound_annual_growth_rate_decrease) ... ok
test_compound_annual_growth_rate_increase (test_RuleOneInvestingCalculations.RuleOneInvestingCalculationsTest.test_compound_annual_growth_rate_increase) ... ok
test_compound_annual_growth_rate_single_negative_decreasing (test_RuleOneInvestingCalculations.RuleOneInvestingCalculationsTest.test_compound_annual_growth_rate_single_negative_decreasing) ... ok
test_compound_annual_growth_rate_single_negative_increasing (test_RuleOneInvestingCalculations.RuleOneInvestingCalculationsTest.test_compound_annual_growth_rate_single_negative_increasing) ... ok
test_max_position_size (test_RuleOneInvestingCalculations.RuleOneInvestingCalculationsTest.test_max_position_size) ... ok
test_payback_time (test_RuleOneInvestingCalculations.RuleOneInvestingCalculationsTest.test_payback_time) ... ok
test_rule_one_margin_of_safety_price (test_RuleOneInvestingCalculations.RuleOneInvestingCalculationsTest.test_rule_one_margin_of_safety_price) ... ok

----------------------------------------------------------------------
Ran 18 tests in 0.002s

OK

With PyTest after renaming, I see 22 tests run:

$ pytest -v
============================= test session starts ==============================
platform darwin -- Python 3.12.4, pytest-8.2.1, pluggy-1.5.0 -- /Users/pkocieln/Library/Caches/pypoetry/virtualenvs/isthisstockgood-Aj2aRiOV-py3.12/bin/python
cachedir: .pytest_cache
rootdir: /Users/pkocieln/Workshop/free/IsThisStockGood
configfile: pyproject.toml
collecting ... collected 22 items

tests/test_DataSources.py::test_msn_money PASSED                         [  4%]
tests/test_DataSources.py::test_yahoo PASSED                             [  9%]
tests/test_MSNMoney.py::MSNMoneyTest::test_parse_pe_ratios_should_properly_calculate_pe_ratios PASSED [ 13%]
tests/test_MSNMoney.py::MSNMoneyTest::test_parse_pe_ratios_should_return_false_if_too_few_pe_ratios PASSED [ 18%]
tests/test_MSNMoney.py::MSNMoneyTest::test_parse_pe_ratios_should_return_false_when_no_data PASSED [ 22%]
tests/test_RuleOneInvestingCalculations.py::RuleOneInvestingCalculationsTest::test_calculate_estimated_future_price PASSED [ 27%]
tests/test_RuleOneInvestingCalculations.py::RuleOneInvestingCalculationsTest::test_calculate_future_eps PASSED [ 31%]
tests/test_RuleOneInvestingCalculations.py::RuleOneInvestingCalculationsTest::test_calculate_future_pe PASSED [ 36%]
tests/test_RuleOneInvestingCalculations.py::RuleOneInvestingCalculationsTest::test_calculate_margin_of_safety PASSED [ 40%]
tests/test_RuleOneInvestingCalculations.py::RuleOneInvestingCalculationsTest::test_calculate_roic PASSED [ 45%]
tests/test_RuleOneInvestingCalculations.py::RuleOneInvestingCalculationsTest::test_calculate_sticker_price PASSED [ 50%]
tests/test_RuleOneInvestingCalculations.py::RuleOneInvestingCalculationsTest::test_compound_annual_growth_rate_both_negative_decrease PASSED [ 54%]
tests/test_RuleOneInvestingCalculations.py::RuleOneInvestingCalculationsTest::test_compound_annual_growth_rate_both_negative_increase PASSED [ 59%]
tests/test_RuleOneInvestingCalculations.py::RuleOneInvestingCalculationsTest::test_compound_annual_growth_rate_decrease PASSED [ 63%]
tests/test_RuleOneInvestingCalculations.py::RuleOneInvestingCalculationsTest::test_compound_annual_growth_rate_increase PASSED [ 68%]
tests/test_RuleOneInvestingCalculations.py::RuleOneInvestingCalculationsTest::test_compound_annual_growth_rate_single_negative_decreasing PASSED [ 72%]
tests/test_RuleOneInvestingCalculations.py::RuleOneInvestingCalculationsTest::test_compound_annual_growth_rate_single_negative_increasing PASSED [ 77%]
tests/test_RuleOneInvestingCalculations.py::RuleOneInvestingCalculationsTest::test_max_position_size PASSED [ 81%]
tests/test_RuleOneInvestingCalculations.py::RuleOneInvestingCalculationsTest::test_payback_time PASSED [ 86%]
tests/test_RuleOneInvestingCalculations.py::RuleOneInvestingCalculationsTest::test_rule_one_margin_of_safety_price PASSED [ 90%]
tests/test_api.py::test_import_app PASSED                                [ 95%]
tests/test_api.py::test_get_data PASSED                                  [100%]

============================== 22 passed in 3.87s ==============================

The tests I added for the API and the data sources are not UnitTest-formatted. That's probably the reason they don't get picked up by UnitTest. I see there's 4 of them:

$ grep -rne "def test_" tests/test_DataSources.py tests/test_api.py
tests/test_DataSources.py:5:def test_msn_money():
tests/test_DataSources.py:35:def test_yahoo():
tests/test_api.py:6:def test_import_app():
tests/test_api.py:16:def test_get_data():

In summary – yes, renaming was enough at this point.

For this to work, all code called by the tests had to be organized into a "package" (in this case – under the name isthisstockgood, installable with pip and poetry). That reorganization was included in #78.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interestingly, with Python and PyTest testing becomes straightforward only after we make the code a package.

With that, in tests we call our logic just like we would use any other third-party package: from isthisstockgood import foo.

Anything simpler than that (including from .src import foo) ends up getting treated differently by different Python versions and requires adding the src directory to import paths, as you did in test_MSNMoney.py:

import os
import sys

app_path = os.path.join(os.path.dirname(__file__), "..", 'src')
sys.path.append(app_path)

I kept using the above trick for a long time, until PIP started to support installing project dependencies from pyproject.toml and Poetry got more reliable. It seems finally we can skip that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied:

diff --git a/tests/test_MSNMoney.py b/tests/test_MSNMoney.py
index d784b35..402dade 100644
--- a/tests/test_MSNMoney.py
+++ b/tests/test_MSNMoney.py
@@ -1,13 +1,8 @@
 """Tests for the MSNMoney.py functions."""


-import os
-import sys
 import unittest

-app_path = os.path.join(os.path.dirname(__file__), "..", 'isthisstockgood')
-sys.path.append(app_path)
-
 from isthisstockgood.Active.MSNMoney import MSNMoney

 class MSNMoneyTest(unittest.TestCase):
diff --git a/tests/test_RuleOneInvestingCalculations.py b/tests/test_RuleOneInvestingCalculations.py
index 8b74bae..8fe9d22 100644
--- a/tests/test_RuleOneInvestingCalculations.py
+++ b/tests/test_RuleOneInvestingCalculations.py
@@ -1,14 +1,9 @@
 """Tests for the app/RuleOneInvestingCalculations.py functions."""


-import os
-import sys
 import unittest

-app_path = os.path.join(os.path.dirname(__file__), "..", 'isthisstockgood')
-sys.path.append(app_path)
-
-import RuleOneInvestingCalculations as RuleOne
+import isthisstockgood.RuleOneInvestingCalculations as RuleOne

 class RuleOneInvestingCalculationsTest(unittest.TestCase):

All 22 tests still run fine:

$ pytest
=========================== test session starts ===========================
platform darwin -- Python 3.12.4, pytest-8.2.1, pluggy-1.5.0
rootdir: /Users/pkocieln/Workshop/free/IsThisStockGood
configfile: pyproject.toml
collected 22 items

tests/test_DataSources.py ..                                        [  9%]
tests/test_MSNMoney.py ...                                          [ 22%]
tests/test_RuleOneInvestingCalculations.py ...............          [ 90%]
tests/test_api.py ..                                                [100%]

=========================== 22 passed in 3.88s ============================

6 changes: 4 additions & 2 deletions isthisstockgood/Active/MSNMoney.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
import json
import isthisstockgood.RuleOneInvestingCalculations as RuleOne



class MSNMoney:
# This key appears to be fixed? So we can use it for now /shrug
_API_KEY = '0QfOX3Vn51YCzitbLaRkTTBadtWpgTN8NZLW0C1SEM'
Expand Down Expand Up @@ -78,7 +80,7 @@ def parse_annual_report_data(self, content):
most_recent_statement = annual_statements[max(annual_statements.keys())]
if not most_recent_statement:
return
self.total_debt = str(float(most_recent_statement.get('longTermDebt', 0)))
self.total_debt = float(most_recent_statement.get('longTermDebt', 0))
self.shares_outstanding = float(most_recent_statement.get('sharesOutstanding', 0))

key_metrics = data.get('analysis', {}).get('keyMetrics', {})
Expand All @@ -92,7 +94,7 @@ def parse_ratios_data(self, content):
return False

self.name = json_content.get('displayName', '')
self.industy = json_content.get('industry', '')
self.industry = json_content.get('industry', '')

# PE Ratios
self._parse_pe_ratios(yearly_data)
Expand Down
28 changes: 28 additions & 0 deletions isthisstockgood/CompanyInfo.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
from dataclasses import dataclass


@dataclass
class CompanyInfo:
ticker_symbol: str
name: str
description: str
industry: str
current_price: float
average_volume: float
market_cap: float
shares_outstanding: int
pe_high: float
pe_low: float
roic: float
roic_averages: [float]
equity: float
equity_growth_rates: [float]
free_cash_flow: float
free_cash_flow_growth_rates: [float]
revenue: float
revenue_growth_rates: [float]
eps: float
eps_growth_rates: [float]
debt_equity_ratio: float
last_year_net_income: float
total_debt: float
18 changes: 0 additions & 18 deletions run_all_tests.py

This file was deleted.

Empty file removed tests/__init__.py
Empty file.
70 changes: 70 additions & 0 deletions tests/test_DataSources.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
from isthisstockgood.CompanyInfo import CompanyInfo
from isthisstockgood.DataFetcher import DataFetcher


def test_msn_money():
test_ticker = 'MSFT'
test_name = 'Microsoft Corp'

data = get_msn_money_data(test_ticker)

assert data.ticker_symbol == test_ticker
assert data.name == test_name
assert data.description != ''
assert data.industry != ''
assert data.current_price > 0.0
assert data.average_volume > 0
assert data.market_cap > 0.0
assert data.shares_outstanding > 0
assert data.pe_high > 0.0
assert data.pe_low > 0.0
assert data.roic != []
assert data.roic_averages != []
assert data.equity != []
assert data.equity_growth_rates != []
assert data.free_cash_flow != []
assert data.free_cash_flow_growth_rates != []
assert data.revenue != []
assert data.revenue_growth_rates != []
assert data.eps != []
assert data.eps_growth_rates != []
assert data.debt_equity_ratio > 0.0
assert data.last_year_net_income > 0.0
assert data.total_debt >= 0.0

def test_yahoo():
test_ticker = 'MSFT'
test_name = 'Microsoft Corp'

data = get_yahoo_data(test_ticker)

assert data.ticker_symbol == test_ticker
assert float(data.five_year_growth_rate) > 0.0

def get_msn_money_data(ticker):
data_fetcher = DataFetcher()
data_fetcher.ticker_symbol = ticker

# Make all network request asynchronously to build their portion of
# the json results.
data_fetcher.fetch_msn_money_data()

# Wait for each RPC result before proceeding.
for rpc in data_fetcher.rpcs:
rpc.result()

return CompanyInfo(**vars(data_fetcher.msn_money))

def get_yahoo_data(ticker):
data_fetcher = DataFetcher()
data_fetcher.ticker_symbol = ticker

# Make all network request asynchronously to build their portion of
# the json results.
data_fetcher.fetch_yahoo_finance_analysis()

# Wait for each RPC result before proceeding.
for rpc in data_fetcher.rpcs:
rpc.result()

return data_fetcher.yahoo_finance_analysis
File renamed without changes.
Loading