-
Notifications
You must be signed in to change notification settings - Fork 680
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Decoding stores that was encrypted by Yahoo! Finance recently #953
base: main
Are you sure you want to change the base?
Conversation
…m Yahoo! Finance as encrypted due to a recent change in their API (around 2 days ago). Fix decodes the data in decrypt_cryptojs_aes() and makes small changes to _read_one_data() to parse it properly later.
Please make this pr a priority. Yahoo api is entirely bricked across related packages. |
I think we need to update requirements.txt with |
Updating with recommendations from @satoshi
Thanks, just updated it now. |
Seems your main failures in the Azure DevOps logs are you are failing both the linter (flake8) and the formatter (black) tests. To pass the formatter, in your project root directory, run: To pass the linter, in your project root directory, run Finally, commit your changes! Good luck :) |
Can't wait till this request gets committed! I use a data reader a lot and this error is causing a lot of problems in my code. |
I meant to say changed 3 files in the above commit, and I also noticed that it got rid of some 'u's from pandas_datareader/tests/io/test_jsdmx.py AND pandas_datareader/tests/yahoo/test_options.py But now I am struggling with the second command using flake8. I installed it with pip and tried to run the above and get the error message: Have been trying to understand what the problem is but im completely unfamiliar with git diff and flake8 |
Me too, my dissertation is using this library and it wont work until this gets accepted :) I have already submitted |
Has anyone reached out to get this merged? And does it work on Windows? |
Asking the same question here. I'm trying to use Tiingo instead but that doesn't seem to be working either? |
If you've already emailed (or contacted on github or whatever) him/her, then all there is to do is wait |
Sometimes I get the following error : """ Process finished with exit code 1 |
ranaroussi/yfinance#1291 (comment) this work for me but keeping unpad block size at 16
|
Hi @raphi6 ! FIrst of all, thank you very much for trying to fix this, I have just starting playinf with this and now is broken 😢 This error you are seeing is happening because the response for that particular stock you are looking for does not have the keys I have tried with 'AAPL', 'GOOGL', 'AMZN' and none of them return the keys And now I want to ask you, how did you get to that NOTE: Also, it feels weird that they are encrypting something... and sharing also the key. That's pointless (or I might be missing something 😅 ) |
Hi @raphf6, Thanks for all your work! In my opinion this isn't an error that occurs sometimes or for specific stocks but something must have been changed on Yahoo's side again. Your solution worked for me until yesterday and since yesterday I get the same error (and I am only requesting one specific fund all the time). |
hellc@87dda3f fixed |
If u cant wait to merge use this |
Encryption genius! Thank you so much Ivan! I will test this out later tonight hopefully. |
Do I have to do any merging? Sorry I am quite new to Git |
Accept this PR into your branch and u would be fine. raphi6#1 |
Decryption issue fix
@CharliesAngel1 What does it mean that these are approved? Do we still have to wait for a merge? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All looks good.
you are a legend bro thanks again |
Just, to be sure - is this solution working for everybody? @hellc's solution does not work for me (out of the box) as it requires packaging version 22 or higher, which is not available for my setting, but it seems the changes were accepted into @raphi6's solution, right? I saw the update but I still get only errors for my request. |
It's still working in BTC pairs so was enough for me. Let us know which pairs is not working for you, maybe we will figure out what else they have changed and fix it too |
I am getting this error for all the pairs now, File "rsi.py", line 14, in |
Can confirm. When I try:
the result is the same for me: |
Also going to bump this I installed pandas-datareader with:
I went into padding_len = bord(padded_data[-1])
if padding_len<1 or padding_len>min(block_size, pdata_len):
raise ValueError("Padding is incorrect.") fails. I'm not familiar with cryptography or whatever it is that is happening here. However, I did this investigation to hopefully get someone on the right track here. Hopefully someone fixes this soon. |
Haven't heard anything in a month. Any status on fixing issue 953/952? Will Pandas-datareader every work again to scrape price data from Yahoo? |
More than likely, Pandas-datareader will eventually become functional again. If you need to access market data right now, I recommend checking out the yfinance library. It's clearly possible to get data from Yahoo! Finance, the question is just when it will be supported inside of pandas again. |
Sorry for any invonvenience, I am new to working on git in such a
professional manor so expect errors with pull request.
Changes:
In pandas-datareader/yahoo/daily.py/
I have tested this on a limited number of stocks on my personal
project and works good.
I have ran the test_yahoo.py and passed 16, failed 4. However, it
still is more tests than the current version on GitHub now due to
Yahoo! Finance new change (I dont think any Yahoo! stocks work atm).
I am unsure of the tests that are failing so some help would be great.
I am sure this can be used just as a temporary fix!
I also don't know how to run the 3rd and 4th bullet points below.
git diff upstream/master -u -- "*.py" | flake8 --diff
black --check pandas_datareader