pvoutput.get_status throws error: NotImplementedError #84
Labels
bug
Something isn't working
Comments
|
It's this specific to this one system_id? |
|
Hi peter,
No.
I query for 225 systems in the Balkans and have noticed 2 systems throwing
this error. See end of github issue. There may be more.
Can you query the same system ids to reproduce the error?
Best regards
Julian
Sent from phone
…On Fri, 14 Jan 2022, 15:23 Peter Dudfield, ***@***.***> wrote:
It's this specific to this one system_id?
—
Reply to this email directly, view it on GitHub
<#84 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/APAUAOI2LFVZXXKSD7WWWZ3UWA5WPANCNFSM5L6XG3MA>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
|
Sorry I havent looked into this yet. |
|
Hi Peter,
I've not downloaded UK data yet.
The error arises from at least one system having the wrong number of fields
in its status.
Happy to share the code of the lightweight PVOutput downloader I have
developed.
best regards
Julian
…On Thu, 3 Feb 2022 at 16:16, Peter Dudfield ***@***.***> wrote:
Sorry I havent looked into this yet.
Does this happen in the UK too?
—
Reply to this email directly, view it on GitHub
<#84 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/APAUAOO2UINA464GBI5BEYTUZKS6JANCNFSM5L6XG3MA>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
--
Julian Briggs
Research Software Developer
Sheffield Solar
Physics & Astronomy
University of Sheffield
In line with government advice, I am currently working remotely.
My preferred method of contact is email.
My normal working hours are 9:30-5 Mon-Thu
|
|
Hi Peter,
here's an example warning about the PVOutput system with Wrong number of
values in row
2022-02-04T09:30:23.838688674Z 2022-02-04 09:30:23,838 downloaderpvoutput
download_insert_readings 416 WARNING Bad status: *Wrong number of values in
row* (12), expected (11): ('20220204', '09:10', '700', '0.156', '3459',
'0', '0.000', 'NaN', 'NaN', '9.5', '2', '227.9').. *pv_system_id: 39266,
date: 2022-02-04*.
I trust you can reproduce this and catch the PVOutput error: *Bad status*
more anon
best
Julian
On Thu, 3 Feb 2022 at 19:30, Julian Briggs ***@***.***>
wrote:
… Hi Peter,
I've not downloaded UK data yet.
The error arises from at least one system having the wrong number of
fields in its status.
Happy to share the code of the lightweight PVOutput downloader I have
developed.
best regards
Julian
On Thu, 3 Feb 2022 at 16:16, Peter Dudfield ***@***.***>
wrote:
> Sorry I havent looked into this yet.
> Does this happen in the UK too?
>
> —
> Reply to this email directly, view it on GitHub
> <#84 (comment)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/APAUAOO2UINA464GBI5BEYTUZKS6JANCNFSM5L6XG3MA>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>
--
Julian Briggs
Research Software Developer
Sheffield Solar
Physics & Astronomy
University of Sheffield
In line with government advice, I am currently working remotely.
My preferred method of contact is email.
My normal working hours are 9:30-5 Mon-Thu
--
Julian Briggs
Research Software Developer
Sheffield Solar
Physics & Astronomy
University of Sheffield
In line with government advice, I am currently working remotely.
My preferred method of contact is email.
My normal working hours are 9:30-5 Mon-Thu
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
pvoutput.get_status throws error: NotImplementedError: file structure not yet supported
To Reproduce
Steps to reproduce the behavior:
pv_system_id = 39266
date = '2022-01-13'
status = self.pv.get_status(pv_system_id, date=date)
See error
NotImplementedError: file structure not yet supported
Expected behavior
Download pv_system status without error
Additional context
I'm seeing pvoutput error: NotImplementedError: file structure not yet supported
when requesting status of system_id 29291: Requesting system status for 2022-01-13
See appended. DEBUG oputput and stack traces.
I have seen a similar error with another system
pvoutput get_status 209 INFO system_id 39266: Requesting system status for 2022-01-13
2022-01-13 15:36:13,539 pvoutput get_status 209 INFO system_id 39266: Requesting system status for 2022-01-13
2022-01-13 15:36:13,543 connectionpool _new_conn 1001 DEBUG Starting new HTTPS connection (1): pvoutput.org:443
2022-01-13 15:36:14,063 connectionpool _make_request 456 DEBUG https://pvoutput.org:443 "GET /service/r2/getstatus.jsp?d=20220113&h=1&limit=288&ext=0&sid1=39266 HTTP/1.1" 200 None
2022-01-13 15:36:14,064 utils _get_response 81 DEBUG response: status_code=200; headers={'Date': 'Thu, 13 Jan 2022 15:36:13 GMT', 'Set-Cookie': 'JSESSIONID=us9myf32fu8ndpm2h7qwsc5k;Path=/; Secure', 'Expires': 'Thu, 01 Jan 1970 00:00:00 GMT', 'X-Rate-Limit-Remaining': '230', 'X-Rate-Limit-Limit': '300', 'X-Rate-Limit-Reset': '1642089600', 'Content-Type': 'text/plain', 'Content-Encoding': 'gzip', 'Vary': 'Accept-Encoding, User-Agent', 'Connection': 'close', 'Server': 'Jetty(7.6.17.v20150415)', 'Strict-Transport-Security': 'max-age=15768000'}
2022-01-13 15:36:14,064 pvoutput _set_rate_limit_params 928 DEBUG {'rate_limit_remaining': 230, 'rate_limit_total': 300, 'rate_limit_reset_time': Timestamp('2022-01-13 16:00:00+0000', tz='UTC')}
Traceback (most recent call last):
File "/home/ph1jb/git/pvoutput/src/downloader.py", line 240, in
nsystems = MAIN.run(sys.argv[1:])
File "/home/ph1jb/git/pvoutput/src/downloader.py", line 223, in run
nsystems = self.insert_system_statuses(database_handler, date, pv_system_ids)
File "/home/ph1jb/git/pvoutput/src/downloader.py", line 194, in insert_system_statuses
status = self.download_system_status(pv_system_id, date)
File "/home/ph1jb/git/pvoutput/src/downloader.py", line 154, in download_system_status
status = self.pv.get_status(pv_system_id, date=date)
File "/home/ph1jb/git/pvoutput/.venv/src/pvoutput/pvoutput/pvoutput.py", line 257, in get_status
pv_system_status = pd.read_csv(
File "/home/ph1jb/git/pvoutput/.venv/lib/python3.8/site-packages/pandas/util/_decorators.py", line 311, in wrapper
return func(*args, **kwargs)
File "/home/ph1jb/git/pvoutput/.venv/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 586, in read_csv
return _read(filepath_or_buffer, kwds)
File "/home/ph1jb/git/pvoutput/.venv/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 488, in _read
return parser.read(nrows)
File "/home/ph1jb/git/pvoutput/.venv/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 1047, in read
index, columns, col_dict = self._engine.read(nrows)
File "/home/ph1jb/git/pvoutput/.venv/lib/python3.8/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 261, in read
raise NotImplementedError("file structure not yet supported")
NotImplementedError: file structure not yet supported
2022-01-13 14:34:06,763 pvoutput get_status 209 INFO system_id 29291: Requesting system status for 2022-01-13
2022-01-13 14:34:06,772 connectionpool _new_conn 1001 DEBUG Starting new HTTPS connection (1): pvoutput.org:443
2022-01-13 14:34:07,310 connectionpool _make_request 456 DEBUG https://pvoutput.org:443 "GET /service/r2/getstatus.jsp?d=20220113&h=1&limit=288&ext=0&sid1=29291 HTTP/1.1" 200 None
2022-01-13 14:34:07,312 utils _get_response 81 DEBUG response: status_code=200; headers={'Date': 'Thu, 13 Jan 2022 14:34:07 GMT', 'Set-Cookie': 'JSESSIONID=10fqtsfp2518apqdw1dfmpdbl;Path=/; Secure', 'Expires': 'Thu, 01 Jan 1970 00:00:00 GMT', 'X-Rate-Limit-Remaining': '163', 'X-Rate-Limit-Limit': '300', 'X-Rate-Limit-Reset': '1642086000', 'Content-Type': 'text/plain', 'Content-Encoding': 'gzip', 'Vary': 'Accept-Encoding, User-Agent', 'Connection': 'close', 'Server': 'Jetty(7.6.17.v20150415)', 'Strict-Transport-Security': 'max-age=15768000'}
2022-01-13 14:34:07,312 pvoutput _set_rate_limit_params 928 DEBUG {'rate_limit_remaining': 163, 'rate_limit_total': 300, 'rate_limit_reset_time': Timestamp('2022-01-13 15:00:00+0000', tz='UTC')}
Traceback (most recent call last):
File "/home/ph1jb/git/pvoutput/src/downloader.py", line 262, in
nsystems = MAIN.run(sys.argv[1:])
File "/home/ph1jb/git/pvoutput/src/downloader.py", line 242, in run
nsystems = self.insert_system_statuses(database_handler, date, pv_system_ids)
File "/home/ph1jb/git/pvoutput/src/downloader.py", line 195, in insert_system_statuses
status = self.download_system_status(pv_system_id, date)
File "/home/ph1jb/git/pvoutput/src/downloader.py", line 155, in download_system_status
status = self.pv.get_status(pv_system_id, date=date)
File "/home/ph1jb/git/pvoutput/.venv/src/pvoutput/pvoutput/pvoutput.py", line 257, in get_status
pv_system_status = pd.read_csv(
File "/home/ph1jb/git/pvoutput/.venv/lib/python3.8/site-packages/pandas/util/_decorators.py", line 311, in wrapper
return func(*args, **kwargs)
File "/home/ph1jb/git/pvoutput/.venv/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 586, in read_csv
return _read(filepath_or_buffer, kwds)
File "/home/ph1jb/git/pvoutput/.venv/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 488, in _read
return parser.read(nrows)
File "/home/ph1jb/git/pvoutput/.venv/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 1047, in read
index, columns, col_dict = self._engine.read(nrows)
File "/home/ph1jb/git/pvoutput/.venv/lib/python3.8/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 261, in read
raise NotImplementedError("file structure not yet supported")
NotImplementedError: file structure not yet supported
The text was updated successfully, but these errors were encountered: