Can get non-json and puke non-informatively

[yarikoptic]

from cron email

$ chronic flock -n -E 0 /home/datalad/.run/tinuous-datalad.lock /mnt/datasets/datalad/ci/logs/tools/cron_job                                                                                                                                                                                         
...
   ...limits were not hit....
                                                                                                                                                                                                                                                                                                                                                                      
2024-07-18T02:40:01-0400 [INFO    ] tinuous: tinuous 0.7.0                                                                                                                                                                                                                                                                                                               
2024-07-18T02:40:02-0400 [INFO    ] tinuous: Fetching resources from github                                                                                                                                                                                                                                                                                              
2024-07-18T02:40:02-0400 [INFO    ] tinuous: Fetching runs newer than 2024-07-17 13:17:21+00:00                                                                                                                                                                                                                                                                          
2024-07-18T02:40:02-0400 [INFO    ] tinuous: Fetching runs for workflow .github/workflows/add-changelog-snippet.yml (Add changelog.d snippet)                                                                                                                                                                                                                            
2024-07-18T02:40:02-0400 [INFO    ] tinuous: Fetching runs for workflow .github/workflows/benchmarks.yml (Benchmarks)                                                                                                                                                                                                                                                    
2024-07-18T02:40:03-0400 [INFO    ] tinuous: Fetching runs for workflow .github/workflows/docbuild.yml (Docs)                                                                                                                                                                                                                                                            
2024-07-18T02:40:04-0400 [INFO    ] tinuous: Fetching runs for workflow .github/workflows/lint.yml (Linters)                                                                                                                                                                                                                                                             
2024-07-18T02:40:05-0400 [INFO    ] tinuous: Fetching runs for workflow .github/workflows/release.yml (Auto-release on PR merge)                                                                                                                                                                                                                                         
2024-07-18T02:40:06-0400 [INFO    ] tinuous: Fetching runs for workflow .github/workflows/shellcheck.yml (Shellcheck on scripts)                                                                                                                                                                                                                                         
2024-07-18T02:40:07-0400 [INFO    ] tinuous: Fetching runs for workflow .github/workflows/test-label.yml (Test for semver label)                                                                                                                                                                                                                                         
2024-07-18T02:40:07-0400 [INFO    ] tinuous: Fetching runs for workflow .github/workflows/test.yml (Test)                                                                                                                                                                                                                                                                
2024-07-18T02:40:08-0400 [INFO    ] tinuous: Run 128 not completed; skipping                                                                                                                                                                                                                                                                                             
2024-07-18T02:40:08-0400 [INFO    ] tinuous: Fetching runs for workflow .github/workflows/test_crippled.yml (CrippledFS)                                                                                                                                                                                                                                                 
2024-07-18T02:40:09-0400 [INFO    ] tinuous: Fetching runs for workflow .github/workflows/test_extensions.yml (Extensions)                                                                                                                                                                                                                                               
2024-07-18T02:40:10-0400 [INFO    ] tinuous: Fetching runs for workflow .github/workflows/test_macos.yml (Test on macOS)                                                                                                                                                                                                                                                 
2024-07-18T02:40:11-0400 [INFO    ] tinuous: Fetching runs for workflow .github/workflows/typing.yml (Type-check)                                                                                                                                                                                                                                                        
2024-07-18T02:40:11-0400 [INFO    ] tinuous: Fetching runs for workflow .github/workflows/update-contributors.yml (allcontributors-auto-detect)                                                                                                                                                                                                                          
2024-07-18T02:40:12-0400 [INFO    ] tinuous: Fetching resources from travis                                                                                                                                                                                                                                                                                              
2024-07-18T02:40:12-0400 [INFO    ] tinuous: Fetching builds newer than 2024-06-18 06:40:12.800405+00:00                                                                                                                                                                                                                                                                 
Traceback (most recent call last):                                                                                                                                                                                                                                                                                                                                       
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.12/site-packages/requests/models.py", line 974, in json                                                                                                                                                                                                                                                    
    return complexjson.loads(self.text, **kwargs)                                                                                                                                                                                                                                                                                                                        
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                                                                                                                                                                        
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.12/site-packages/simplejson/__init__.py", line 514, in loads                                                                                                                                                                                                                                               
    return _default_decoder.decode(s)                                                                                                                                                                                                                                                                                                                                    
           ^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                                                                                                                                                                                    
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.12/site-packages/simplejson/decoder.py", line 386, in decode                                                                                                                                                                                                                                               
    obj, end = self.raw_decode(s)                                                                                                                                                                                                                                                                                                                                        
               ^^^^^^^^^^^^^^^^^^                                                                                                                                                                                                                                                                                                                                        
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.12/site-packages/simplejson/decoder.py", line 416, in raw_decode                                                                                                                                                                                                                                           
    return self.scan_once(s, idx=_w(s, idx).end())                                                                                                                                                                                                                                                                                                                       
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                                                                                                                                                                       
simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)                                                                                                                                                                                                                                                                                             
                                                                                                                                                                                                                                                                                                                                                                         
During handling of the above exception, another exception occurred:                                                                                                                                                                                                                                                                                                      
                                                                                                                                                                                                                                                                                                                                                                         
Traceback (most recent call last):                                                                                                                                                                                                                                                                                                                                       
  File "/home/datalad/miniconda3/envs/tinuous-dev/bin/tinuous", line 8, in <module>                                                                                                                                                                                                                                                                                      
    sys.exit(main())                                                                                                                                                                                                                                                                                                                                                     
             ^^^^^^                                                                                                                                                                                                                                                                                                                                                      
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.12/site-packages/click/core.py", line 1157, in __call__                                                                                                                                                                                                                                                    
    return self.main(*args, **kwargs)                                                                                                                                                                                                                                                                                                                                    
           ^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                                                                                                                                                                                    
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.12/site-packages/click/core.py", line 1078, in main                                                                                                                                                                                                                                                        
    rv = self.invoke(ctx)                                                                                                                                                                                                                                                                                                                                                
         ^^^^^^^^^^^^^^^^                                                                                                                                                                                                                                                                                                                                                
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.12/site-packages/click/core.py", line 1688, in invoke                                                                                                                                                                                                                                                      
    return _process_result(sub_ctx.command.invoke(sub_ctx))                                                                                                                                                                                                                                                                                                              
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                                                                                                                                                               
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.12/site-packages/click/core.py", line 1434, in invoke                                                                                                                                                                                                                                                      
    return ctx.invoke(self.callback, **ctx.params)                                                                                                                                                                                                                                                                                                                       
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                                                                                                                                                                       
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.12/site-packages/click/core.py", line 783, in invoke                                                                                                                                                                                                                                                       
    return __callback(*args, **kwargs)                                                                                                                                                                                                                                                                                                                                   
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                                                                                                                                                                                   
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.12/site-packages/click/decorators.py", line 45, in new_func                                                                                                                                                                                                                                                
    return f(get_current_context().obj, *args, **kwargs)                                                                                                                                                                                                                                                                                                                 
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                                                                                                                                                                 
  File "/mnt/datasets/datalad/ci/tinuous/src/tinuous/__main__.py", line 118, in fetch                                                                                                                                                                                                                                                                                    
    for obj in ci.get_build_assets(                                                                                                                                                                                                                                                                                                                                      
  File "/mnt/datasets/datalad/ci/tinuous/src/tinuous/travis.py", line 99, in get_build_assets                                                                                                                                                                                                                                                                            
    for build in self.paginate(                                                                                                                                                                                                                                                                                                                                          
  File "/mnt/datasets/datalad/ci/tinuous/src/tinuous/travis.py", line 82, in paginate                                                                                                                                                                                                                                                                                    
    data = self.client.get(path, params=params).json()                                                                                                                                                                                                                                                                                                                   
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                                                                                                                                                                   
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.12/site-packages/requests/models.py", line 978, in json                                                                                                                                                                                                                                                    
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)                                                                                                                                                                                                                                                                                                                   
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)                 

am I right that it is happening at requests level and thus we likely can't do anything to make crash more informative?

[jwodder]

@yarikoptic Depending on what information you want to see (presumably the non-JSON response body?), we could just call json.loads() on the body ourselves instead of having requests do it.

For the record, the fact that the error states line 1 column 1 (char 0) suggests that the response bodies received were empty.

[yarikoptic]

@yarikoptic Depending on what information you want to see (presumably the non-JSON response body?), we could just call json.loads() on the body ourselves instead of having requests do it.

ah, cool... let's see if we ever run into such a case again first which would indeed require

For the record, the fact that the error states line 1 column 1 (char 0) suggests that the response bodies received were empty.

not necessarily I think:

In [5]: json.loads("abracadabra")
---------------------------------------------------------------------------
JSONDecodeError                           Traceback (most recent call last)
Cell In[5], line 1
----> 1 json.loads("abracadabra")

File /usr/lib/python3.11/json/__init__.py:346, in loads(s, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    341     s = s.decode(detect_encoding(s), 'surrogatepass')
    343 if (cls is None and object_hook is None and
    344         parse_int is None and parse_float is None and
    345         parse_constant is None and object_pairs_hook is None and not kw):
--> 346     return _default_decoder.decode(s)
    347 if cls is None:
    348     cls = JSONDecoder

File /usr/lib/python3.11/json/decoder.py:337, in JSONDecoder.decode(self, s, _w)
    332 def decode(self, s, _w=WHITESPACE.match):
    333     """Return the Python representation of ``s`` (a ``str`` instance
    334     containing a JSON document).
    335 
    336     """
--> 337     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    338     end = _w(s, end).end()
    339     if end != len(s):

File /usr/lib/python3.11/json/decoder.py:355, in JSONDecoder.raw_decode(self, s, idx)
    353     obj, end = self.scan_once(s, idx)
    354 except StopIteration as err:
--> 355     raise JSONDecodeError("Expecting value", s, err.value) from None
    356 return obj, end

JSONDecodeError: Expecting value: line 1 column 1 (char 0)