Malformed EUPMC query doesn't terminate, nor give error message

[rossmounce]

Perhaps related to #114 I noted recently that a subtly malformed EUPMC query gets stuck at "Searching using eupmc API" without terminating (ever), or giving an error message. This could lead some users to wait 10 minutes assuming it's loading a big query.

The user error causing this is missing the colon. It should be (FIRST_PDATE:[2013-01-01 TO 2016-12-05]) but instead (FIRST_PDATE[2013-01-01 TO 2016-12-05]) is given.

It would be nice (although perhaps hard), if this could be detected & some kind of informative error message supplied.

getpapers -V
0.4.10

getpapers -q '"arbuscular mycorrhizae" (FIRST_PDATE[2013-01-01 TO 2016-12-05])' -o malformedquery
info: Searching using eupmc API

[blahah]

I think we have to rely on the eupmc to tell us that a query is malformed. It's possible that they already do this and we aren't handling the response properly.

[tarrow]

How long did it run for?

Basically the problem we have/had is that (with a changing likelihood of it happening) EuPMC sometimes responds with an error page or simply no results when there are actually results to be delivered.

For this reason we now very aggressively retry until we get a successful response but not forever. I would expect it to eventually fail; probably in under 10mins but perhaps longer.

I'd like to retry less aggressively but I was going to wait and see if we have a few more months of relative stability before I turn it down.

[rossmounce]

My impression was that it would run indefinitely. But I am running it again now with time to see if it does in fact terminate at some point...

[rossmounce]

Apologies, it seems I was being impatient:

time getpapers -q '"arbuscular mycorrhizae" (FIRST_PDATE[2013-01-01 TO 2016-12-05])' -o malformedquery
info: Searching using eupmc API
warn: We had to retry the last request 50 times.
error: Malformed or empty response from EuropePMC. Try running again. Perhaps your query is wrong.

real	4m11.392s
user	0m1.376s
sys	0m0.104s

[tarrow]

I'll now knock this down to 5 times.

This is now causing a problem itself because EuPMC are now reporting incorrect number of results even in one page. e.g. say there are 10 results. Only return 9 and then we retry forever trying to get that 'last' one that isn't there.

I reported this bug to them before but obviously it has reoccurred.