better monitoring for rss-puller (and all Producer/Queuer classes)

[philbudne]

Since we're no longer always/only fetching RSS files a fixed interval in the past (two days ago), it would be nice to have a way to see the date of the most recently fetched story that has been queued.

Grafana appears to have axis labeling options for date/times: Investigate what it wants: decimal numbers like YYYYMMDD or epoch seconds (since 1970-01-01)?

Investigate:

• a single gauge for the latest fetch_date queued in the current batch (add record keeping to Producer.queue_story?)
• two gauges: one for latest, one for oldest (since hist-queuer goes backwards in time, and so do the historical CSV files?)
• use statsd timer type (which keeps highest/lowest values for all data points in a reporting period). cons: would keep meaningless things like mean/std dev!, graphs would only show data for reporting periods where data was queued, while gauge values "stick" and continue to report whether or not new data has been reported.

[philbudne]

Footnote:

I suppose min/max date metrics could be reported for ALL Workers (gathering the info in StorySender.send_story), which would be useful if multiple queues were congested, so you could see what range of dates were being handled where in the pipeline?!