Skip to content

Releases: m-lab/etl-gardener

Add TCPInfo instance, various other fixes.

13 Jun 11:33
b7b655c
Compare
Choose a tag to compare

This release has many changes from April 1 to June 12.
Primary changes are:

  1. Change table detail query to count distinct rows and tasks. This should improve sanity check reliability.
  2. Add tcpinfo config, and queries for dedup and sanity check.
  3. Update table detail query to support tcpinfo, which does not have test_id or task_filename fields.

The tcpinfo instance has been working in sandbox. Not clear whether it is WAI in staging. It successfully processed one date to completion, but then the queues had to be cleared and instance restarted.

Travis best practices, sidestream dedup query

29 Apr 18:06
f77f498
Compare
Choose a tag to compare

#149 Add ca-certificate to Dockerfile
#147 Add version to the build and convert travis to best practices
#146 package moved
#145 dedup chooses latest parse_time (sidestream)
#142 promauto and prometheusx

Release: Gardener with updated de-dup query

27 Feb 20:57
8e1b000
Compare
Choose a tag to compare

#140 update dedup query and table detail query

Should reduce lost rows and work around duplicate historical archives.

Increase recheck delay to 60 minutes

15 Feb 14:39
4c7d0fd
Compare
Choose a tag to compare

#136 Increase delay to 60 minutes

Don't trust StreamingBuffer status

13 Feb 19:49
9731011
Compare
Choose a tag to compare

This release includes critical change to recheck StreamingBuffer status, which is unreliable.
Advancing minor version number, as this should correct a major bug and allow correct behavior.

#133 Use histogram for time-in-state
#131 wait 5 minutes for stable table (actually 10 minutes)
#132 fix restart with dateSkip

Retain templated table on error

11 Feb 18:13
7eeaa33
Compare
Choose a tag to compare

#130 delay deleting dedup source
This allows examination of the dedup source when sanitycheck fails.

Prod - preserve error states in datastore

07 Feb 01:52
90573e6
Compare
Choose a tag to compare

Primarily changes the retention of datastore entries when errors occur.
126 and 127 shouldn't impact prod
125 is a minor tweak to queue behavior.

#128 Preserve error states 23 hours ago
#127 fast disco a day ago
#126 Sandbox fast gardener hack 13 days ago
#125 Sandbox tweak queue wait 14 days ago

Fix task queue false "empty queue" triggers

11 Jan 00:55
46a477c
Compare
Choose a tag to compare

The major update is to fix false IsEmpty checks due to occasional TQ bug.
Also includes improvements to logging, and a new metric to track suspicious TQ empty stats.

Change START_TIME to include complete history

24 Oct 17:27
1ef38d1
Compare
Choose a tag to compare
Merge pull request #116 from m-lab/sandbox-all

Set correct starting dates for all 4 data types

Production release - add traceroute dataset

23 Oct 23:27
41cacb1
Compare
Choose a tag to compare

This release adds the traceroute gardener to production.

As well, this release changes the source bucket for all gardeners to archive-measurement-lab.