diff --git a/_includes/schema_ndt5resultrowv2.md b/_includes/schema_ndt5resultrowv2.md index 026671a3..34e145a4 100644 --- a/_includes/schema_ndt5resultrowv2.md +++ b/_includes/schema_ndt5resultrowv2.md @@ -6,15 +6,17 @@ | a.**TestTime** | TIMESTAMP | The date and time of the measurement in UTC. | | a.**CongestionControl** | STRING | The congestion control algorithm used for connection. | | a.**MeanThroughputMbps** | FLOAT | The measured rate as calculated by the server. Presented in megabits per second, or Mbit/s, this value is the average of tcp-info snapshots taken at the beginning and end of an ndt7 measurement. Therefore it is identified as "MeanThroughputMbps". | -| a.**MinRTT** | FLOAT | The minimum Round Trip Time observed during the measurement, recorded in milliseconds. | +| a.**MinRTT** | FLOAT | The minimum Round Trip Time observed during the measurement, recorded in milliseconds. Derived from TCPInfo.MinRTT after 2020-06-18. | | a.**LossRate** | FLOAT | Loss rate from the lifetime of the connection. | | **parser** | RECORD | Metadata about how the parser processed this measurement row. | | parser.**Version** | STRING | Version is the symbolic version (if any) of the running server code that produced this measurement. | | parser.**Time** | TIMESTAMP | The time that the parser processed this row. | | parser.**ArchiveURL** | STRING | The Google Cloud Storage URL to the archive containing the Filename for this row. | -| parser.**Filename** | STRING | | +| parser.**Filename** | STRING | The name of the file within the ArchiveURL originally created by the measurement service. Results in the raw record are derived from measurements in this file. | | parser.**Priority** | INTEGER | | -| parser.**GitCommit** | STRING | | +| parser.**GitCommit** | STRING | The git commit of this build of the parser. | +| parser.**ArchiveSize** | INTEGER | The original archive size as found in GCS. | +| parser.**FileSize** | INTEGER | The size of the file data provided to the parser for this row. | | **date** | DATE | Date is used by BigQuery to partition data to improve query performance. | | **raw** | RECORD | Fields from the raw data. | | raw.**GitShortCommit** | STRING | GitShortCommit is the Git commit (short form) of the running server code that produced this measurement. | @@ -54,10 +56,10 @@ | raw.S2C.**StartTime** | TIMESTAMP | The date and time when the measurement began in UTC. | | raw.S2C.**EndTime** | TIMESTAMP | The date and time when the measurement ended in UTC. | | raw.S2C.**MeanThroughputMbps** | FLOAT | The measured rate as calculated by the server. Presented in megabits per second, or Mbit/s, this value is the average of tcp-info snapshots taken at the beginning and end of an ndt5 measurement. Therefore it is identified as "MeanThroughputMbps". | -| raw.S2C.**MinRTT** | INTEGER | The minimum RTT observed during the download measurement, recorded in milliseconds. | -| raw.S2C.**MaxRTT** | INTEGER | The maximum sampled round trip time, recorded in milliseconds. | -| raw.S2C.**SumRTT** | INTEGER | The sum of all sampled round trip times, recorded in milliseconds. | -| raw.S2C.**CountRTT** | INTEGER | The number of round trip time samples included in S2C.SumRTT, reported in milliseconds. | +| raw.S2C.**MinRTT** | INTEGER | The application measured minimum observed round trip time, recorded in nanoseconds. | +| raw.S2C.**MaxRTT** | INTEGER | The application measured maximum sampled round trip time, recorded in nanoseconds. | +| raw.S2C.**SumRTT** | INTEGER | The sum of all sampled round trip times, recorded in nanoseconds. | +| raw.S2C.**CountRTT** | INTEGER | The number of round trip time samples included in S2C.SumRTT. | | raw.S2C.**ClientReportedMbps** | FLOAT | The download rate as calculated by the client, in megabits per second, or Mbit/s. Not all clients report this value. | | raw.S2C.**TCPInfo** | RECORD | The TCPInfo record provides results from the TCP_INFO netlink socket. These are the same values returned to clients at the end of the download (S2C) measurement. | | raw.S2C.TCPInfo.**State** | INTEGER | TCP state is nominally 1 (Established). Other values reflect transient states having incomplete rows.
Kernel: See TCP_ESTABLISHED in include/net/tcp_states.h | @@ -99,7 +101,7 @@ | raw.S2C.TCPInfo.**SegsOut** | INTEGER | The number of segments transmitted. Includes data and pure ACKs.
Kernel: segs_out in include/linux/tcp.h | | raw.S2C.TCPInfo.**SegsIn** | INTEGER | The number of segments received. Includes data and pure ACKs.
Kernel: segs_in in include/linux/tcp.h | | raw.S2C.TCPInfo.**NotsentBytes** | INTEGER | Number of bytes queued in the send buffer that have not been sent.
Kernel: tcpi_notsent_bytes() in net/ipv4/tcp.c | -| raw.S2C.TCPInfo.**MinRTT** | INTEGER | Minimum Round Trip Time. From an older, pre-BBR algorithm.
Kernel: tcp_min_rtt in include/net/tcp.h | +| raw.S2C.TCPInfo.**MinRTT** | INTEGER | Minimum Round Trip Time. From an older, pre-BBR algorithm. Recorded in microseconds.
Kernel: tcp_min_rtt in include/net/tcp.h | | raw.S2C.TCPInfo.**DataSegsIn** | INTEGER | Input segments carrying data (len>0).
Kernel: data_segs_in in include/net/tcp.h | | raw.S2C.TCPInfo.**DataSegsOut** | INTEGER | Transmitted segments carrying data (len>0).
Kernel: data_segs_out in include/net/tcp.h | | raw.S2C.TCPInfo.**DeliveryRate** | INTEGER | Observed Maximum Delivery Rate.
Kernel: tcp_compute_delivery_rate() in net/ipv4/tcp.c | diff --git a/_includes/schema_ndt7resultrow.md b/_includes/schema_ndt7resultrow.md index b6e46005..a86a62db 100644 --- a/_includes/schema_ndt7resultrow.md +++ b/_includes/schema_ndt7resultrow.md @@ -12,9 +12,11 @@ | parser.**Version** | STRING | Version is the symbolic version (if any) of the running server code that produced this measurement. | | parser.**Time** | TIMESTAMP | The time that the parser processed this row. | | parser.**ArchiveURL** | STRING | The Google Cloud Storage URL to the archive containing the Filename for this row. | -| parser.**Filename** | STRING | | +| parser.**Filename** | STRING | The name of the file within the ArchiveURL originally created by the measurement service. Results in the raw record are derived from measurements in this file. | | parser.**Priority** | INTEGER | | -| parser.**GitCommit** | STRING | | +| parser.**GitCommit** | STRING | The git commit of this build of the parser. | +| parser.**ArchiveSize** | INTEGER | The original archive size as found in GCS. | +| parser.**FileSize** | INTEGER | The size of the file data provided to the parser for this row. | | **date** | DATE | Date is used by BigQuery to partition data to improve query performance. | | **raw** | RECORD | Fields from the raw data. | | raw.**GitShortCommit** | STRING | GitShortCommit is the Git commit (short form) of the running server code that produced this measurement. | @@ -33,9 +35,13 @@ | raw.Upload.ServerMeasurements.**AppInfo** | RECORD | Server measurements performed outside of the kernel | | raw.Upload.ServerMeasurements.AppInfo.**NumBytes** | INTEGER | The number of bytes written to or read from the socket during the measurement. | | raw.Upload.ServerMeasurements.AppInfo.**ElapsedTime** | INTEGER | The duration of the measurement as measured by the M-Lab server in milliseconds. | +| raw.Upload.ServerMeasurements.**ConnectionInfo** | RECORD | | +| raw.Upload.ServerMeasurements.ConnectionInfo.**Client** | STRING | | +| raw.Upload.ServerMeasurements.ConnectionInfo.**Server** | STRING | | +| raw.Upload.ServerMeasurements.ConnectionInfo.**UUID** | STRING | UUID for TCP connection for this measurement. | | raw.Upload.ServerMeasurements.**BBRInfo** | RECORD | Instrumentation in the BBR TCP module in the kernel. | | raw.Upload.ServerMeasurements.BBRInfo.**BW** | INTEGER | The maximum end-to-end bandwidth from the server to the client as measured by BBR. | -| raw.Upload.ServerMeasurements.BBRInfo.**MinRTT** | INTEGER | The minimum round trip time as measured by BBR. | +| raw.Upload.ServerMeasurements.BBRInfo.**MinRTT** | INTEGER | The minimum round trip time as measured by BBR. Recorded in microseconds. | | raw.Upload.ServerMeasurements.BBRInfo.**PacingGain** | INTEGER | Fixed point multiplier used to set the pacing rate from the maximum bandwidth. The binary point varies by kernel version but the statistical mode is always 1.0. | | raw.Upload.ServerMeasurements.BBRInfo.**CwndGain** | INTEGER | Fixed point multiplier used to set the maximum window size from BW*MinRTT. The denominator varies by kernel version. | | raw.Upload.ServerMeasurements.BBRInfo.**ElapsedTime** | INTEGER | The duration of the measurement as measured by the M-Lab server in milliseconds. | @@ -79,7 +85,7 @@ | raw.Upload.ServerMeasurements.TCPInfo.**SegsOut** | INTEGER | The number of segments transmitted. Includes data and pure ACKs.
Kernel: segs_out in include/linux/tcp.h | | raw.Upload.ServerMeasurements.TCPInfo.**SegsIn** | INTEGER | The number of segments received. Includes data and pure ACKs.
Kernel: segs_in in include/linux/tcp.h | | raw.Upload.ServerMeasurements.TCPInfo.**NotsentBytes** | INTEGER | Number of bytes queued in the send buffer that have not been sent.
Kernel: tcpi_notsent_bytes() in net/ipv4/tcp.c | -| raw.Upload.ServerMeasurements.TCPInfo.**MinRTT** | INTEGER | Minimum Round Trip Time. From an older, pre-BBR algorithm.
Kernel: tcp_min_rtt in include/net/tcp.h | +| raw.Upload.ServerMeasurements.TCPInfo.**MinRTT** | INTEGER | Minimum Round Trip Time. Recorded in microseconds.
Kernel: tcp_min_rtt in include/net/tcp.h | | raw.Upload.ServerMeasurements.TCPInfo.**DataSegsIn** | INTEGER | Input segments carrying data (len>0).
Kernel: data_segs_in in include/net/tcp.h | | raw.Upload.ServerMeasurements.TCPInfo.**DataSegsOut** | INTEGER | Transmitted segments carrying data (len>0).
Kernel: data_segs_out in include/net/tcp.h | | raw.Upload.ServerMeasurements.TCPInfo.**DeliveryRate** | INTEGER | Observed Maximum Delivery Rate.
Kernel: tcp_compute_delivery_rate() in net/ipv4/tcp.c | @@ -99,9 +105,13 @@ | raw.Upload.ClientMeasurements.**AppInfo** | RECORD | Server measurements performed outside of the kernel | | raw.Upload.ClientMeasurements.AppInfo.**NumBytes** | INTEGER | The number of bytes written to or read from the socket during the measurement. | | raw.Upload.ClientMeasurements.AppInfo.**ElapsedTime** | INTEGER | The duration of the measurement as measured by the M-Lab server in milliseconds. | +| raw.Upload.ClientMeasurements.**ConnectionInfo** | RECORD | | +| raw.Upload.ClientMeasurements.ConnectionInfo.**Client** | STRING | | +| raw.Upload.ClientMeasurements.ConnectionInfo.**Server** | STRING | | +| raw.Upload.ClientMeasurements.ConnectionInfo.**UUID** | STRING | UUID for TCP connection for this measurement. | | raw.Upload.ClientMeasurements.**BBRInfo** | RECORD | Instrumentation in the BBR TCP module in the kernel. | | raw.Upload.ClientMeasurements.BBRInfo.**BW** | INTEGER | The maximum end-to-end bandwidth from the server to the client as measured by BBR. | -| raw.Upload.ClientMeasurements.BBRInfo.**MinRTT** | INTEGER | The minimum round trip time as measured by BBR. | +| raw.Upload.ClientMeasurements.BBRInfo.**MinRTT** | INTEGER | The minimum round trip time as measured by BBR. Recorded in microseconds. | | raw.Upload.ClientMeasurements.BBRInfo.**PacingGain** | INTEGER | Fixed point multiplier used to set the pacing rate from the maximum bandwidth. The binary point varies by kernel version but the statistical mode is always 1.0. | | raw.Upload.ClientMeasurements.BBRInfo.**CwndGain** | INTEGER | Fixed point multiplier used to set the maximum window size from BW*MinRTT. The denominator varies by kernel version. | | raw.Upload.ClientMeasurements.BBRInfo.**ElapsedTime** | INTEGER | The duration of the measurement as measured by the M-Lab server in milliseconds. | @@ -145,7 +155,7 @@ | raw.Upload.ClientMeasurements.TCPInfo.**SegsOut** | INTEGER | The number of segments transmitted. Includes data and pure ACKs.
Kernel: segs_out in include/linux/tcp.h | | raw.Upload.ClientMeasurements.TCPInfo.**SegsIn** | INTEGER | The number of segments received. Includes data and pure ACKs.
Kernel: segs_in in include/linux/tcp.h | | raw.Upload.ClientMeasurements.TCPInfo.**NotsentBytes** | INTEGER | Number of bytes queued in the send buffer that have not been sent.
Kernel: tcpi_notsent_bytes() in net/ipv4/tcp.c | -| raw.Upload.ClientMeasurements.TCPInfo.**MinRTT** | INTEGER | Minimum Round Trip Time. From an older, pre-BBR algorithm.
Kernel: tcp_min_rtt in include/net/tcp.h | +| raw.Upload.ClientMeasurements.TCPInfo.**MinRTT** | INTEGER | Minimum Round Trip Time. Recorded in microseconds.
Kernel: tcp_min_rtt in include/net/tcp.h | | raw.Upload.ClientMeasurements.TCPInfo.**DataSegsIn** | INTEGER | Input segments carrying data (len>0).
Kernel: data_segs_in in include/net/tcp.h | | raw.Upload.ClientMeasurements.TCPInfo.**DataSegsOut** | INTEGER | Transmitted segments carrying data (len>0).
Kernel: data_segs_out in include/net/tcp.h | | raw.Upload.ClientMeasurements.TCPInfo.**DeliveryRate** | INTEGER | Observed Maximum Delivery Rate.
Kernel: tcp_compute_delivery_rate() in net/ipv4/tcp.c | @@ -175,9 +185,13 @@ | raw.Download.ServerMeasurements.**AppInfo** | RECORD | Server measurements performed outside of the kernel | | raw.Download.ServerMeasurements.AppInfo.**NumBytes** | INTEGER | The number of bytes written to or read from the socket during the measurement. | | raw.Download.ServerMeasurements.AppInfo.**ElapsedTime** | INTEGER | The duration of the measurement as measured by the M-Lab server in milliseconds. | +| raw.Download.ServerMeasurements.**ConnectionInfo** | RECORD | | +| raw.Download.ServerMeasurements.ConnectionInfo.**Client** | STRING | | +| raw.Download.ServerMeasurements.ConnectionInfo.**Server** | STRING | | +| raw.Download.ServerMeasurements.ConnectionInfo.**UUID** | STRING | UUID for TCP connection for this measurement. | | raw.Download.ServerMeasurements.**BBRInfo** | RECORD | Instrumentation in the BBR TCP module in the kernel. | | raw.Download.ServerMeasurements.BBRInfo.**BW** | INTEGER | The maximum end-to-end bandwidth from the server to the client as measured by BBR. | -| raw.Download.ServerMeasurements.BBRInfo.**MinRTT** | INTEGER | The minimum round trip time as measured by BBR. | +| raw.Download.ServerMeasurements.BBRInfo.**MinRTT** | INTEGER | The minimum round trip time as measured by BBR. Recorded in microseconds. | | raw.Download.ServerMeasurements.BBRInfo.**PacingGain** | INTEGER | Fixed point multiplier used to set the pacing rate from the maximum bandwidth. The binary point varies by kernel version but the statistical mode is always 1.0. | | raw.Download.ServerMeasurements.BBRInfo.**CwndGain** | INTEGER | Fixed point multiplier used to set the maximum window size from BW*MinRTT. The denominator varies by kernel version. | | raw.Download.ServerMeasurements.BBRInfo.**ElapsedTime** | INTEGER | The duration of the measurement as measured by the M-Lab server in milliseconds. | @@ -221,7 +235,7 @@ | raw.Download.ServerMeasurements.TCPInfo.**SegsOut** | INTEGER | The number of segments transmitted. Includes data and pure ACKs.
Kernel: segs_out in include/linux/tcp.h | | raw.Download.ServerMeasurements.TCPInfo.**SegsIn** | INTEGER | The number of segments received. Includes data and pure ACKs.
Kernel: segs_in in include/linux/tcp.h | | raw.Download.ServerMeasurements.TCPInfo.**NotsentBytes** | INTEGER | Number of bytes queued in the send buffer that have not been sent.
Kernel: tcpi_notsent_bytes() in net/ipv4/tcp.c | -| raw.Download.ServerMeasurements.TCPInfo.**MinRTT** | INTEGER | Minimum Round Trip Time. From an older, pre-BBR algorithm.
Kernel: tcp_min_rtt in include/net/tcp.h | +| raw.Download.ServerMeasurements.TCPInfo.**MinRTT** | INTEGER | Minimum Round Trip Time. Recorded in microseconds.
Kernel: tcp_min_rtt in include/net/tcp.h | | raw.Download.ServerMeasurements.TCPInfo.**DataSegsIn** | INTEGER | Input segments carrying data (len>0).
Kernel: data_segs_in in include/net/tcp.h | | raw.Download.ServerMeasurements.TCPInfo.**DataSegsOut** | INTEGER | Transmitted segments carrying data (len>0).
Kernel: data_segs_out in include/net/tcp.h | | raw.Download.ServerMeasurements.TCPInfo.**DeliveryRate** | INTEGER | Observed Maximum Delivery Rate.
Kernel: tcp_compute_delivery_rate() in net/ipv4/tcp.c | @@ -241,9 +255,13 @@ | raw.Download.ClientMeasurements.**AppInfo** | RECORD | Server measurements performed outside of the kernel | | raw.Download.ClientMeasurements.AppInfo.**NumBytes** | INTEGER | The number of bytes written to or read from the socket during the measurement. | | raw.Download.ClientMeasurements.AppInfo.**ElapsedTime** | INTEGER | The duration of the measurement as measured by the M-Lab server in milliseconds. | +| raw.Download.ClientMeasurements.**ConnectionInfo** | RECORD | | +| raw.Download.ClientMeasurements.ConnectionInfo.**Client** | STRING | | +| raw.Download.ClientMeasurements.ConnectionInfo.**Server** | STRING | | +| raw.Download.ClientMeasurements.ConnectionInfo.**UUID** | STRING | UUID for TCP connection for this measurement. | | raw.Download.ClientMeasurements.**BBRInfo** | RECORD | Instrumentation in the BBR TCP module in the kernel. | | raw.Download.ClientMeasurements.BBRInfo.**BW** | INTEGER | The maximum end-to-end bandwidth from the server to the client as measured by BBR. | -| raw.Download.ClientMeasurements.BBRInfo.**MinRTT** | INTEGER | The minimum round trip time as measured by BBR. | +| raw.Download.ClientMeasurements.BBRInfo.**MinRTT** | INTEGER | The minimum round trip time as measured by BBR. Recorded in microseconds. | | raw.Download.ClientMeasurements.BBRInfo.**PacingGain** | INTEGER | Fixed point multiplier used to set the pacing rate from the maximum bandwidth. The binary point varies by kernel version but the statistical mode is always 1.0. | | raw.Download.ClientMeasurements.BBRInfo.**CwndGain** | INTEGER | Fixed point multiplier used to set the maximum window size from BW*MinRTT. The denominator varies by kernel version. | | raw.Download.ClientMeasurements.BBRInfo.**ElapsedTime** | INTEGER | The duration of the measurement as measured by the M-Lab server in milliseconds. | @@ -287,7 +305,7 @@ | raw.Download.ClientMeasurements.TCPInfo.**SegsOut** | INTEGER | The number of segments transmitted. Includes data and pure ACKs.
Kernel: segs_out in include/linux/tcp.h | | raw.Download.ClientMeasurements.TCPInfo.**SegsIn** | INTEGER | The number of segments received. Includes data and pure ACKs.
Kernel: segs_in in include/linux/tcp.h | | raw.Download.ClientMeasurements.TCPInfo.**NotsentBytes** | INTEGER | Number of bytes queued in the send buffer that have not been sent.
Kernel: tcpi_notsent_bytes() in net/ipv4/tcp.c | -| raw.Download.ClientMeasurements.TCPInfo.**MinRTT** | INTEGER | Minimum Round Trip Time. From an older, pre-BBR algorithm.
Kernel: tcp_min_rtt in include/net/tcp.h | +| raw.Download.ClientMeasurements.TCPInfo.**MinRTT** | INTEGER | Minimum Round Trip Time. Recorded in microseconds.
Kernel: tcp_min_rtt in include/net/tcp.h | | raw.Download.ClientMeasurements.TCPInfo.**DataSegsIn** | INTEGER | Input segments carrying data (len>0).
Kernel: data_segs_in in include/net/tcp.h | | raw.Download.ClientMeasurements.TCPInfo.**DataSegsOut** | INTEGER | Transmitted segments carrying data (len>0).
Kernel: data_segs_out in include/net/tcp.h | | raw.Download.ClientMeasurements.TCPInfo.**DeliveryRate** | INTEGER | Observed Maximum Delivery Rate.
Kernel: tcp_compute_delivery_rate() in net/ipv4/tcp.c | diff --git a/_includes/schema_pcaprow.md b/_includes/schema_pcaprow.md index 7e45965e..09c2be12 100644 --- a/_includes/schema_pcaprow.md +++ b/_includes/schema_pcaprow.md @@ -5,7 +5,9 @@ | parser.**Version** | STRING | Version is the symbolic version (if any) of the running server code that produced this measurement. | | parser.**Time** | TIMESTAMP | The time that the parser processed this row. | | parser.**ArchiveURL** | STRING | The Google Cloud Storage URL to the archive containing the Filename for this row. | -| parser.**Filename** | STRING | | +| parser.**Filename** | STRING | The name of the file within the ArchiveURL originally created by the measurement service. Results in the raw record are derived from measurements in this file. | | parser.**Priority** | INTEGER | | -| parser.**GitCommit** | STRING | | +| parser.**GitCommit** | STRING | The git commit of this build of the parser. | +| parser.**ArchiveSize** | INTEGER | The original archive size as found in GCS. | +| parser.**FileSize** | INTEGER | The size of the file data provided to the parser for this row. | | **date** | DATE | Date is used by BigQuery to partition data to improve query performance. |