update status of all tests and core services (#542)

* update status of all tests and core services; adds pages for new tests and core services * removed missing schema referece * added page for pcap service, updated tests, retired sidecar services, imported new auto-generated schema for ndt & tcpinfo * add new & updated test icons; update ndt, wehe, neubot text; test descriptions; add image attribution variable and added it to alt & title tags for images on the tests page * updated SK status * corrected incorrectly formatted link * revised SK text * revised SK text per final reviewer comments
m-lab · Feb 14, 2020 · aedbe89 · aedbe89
1 parent 78737e9
commit aedbe89
Show file tree

Hide file tree

Showing 18 changed files with 452 additions and 350 deletions.
diff --git a/_includes/grid.html b/_includes/grid.html
@@ -44,13 +44,13 @@ <h2 class="grid-header col-xs-12" id="{{ grid.group-heading | downcase | replace
           {% endif %}
 
             {% if grid.group-list-class == "l-grid l-3col" %}
-            <div class="grid-img{{ grid.grid-img-class }}"><img src="{{ site.baseurl }}/images{{ img_dir }}/{{ item.image }}" alt="{{ item.heading }}" class=""></div>
+            <div class="grid-img{{ grid.grid-img-class }}"><img src="{{ site.baseurl }}/images{{ img_dir }}/{{ item.image }}" alt="{{ item.heading }} - {{ item.img_attribution }}" title="{{ item.heading }} - {{ item.img_attribution }}" class=""></div>
             <div class="grid-info">
               <h3 class="grid-heading">{{ item.heading }}</h3>
               <div class="grid-description"><p>{{ item.desc }}</p></div>
             </div>
             {% else %}
-            <img src="{{ site.baseurl }}/images{{ img_dir }}/{{ item.image }}" alt="{{ item.alt }}" class="grid-link is-plain">
+            <img src="{{ site.baseurl }}/images{{ img_dir }}/{{ item.image }}" alt="{{ item.alt }} - {{ item.img_attribution }}" title="{{ item.heading }} - {{ item.img_attribution }}" class="grid-link is-plain">
             {% endif %}
 
           {% if item.link %}

diff --git a/_includes/schema_ndtresultrow.md b/_includes/schema_ndtresultrow.md
diff --git a/_includes/schema_tcprow.md b/_includes/schema_tcprow.md
diff --git a/_pages/05-tests.md b/_pages/05-tests.md
@@ -7,72 +7,83 @@ menu-item: true
 breadcrumb: tests
 grid-section:
   - group-heading: "Current Tests"
-    group-description: "Performance tests measure basic and advanced performance characteristics of your network, such as speed, latency, jitter, and much more."
+    group-description: ""
     group-list-class: "l-grid l-3col"
     grid-items:
-      - link: "/tests/bismark/"
-        image: "neubot.png"
-        heading: "BISmark"
-        desc: "Allows you to apply to host a home router that also tests network performance over time."
-        categories: "performance"
-      - link: "/tests/mobiperf/"
-        image: "mobile.png"
-        heading: "MobiPerf"
-        desc: "Measures network performance on mobile platforms."
-        categories: "performance"
       - link: "/tests/ndt/"
         image: "ndt.png"
+        img_attribution: ""
         heading: "NDT (Network Diagnostic Tool)"
         desc: "Tests your connection speed, and provides a sophisticated diagnosis of problems limiting speed."
-        categories: "performance"
-      - link: "/tests/paris_traceroute/"
-        image: "arrow-swoop.png"
-        heading: "Paris Traceroute"
+      - link: "/tests/neubot/"
+        image: "neubot.png"
+        img_attribution: ""
+        heading: "Neubot DASH"
+        desc: "DASH is designed to measure the quality of tested networks by emulating a video streaming player."
+      - link: "/tests/wehe/"
+        image: "wehe.png"
+        img_attribution: ""
+        heading: "WeHe"
+        desc: "Wehe uses your device to exchange Internet traffic recorded from real, popular apps like YouTube and Spotify, and attempts to tell you whether your ISP is giving different performance to an app's network traffic."
+  - group-heading: "Current M-Lab Core Services"
+    group-description: ""
+    group-list-class: "l-grid l-3col"
+    grid-items:
+      - link: "/tests/tcp-info/"
+        image: "tcpinfo.png"
+        img_attribution: "Created by Pravin Unagar from Noun Project"
+        heading: "TCP INFO"
+        desc: "Collects statistics about the TCP connections running on the M-Lab platform using tcp-info."
+      - link: "/tests/pcap/"
+        image: "pcap.png"
+        img_attribution: "Created by Jo Santos from Noun Project"
+        heading: "Packet Headers"
+        desc: "Collects packet headers for all incoming TCP flows and saves each stream of packet captures into a per-stream .pcap file."
+      - link: "/tests/traceroute/"
+        image: "traceroute.png"
+        img_attribution: "Created by Popular from Noun Project"
+        heading: "Traceroute"
         desc: "Collects network path information for every connection to the M-Lab platform."
-        categories: "network state"
-      - link: "/tests/samknows/"
-        image: "samknows.png"
-        heading: "SamKnows"
-        desc: "The SamKnows performance testing platform includes several performance and quality of service tests."
-      - link: "/tests/sidestream/"
-        image: "sidestream.png"
-        heading: "SideStream"
-        desc: "Collects statistics about the TCP connections running on the M-Lab platform."
-        categories: "performance"
+  - group-heading: "Inactive Tests"
+    group-description: "Tests that were recently active on the M-Lab 1.0 platform, but have not yet been redesigned by their developers to run on the M-Lab 2.0 platform."
+    group-list-class: "l-grid l-3col"
+    grid-items:
       - link: "/tests/reverse_traceroute/"
         image: "arrow-swoop.png"
         heading: "Reverse Traceroute"
         desc: "Measures the network path back to a user from selected network endpoints."
-        categories: "network state"
   - group-heading: "Retired Tests"
     group-description: "Tests in this section were once hosted with M-Lab but have since been retired. The data collected by these tests while they were hosted on with M-Lab remains available. Please see each individual test's page for more information."
     group-list-class: "l-grid l-3col"
     grid-items:
+      - link: "/tests/bismark/"
+        image: "neubot.png"
+        heading: "BISmark"
+        desc: "Allowed people to host a home router that also tests network performance over time."
       - link: "/tests/glasnost/"
         image: "glasnost.png"
         heading: "Glasnost"
         desc: "Glasnost tested for application-specific blocking or throttling, and was decommissioned on 07/07/2017. The source code is still available."
-        categories: "performance, transparency"
-      - link: "/tests/neubot/"
-        image: "neubot.png"
-        heading: "Neubot"
-        desc: "Performs periodic tests to measure network performance and traffic throttling."
-        categories: "performance, transparency"
       - link: "/tests/npad/"
         image: "npad.png"
         heading: "NPAD (Network Path &amp; Application Diagnostics)"
         desc: "Diagnosed common problems that impact last-mile broadband networks."
-        categories: "performance"
       - link: "/tests/ooni/"
         image: "ooni.png"
         heading: "OONI Probe"
         desc: "Measured specific use cases of network interference."
-        categories: "transparency"
+      - link: "/tests/mobiperf/"
+        image: "mobile.png"
+        heading: "MobiPerf"
+        desc: "Measured network performance on mobile platforms."
       - link: "/tests/pathload2/"
         image: "pathload2.png"
         heading: "Pathload2"
         desc: "Tested the available bandwidth of an Internet connection until it was decommissioned from the M-Lab platform on 12/21/2012. However, the data and source code are still available."
-        categories: "performance"
+      - link: "/tests/samknows/"
+        image: "samknows.png"
+        heading: "SamKnows"
+        desc: "M-Lab hosted the off-net servers for SamKnows performance testing used in the FCC's Measuring Broadband America Program from 2009-2019."
       - link: "/tests/shaperprobe/"
         image: "arrow-swoop.png"
         heading: "Shaperprobe"
@@ -81,7 +92,18 @@ grid-section:
         image: "mobile.png"
         heading: "Windrider"
         desc: "Attempted to detect whether a mobile provider was performing application or service specific differentiation until it was decommissioned on 01/17/2013. The source code is still available."
-        categories: "transparency"
+  - group-heading: "Retired M-Lab Core Services"
+    group-description: ""
+    group-list-class: "l-grid l-3col"
+    grid-items:
+      - link: "/tests/sidestream/"
+        image: "sidestream.png"
+        heading: "SideStream"
+        desc: "Collected web100 statistics about the TCP connections running on the M-Lab 1.0 platform."
+      - link: "/tests/paris_traceroute/"
+        image: "arrow-swoop.png"
+        heading: "Paris Traceroute"
+        desc: "Collected network path information for every connection to the M-Lab 1.0 platform."
 
 ---
 

diff --git a/_pages/tests/bismark.md b/_pages/tests/bismark.md
@@ -7,16 +7,8 @@ breadcrumb: tests
 
 # BISmark
 
-BISmark is an OpenWRT-based (http://openwrt.org) platform that measures Internet service provider (ISP) performance and traffic inside home networks. BISmark takes the place of your home router, performing all of your router's normal functions in addition to periodic network performance measurements. BISmark users gain access to a personalized dashboard showing performance over time and comparing performance across ISPs and regions.
-
-[Apply to host a BISmark gateway](http://projectbismark.net/){:target="_blank"}
-
-Please cite this data set as follows: **The M-Lab BISmark Data Set, &lt;date range used&gt;. https://measurementlab.net/tests/bismark**
-
-**Data** collected by BISmark is available in raw format at [http://uploads.projectbismark.net/](http://uploads.projectbismark.net/){:target="_blank"}.
+BISmark was an OpenWRT-based (http://openwrt.org) platform that measured Internet service provider (ISP) performance and traffic inside home networks. BISmark was hosted by M-Lab until December 2019.
 
 **Source code** is available at [https://github.com/projectbismark/](https://github.com/projectbismark/){:target="_blank"}.
 
-The BISmark data format is described at [https://github.com/projectbismark/bismark-management-server/blob/master/xml_description](https://github.com/projectbismark/bismark-management-server/blob/master/xml_description){:target="_blank"}.
-
-**Get more information** at [http://projectbismark.net/](http://projectbismark.net/){:target="_blank"}.
+**Get more information** by contacting the BISmark team at [[email protected]](mailto:[email protected]).
diff --git a/_pages/tests/ndt/ndt.md b/_pages/tests/ndt/ndt.md
@@ -52,6 +52,12 @@ or, in [BibTeX](https://en.wikipedia.org/wiki/BibTeX){:target="_blank"} format:
 }
 ```
 
+## Policies & Support Information
+
+NDT is maintained by Measurement Lab, and is governed by the [M-Lab Privacy Policy]({{ site.baseurl }}/privacy/).
+
+Get support for NDT emailing [[email protected]](mailto:[email protected]).
+
 ## Data Collected by NDT
 
 When you run NDT, the IP address provided by your Internet Service Provider will be collected along with your measurement results. M-Lab conducts the test and publishes all test results to promote Internet research. NDT does not collect any information about you as an Internet user.

diff --git a/_pages/tests/neubot.md b/_pages/tests/neubot.md
@@ -5,12 +5,44 @@ title: "Neubot"
 breadcrumb: tests
 ---
 
-# Neubot
+# Neubot Experiment & Dash Streaming Test
 
-Neubot (the network neutrality bot) was a free software Internet bot, developed and maintained by the Nexa Center for Internet & Society, that gathered network performance data useful to investigate network neutrality. The Neubot client was [retired in January 2019](http://www.neubot.org/2019/01/retiring-neubot-client.html){:target="_blank"}. If you are interested in running tests similar to Neubot, M-Lab recommends using [OONI Probe](https://ooni.torproject.org/install/){:target="_blank"}. _M-Lab will continue hosting the Neubot server until our [platform modernization project]({{ site.baseurl }}/blog/modernizing-mlab/#modernizing-the-m-lab-platform) is completed, after which it will be retired._
+The Neubot experiment was originally developed as a free software Internet bot developed and maintained by the Nexa Center for Internet & Society, that gathered network performance data useful to investigate network neutrality. The original Neubot client was [retired in January 2019](http://www.neubot.org/2019/01/retiring-neubot-client.html){:target="_blank"}.
 
-**Data** collected by Neubot while hosted on M-Lab remains available in raw format at [https://console.cloud.google.com/storage/browser/archive-measurement-lab/neubot](https://console.cloud.google.com/storage/browser/archive-measurement-lab/neubot){:target="_blank"}.
+M-Lab continues to host Neubot's DASH streaming test, one of the original suite of Neubot tests.
 
-Please cite this data set as follows: **The M-Lab Neubot Data Set, &lt;date range used&gt;. https://measurementlab.net/tests/neubot**
+## DASH Test
 
-**Get more information** at [http://www.neubot.org/](http://www.neubot.org/){:target="_blank"}.
+DASH is designed to measure the quality of tested networks by emulating a video streaming player. This test is called DASH because it uses the DASH ([Dynamic Adaptive Streaming over HTTP](https://en.wikipedia.org/wiki/Dynamic_Adaptive_Streaming_over_HTTP){:target="_blank"}) streaming technique.
+
+DASH can be useful to understand the baseline streaming performance of a specific network connection. It measures video-related metrics as well as network metrics that are key to understand the reason of performance issues.
+
+When you run the test, it emulates the streaming of a thirty-second video from an M-Lab server. The video is divided in fifteen two seconds segments. When the client requests a video segment, it must also specify the video quality (e.g., SD, HD, Super HD). The higher the request quality, the bigger the returned segment. During the streaming, the client seeks to use the higher quality that does not load the network, creating queues, so that the streaming can continue smoothly.
+
+The player is simple in that it does not employ algorithms that real players (e.g. YouTube, Netflix) implement to keep the streaming quality stable and to avoid stalls. This simplicity is, however, key to understanding the contribution of the network to streaming quality, which otherwise could be masked by smart players’ behavior.
+
+As a result, it is expected that real players will be generally faster than this test, because they implement more optimization techniques. However, if the [throttling of video is caused by congestion at interconnection points](https://arstechnica.com/tech-policy/2010/12/comcastlevel3/){:target="_blank"}, this test may result faster when the network path from the client to the M-Lab server does not pass through the congested interconnection point.
+
+## Source code
+
+* [Neubot M-Lab Server & Client](https://github.com/m-lab/dash){:target="_blank"}
+
+## Citing the M-Lab Neubot Dataset
+
+Please cite this data set as follows: **The M-Lab Neubot - Dash Data Set, &lt;date range used&gt;. https://measurementlab.net/tests/neubot**
+
+## Policies & Support Information
+
+**Get more information** at [http://www.neubot.org/](http://www.neubot.org/){:target="_blank"}.
+
+## Data Collected by DASH in Raw Format
+
+Data collected by Neubot's DASH test while hosted on M-Lab is available in raw format at [https://console.cloud.google.com/storage/browser/archive-measurement-lab/neubot/dash](https://console.cloud.google.com/storage/browser/archive-measurement-lab/dash){:target="_blank"}.
+
+## Data Collected by Neubot in Raw Format
+
+Data collected by Neubot while hosted on M-Lab is available in raw format at [https://console.cloud.google.com/storage/browser/archive-measurement-lab/neubot](https://console.cloud.google.com/storage/browser/archive-measurement-lab/neubot){:target="_blank"}.
+
+## Neubot & DASH Data in BigQuery
+
+Neubot and DASH test data are not published to BigQuery at this time.
diff --git a/_pages/tests/paris_traceroute/paris_traceroute.md b/_pages/tests/paris_traceroute/paris_traceroute.md
@@ -7,7 +7,7 @@ breadcrumb: tests
 
 # Paris Traceroute
 
-Paris Traceroute collects network path information for every connection used by the measurement tests running on the M-Lab platform. In addition to the route and network topology data provided by regular traceroute, Paris Traceroute detects load balancing, noting when a transmission is split between two paths. Like SideStream, Paris Traceroute runs when another M-Lab test makes a connection with the platform.
+Paris Traceroute was used by M-Lab to collect network path information for every connection used by the measurement tests running on the M-Lab platform until the fourth quarter of 2019 (Q42019). At that time, M-Lab replaced Paris-traceroute with the [Scamper](https://www.caida.org/tools/measurement/scamper/){:target="_blank"} traceroute tool provided by the [Center for Applied Internet Data Analysis, CAIDA](https://www.caida.org){:target="_blank"}. In addition to the route and network topology data provided by regular traceroute, Paris Traceroute detected load balancing, noting when a transmission was split between two paths.
 
 More information about Paris Traceroute is available at: [http://www.paris-traceroute.net](http://www.paris-traceroute.net/){:target="_blank"}.
 
@@ -22,13 +22,12 @@ Please cite this data set as follows: **The M-Lab Paris Traceroute Data Set, &lt
 
 ## Paris Traceroute Data in Raw Format
 
-Data collected by Paris Traceroute is available in raw format in Google Cloud Storage: [https://console.cloud.google.com/storage/browser/archive-measurement-lab/paris-traceroute/](https://console.cloud.google.com/storage/browser/archive-measurement-lab/paris-traceroute/){:target="_blank"}.
+Data collected by Paris Traceroute remains available in raw format in Google Cloud Storage: [https://console.cloud.google.com/storage/browser/archive-measurement-lab/paris-traceroute/](https://console.cloud.google.com/storage/browser/archive-measurement-lab/paris-traceroute/){:target="_blank"}.
 
 ## Paris Traceroute Data in BigQuery
 
-M-Lab parses all Paris Traceroute data into BigQuery tables and views, and makes query access available for free by subscription to a Google Group. Find out more about how to get access on our [BigQuery QuickStart page]({{ site.baseurl }}/data/docs/bq/quickstart/).
+M-Lab parsed all Paris Traceroute data into BigQuery tables and views, and makes query access available for free by subscription to a Google Group. Find out more about how to get access on our [BigQuery QuickStart page]({{ site.baseurl }}/data/docs/bq/quickstart/).
 
 BigQuery Tables/Views/Schema(s) for Paris Traceroute Data:
 
 * [Paris Traceroute data in BigQuery](https://console.cloud.google.com/bigquery?project=measurement-lab&p=measurement-lab&d=aggregate&t=traceroute&page=table){:target="_blank"}
-* _Paris Traceroute schema_