Skip to content

Mobility Data Manipulation

Jump to bottom
Joshua Selsky edited this page Aug 19, 2013 · 46 revisions

Mobility Upload

What does it do?

Allows users to upload mobility data to the system. Mobility data is not attached to a campaign or a class.

URI

mobility/upload

Access Rules

Any authenticated ohmage user may upload mobility data.

Input Parameters

  • (r) user = A valid username.
  • (r) password = The hashed password for the above user.
  • (r) client = A short description of the software client performing the upload.
  • (r) data = A JSON Array of mobility data points.

Data Point JSON Syntax

Mobility data points come in two flavors: mode_only and sensor_data. The former contains only a mobility mode while the latter contains the mobility mode and the sensor data belonging to that mode.

mode_only subtype

JSON Key Usage/Description
id A UUID for this record
time The integer milliseconds since the UNIX epoch representing the time when this data point was created.
timezone The timezone ID for the timezone on the device when this data point was created.
location_status A string describing location status. Must be one of: unavailable, valid, inaccurate, stale. If the status is unavailable, it is an error to send a location object.
location A JSON object for housing location data.
latitude A double representing latitude.
longitude A double representing longitude.
provider A non-empty string representing the location provider. (e.g., GPS or network).
accuracy A float representing accuracy in meters from actual location.
time A long value representing the milliseconds since the UNIX epoch at which time this location value was collected.
timezone The timezone ID for the timezone of the device when this location value was collected.
subtype Must be mode_only.
mode A string defining a mobility mode. Must be one of: still, walk, run, bike, drive

JSON Example

"data":[
    {
        "id":"eba1e960-8b0a-11e1-88a7-0050563cd332",
        "time":1328659200,
        "timezone":"PT",
        "subtype":"mode_only",
        "location_status":"valid"
        "location": {
            "latitude":41.891,
            "longitude":-87.628,
            "accuracy":10,
            "provider":"GPS",
            "time":1328659369,
            "timezone":"PT"
        },
        "mode":"still"
    },
    ...
]

sensor_data subtype

JSON Key Usage/Description
time The integer milliseconds since the UNIX epoch representing the time when this data point was created.
timezone The timezone ID for the timezone on the device when this data point was created.
location_status A string describing location status. Must be one of: unavailable, valid, inaccurate, stale. If the status is unavailable, it is an error to send a location object.
location A JSON object for housing location data.
latitude A double representing latitude.
longitude A double representing longitude.
provider A non-empty string representing the location provider. (e.g., GPS or network).
accuracy A float representing accuracy in meters from actual location.
time An integer value representing the milliseconds since the UNIX epoch at which time this location value was collected.
timezone The timezone ID for the timezone of the device when this location value was collected.
mode A string defining a mobility mode. Must be one of: still, walk, run, bike, drive.
subtype Must be sensor_data.
data This is the sensor data:
mode The mode that was calculated based on this information.
speed A double representing device speed.
accel_data A JSON Array of JSON Objects of the form [{"x":0.0,"y":0.0,"z":0.0}, ...].
wifi_data A JSON object for housing wi-fi scan data.
time An integer value representing the milliseconds since the UNIX epoch at which time this wifi data value collected.
timezone A string value defining the timezone in which the scan took place.
scan A JSON Array of JSON Objects of the form [{"ssid":"ssid_name", "strength":0.0},...]

JSON Example

"data":[
    {
        "time":1328659200,
        "timezone":"PT",
        "subtype":"sensor_data",
        "location_status":"valid"
        "location": {
            "latitude":41.891,
            "longitude":-87.628,
            "accuracy":10,
            "provider":"GPS",
            "time":1328659369,
            "timezone":"PT"
        },
        "mode":"still",
        "data":{
            "mode":"still",
            "speed":0.0,
            "accel_data":[{"x":0.0,"y":0.0,"z":0.0},...],
            "wifi_data":{
                 "time":1328660189,
                 "timezone":"PT",
                 "scan":[{"ssid":"ssid_name", "strength":0.0},...]
            }
        }
    },
    ...
]

Example POST

POST /app/mobility/upload HTTP/1.1
 Host: dev.ohmage.org
 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.77 Safari/535.7
 Content-Length: byte-length-of-content
 Content-Type: application/x-www-form-urlencoded

  user=username
  &password=hashed_password
  &client=curl
  &data=<A JSON Array of JSON Objects matching either the mode_only or sensor_data syntax.>

cURL Examples

Uncompressed

curl --trace-ascii ~/curltrace.txt -d @/mobility_data_and_auth_params.txt http://localhost:8080/app/mobility/upload

Compressed

curl --trace-ascii ~/curltrace.txt -H "Content-Encoding: gzip" --data-binary @/mobility_data_and_auth_params.tgz http://localhost:8080/app/mobility/upload

Output Format

Success

{
   "result" : "success"
}

Failure

See the error page for a description of error codes and their associated descriptions.

↑ Back to Top

Mobility Read

What does it do?

Allows users to read mobility data.

URI

mobility/read

Access Rules

Mobility data is private to the user uploading the data. There is a configuration flag in ohmage that allows privileged class users to view other user's mobility data provided that the two users have a class in common and the requesting user is privileged in the common class (or classes).

Input Parameters

Supports Token-Based Authentication ...

  • (r) auth_token = A valid authentication token from user/auth_token. May also be set using the Set-Cookie header.

... or User-Password Authentication

  • (r) user = A username of the user attempting to login
  • (r) password = A password for the above user.

Additional Parameters

  • (r) client = A short description of the software client performing the upload.
  • (r) date = An ISO8601-formatted date from which to retrieve mobility data points.
  • (o) username = The username of the user whose data is desired. This is only applicable if the requesting user is an admin or if the server allows it (the "mobility_enabled" flag from config/read) and the requesting user is privileged in any class to which the desired user belongs.
  • (o) with_sensor_data = true/false Indicates whether or not to return the sensor data with the regular data. The default is false. Deprecated in favor of 'column_list'.
  • (o) column_list = A comma-separated list of columns. If any column header is given, for example sensor:accel_data, and any of its sub-columns are given, for example accel:x, the result will be the greater of the columns. In this example, all accelerometer data columns would be returned because sensor:accel_data exists, and accel:x would be effectively ignored.

Example POST

POST /app/mobility/read HTTP/1.1
 Host: dev.ohmage.org
 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.77 Safari/535.7
 Content-Length: byte-length-of-content
 Content-Type: application/x-www-form-urlencoded

  user=username
  &password=hashed_password
  &client=curl
  &date=2012-02-29

cURL Example

curl -v -d "client=curl&date=2012-02-29&auth_token=abcfcd36-ab25-4494-8434-7798cb1d718e" https://dev.andwellness.org/app/mobility/read

Output Format

Success

The output is "minified" JSON due to the possible high frequency of mobility data collection. The typical number of points is 1440 per day (once per minute).

The output contains the standard result element along with a JSON Array of JSON Objects.

The following is an example output where the following keys were given, "mobility:id, mobility:mode, mobility:timestamp, mobility:time, mobility:timezone, location, mobility:sub_type, sensor_data, sensor_data:speed, classification:mode". Note that "sensor_data:speed" is overridden by "sensor_data".

{
   "result" : "success"
   "data" : [
       {
           "id":"81DB3AE9-16CB-4B63-B2F6-9E30404B7C65",
           "m":"still",
           "ts":"2012-02-07 18:07:44",
           "t":1328666862,
           "tz":"PT",
           "ls":"valid",
           "l":{
               "la":41.891,
               "lo":-87.628,
               "pr":"GPS",
               "t":1328666862,
               "tz":"PT"
               "ac":30
           },
           "st":"sensor_data",
           "data":{
               "sp":83.4389,
               "wd":{
                   "t":1328666321,
                   "tz":"America/Los_Angeles",
                   "sc":[
                       {
                           "ss":"72:3b:39:11:8a",
                           "st":-3.389289
                       },
                       ...
                   ]
               },
               "ad":[
                   {
                       "x":3982.3902,
                       "y":89.8928094,
                       "z":39.93092899
                   },
                   ...
               ],
           },
         "cd":{
           "m":"walking"
         }
       },
       ...
   ]
}

Failure

See the error page for a description of error codes and their associated descriptions.

↑ Back to Top

Mobility Read - CSV

What does it do?

Allows users to read mobility data and has it returned as a CSV file.

URI

mobility/read/csv

Access Rules

Mobility data is private to the user uploading the data. There is a configuration flag in ohmage that allows privileged class users to view other user's mobility data provided that the two users have a class in common and the requesting user is privileged in the common class (or classes).

Input Parameters

Supports Token-Based Authentication ...

  • (r) auth_token = A valid authentication token from user/auth_token. May also be set using the Set-Cookie header.

... or User-Password Authentication

  • (r) user = A username of the user attempting to login
  • (r) password = A password for the above user.

Additional Parameters

  • (r) client = A short description of the software client performing the upload.
  • (r) start_date = An ISO8601-formatted date from which to begin retrieving mobility data points.
  • (r) end_date = An ISO8601-formatted date from which to stop retrieving mobility data points.
  • (o) username = The username of the user whose data is desired. This is only applicable if the requesting user is an admin or if the server allows it (the "mobility_enabled" flag from config/read) and the requesting user is privileged in any class to which the desired user belongs.
  • (o) column_list = A comma-separated list of columns. If any column header is given, for example sensor:accel_data, and any of its sub-columns are given, for example accel:x, this will return an error. Because the columns are returned in the same order that they are requested and because sensor:accel_data would be exploded into its concrete sub-columns, this would result in a duplicate accel:x column. Duplicate columns in this API are an error.

Example POST

POST /app/mobility/read/csv HTTP/1.1
 Host: dev.ohmage.org
 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.77 Safari/535.7
 Content-Length: byte-length-of-content
 Content-Type: application/x-www-form-urlencoded

  user=username
  &password=hashed_password
  &client=curl
  &start_date=2012-02-29
  &end_date=2012-03-02

cURL Example

curl -v -d "client=curl&start_date=2012-03-15&end_date=2012-03-20&auth_token=abcfcd36-ab25-4494-8434-7798cb1d718e" https://dev.andwellness.org/app/mobility/read/csv

Output Format

Success

The HTTP response will contain a CSV file where the first row is the column names separated by a comma and the subsequent rows, if any, are the actual data corresponding to their respective columns.

Columns

Column Name JSON Key Data Returned
mobility:id id A string representing this point's unique identifier.
mobility:time t A long representing the number of milliseconds since the Unix epoch when this point was generated. This is in UTC.
mobility:timestamp ts An ISO8601-formatted string representing when this point was generated based on the time zone. This is in the given timezone, see "mobility:timezone".
mobility:timezone tz The time zone of the device when this point was generated.
location l A JSON object representing the location information.
location:status location_status The status of the location, one of "valid", "network", "inaccurate", "stale", "unavailable". This key-value pair will be placed in the parent's container rather than this JSON.
location:time t A long representing the milliseconds since the Unix epoch when this location information was aggregated. This is in UTC.
location:timezone tz A string representing the time zone of the device.
location:latitude la A double representing the latitude of the device.
location:longitude lo A double representing the longitude of the device.
location:accuracy ac A double representing the accuracy of the reading.
location:provider pr A string describing who generated this location reading.
mobility:mode m The mode of this point as generated by the user's device.
mobility:subtype st The sub-type of this Mobility point. One of "mode_only" or "sensor_data".
mobility:sensor_data data The sensor data representing this point if this point's sub-type is "sensor_data". If not, this will be omitted from the JSON.
sensor:speed sp The speed of the device as reported by the GPS.
sensor:accel_data ad The accelerometer data recorded just before this point was generated. The format is a JSON array of JSON objects, where each object contains a single (x, y, z) reading of the accelerometer, and the array represents all of the readings for this point.
accel:x x A double representing the x component of the accelerometer reading.
accel:y y A double representing the y component of the accelerometer reading.
accel:z z A double representing the z component of the accelerometer reading.
sensor:wifi_data wd The WiFi data recorded just before this point was generated. The format is a JSON object.
wifi:time t A long representing the number of milliseconds since the Unix epoch when this information was aggregated. This is in UTC.
wifi:timestamp ts An ISO8601-formatted string representing when this information was aggregated based on the "wifi:timezone".
wifi:timezone tz The time zone of the device when this information was aggregated.
wifi:scan sc The information aggregated from this scan. The format is a JSON array of JSON objects where each object is the SSID and strength of a WiFi access point and the array is the collection of all access points heard in this scan.
wifi:ssid ss The SSID of the access point.
wifi:strength st The strength of the signal from this access point.
mobility:classifier_data cd The classifier data generated for this point by the server if this point's sub-type is "sensor_data" and its more is not "error". If this mode is not a "sensor_data" sub-type or its mode is "error", it will be omitted from the JSON.
classification:fft fft A JSON array of doubles representing the fast Fourier transform of this sensor data.
classification:variance v The variance of the sensor data.
classification:average a The average value of the sensor data.
classification:mode m The mode calculated by server-side classifier.
  • The mode may be "ERROR" meaning that any of the data in the sensor data fields may not be available and that the classification information will definitely be missing. Whenever any data is missing, its corresponding key will be omitted.

Failure

See the error page for a description of error codes and their associated descriptions.

Back to Top

Mobility Read Chunked

What does it do?

Allows users to read a "chunked" version of mobility data over a date range. Mobility data is aggregated into ten-point chunks where the count of the mobility modes is returned in each chunk.

URI

mobility/read/chunked

Access Rules

See access rules for mobility/read above.

Input Parameters

Supports Token-Based Authentication ...

  • (r) auth_token = A valid authentication token from user/auth_token. May also be set using the Set-Cookie header.

... or User-Password Authentication

  • (r) user = A username of the user attempting to login
  • (r) password = A password for the above user.

Additional Parameters

  • (r) client = A short description of the software client performing the upload.
  • (r) start_date = An ISO8601-formatted date indicating the start of a date range.
  • (r) end_date = An ISO8601-formatted date indicating the end of a date range.
  • (o) chunk_mins = The number of minutes per chunk. The default is 10.

Example POST

POST /app/mobility/read/chunked HTTP/1.1
 Host: dev.ohmage.org
 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.77 Safari/535.7
 Content-Length: byte-length-of-content
 Content-Type: application/x-www-form-urlencoded

  user=username
  &password=hashed_password
  &client=curl
  &start_date=2012-02-29
  &end_date=2012-03-15

cURL Example

curl -v -d "start_date=2012-03-15&end_date=2012-03-16&auth_token=abcfcd36-ab25-4494-8434-7798cb1d718e&client=curl" https://dev.andwellness.org/app/mobility/read/chunked

Output Format

The output is "minified" JSON due to the possible high frequency of mobility data collection.

The output contains the standard result element along with a JSON Array of JSON Objects with the following schema.

JSON Key Usage/Description
mc A JSON Object defining mobility modes and their counts in the format {"still:1,"walk":2,"run":7} TODO - need to verify the modes allowed in the object -- we used to have an INVALID mode indicating that for some portion of the chunk there was no data.
d An integer representing the chunk duration in milliseconds.
tz A string timezone ID for the timezone on the device when this data point was created.
ts An ISO8601-formatted date-time string representing the time at which the earliest point in this chunk was generated.
ls A string describing location status of the first point in the chunk. Must be one of: unavailable, valid, inaccurate, stale. If the status is unavailable, the location object below will not be present.
l A JSON object for housing location data for the first point in the chunk. If the location status is "unavailable", this will not exist.
la A double representing latitude.
lo A double representing longitude.
pr A non-empty string representing the location provider. (e.g., GPS or network).
ac A float representing accuracy in meters from actual location.
t An integer value representing the milliseconds since the UNIX epoch at which time this location value was collected.
tz The timezone ID for the timezone of the device when this location value was collected.

Success

{
   "result" : "success"
   "data" : [
       {
           "mc":{"still:1,"walk":2,"run":7},
           "d":600000,
           "ts":"2012-02-07 18:07:44",
           "tz":"PT",
           "ls":"valid",
           "l":{
               "la":41.891,
               "lo":-87.628,
               "pr":"GPS",
               "t":1328666862,
               "tz":"PT",
               "ac":30
           },
       },..
   ]

}

Failure

See the error page for a description of error codes and their associated descriptions.

↑ Back to Top

Mobility Read Aggregate

What does it do?

Allows users to read the Mobility data and have it aggregate the number of milliseconds per mode for a given duration over a date range. The user specifies a date range of Mobility data. Then, they specify a "chunk duration" in terms of days. Then, the data is "chunked" by starting at the start date, creating a chunk of the number of days, and then creating new chunks where the last chunk left off until it reaches the end date. This is in contrast to mobility/read/chunked, where a new chunk is based on the date and time of the first point in the chunk. For each chunk, it counts the number of milliseconds for each mode. If two points are separated by less than 6 minutes, the time between the points is all considered to be the same made as the latter of the two points. If a point doesn't have another point within 6 minutes before it, then the duration for that point is exactly one minute.

URI

mobility/aggregate/read

Access Rules

See access rules for mobility/read above.

Input Parameters

Supports Token-Based Authentication ...

  • (r) auth_token = A valid authentication token from user/auth_token. May also be set using the Set-Cookie header.

... or User-Password Authentication

  • (r) user = A username of the user attempting to login
  • (r) password = A password for the above user.

Additional Parameters

  • (r) client = A short description of the software client performing the upload.
  • (r) start_date = An ISO8601-formatted date indicating the start of a date range.
  • (r) end_date = An ISO8601-formatted date indicating the end of a date range.
  • (r) duration = The number of days per chunk.
  • (o) username = A user's username on whom the data should be aggregated.

Example POST

POST /app/mobility/aggregate/read HTTP/1.1
 Host: dev.ohmage.org
 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.77 Safari/535.7
 Content-Length: byte-length-of-content
 Content-Type: application/x-www-form-urlencoded

  user=username
  &password=hashed_password
  &client=curl
  &start_date=2012-02-29
  &end_date=2012-03-15
  &duration=1

cURL Example

curl -v -d "start_date=2012-03-15&end_date=2012-03-16&duration=1&auth_token=abcfcd36-ab25-4494-8434-7798cb1d718e&client=curl" https://dev.andwellness.org/app/mobility/aggregate/read

Output Format

The output contains the standard result element along with a JSON Array of JSON Objects with the following schema.

JSON Key Description
timestamp The timestamp of this chunk.
data A JSON array of JSON objects where each object represents a mode and a duration.
JSON Key Description
mode This chunk's mode.
duration The number of milliseconds the user was in this mode for this chunk.

Success

{
   "result" : "success"
   "data" : [
       {
           "timestamp":"2012-02-07",
           "data":[
               {
                   "mode":"walk",
                   "duration":2378946
               },
               ...
           ]
       },
       ...
   ]

}

Failure

See the error page for a description of error codes and their associated descriptions.

↑ Back to Top

Mobility Dates Read

What does it do?

Allows users to read the dates on which they uploaded Mobility data.

URI

mobility/dates/read

Access Rules

A user may read information about another user if the requesting user is an admin or if the server is configured to allow privileged users in a class to read other users' Mobility information (this flag can be obtained from config/read) and the requesting user is an admin in at least one class to which the user whose information is desired belongs.

Input Parameters

Supports Token-Based Authentication ...

  • (r) auth_token = A valid authentication token from user/auth_token. May also be set using the Set-Cookie header.

... or User-Password Authentication

  • (r) user = A username of the user attempting to login
  • (r) password = A password for the above user.

Additional Parameters

  • (r) client = A short description of the software client performing the upload.
  • (o) start_date = An ISO8601-formatted date which limits the results to only those dates on or after this one.
  • (o) end_date = An ISO8601-formatted date which limits the results to only those dates on or before this one.
  • (o) username = The username of the user whose data is desired. This is only applicable if the requesting user is an admin or if the server allows it and the requesting user is privileged in any class to which the desired user belongs.

Example POST

POST /app/mobility/dates/read HTTP/1.1
 Host: dev.ohmage.org
 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.77 Safari/535.7
 Content-Length: byte-length-of-content
 Content-Type: application/x-www-form-urlencoded
 Set-Cookie: abcfcd36-ab25-4494-8434-7798cb1d718e

  user=username
  &client=curl
  &start_date=2012-02-29

cURL Example

curl -v -d "client=curl&start_date=2012-03-15&auth_token=abcfcd36-ab25-4494-8434-7798cb1d718e" https://dev.andwellness.org/app/mobility/dates/read

Output Format

The output is a JSON object with two keys. One will always be "result", and its value will always be one of "success" or "failure". For success, the second key will be "data", and its value will be a JSON array of ISO8601-formatted dates.

Success

{
   "result" : "success"
   "data" : [
       "2012-02-29",
       "2012-03-03",
       ...
   ]

}

Failure

See the error page for a description of error codes and their associated descriptions.

↑ Back to Top

Clone this wiki locally