3 Data Stream API
The Data Stream API is the main component of the API Factory and enables users to browse, push and pull objects from their datasets as well as make complex queries, updates and deletes on datasets. Before interacting with a dataset, users should ensure they have a key with appropriate permissions on the dataset they are working with. Keys are generated and managed using the Management API.
API methods are documented and can be tested using the Swagger web tools provided at:
http://<api-factory location>/
Before testing the API methods, use the Authorize button at the top right of the page. You should authenticate using the specific dataset key as both the username and password.
The following API functionality is currently supported:
Browse
Write
Read
Update
Delete
The API’s browse endpoint is used for browsing datasets, with full control over filters, pagination and sorting of result sets.
Pushing new data to a dataset is performed through a HTTP POST request, using Basic Authentication, with the dataset access key being supplied as both the authentication username and password. The URL for pushing data using the API is as follows:
https://<api-factory location>/object/<dataset-id>/
Data should be formatted as a JSON document and passed as the HTTP request body, using application/json as a Content-Type HTTP request header. A typical JSON document to be added to a dataset might look as follows:
{
"sensorId": "Sensor123",
"sensorStatus": "on",
"sensorType": "humidity",
"humidity": 68
}Note that your datapoint may be given a unique ID of your choosing, by specifying the attribute _id within your document body. If the _id field is omitted, a unique value will be assigned to your datapoint automatically.
curl -X POST \
https://api.mksmart.org/data/object/baff61f3-47f7-4f8b-81db-9588c7e44762 \
-H 'Content-Type: application/json' \
-d '{"sensorId": "Sensor123", "sensorStatus": "on", "sensorType": "humidity", "humidity": 68}' \
-u 3486133f-fbb2-428d-8e83-d7aa39860cf5
$curl = curl_init();
$username = "3486133f-fbb2-428d-8e83-d7aa39860cf5";
$password = "";
curl_setopt_array($curl, array(
CURLOPT_URL => "https://api.mksmart.org/data/object/baff61f3-47f7-4f8b-81db-9588c7e44762",
CURLOPT_USERPWD => ($username . ":" . $password),
CURLOPT_RETURNTRANSFER => true,
CURLOPT_ENCODING => "",
CURLOPT_MAXREDIRS => 10,
CURLOPT_TIMEOUT => 30,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_CUSTOMREQUEST => "POST",
CURLOPT_POSTFIELDS => "{\"sensorId\": \"Sensor123\", \"sensorStatus\": \"on\",\"sensorType\": \"humidity\",\"humidity\": 68}",
CURLOPT_HTTPHEADER => array(
"Content-Type: application/json"
),
));
$response = curl_exec($curl);
$err = curl_error($curl);
curl_close($curl);
if ($err) {
echo "cURL Error #:" . $err;
} else {
echo $response;
}import requests
url = "https://api.mksmart.org/data/object/baff61f3-47f7-4f8b-81db-9588c7e44762"
auth = ("3486133f-fbb2-428d-8e83-d7aa39860cf5","")
payload = "{\"sensorId\": \"Sensor123\", \"sensorStatus\": \"on\", \"sensorType\": \"humidity\",\"humidity\": 68,}"
headers = {
'Content-Type': "application/json"
}
response = requests.post(url, data=payload, headers=headers, auth=auth)
print(response.text)Note that, as data gets added to your dataset, the API will automatically annotate your JSON documents with some additional values. These help add context to your data and can be useful when constructing queries to return specific ranges of data. These additional fields can be distinguished from your own fields as they are preceded with an underscore:
_datasetid: The UUID of the dataset the JSON document was added to
_timestamp: The UNIX timestamp indicating when the data point was added to the dataset
_timestamp_year: The year component of the timestamp
_timestamp_month: The month component of the timestamp
_timestamp_day: The day (of the month) component of the timestamp
_timestamp_hour: The hour component of the timestamp
_timestamp_minute: The minute component of the timestamp
_timestamp_second: The second component of the timestamp
Be aware that the timestamp values added to your data reflect the time at which the JSON document was added to your dataset via the API, not the time at which a sensor value, for example, was read from a sensor. This should be considered when batch processing large amounts of data and writing to your dataset in bulk.
A data point that has been annotated with additional fields may look as follows:
{
"sensorId": "Sensor123",
"sensorStatus": "on",
"sensorType": "humidity",
"humidity": 68,
"_datasetid": "707d796b-7311-4c74-9255-02bced129463",
"_timestamp": 1556110378,
"_timestamp_year": "2019",
"_timestamp_month": "04",
"_timestamp_day": "24",
"_timestamp_hour": "13",
"_timestamp_minute": "52",
"_timestamp_second": "58"
}A dataset can be queried through a HTTP POST request with Basic Authentication, with the dataset access key being supplied as both the authentication username and password. The dataset query URL is as follows:
https://\<api-factory location\>/query/\<dataset-uuid\>/
Making a simple POST request to this URL, with the appropriate authentication, will return all the data points from this dataset as an array of JSON documents.
curl -X POST https://api.mksmart.org/data/query/707d796b-7311-4c74-9255-02bced129463 -u 3486133f-fbb2-428d-8e83-d7aa39860cf5
$curl = curl_init();
$username = "3486133f-fbb2-428d-8e83-d7aa39860cf5";
$password = "";
curl_setopt_array($curl, array(
CURLOPT_URL => "https://api.mksmart.org/data/query/707d796b-7311-4c74-9255-02bced129463/",
CURLOPT_USERPWD => ($username . ":" . $password),
CURLOPT_RETURNTRANSFER => true,
CURLOPT_ENCODING => "",
CURLOPT_MAXREDIRS => 10,
CURLOPT_TIMEOUT => 30,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_CUSTOMREQUEST => "POST",
CURLOPT_POSTFIELDS => "",
CURLOPT_HTTPHEADER => array(
"Content-Type: application/json"
),
));
$response = curl_exec($curl);
$err = curl_error($curl);
curl_close($curl);
if ($err) {
echo "cURL Error #:" . $err;
} else {
echo $response;
}url = "https://api.mksmart.org/data/query/707d796b-7311-4c74-9255-02bced129463/"
auth = ("3486133f-fbb2-428d-8e83-d7aa39860cf5","")
payload = ""
headers = {
'Content-Type': "application/json"
}
response = requests.post(url, data=payload, headers=headers, auth=auth)
print(response.text)A reading request can be refined using a query over the fields of data points contained in the specified dataset. The Object Stream API allows the use of MongoDB’s query language to filter requests. The query document should be constructed and passed in the body of the HTTP POST request, specifying application/json as a Content-Type HTTP request header.
The simplest query, an empty JSON document, will return all data points:
{}To specify equality conditions, use <field>:<value> expressions in the query filter document:
{
"sensorStatus": "on"
}Multiple equality conditions can be specified, using commas to separate them:
{
"sensorStatus": "on",
"sensorType": "humidity"
}Query operators can be used to construct more sophisticated queries and filter on a range of conditions. For example, to select values greater than or equal to a particular value, the $gte operator can be used:
{
"sensorStatus": "on",
"sensorType": "humidity",
"humidity": {
"$gte": 65
}
}Putting the above query into practice within a script would look like the following:
curl -X POST \
https://api.mksmart.org/data/query/707d796b-7311-4c74-9255-02bced129463/ \
-u 3486133f-fbb2-428d-8e83-d7aa39860cf5
-H 'Content-Type: application/json' \
-d '{
"sensorStatus": "on",
"sensorType": "humidity",
"humidity": {
"$gte": 65
}
}'
$curl = curl_init();
$username = "3486133f-fbb2-428d-8e83-d7aa39860cf5";
$password = "";
curl_setopt_array($curl, array(
CURLOPT_URL => "https://api.mksmart.org/data/query/707d796b-7311-4c74-9255-02bced129463/",
CURLOPT_USERPWD => ($username . ":" . $password),
CURLOPT_RETURNTRANSFER => true,
CURLOPT_ENCODING => "",
CURLOPT_MAXREDIRS => 10,
CURLOPT_TIMEOUT => 30,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_CUSTOMREQUEST => "POST",
CURLOPT_POSTFIELDS => "{ \"sensorStatus\": \"on\", \"sensorType\": \"humidity\", \"humidity\": { \"$gte\": 65 } }",
CURLOPT_HTTPHEADER => array(
"Content-Type: application/json"
),
));
$response = curl_exec($curl);
$err = curl_error($curl);
curl_close($curl);
if ($err) {
echo "cURL Error #:" . $err;
} else {
echo $response;
}import requests
url = "https://api.mksmart.org/data/query/707d796b-7311-4c74-9255-02bced129463/"
auth = ("3486133f-fbb2-428d-8e83-d7aa39860cf5","")
payload = "{\"sensorStatus\": \"on\", \"sensorType\": \"humidity\", \"humidity\": { \"$gte\": 65 } }"
headers = {
'Content-Type': "application/json"
}
response = requests.post(url, data=payload, headers=headers, auth=auth)
print(response.text)MongoDB provides a sophisticated query language that offers similar functionality to that available in SQL. We recommend taking a look at the full documentation on constructing MongoDB-style query documents which is available here.
By default, the API is configured to only return 100 objects when a read request is made. The query limit can be changed by appending a query parameter to the URL when making the request. For example, this cURL request has been modified to increase the query limit to 200:
curl -X POST https://api.mksmart.org/data/query/707d796b-7311-4c74-9255-02bced129463?query=200 -u 3486133f-fbb2-428d-8e83-d7aa39860cf5
Individual datapoints can be updated, using the HTTP PUT method against the following URI
https://\<api-factory location\>/object/\<dataset-uuid\>/\<_id\>
Use Basic Authentication to authenticate the request, with the dataset access key being supplied as both the authentication username and password. Once again, as with creating the original datapoint, the new datapoint should be formatted as a JSON document and passed as the HTTP request body, using application/json as a Content-Type HTTP request header.
Note that the _id attribute of a datapoint cannot be changed during the update process – any new _id attribute included in the new JSON document will be ignored.
Individual datapoints can be permanently deleted, using the HTTP DELETE method against the following URI
https://\<api-factory location\>/object/\<dataset-uuid\>/\<_id\>