Integrations & APIs

The Qatium Ingest API enables you to upload external data – asset status, flow, pressure and tank level variables – to your model into Qatium.

Preparing the Data

Qatium assigns a live data point to the asset defined in the body of the request. The mapping sensor-asset should be handled before the data is sent through. Qatium does not store sensor ID, name or label information as part of a mapping to the asset.
The mapping sensor-asset should be handled as shown in the table below:

Variable	Metric value in the API	Asset to associate
flow	flow	pipe
pressure	pressure	junction
pump status	status	pump
pump relative speed	setting	pump
valve status	status	valve
valve setting	setting	valve
tank level	level	tank
demand	demand	junction

Qatium Ingest API does not support null values, please refrain from sending null values through the API.
Please note that any information sent outside of those guidelines will be discarded by the application. Data can be sent as a body in JSON format or as a JSON or CSV file as shown in the examples below.
1. JSON Body
Headers:

Authorization: Bearer <TOKEN>
  Content-Type: application/json; charset=utf-8

Body:

[
    {
      "time": <string>,
      "asset": <string>,
      "value": <number>,
      "metric": <string>,
      "unit": <string>
    },
    ...
  ]

  1.2 JSON file
Headers:

Authorization: Bearer <TOKEN>
  Content-Type: multipart/form-data

Form:

Name: file
  Type: file

File content:

[
    {
      "time": <string>,
      "asset": <string>,
      "value": <number>,
      "metric": <string>,
      "unit": <string>
    },
    ...
  ]

  1.2 CSV file
You can use CSV files with or without headers.

• With headers: No specific requirement.
• Without headers: The content needs to be placed in the order defined in the File Headers below:

Headers:

Authorization: Bearer <TOKEN>
  Content-Type: multipart/form-data

Form:

Name: file
  Type: file

File Headers (optional):

"time","asset","metric","unit","value","test"

Example of file content with headers:

"time","asset","metric","unit","value"
  <string>,<string>,<string>,<string>,<number>,<string>
  ...

Example of file content without headers:

  "2021-12-31T23:45:00Z","Pipe1","flow","l/s",2.75
  ...

 

Variables	Legend	Format	Example
time	The time when the reading was taken.	string (timestamp in ISO 8601 format)	“2021-11-03T15:00:00Z”
asset	The ID of the asset associated with the reading. Take into account that the ID must be the same as the one in Qatium.	string (case should be respected)	“Pipe1”
value	The value of the readings. Valves and pump statuses should be a number, 0 (closed) or 1 (fully open). Pump settings should be a ratio of the pump current speed by pump nominal speed. Tank level minimum value is set to 0.	number  (decimal separator must be a dot “.”)	1.7
metric	The variable of the reading value.	string, one of the following values: pressure, flow, level or statusMore information in the mapping table	“flow”
unit	There is no unit conversion. The value sent needs to be identical to the one in the model. No unit conversion will be performed. Hence, the format list includes the unit supported by the hydraulic engine.	string, either “gpm”, “cfs”, “mgd”, “imgd”, “afd”, “lps”, “l/s”, “lpm”, “l/m”, “mld”, “cmh”, “cmd”,”mwc”, ”psi”, “ft” or “m”  (data will not be converted, the unit should match with the one used in the model)	“l/s”

 
In addition, validation is applied for the following metrics:

Valid metric	Valid unit (International System)	Valid unit (US-Customary)	Valid value
pressure	“m”, “mca”, “mwc”	“psi”	Numeric
flow	“Ml/d”, “cmd”, “cmh”, “l/m”, “l/min”, “l/s”,“lpm”, “lps”, “m^3/d”, “m^3/h”, “mld”	“Mgal/d”, “Mgallon-imp/d”, “acre ft/d”, “afd”, “cfs”, “ft^3/s”, “gal/min”, “gpm”, “imgd”, “mgd”	Numeric
level	“m”, “m3”, “m^3”	“ft”, “ft3”, “ft^3”	Numeric
status	none	none	Numeric (0 or 1)

Sending the data

1. Authentication
In order to send data to the Ingest API, you must use an auth token with the right permissions. An authorization token and endpoint url will be provided upon request. Please provide the network ID when requesting your token. The network ID can be found in the URL when visiting the network. i.e. 1a23b45c-1a23-12a3-1a23-1234a567bc89, located in the URL: https://qatium.app/network/NetworkID/NetworkName
Please contact Q to request your token and end point URL.
2. HTTP Method
The API uses the standard HTTP method POST to append a new datapoint in Qatium.
Please note that there is currently a 20 request per minute limit and two ingest formats are accepted:

• Body with a JSON object, we recommend limit the size of the files to 50MB.
• Multipart/form-data with a JSON or CSV file, we recommend limit the size of the files to 1.5GB.

The API is designed for realtime and near-real time updates, for historical data uploads please contact Q.

Understanding the response

When sending data to the Ingest API, you will get a response.

Code	Description
200	No error
400	Bad request
401	Authentication token missing or invalid
403	Forbidden
404	Not found
413	The message body is too large
429	Too many requests. When over the rate limits
500	Unexpected error

When a data upload is successful, the response will include the status of the data in percentages, just as the example below:

...
"status": {
  "successful": 80,
  "quarantined": 20,
  "pending": 0
}
...

In the example above, 80% of the rows have been successfully ingested and 20% are in quarantine

Job status API

In order to retrieve the job status of a request, you must use an authorization token with the right permissions. An authorization token and endpoint url will be provided upon request. Please provide the network ID when requesting access to the API. The network ID can be found in the URL when visiting the network. i.e. https://qatium.app/network/NetworkID/NetworkName If you do not have a token, please fill out the token request form to request access to the API.

Calling the API

The API uses the standard HTTP GET method to retrieve a request job status. To identify the file to download, you must use the job_id obtained in response to an Ingest request. Example of download

{
    "job_id": "12345ab1-a1b2-1ab2-ab12-1234567a8b9c",
    "message": "Your request has been processed with warnings so 2 rows have been appended and 1 rows could 
not be appended."
}

Understanding the API response

When sending data to the Ingest API, you will get a status response and a code.

Status	Description
Ok	The ingest has been processed correctly.
Warning	There is data that has been processed correctly and data with errors.
Invalid	No data has been processed correctly.
Processing	The data has not yet been fully processed.
Pending	The data has not started to be processed.

Code	Description
200	Success
201	Success
401	Unauthorized
404	Not found
500	Unexpected error

Job details API

You need an authorization token with the right permissions to retrieve detailed information about a job. An authorization token and endpoint url will be provided upon request. Please provide the network ID when requesting access to the API. The network ID can be found in the URL when visiting the network. i.e.https://qatium.app/network/NetworkID/NetworkName If you do not have a token, please fill out the token request form to request access to the API.

Calling the API

The API uses the standard HTTP GET method to retrieve a request job details. You must specify the job_id obtained in response to an ingest. If the provided jobs don’t have any rows in quarantine, the response will be an empty array. Otherwise the response will look similar to the one in the example below: Example of download

{
    "network": "00000000-0000-0000-0000-000000000000",
    "job_id": "12345ab1-a1b2-1ab2-ab12-1234567a8b9c",
    "job_status": "invalid",
    "message": "Your request have been processed with errors so no rows have been appended.",
    "errors": ["value 'undefined' on column 'value' is not Float32"],
    "insertion_date": "2022-08-22 11:45:30"
}

Understanding the API response

When sending data to the Ingest API, you will get a response.

Code	Description
200	Success
401	Unauthorized
500	Unexpected error