- Data points
- Time-series data upload API
- Data visualization
- Data storage
- Data retention
- Data durability
- Rule engine
- Data Query REST API
- WebSocket API
ThingsBoard provides a rich set of features related to time-series data:
- Collect data from devices using various protocols and integrations;
- Store time series data in SQL (PostgreSQL) or NoSQL (Cassandra or Timescale) databases;
- Query the latest time series data values or all data within the specified time range with flexible aggregation;
- Subscribe to data updates using WebSockets for visualization or real-time analytics;
- Visualize time series data using configurable and highly customizable widgets and dashboards;
- Filter and analyze data using flexible Rule Engine;
- Generate alarms based on collected data;
- Forward data to external systems using External Rule Nodes (e.g. Kafka or RabbitMQ Rule Nodes).
This guide provides an overview of the features listed above, and some useful links to get more details.
Data points
ThingsBoard internally treats time-series data as timestamped key-value pairs. We call single timestamped key-value pair a data point. Flexibility and simplicity of the key-value format allow easy and seamless integration with almost any IoT device on the market. Key is always a string and is basically a data point key name, while the value can be either string, boolean, double, integer or JSON.
The following JSON contains 5 data points: temperature (double), humidity (integer), hvacEnabled (boolean), hvacState (string) and configuration (JSON):
1
2
3
4
5
6
7
8
9
10
11
{
"temperature": 42.2,
"humidity": 70,
"hvacEnabled": true,
"hvacState": "IDLE",
"configuration": {
"someNumber": 42,
"someArray": [1,2,3],
"someNestedObject": {"key": "value"}
}
}
You may notice that the JSON listed above does not have a timestamp information. In such case, ThingsBoard uses current server timestamp. However, you may include timestamp information into the message. See example below:
1
2
3
4
5
6
7
{
"ts": 1527863043000,
"values": {
"temperature": 42.2,
"humidity": 70
}
}
Time-series data upload API
You may use built-in transport protocol implementations:
Most of the protocols above support JSON, Protobuf or own data format. For other protocols, please review “How to connect your device?” guide.
Data visualization
We assume you have already pushed time-series data to ThingsBoard. Now you may use it in your dashboards. We recommend dashboards overview to get started. Once you are familiar how to create dashboards and configure data sources, you may use widgets to visualize either latest values or real-time changes and historical values. Good examples of widgets that visualize latest values are digital and analog gauges, or cards. Charts are used to visualize historical and real-time values and maps to visualize movement of devices and assets.
You may also use input widgets to allow dashboard users to input new time-series values using the dashboards.
Data storage
ThingsBoard Cloud stores time-series data in the Cassandra database with replication factor of 3. The on-prem installation of ThingsBoard support storage of time-series data in SQL (PostgreSQL) or NoSQL (Cassandra or Timescale) databases.
Data retention
ThingsBoard Cloud stores data with configurable time-to-live (TTL) parameter. The value of the parameter is part of the Subscription plan. You may overwrite the default value in the “Save Timeseries” rule node or using “TTL” metadata field of your message. This allows you to optimize storage consumption. The maximum allowed value of TTL is 5 years. For example, you may store “raw” data for 3 month and aggregated data for 3 years.
Data durability
The device that sends message with time-series data to ThingsBoard will receive confirmation once the message is successfully stored into the Rule Engine Queue that is configured for particular device profile.
As a tenant administrator, you may configure processing strategy for the queue. You may configure the queue either to reprocess or ignore the failures of the message processing. This allows granular control on the level of durability for the time-series data and all other messages processed by the rule engine.
Rule engine
The Rule Engine is responsible for processing all sorts of incoming data and event. You may find most popular scenarios of using attributes within rule engine below:
Generate alarms based on the logical expressions against time-series values
Use alarm rules to configure most common alarm conditions via UI or use filter nodes to configure more specific use cases via custom JS functions.
Modify incoming time-series data before they are stored in the database
Use message type switch rule node to filter messages that contain “Post telemetry” request. Then, use transformation rule nodes to modify a particular message.
Calculate delta between previous and current time-series value
Use calculate delta rule node to calculate power, water and other consumption based on smart-meter readings.
Fetch previous time-series values to analyze incoming telemetry from device
Use originator telemetry rule node to enrich incoming time-series data message with previous time-series data of the device.
Fetch attribute values to analyze incoming telemetry from device
Use enrichment rule nodes to enrich incoming telemetry message with attributes of the device, related asset, customer or tenant. This is extremely powerful technique that allows to modify processing logic and parameters based on settings stored in the attributes.
Use analytics rule nodes to aggregate data for related assets
Use analytics rule nodes to aggregate data from multiple devices or assets.
Useful to calculate total water consumption for the building/district based on data from multiple water meters.
Data Query REST API
ThingsBoard provides following REST API to fetch entity data:
Get time-series data keys for specific entity
You can fetch list of all time-series data keys for particular entity type and entity id using GET request to the following URL
1
http(s)://host:port/api/plugins/telemetry/{entityType}/{entityId}/keys/timeseries
|
|
Supported entity types are: TENANT, CUSTOMER, USER, DASHBOARD, ASSET, DEVICE, ALARM, ENTITY_VIEW
Get latest time-series data values for specific entity
You can fetch list of latest values for particular entity type and entity id using GET request to the following URL
1
http(s)://host:port/api/plugins/telemetry/{entityType}/{entityId}/values/timeseries?keys=key1,key2,key3
|
|
Supported entity types are: TENANT, CUSTOMER, USER, DASHBOARD, ASSET, DEVICE, ALARM, ENTITY_VIEW
Get historical time-series data values for specific entity
You can also fetch list of historical values for particular entity type and entity id using GET request to the following URL
1
http(s)://host:port/api/plugins/telemetry/{entityType}/{entityId}/values/timeseries?keys=key1,key2,key3&startTs=1479735870785&endTs=1479735871858&interval=60000&limit=100&agg=AVG
The supported parameters are described below:
- keys - comma-separated list of telemetry keys to fetch.
- startTs - Unix timestamp that identifies the start of the interval in milliseconds.
- endTs - Unix timestamp that identifies the end of the interval in milliseconds.
- interval - the aggregation interval, in milliseconds.
- agg - the aggregation function. One of MIN, MAX, AVG, SUM, COUNT, NONE.
- limit - the max amount of data points to return or intervals to process.
ThingsBoard will use startTs, endTs, and interval to identify aggregation partitions or sub-queries and execute asynchronous queries to DB that leverage built-in aggregation functions.
|
|
Supported entity types are: TENANT, CUSTOMER, USER, DASHBOARD, ASSET, DEVICE, ALARM, ENTITY_VIEW
WebSocket API
WebSockets are actively used by Thingsboard Web UI. WebSocket API duplicates REST API functionality and provides the ability to subscribe to device data changes. You can open a WebSocket connection to a telemetry service using the following URL
1
ws(s)://host:port/api/ws
Once opened, you need to authenticate the session within 10 seconds with auth command:
1
2
3
4
5
6
{
"authCmd": {
"cmdId": 0,
"token": "$JWT_TOKEN"
}
}
Then you can send subscription commands and receive subscription updates:
where
- cmdId - unique command id (within corresponding WebSocket connection)
- entityType - unique entity type. Supported entity types are: TENANT, CUSTOMER, USER, DASHBOARD, ASSET, DEVICE, ALARM
- entityId - unique entity identifier
- keys - a comma-separated list of data keys
- timeWindow - fetch interval for time series subscriptions, in milliseconds. Data will be fetch within following interval [now()-timeWindow, now()]
- startTs - start time of fetch interval for historical data query, in milliseconds.
- endTs - end time of fetch interval for historical data query, in milliseconds.
Example
Change values of the following variables :
-
token - to the JWT token which you can get using the following link.
-
entityId - to your device id.
In case of live-demo server :
- replace host:port with demo.thingsboard.io and choose secure connection - wss://
In case of local installation :
- replace host:port with 127.0.0.1:8080 and choose ws://
|