Stand with Ukraine flag
Try it now Pricing
IoT Gateway
Community Edition Professional Edition Cloud Edge PE Edge IoT Gateway License Server Trendz Analytics Mobile Application PE Mobile Application MQTT Broker
Documentation > Configuration guides > Request Connector
Getting Started
Installation
On this page

Request Connector Configuration

This guide will help you to get familiar with Request Connector configuration for ThingsBoard IoT Gateway.
Use general configuration guide to enable this Connector.
The purpose of this Connector is to connect to external HTTP(S) API endpoints and get data from them.
Connector is also able to push data to external HTTP(S) API based on the updates/commands from ThingsBoard.

This connector is useful when you have some HTTP(S) API endpoints in your device or some data in external resource and you would like to push this data to the ThingsBoard.

We will describe connector configuration file below.

Connector configuration: request.json

Connector configuration is a JSON file that contains information about how to connect to external API endpoints, what urls to use when reading data and how to process the data.
Let’s review the format of the configuration file using example below.

Example of Request Connector config file.

Example listed below will connect to server on a localhost with 5000 port.
Connector will use basic HTTP authorization using username and password.
Then, connector will read data from a list of endpoints using urls from mapping section. See more info in a description below.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116

{
  "host": "http://127.0.0.1:5000",
  "SSLVerify": true,
  "security": {
    "type": "basic",
    "username": "user",
    "password": "password"
  },
  "mapping": [
    {
      "url": "getdata",
      "httpMethod": "GET",
      "httpHeaders": {
        "ACCEPT": "application/json"
      },
      "allowRedirects": true,
      "timeout": 0.5,
      "scanPeriod": 5,
      "converter": {
        "type": "json",
        "deviceNameJsonExpression": "SD8500",
        "deviceTypeJsonExpression": "SD",
        "attributes": [
          {
            "key": "serialNumber",
            "type": "string",
            "value": "${serial}"
          }
        ],
        "telemetry": [
          {
            "key": "Maintainer",
            "type": "string",
            "value": "${Developer}"
          }
        ]
      }
    },
    {
      "url": "get_info",
      "httpMethod": "GET",
      "httpHeaders": {
        "ACCEPT": "application/json"
      },
      "allowRedirects": true,
      "timeout": 0.5,
      "scanPeriod": 100,
      "converter": {
        "type": "custom",
        "deviceNameJsonExpression": "SD8500",
        "deviceTypeJsonExpression": "SD",
        "extension": "CustomRequestUplinkConverter",
        "extension-config": [
          {
            "key": "Totaliser",
            "type": "float",
            "fromByte": 0,
            "toByte": 4,
            "byteorder": "big",
            "signed": true,
            "multiplier": 1
          },
          {
            "key": "Flow",
            "type": "int",
            "fromByte": 4,
            "toByte": 6,
            "byteorder": "big",
            "signed": true,
            "multiplier": 0.01
          }
        ]
      }
    }
  ],
  "attributeUpdates": [
      {
        "httpMethod": "POST",
        "httpHeaders": {
          "CONTENT-TYPE": "application/json"
        },
        "timeout": 0.5,
        "tries": 3,
        "allowRedirects": true,
        "deviceNameFilter": "SD.*",
        "attributeFilter": "send_data",
        "requestUrlExpression": "sensor/${deviceName}/${attributeKey}",
        "valueExpression": "{\"${attributeKey}\":\"${attributeValue}\"}"
      }
  ],
  "serverSideRpc": [
    {
      "deviceNameFilter": ".*",
      "methodFilter": "echo",
      "requestUrlExpression": "sensor/${deviceName}/request/${methodName}/${requestId}",
      "responseTimeout": 1,
      "httpMethod": "GET",
      "valueExpression": "${params}",
      "timeout": 0.5,
      "tries": 3,
      "httpHeaders": {
        "Content-Type": "application/json"
      }
    },
    {
      "deviceNameFilter": ".*",
      "methodFilter": "no-reply",
      "requestUrlExpression": "sensor/${deviceName}/request/${methodName}/${requestId}",
      "httpMethod": "POST",
      "valueExpression": "${params}",
      "httpHeaders": {
        "Content-Type": "application/json"
      }
    }
  ]
}

General section

Parameter Default value Description
host http://127.0.0.1:5000 Domain address or ip of the server.
SSLVerify true Verify or no SSL certificate on the server if available.

Security section

This section provides configuration for client authorization at the external server.

BasicRecommended
AnonymousNo security

One type of security configuration is Basic authentication. The Request Connector sends HTTP requests with the Authorization header that contains the word Basic word followed by a space and a base64-encoded string username:password.

Parameter Default value Description
type basic Type of authorization.
username username Username for authorization.
password password Password for authorization.

Security section in configuration file will look like this:

1
2
3
4
5

    "security": {
      "type": "basic",
      "username": "username",
      "password": "password"
    }

Anonymous auth is the most simple option. It is useful for testing.

Parameter Default value Description
type anonymous Type of authorization.

Security subsection in configuration file will look like this:

1
2
3

    "security": {
      "type": "anonymous"
    }

Mapping section

This configuration section contains array of objects with endpoints that the gateway will try to read after connecting to the server.
Also this section contains settings about processing incoming messages (converter).
After request, each response from that url is analyzed to extract device name, type and data (attributes and/or timeseries values).
By default, the gateway uses Json converter, but it is possible to provide custom converter. See example in the source code.

Note: You can specify multiple mapping objects inside the array.

Parameter Default value Description
url getdata Url address for sending request.
httpMethod GET HTTP method for request (GET, POST etc.).
httpHeaders { “ACCEPT”: “application/json” } Object contains additional HTTP headers for request.
allowRedirects true Allow request redirection.
timeout 0.5 Timeout for request.
scanPeriod 5 Rescan rate.

Converter subsection

This subsection contains configuration for processing incoming messages.

Types of request converters:

  1. json – Default converter
  2. custom – Custom converter (You can write it by yourself, and it will use to convert incoming data from the response.)
jsonRecommended if json will be received in response
customRecommended if bytes or anything else will be received in response

Json converter is default converter, it looks for deviceName, deviceType, attributes and telemetry in the incoming message from the broker, with rules, described in this subsection:

Parameter Default value Description
type json Provides information to connector that default converter will be uses for converting data from response.
deviceNameJsonExpression SD8500 Simple JSON expression, uses for looking device name in the incoming message (string “SD8500” will be used as device name).
deviceTypeJsonExpression SD Simple JSON expression, uses for looking device type in the incoming message (string “SD” will be used as device type).
timeout 60000 Timeout for triggering “Device Disconnected” event
attributes   This subsection contains parameters of the incoming message, that will be interpreted as attributes for the device.
… type string Type of incoming data for a current attribute.
… key serialNumber Attribute name, that will sends to ThingsBoard instance.
… value ${serial} Simple JSON expression, uses for looking value in the incoming message, that will send to ThingsBoard instance as value of key parameter.
timeseries   This subsection contains parameters of the incoming message, that will be interpreted as telemetry for the device.
… type string Type of incoming data for a current telemetry.
… key Maintainer Telemetry name, that will sends to ThingsBoard instance.
… value ${Developer} Simple JSON expression, uses for looking value in the incoming message, that will send to ThingsBoard instance as value of key parameter.
Doc info icon

Parameters in attributes and telemetry section may differ from those presented above, but will have the same structure.

Mapping subsection will look like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

    {
      "url": "getdata",
      "httpMethod": "GET",
      "httpHeaders": {
        "ACCEPT": "application/json"
      },
      "allowRedirects": true,
      "timeout": 0.5,
      "scanPeriod": 5,
      "converter": {
        "type": "json",
        "deviceNameJsonExpression": "SD8500",
        "deviceTypeJsonExpression": "SD",
        "attributes": [
          {
            "key": "serialNumber",
            "type": "string",
            "value": "${serial}"
          }
        ],
        "telemetry": [
          {
            "key": "Maintainer",
            "type": "string",
            "value": "${Developer}"
          }
        ]
      }
    }

Also, you can combine values from MQTT message in attributes, telemetry and serverSideRpc section, for example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

{
      "url": "getdata",
      "httpMethod": "GET",
      "httpHeaders": {
        "ACCEPT": "application/json"
      },
      "allowRedirects": true,
      "timeout": 0.5,
      "scanPeriod": 5,
      "converter": {
        "type": "json",
        "deviceNameJsonExpression": "SD8500",
        "deviceTypeJsonExpression": "SD",
        "attributes": [],
        "telemetry": [
          {
            "type": "double",
            "key": "temperature",
            "value": "${temp}"
          },
          {
            "type": "double",
            "key": "humidity",
            "value": "${hum}"
          },
          {
            "type": "string",
            "key": "combine",
            "value": "${hum}:${temp}"
          }
        ]
      }
    }

A custom converter is converter written for some device:

Parameter Default value Description
type custom Provides information to connector that custom converter will be uses for converting data from response.
extension CustomRequestUplinkConverter Name of custom converter class.
extension-config   This subsection is a configuration for the custom converter. In default example it contains setting for parsing response string.
key Totaliser In this example value “Totaliser” will be interpreted as a telemetry key in ThingsBoard instance.
type float In this example type of the data.
fromByte 0 In this example start byte position in the response string.
toByte 4 In this example start byte position in the response string.
byteorder big In this example byteorder for response string.
signed true In this example using to indicate is the data in response signed.
multiplier 1 In this example every data from in response should be multiplied to some coefficient.
Doc info icon


Custom converter usually needed if you want to collect data from some device with not regular structure in response or when the data needs some processing before sending it to the ThingsBoard.

Mapping subsection in the configuration will look like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

    {
      "url": "get_info",
      "httpMethod": "GET",
      "httpHeaders": {
        "ACCEPT": "application/json"
      },
      "allowRedirects": true,
      "timeout": 0.5,
      "scanPeriod": 100,
      "converter": {
        "type": "custom",
        "deviceNameJsonExpression": "SD8500",
        "deviceTypeJsonExpression": "SD",
        "extension": "CustomRequestUplinkConverter",
        "extension-config": [
          {
            "key": "Totaliser",
            "type": "float",
            "fromByte": 0,
            "toByte": 4,
            "byteorder": "big",
            "signed": true,
            "multiplier": 1
          },
          {
            "key": "Flow",
            "type": "int",
            "fromByte": 4,
            "toByte": 6,
            "byteorder": "big",
            "signed": true,
            "multiplier": 0.01
          }
        ]
      }
    }

Attribute update section

Configuration in this section are optional.
ThingsBoard allows to provision device attributes and fetch some of them from the device application. You can treat this as a remote configuration for devices. Your devices are able to request shared attributes from ThingsBoard. See user guide for more details.

The “attributeRequests” configuration allows configuring the format of the corresponding attribute request and response messages.

Parameter Default value Description
httpMethod GET HTTP method for request (GET, POST etc.).
httpHeaders { “CONTENT-TYPE”: “application/json” } Object contains additional HTTP headers for request.
timeout 0.5 Timeout for request.
tries 3 Count of tries to send data
allowRedirects true Allow request redirection.
deviceNameFilter SD.* Regular expression device name filter, uses to determine, which function to execute.
attributeFilter send_data Regular expression attribute name filter, uses to determine, which function to execute.
requestUrlExpression sensor/${deviceName}/${attributeKey} JSON-path expression uses for creating url address to send a message.
valueExpression {\”${attributeKey}\”:\”${attributeValue}\”} JSON-path expression uses for creating the message data that will send to url.

The attributeUpdates section will look like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17


  "attributeUpdates": [
      {
        "httpMethod": "POST",
        "httpHeaders": {
          "CONTENT-TYPE": "application/json"
        },
        "timeout": 0.5,
        "tries": 3,
        "allowRedirects": true,
        "deviceNameFilter": "SD.*",
        "attributeFilter": "send_data",
        "requestUrlExpression": "sensor/${deviceName}/${attributeKey}",
        "valueExpression": "{\"${attributeKey}\":\"${attributeValue}\"}"
      }
  ]

Server side RPC section

ThingsBoard allows sending RPC commands to the device that is connected to ThingsBoard directly or via Gateway.

Configuration, provided in this section uses for sending RPC requests from ThingsBoard to device.

Parameter Default value Description
deviceNameFilter SmartMeter.* Regular expression device name filter, uses to determine, which function to execute.
methodFilter echo Regular expression method name filter, uses to determine, which function to execute.
requestUrlExpression sensor/${deviceName}/request/${methodName}/${requestId} JSON-path expression, uses to create url address to send RPC request.
responseTimeout 0.5 Timeout for request.
httpMethod GET HTTP method for request (GET, POST etc.).
valueExpression ${params} JSON-path expression, uses for creating data for sending to the external endpoint.
timeout 0.5 Timeout for request.
tries 3 Count of tries to send data
httpHeaders { “CONTENT-TYPE”: “application/json” } Object contains additional HTTP headers for request.
Doc info icon

There are 2 types of the RPC calls:

  1. With reply, after sending request the gateway will wait for response and send it to ThingsBoard.
  2. With no reply, after sending request the gateway will not wait for response.

Examples for both methods provided below.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

  "serverSideRpc": [
    {
      "deviceNameFilter": ".*",
      "methodFilter": "echo",
      "requestUrlExpression": "sensor/${deviceName}/request/${methodName}/${requestId}",
      "responseTimeout": 1,
      "httpMethod": "GET",
      "valueExpression": "${params}",
      "timeout": 0.5,
      "tries": 3,
      "httpHeaders": {
        "Content-Type": "application/json"
      }
    },
    {
      "deviceNameFilter": ".*",
      "methodFilter": "no-reply",
      "requestUrlExpression": "sensor/${deviceName}/request/${methodName}/${requestId}",
      "httpMethod": "POST",
      "valueExpression": "${params.hum}",
      "httpHeaders": {
        "Content-Type": "application/json"
      }
    }
  ]

Next steps

Explore guides related to main ThingsBoard features: