Data Pushing module is an easy way to collect data from industrial control system and push it to the other IT, IoT,  or Enterprise services. This is can be used for OT/IT Integration and many other modern applications.


You can collect data from any supported data source (such as OPC and Modbus) and push the collected data to any of the following destinations:


  1. MQTT Broker.
  2. Web services
  3. Azure IoT Hub.
  4. Google IoT Core.
  5. AWS IoT
  6. Stream Platform/SCADA/Gateway


You can specify the pushing intervals, and you can also group data to be sent as a bulk.


From Stream Explorer, select Data Pushing under Integration and click on the Add button to add a new data pushing job.


Main Tab:



Basic

  • Active: Enables or disables this data pushing job.
  • Name: Unique name for the data pushing job.    
  • Description: (Optional). Use it to add meaningful note about this job.


Push Trigger

  • Type: Select one of the two options:


a. OnValueChange

    • Minimum Value Change: Push the data only when the Tag's value changes by a specific absolute value.
    • On Status Change: Push the data when Tag's status changes (e.g. from Normal to Hi or vise versa).
    • On Timestamp Change: Push the data when Tag's timestamp changes (i.e. updated from its data source).
    • Push Interval: At first, Tags values are stored in the application internal memory in a queue. Later, it's pushed to its destination after the (Push Interval) is elapsed


b. Periodic



    • Sample Interval: Push the data cyclically each specified number of seconds.
    • Push Interval: At first, Tags values are stored in the application internal memory in a queue. Later, it's pushed to its destination after the (Push Interval) is elapsed



Destination Tab:



  • Type: Select any of the supported destination to push the data to. Following are the details of each type:


MQTT Broker: 

       

- Click Manage to specify the required information of the MQTT Broker (IP Address, User name, password .. etc).    

- Client Id: Although it's not mandatory for all MQTT brokers, but it's useful to identification and filtration at the subscriber side.

- Topic Name: Each MQTT message is called a topic. And each topic requires a name (as a header) and a body (actual data for Tags).


Web Service: 

       

- Click Manage to specify the required information of the web service (IP Address and other settings).


Data: 


By default, Tag's data are pushed as array of JSON objects as the following:


[

  {

    "MsgSource": "SIMULATOR",

    "TagName": "SIM.RAMP1",

    "RawValue": "41",

    "ScaledValue": "41",

    "Status": "Normal",

    "ComStatus": null,

    "TimeStamp": "2020-02-03 12:21:14"

  },

  {

    "MsgSource": "SIMULATOR",

    "TagName": "SIM.RANDOM1",

    "RawValue": "88",

    "ScaledValue": "88",

    "Status": "Normal",

    "ComStatus": null,

    "TimeStamp": "2020-02-03 12:21:14"

  }

]


How to send additional data (static data) with the payload?


First: in Tag Editor, for SIM.RAMP1 Tag, Setup user defined fields




Second: In the integration model editor, Select Send user defined fields with values.



The JSON message will look like the following:

[

  {

    "MsgSource": "SIMULATOR",

    "TagName": "SIM.RAMP1",

    "RawValue": "41",

    "ScaledValue": "41",

    "Status": "Normal",

    "ComStatus": null,

    "TimeStamp": "2020-02-03 12:21:14",

    "Location": "area1"

  },

  {

    "MsgSource": "SIMULATOR",

    "TagName": "SIM.RANDOM1",

    "RawValue": "88",

    "ScaledValue": "88",

    "Status": "Normal",

    "ComStatus": null,

    "TimeStamp": "2020-02-03 12:21:14"

  }

]



Advanced Tab:



Retries

    • Max Number of Retries: Sets the maximum number of retry attempts for pushing data to the destination. This helps ensure data is pushed successfully in cases of temporary connectivity issues.
    • Retry Delay (Sec): Specifies the delay, in seconds, between each retry.


Store and Forward

    • Store and Forward Enabled: Enables the store-and-forward feature. When Enabled, data that fails to push (due to network issue for example) will be stored locally in a SQLite database file and attempted to be resent at a later time.
    • Chunk Size (Records count): Specifies the number of records to store in each local SQLite file. Each file created will contain a chunk of up to this number of records.
    • Old Data Push Interval (Sec): Defines the interval, in seconds, for attempting to push stored data from local files to the remote destination. This interval controls how often the system tries to resend failed data. And this process runs in parallel to the main thread of pushing new fresh data.



Status Tab:




    • Last Push Status Tag: This assigned Boolean tag will equal 1 if the last push operation is successful, otherwise will become 0.
    • Timestamp of Last Successful Push Tag: This assigned String tag will hold the timestamp of the last successful Push operation.
    • Last Status Message Tag: This assigned String tag will hold the status message of the last Push operation. (either "OK" or error message).
    • Log Last Message To File: This enables logging the payload to a file for further debugging.


Note: You can add as many integration data pushing models as needed. Each will run on it's separate thread independently.


Note: Any defined integration model will appear as an option in integration configuration of tags as shown in the below screen shot.


             



Note: Tags that have status of "Not Initialized" are not pushed until their status change to something else (e.g. Normal or any alarm state)