Stream UDP Integration Protocol Specification

1. Overview

This document specifies the binary protocol used by the Stream platform for UDP-based data integration. It enables external systems to receive real-time tag data from Stream controllers via UDP packets.

Integrity: Each packet includes a CRC-16 checksum.

Conventions

  • Byte order: Big-endian for all multi-byte values
  • Strings: UTF-8 encoded

Important: All multi-byte integers are big-endian.


2. Packet Layout

Each UDP packet has three parts:

┌───────────────┬───────────────┬─────────┐

│    HEADER     │     BODY      │  CRC16  │

│   (28 bytes)  │   (variable)  │ (2 B)   │

└───────────────┴───────────────┴─────────┘

Key specifications

  • Maximum packet size: 1200 bytes
  • CRC coverage: Header + Body (CRC not included)

3. Header (28 bytes)

Field

Type

Size

Description

Magic

String

4

Literal "SCTL" (ASCII)

PacketType

Byte

1

0 = Data, 2 = Reserved

Flags

Byte

1

Control flags (currently unused; set to 0)

StreamId

Int16

2

Stream identifier (big-endian)

Sequence

Int64

8

Packet sequence number (big-endian)

BodyLength

Int16

2

Length of the Body in bytes (big-endian)

Reserved

Bytes

10

Reserved for future use (set to zeros)


Validation checklist

  • Magic is "SCTL"
  • BodyLength matches actual parsed body length


4. Body

The Body contains a collection of items.

Field

Type

Size

Description

ItemCount

UInt16

2

Number of items (big-endian)

Items

Item[]

Variable

Sequence of Item records

4.1 Item

Field

Type

Size

Description

NameLength

UInt16

2

Length of tag name in bytes (big-endian)

NameUtf8

String

Variable

Tag name (UTF-8)

ValueType

Byte

1

Data type identifier (see §5 Value Types)

Timestamp

Int64

8

UNIX timestamp in milliseconds (big-endian)

Value

Variant

Variable

Value payload; format depends on ValueType

Timestamp precision: Milliseconds since Unix epoch (January 1, 1970).

5. Value Types

Type ID

Name

Value Format

Description

0

Bool

1 byte

0 = false, 1 = true

1

Int16

2 bytes (big-endian)

16-bit signed integer

2

Real32

4 bytes (IEEE-754)

32-bit floating-point

3

String

2 + N bytes

Length-prefixed UTF-8 string

4

Int32

4 bytes (big-endian)

32-bit signed integer

5

Int64

8 bytes (big-endian)

64-bit signed integer

5.1 String Value Format

Field

Type

Size

Description

Length

UInt16

2

String length in bytes (BE)

StringData

String

Variable

UTF-8 string bytes

Practical limits: ~1150 bytes per string or tag name (constrained by packet size).

6. CRC-16

A 16-bit CRC is appended at the end of the packet.

  • Algorithm: CRC-16-CCITT
  • Polynomial: 0x1021
  • Initial value: 0xFFFF
  • Covers: Header + Body (excludes the CRC field itself)
  • Format: 2 bytes, big-endian

Tip: Compute CRC over exactly the first (Header + BodyLength) bytes.

7. Fragmentation

When data exceeds 1200 bytes, split across multiple packets.

  • Each fragment:
    • Has a unique Sequence number (increment for each fragment)
    • Uses the same StreamId for the logical stream
    • Is independently validated via CRC-16

Note: Packets may arrive out of order. Use Sequence and StreamId to detect gaps and associate fragments.

8. Example

Scenario: Two items

  1. Temperature = 23.5 (Real32) at 1672531200000
  2. Pressure = 1013 (Int32) at 1672531200001

Hex dump (annotated):

53 43 54 4C                      # Magic "SCTL"

00                               # PacketType (Data)

00                               # Flags

00 01                            # StreamId = 1

00 00 00 00 00 00 00 01          # Sequence = 1

00 2D                            # BodyLength = 45

00 00 00 00 00 00 00 00 00 00    # Reserved (10 bytes)


00 02                            # ItemCount = 2


# Item 1: Temperature

00 0B                            # NameLength = 11

54 65 6D 70 65 72 61 74 75 72 65 # "Temperature"

02                               # ValueType = Real32

00 00 01 85 94 4E 00 00          # Timestamp = 1672531200000

41 BC 00 00                      # Value = 23.5 (IEEE-754 float)


# Item 2: Pressure

00 08                            # NameLength = 8

50 72 65 73 73 75 72 65          # "Pressure"

04                               # ValueType = Int32

00 00 01 85 94 4E 00 01          # Timestamp = 1672531200001

00 00 03 F5                      # Value = 1013


A1 B2                            # CRC-16




9. Integration Guidelines

9.1 UDP Socket Setup

  • Listen on the configured UDP port.
  • Buffer size ≥ 1200 bytes.
  • Anticipate packet loss and duplication (UDP is unreliable).


9.2 Packet Validation

  • Verify Magic = "SCTL".
  • Ensure BodyLength matches parsed body bytes.
  • Verify CRC-16.
  • Check timestamps for reasonable ranges (e.g., not far in future/past).


9.3 Data Processing

  • Parse items sequentially.
  • Respect variable-length fields (names and string values).
  • Convert timestamps from UNIX milliseconds as needed.
  • Parse Real32 using IEEE-754.


9.4 Ordering & Loss

  • Use Sequence to detect gaps/missing packets.
  • StreamId differentiates streams.
  • Tolerate out-of-order arrival.


10. Constraints & Notes

  • Maximum packet size: 1200 bytes
  • Maximum items per packet: Depends on item sizes (names + values)
  • Name & string limits: Practically ~1150 bytes each
  • Timestamp precision: Milliseconds (Int64)
  • Flags: Currently unused; set to 0 for compatibility