Hologram’s Data Engine is a message queue and protocol translation layer designed to route data from your embedded devices to other internet-connected services. It features a low-bandwidth ingestion protocol designed for resource-constrained microcontrollers.
Messages
A message consists of a data payload and metadata. The payload is typically a text string, but can be arbitrary binary data. Most metadata is automatically added on the server based on the message source, but you can also specify one or more topics to control how Data Engine Messages interact with Hologram Routes.
Topics
Topics provide the Data Engine with context on what the data represents, where the data originated, or why the data was generated. The Data Engine uses topics to conditionally route messages to their proper destinations via Routes.
Topics are arbitrary strings, and a message can have more than one topic associated with it. You should develop your own conventions on how to use topics. Thoughtful use of topics can make it easier to set up new routing rules and integrations without needing to update device firmware.
Note: User topics cannot begin with an underscore character. The Data Engine reserves these names for system topics such as _DEVICE_ID_.
Here are some examples of good topics to add to messages:
-
DEVICE_ERROR– For messages containing information about an error that occurred on the device. You might route these messages to a monitoring service to track the health of your device fleet. -
SENSOR_DATA– Messages containing readings from sensors. You might route these to a time-series database or other storage service for later analysis. -
CUSTOMER_XYZ– It may make sense to segment messages based on criteria such as customer ID, geographic region, or device version. You might use topics for this, even if you don’t have an immediate need to route these messages to different places.
Note: It's not always obvious whether to include some information as a topic or encoded in the message payload itself. The Routes are only able to route messages based on topics, and messages to downstream services can include topics. Therefore, for the most flexibility, it's a good idea to use topics for classifying messages.
System-Generated Topics
In addition to the topics you specify when writing a message, the Data Router will automatically append certain topics based on the source and protocol of the message.
All system topics begin and end with an underscore character, e.g. _SIMPLESTRING_. For security and consistency reasons, you are not able to specify these system topics explicitly.
The most important system-generated topic is the device topic. Every message written to the Data Engine from a device gets a topic in the form of _DEVICE_1234_, where 1234 is the Hologram device ID.
For a complete listing of system tags that the Data Engine may add to your messages, please refer to the System Topics reference.
Hologram Data Engine Credentials
When sending a message to Hologram’s Data Engine via the Embedded API, you must authenticate as a specific device. The authentication credentials consist of a 8-byte device key.
If you believe that a device’s credentials may have been compromised, you can regenerate them. Each device has one active set of credentials at a time, so the old credentials will no longer work after regenerating.
Note: The TCP API also supports a separate authentication method based off of the SIM card. If you use the Hologram Python SDK in its default sending mode the separate authentication method is used. With this authentication method, there is no need to generate credentials on the dashboard.