Before each device is able to send data to an application server, it needs to communicate with the mobile network to ensure that the data is allowed to be sent over the network. This communication happens through signaling events which are usually hidden to the application. The EMnify DataStreamer makes this connectivity metadata and device data usage available in real-time in your Datadog account for detailed insights on the consumption and state of your devices.
The following guide will cover the steps necessary to create a DataStream integration from the EMnify platform to quickly and securely deliver usage records to Datadog. For this example integration, we will build the following features:
1. In your Datadog account, navigate to Integrations → APIs from the left menu
2. Create a new Application Key and give it a name
3. Note the location of the API Key and Application Key
4. Log in to the EMnify portal and navigate to the Technical Settings page to create, inspect and manage Data Streams:
5. Click Add New Stream and under DataDog click ADD
Newly-created data streams are immediately active and should show a green Running status indicating a 200 HTTP response code when correctly configured.
If the configuration is invalid, a warning message will be displayed indicating incorrect configuration.
Navigate to Metrics -> Summary to configure the units of data consumed to be reported as bytes and count as the metric type. This can be configured for
To generate a dashboard with graphs for each device in your fleet, navigate to Metrics -> Explorer
The next section details how to create new graphs for a dashboard which highlight specific metrics that are important for our use case.
The graph is now visible on the dashboard and can be interacted with. Clicking on the expand icon will open the widget in fullscreen and allows for an overview of the displayed metrics with a table showing a breakdown with average, minimum and maximum reported values.
Showing the top devices by data usage is a simple and effective way to identify devices which are using more data than expected. This can be especially helpful for troubleshooting misconfigured devices which may consume a large amount of data.
Showing the change in data usage can act as an additional gauge in unexpected changes relating to data consumption. To measure changes in data usage in the past hour:
Clicking on one of the devices in this chart will now contain a contextual menu item for displaying the device in the EMnify UI to cross-reference device configuration.
Using the $var value in graphs allows for specifying the endpoint id (host) or the ICCID as a filter value for multiple graphs. To enable filtering across a dashboard in this way:
The graph that previously displayed metrics for all endpoints now shows only the selected ICCID
As connectivity data is now arriving in real-time, the next useful steps that can be configured is alerting via the Monitoring features of Datadog.
A monitor with alerting has been created and can be explored in the Manage Monitors tab with a detailed history of when alerts were triggered relative to the activity of your device fleet. The new monitor shows that we are above our warning threshold and the evaluation graph shows progression of the alert between warn states:
The monitors configuration has an effective detection method for our use case called Anomaly detection. This is especially useful for identifying unusual large spikes of data usage. Using the default configuration, this monitor does not accept threshold values but will measure the average minimum and maximum values of data usage of the device fleet and highlight sudden peaks:
When configuring a monitor, the Notify your team section allows for adding Datadog users or teams to a list of recipients. The default channel for alert notifications is to use the email address associated with their Datadog account.
In this guide, we have learned how to create simple dashboards which provide actionable insights into the data consumption of your devices. A useful integration that can improve communication of these insights and alerts with other teams such as operations or support is to add a Slack integration. This is quickly and easily configured within Slack by creating a webhook URL in your workspace and configuring Datadog to use this channel for delivering notifications in individual monitors instead of email. The Datadog documentation describes the steps necessary to enable this functionality to get graphs and alerts delivered to your private or public Slack channels.