Stream connectivity metadata directly to Azure in real-time

Before each device is able to send data to the application it needs to communicate with the mobile network to ensure that the data is allowed to be sent over the network. This communication happens through signaling events which are usually hidden to the application.  The EMnify DataStreamer makes this connectivity metadata available in real-time to your cloud platform for detailed insights on the event and usage data of your devices. 

Note: this integration is in a beta testing phase as of March 2020 and therefore does not have UI components for self-service configuration in the EMnify portal. Customers who want to opt-in for testing this integration may get in touch directly for an early-access trial at christian.henke@emnify.com.

Prerequisites

  • A device with a cellular modem and EMnify SIM card
  • An Azure account

Benefits 

  • connectivity metadata integrated into operational dashboards for service teams
  • allows faster triaging between device, connectivity and application issues
  • insights about network attach, data and roaming issues or when device is above service limit
  • view on service usage and cost per device 
  • directly delivered to Azure event hubs without need for managing public IPs

Integration steps

To create an Event Hub, users in Azure need to create a namespace which is used to group multiple event hubs.

1. In the Azure portal, navigate to Event Hubs and click + Add or Create event hubs namespace:

2. Provide a name and create a resource group for the namespace and click Create

3. Select the newly-created namespace and click + Event Hub

5. Give the event hub a name and click Create

6. After the Event Hub has been created, the next step that needs to be done is configuring access for the EMnify Datastreamer to be able to publish incoming events.
To set this up, click Shared Access Policies

7.  Click + Add to create a new access policy with a relevant name, this policy must allow allow send permissions:

8. The access policy will be updated in-place to display connection credentials that the EMnify Datastreamer requires for publishing events.
Copy the Connection String Primary Key:

 

 

The event data arrives directly into the Azure Event Hub for further processing by other services:

Verifying the integration

When incoming messages are successfully sent to an Azure Event Hub, clicking on Process Data allows for users to run SQL queries on data as it arrives in real-time.
The default query will list all messages, but the advantages of being able to run queries on live EMnify streaming data could easily be leveraged to get an insight on top devices which consume data within a live time window, aggregate data consumption by hour, per region, tagged EMnify endpoints, and more.

Next Steps

As connectivity data is now arriving in real-time, it's simple to implement other services which can react to any events which are of interest. One useful integration that can be quickly launched is the Capture service which will automatically deliver the streaming data in Event Hubs to an Azure Blob storage or Azure Data Lake Store account. To leverage and make sense of the streaming data, a Stream Analytics job can retrieve the data and feed a Power BI visualization to easily and quickly gain insights.