« All blog posts

Connecting OPC UA Publisher to Azure with MQTT

14.07.2021


Having implemented OPC UA PubSub (Publisher/Subscriber) functionality in our SDK for Java gave us a good opportunity to update our Azure demo to utilize these latest features. Our earlier work with this project is described in two previous blog posts: one and two.

In our previous iteration we used a Microsoft OPC publisher-module to transfer data from an OPC UA server to Azure. Since Azure’s IoT Hubs also allow direct connections using the MQTT protocol, we wanted to replace that component with our own PubSub publisher. As our old Raspberry Pi powered weather station was also starting to show its age, we decided to take it down and replace it with a simulated data source.

NOTE: This implementation requires OPC UA SDK for Java Version 4.5.8

Azure IoT Hub via MQTT

Sending data to Azure with MQTT

Azure IoT Hub provides endpoints allowing IoT devices to communicate with the cloud using the MQTT protocol. While the IoT Hub is not a fully featured MQTT broker, it still supports receiving OPC UA PubSub messages. Connecting to the IoT Hub’s events allows further processing of the data sent in the device-to-cloud messages.

OPC UA Publisher

OPC UA PubSub is a new communication mechanism defined as an alternative to the basic OPC UA Client/Server communication. PubSub enables a few different alternative transport protocols. MQTT is suited for cloud communications and is widely supported by most cloud products.

OPC UA Publisher is an application that is configured to send fixed datasets to MQTT Brokers, for example, which can then distribute it to several subscribers.

Setting up an OPC UA MQTT publisher that connects to an Azure IoT Hub using our SDK for Java is easy: the SDK includes a sample publisher that can be used to get started quicky. Simply providing the publisher with the right credentials corresponding to a device defined in the IoT Hub makes it ready to send JSON encoded OPC UA data using a MQTT connection secured with TLS. In the SamplePubSubPublisherServer this can be done by starting the server with the following command line parameters:

>samplepubsubpublisherserver.bat 
  --address ssl://<IoT Hub hostname>:8883 
  --encoding json 
  --username <IoT Hub hostname>/<device id>/?api-version=2018-06-30 
  --password <device SAS token> 
  --client-id <device id> 
  --queue-name devices/<device id>/messages/events/ 
  --metadata-queue-name devices/<device id>/messages/events/
  --no-network-headers

Type it all in one line and in Linux and macOS, use samplepubsubpublisherserver.sh instead of .bat. Replace <IoT Hub hostname> with the actual hostname of your IoT Hub instance and <device id> with the ID of the device as it is identified in Azure. <device SAS token> is the IoT Hub Security Token aka ‘Shared Access Signature (SAS)’ of the device.

The Publisher will now send messages in the standard OPC UA PubSub JSON format. By default, the sample publisher will include OPC UA network headers that could be used to filter out some of the data, but with Azure we need to use the --no-network-headers argument, which will make the publisher to send plain name/value pairs, for example as

[{
  "MyLevel":{
    "Value":73.0,
    "SourceTimestamp":"2021-07-07T17:36:29+0300"
  },
  "MyLevelDisplayName":{
    "Value":"MyLevel"
  }
}]

.NET Core Event handler

Using the Azure Event Hubs client library, we created a .NET core application with a BackgroundService to receive and process device messages through the IoT Hub Event Hub compatible endpoint.

protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
		string consumerGroup = "azure";

		// Create a blob container client that the event processor will use 
		BlobContainerClient storageClient =
				new BlobContainerClient(_blobStorageConnectionString, <CONTAINER_NAME>);

		// Create an event processor client to process events in the event hub
		EventProcessorClient processor =    
				new EventProcessorClient(storageClient, consumerGroup, _eventHubConnectionString, <EVENT HUB NAME>);

		// Register handlers for processing events and handling errors
		processor.ProcessEventAsync += ProcessEventHandler;
		processor.ProcessErrorAsync += ProcessErrorHandler;
		_logger.LogInformation("Starting event processor");
		// Start the processing
		await processor.StartProcessingAsync(stoppingToken);

}

Received messages are processed into a compatible format and then sent to our Power BI dataset push API-endpoint as a POST request.

async Task ProcessEventHandler(ProcessEventArgs eventArgs)
{
		var deviceMessagesJsonString = Encoding.UTF8.GetString(eventArgs.Data.Body.ToArray());

		List<DeviceMessage> deviceMessages;
		try
		{
				deviceMessages = JsonSerializer.Deserialize<List<DeviceMessage>>(deviceMessagesJsonString);
		}
		catch(Exception e)
		{
				_logger.LogError("failed to deserialize into normal device message");
				return;
		}

		var result = new List<MyLevel>();
		foreach (var message in deviceMessages)
		{
				//Only process recent messages as PowerBi push-datasets have limit of 1 req/second
				if (message.MyLevel != null && eventArgs.Data.EnqueuedTime.AddSeconds(20) > DateTime.UtcNow)
				{
						result.Add(message.MyLevel);
				}
		}

		if (result.Count == 0)
		{
				return;
		}
		var content = new StringContent(
				JsonSerializer.Serialize(result),
				Encoding.UTF8,
				"application/json");

		var client = _clientFactory.CreateClient();
		var res = await client.PostAsync(
				<POWERBI_PUSH_URL>,
				content);
		_logger.LogInformation(res.StatusCode.ToString());
}

We used Azure App Service to host the application. Full instructions on how to use the Event Hubs client library to receive and process events can be found here.

Power BI

Power BI provides a variety of means to turn your data into different kinds of real time graphs, dashboards, and reports that can be embedded into either your own applications or other Microsoft services such as SharePoint. Since real time updating dashboards cannot be shared to users who do not own a Power BI Pro license without additional cost to us, we have decided to instead generate a static graph in Power BI which can be embedded to a website with a simple html tag. Publish to web graph data is cached and updated roughly every hour, meaning it won’t necessarily provide the most recent datapoints when viewed on our webpage. The graph generated from our data can be viewed here.

Conclusion

We have shown that our OPC UA SDK for Java is capable of publishing OPC UA data directly to Azure, from where it can be sent to Power BI to be visualized. While our small demonstration remains relatively basic, the implementation can be extended to cover far more complex use cases.

Do you want to know more?

If you are interested in developing your own Azure or other cloud based system with OPC UA connectivity, feel free to contact us. We can provide you software tools and professional services that enable fast development in this rapidly growing new market.

You can send us email (sales@prosysopc.com) or use the contact form.

Luukas Lusetti

Luukas Lusetti

Software Engineer

Email: luukas.lusetti@prosysopc.com

Expertise and responsibility areas: OPC & OPC UA product development and project work

Ari-Pekka Soikkeli

Ari-Pekka Soikkeli

Software Engineer

Email: ari-pekka.soikkeli@prosysopc.com

Tags: Azure, IoT, Cloud, OPC UA, OPC UA PubSub, MQTT, Java, Power BI

comments powered by Disqus

About Prosys OPC Ltd

Prosys OPC is a leading provider of professional OPC software and services with over 20 years of experience in the field. OPC and OPC UA (Unified Architecture) are communications standards used especially by industrial and high-tech companies.

Read more about us »

Newest blog posts

Connecting OPC UA Publisher to Azure with MQTT

Prosys OPC UA SDK for Java allows publishing OPC UA data to Azure using the MQTT protocol.

Introduction to OPC UA Online Course - Instructions

The blog post describes step-by-step TalentLMS sign-up and course payment process

Update to importing Information Models, December 2020

Updates to the blog post on importing Information Models from NodeSet files

View all blog posts »