Table of Contents
Qt und Azure IoT Hub

Azure IoT Hub: Connecting a Qt Application with Azure (Part 1 of 4)

Qt und Azure IoT Hub

Introduction

IoT devices require a central hub (in our case: the Azure IoT Hub) to upload their data to and to retrieve new configurations or firmware updates from. For some scenarios, this system must be able to handle millions of connections and large amounts of incoming data.

Such a system must be able to authenticate clients and should offer a way to block a device if it is misbehaving or has been stolen. Due to the expected amount of IoT devices, the registration of new devices and configuration management must be easy to automate.

To manage device configurations, it is desirable to have an asynchronous bidirectional communication channel between the IoT device and the hub where settings can be stored so the device can retrieve them on the next connect.

Azure IoT Hub basics

IoT Hub is a managed service running on Microsoft Azure which offers bidirectional messaging and device management for IoT devices. It is highly scalable and integrates well with other Azure components that can be used for message processing.

Transports and authentication

IoT devices can choose between different transports for their connecting to IoT Hub

  • MQTT, MQTT over WebSockets
  • AMQP, AMQP over WebSockets
  • HTTPS

Authentication is done using a symmetric key or X.509 certificates which can be either self-signed or signed by a CA. There is also a way to authenticate devices using a TPM. This requires the Azure Device Provisioning Service which will be covered in a follow up post.

Any device registered to an IoT Hub can be blocked from connecting if it misbehaves or is no longer wanted. The block is not permanent and can be lifted at any time.

Device to cloud messaging

IoT devices can send messages with a size of up to 256 KB. The message format can be text or binary.

The messages are made available on an EventHub compatible endpoint of the IoT Hub by default where they are kept for up to seven days until they are consumed. Messages can also be routed to custom targets like Event Hubs, Azure Storage, Service Bus Queues or Service Bus Topics.

For data with a size of more than 256 KB, there is also a file upload feature which enables the IoT device to upload a file into a blob storage.

Cloud to device messaging

IoT Hub supports two different ways of cloud to device messaging.

Normal cloud to device messages can be up to 64 KB in size. If a message can not be delivered to the IoT device within 48 hours, it is discarded.

Direct methods are used to invoke a method on an IoT device from the cloud and receive the results as soon as the method call is finished. The payload size is up to 128 KB. If a device is not currently connected, the method call fails right away.

The Device Twin

The retention periods of the cloud to device messaging features described above makes them unsuitable for scenarios where a configuration change is requested by the cloud and must be applied when the client comes back online.

The device twin is a JSON document IoT Hub keeps for every registered device. The document can be updated from the cloud and by the IoT device.

{
    "deviceId": "MyDemoDevice",
    "etag": "AAAAAAAAAAM=",
    "deviceEtag": "MTY1MTM4MzE5",
    "status": "enabled",
    "statusUpdateTime": "0001-01-01T00:00:00Z",
    "connectionState": "Disconnected",
    "lastActivityTime": "2020-02-27T14:30:07.854667Z",
    "cloudToDeviceMessageCount": 0,
    "authenticationType": "sas",
    "x509Thumbprint": {
      "primaryThumbprint": null,
      "secondaryThumbprint": null
    },
    "version": 4,
    "tags": {
      "location": "Darmstadt"
    },
    "properties": {
      "desired": {
        "sendInterval": 300,
        "$metadata": {
          "$lastUpdated": "2020-04-29T10:31:34.9028815Z",
          "$lastUpdatedVersion": 3,
          "sendInterval": {
            "$lastUpdated": "2020-04-29T10:31:34.9028815Z",
            "$lastUpdatedVersion": 3
          }
        },
        "$version": 3
      },
      "reported": {
        "$metadata": {
          "$lastUpdated": "2019-12-11T08:09:25.0658234Z"
        },
        "$version": 1
      }
    },
    "capabilities": {
      "iotEdge": false
    }
  } 

The cloud side can write the desired object to set configuration parameters for the IoT device, in this example the sendInterval property. If the device is online while the changes are being applied, it will receive an update of the device twin. If the device is offline, it will receive the changed device twin on its next connect.

Only the IoT device can write the reported object. Its main purpose is to report on the success when applying configuration or to report the current software and hardware version.

If a property is not included in the desired or reported object, the old value is kept. To remove a property, it must be explicitly set to null.

The metadata objects contain information on when and how often a property has been updated.

The desired and reported properties can each be up to 32 KB in payload size and be nested up to 10 levels deep

A device twin can have tags which are only visible on the cloud side and can be used to organize devices in groups.

Device management

Tags and properties from the device twin can be used as filter criteria for device management. For example, one could create a configuration to set the desired property sendInterval to 600 for all devices on the IoT hub that have the location tag set to “Darmstadt” and the firmwareVersion in the reported property set to “1.2.3”.

IoT Hub tiers

Azure offers different service tiers for IoT Hub. The Basic tier of IoT Hub is much cheaper than the Standard tier but doesn‘t support device twins, device management and cloud to device messages.

For evaluation purposes, there is also a Free tier of IoT Hub which has all the features from the Standard tier but only handles 8000 messages per day.

Upgrading from Basic to Standard is possible but there is no way to update from Free to Standard. Keep this in mind before building a complex setup with several devices on a Free instance.

Service and device SDKs

Microsoft offers SDKs for several programming languages on several platforms. The SDKs consist of two parts. The service SDK is used for interaction with the IoT Hub service, for example device registration or device management. The device SDK is used to create the software for the IoT device and offers functionality for establishing a connection to IoT Hub, sending and receiving messages and reading and updating the device twin.

It is also possible to interact with IoT Hub without an SDK by calling the APIs or establishing an MQTT or AMQP connection directly, but using one of the SDKs is recommended.

IoT device example in C++

The following example shows how to build a simple Qt based telemetry sender application with device twin support using the Azure IoT SDK C. The SDK can be built with CMake, the default settings are sufficient.

Running this example requires a Free or Standard tier IoT Hub with at least one device with shared key authentication.

The full example code is hosted on github.

$ git clone

Walkthrough

The IotHubClient class uses the low level client from the Azure IoT Device SDK to communicate with IoT Hub. Sending messages and updating the reported object in the device twin is done via method calls. Changes to the connection status, the success of sending messages, the device twin update success and changes to the desired object of the device twin are exposed as Qt signals.
#include <QCoreApplication>
#include <QDateTime>
#include <QDebug>
#include <QJsonDocument>
#include <QJsonObject>
#include <QTimer>

#include <azureiot/iothub.h>
#include <azureiot/iothub_device_client_ll.h>
#include <azureiot/iothub_client_options.h>
#include <azureiot/iothub_message.h>
#include <azureiot/iothubtransportamqp.h>

class IotHubClient : public QObject
{
    Q_OBJECT

public:
    IotHubClient(QObject *parent = nullptr);
    ~IotHubClient();

    bool init(const QString &connectionString);

    bool sendMessage(int id, const QByteArray &data);
    bool updateDeviceTwin(const QJsonObject &reported);

    bool connected() const;

signals:
    void connectedChanged(bool connected);
    void deviceTwinUpdated(int statusCode);
    void messageStatusChanged(int id, bool success);
    void desiredObjectChanged(QJsonObject desired);

private:
    // Callbacks for the IoT Hub client
    static void connectionStatusCallback(IOTHUB_CLIENT_CONNECTION_STATUS result,
                                         IOTHUB_CLIENT_CONNECTION_STATUS_REASON reason,
                                         void* context);
    static void sendConfirmCallback(IOTHUB_CLIENT_CONFIRMATION_RESULT result, void* context);
    static void deviceTwinCallback(DEVICE_TWIN_UPDATE_STATE updateState, const unsigned char* payload,
                                   size_t size, void* context);
    static void reportedStateCallback(int statusCode, void* context);

    // Call the IOT SDK's DoWork function
    void doWork();

    void setConnected(bool connected);

    struct MessageContext {
        MessageContext(IotHubClient *client, int id, IOTHUB_MESSAGE_HANDLE message) {
            this->client = client;
            this->id = id;
            this->message = message;
        }
        IotHubClient *client;
        int id;
        IOTHUB_MESSAGE_HANDLE message;
    };

    QTimer mDoWorkTimer;
    IOTHUB_DEVICE_CLIENT_LL_HANDLE mDeviceHandle = nullptr;
    bool mConnected = false;
};

IotHubClient::IotHubClient(QObject *parent)
    : QObject(parent)
{
    QObject::connect(&mDoWorkTimer, &QTimer::timeout, this, &IotHubClient::doWork);
    mDoWorkTimer.setInterval(200);
}

IotHubClient::~IotHubClient()
{
    mDoWorkTimer.stop();
    if (mDeviceHandle)
        IoTHubDeviceClient_LL_Destroy(mDeviceHandle);
    IoTHub_Deinit();
}

void IotHubClient::doWork()
{
    if (mDeviceHandle)
        IoTHubDeviceClient_LL_DoWork(mDeviceHandle);
}

bool IotHubClient::connected() const
{
    return mConnected;
}

void IotHubClient::setConnected(bool connected)
{
    if (mConnected != connected) {
        mConnected = connected;
        emit connectedChanged(mConnected);
    }
} 

Initialization of the SDK and client setup

The SDK must be initialized by calling IoTHub_Init() before it can be used. IoTHub_Deinit() deinitializes the SDK and is called in the destructor. There are different methods to create a client, depending on the authentication method used. In our example, we use a connection string with a shared key included and have to call IoTHubDeviceClient_LL_CreateFromConnectionString(). If nullptr is returned, this indicates that something is wrong with the connection string. IoTHubDeviceClient_LL_SetConnectionStatusCallback() and IoTHubDeviceClient_LL_SetDeviceTwinCallback() add callbacks that are not specific to a certain operation of the client. After the client has been initialized, the function IoTHubDeviceClient_LL_DoWork() must be called as often as possible. We use the Qt event loop for this and set up a QTimer which triggers every 200ms.
bool IotHubClient::init(const QString &connectionString)
{
    if (mDeviceHandle) {
        qDebug() << "Client is already initialized";
        return false;
    }

    auto result = IoTHub_Init();

    if (result != IOTHUB_CLIENT_OK) {
        qWarning() << "IoTHub_Init failed with result" << result;
        return false;
    }

    mDeviceHandle = IoTHubDeviceClient_LL_CreateFromConnectionString(connectionString.toUtf8().constData(),
                                                                     AMQP_Protocol);

    if (!mDeviceHandle) {
        qWarning() << "Failed to create client from connection string";
        return false;
    }

    result = IoTHubDeviceClient_LL_SetConnectionStatusCallback(mDeviceHandle, connectionStatusCallback, this);

    if (result != IOTHUB_CLIENT_OK) {
        qWarning() << "Failed to set connection status callback with result" << result;
        return false;
    }

    result = IoTHubDeviceClient_LL_SetDeviceTwinCallback(mDeviceHandle, deviceTwinCallback, this);

    if (result != IOTHUB_CLIENT_OK) {
        qWarning() << "Failed to set device twin callback with result" << result;
        return false;
    }

    mDoWorkTimer.start();

    return true;
}
 

Connection status callback

ConnectionStatusCallback is called on changes to the client’s connection status. It is called with a status and a reason, for example IOTHUB_CLIENT_CONNECTION_UNAUTHENTICATED due to IOTHUB_CLIENT_CONNECTION_NO_PING_RESPONSE. In our example, the callback sets the mConnected variable and emits the connectedChanged() signal.
void IotHubClient::connectionStatusCallback(IOTHUB_CLIENT_CONNECTION_STATUS result,
                                            IOTHUB_CLIENT_CONNECTION_STATUS_REASON reason,
                                            void *context)
{
    qDebug() << "Connection status changed to" << MU_ENUM_TO_STRING(IOTHUB_CLIENT_CONNECTION_STATUS, result) <<
                "with reason" << MU_ENUM_TO_STRING(IOTHUB_CLIENT_CONNECTION_STATUS_REASON, reason);

    const auto client = static_cast<IotHubClient *>(context);

    client->setConnected(result == IOTHUB_CLIENT_CONNECTION_AUTHENTICATED);
} 

Send confirm callback

SendConfirmCallback is called if sending a message succeeds or fails. For example, IOTHUB_CLIENT_CONFIRMATION_BECAUSE_DESTROY indicates that the message could not be sent before the client was destroyed. Our implementation emits the messageStatusChanged() signal which informs the connected slots about success or failure.
void IotHubClient::sendConfirmCallback(IOTHUB_CLIENT_CONFIRMATION_RESULT result, void *context)
{
    const auto messageContext = static_cast<IotHubClient::MessageContext *>(context);

    qDebug() << "Send confirm callback for message" << messageContext->id << "with" <<
                MU_ENUM_TO_STRING(IOTHUB_CLIENT_CONFIRMATION_RESULT, result);

    emit messageContext->client->messageStatusChanged(messageContext->id, result == IOTHUB_CLIENT_CONFIRMATION_OK);

    IoTHubMessage_Destroy(messageContext->message);
    delete messageContext;
} 

Device twin callback

DeviceTwinCallback is called if the initial full device twin is received and on every subsequent update of the desired object. The example callback decodes the JSON string and emits the desiredObjectChanged() signal with the new content of the desired object.
void IotHubClient::deviceTwinCallback(DEVICE_TWIN_UPDATE_STATE updateState, const unsigned char *payload,
                                      size_t size, void *context)
{
    qDebug() << "Received" << (updateState == DEVICE_TWIN_UPDATE_PARTIAL ? "partial" : "full") << "device twin update";

    const auto client = static_cast<IotHubClient *>(context);

    const auto data = QByteArray::fromRawData(reinterpret_cast<const char *>(payload), size);

    QJsonParseError parseError;
    const auto document = QJsonDocument::fromJson(data, &parseError);

    if (parseError.error) {
        qWarning() << "Failed to parse JSON document:" << parseError.errorString();
        return;
    }

    if (!document.isObject()) {
        qWarning() << "JSON document is not an object";
        return;
    }

    const auto deviceTwinObject = document.object();
    qDebug() << "Device twin content:" << deviceTwinObject;

    auto desired = deviceTwinObject;

    // Partial updates contain the desired object without an enclosing object
    if (updateState == DEVICE_TWIN_UPDATE_COMPLETE) {
        const auto jsonIterator = deviceTwinObject.constFind(QLatin1String("desired"));
        if (jsonIterator == deviceTwinObject.constEnd() || !jsonIterator->isObject()) {
            qWarning() << "The desired property is missing or invalid";
            return;
        }
        desired = jsonIterator->toObject();
    }

    emit client->desiredObjectChanged(desired);
} 

Reported state callback

ReportedStateCallback is called if a device twin update either succeeds or fails. The statusCode parameter is the HTTP status code of the update operation which is emitted as payload of the deviceTwinUpdated() signal.
void IotHubClient::reportedStateCallback(int statusCode, void *context)
{
    const auto client = static_cast<IotHubClient *>(context);
    emit client->deviceTwinUpdated(statusCode);
} 

Sending a message

The sendMessage() method implements sending a message to IoT Hub.

The id can be chosen by the caller and is used in the messageStatusChanged() signal when reporting on success or failure of a message sending operation.

For sending a message, we first have to create a message by calling IoTHubMessage_CreateFromByteArray() which is then put into the client’s send queue using IoTHubDeviceClient_LL_SendEventAsync().

The MessageContext struct is passed to the send confirm callback and contains the necessary meta information for the messageStatusChanged() signal.

bool IotHubClient::sendMessage(int id, const QByteArray &data)
{

    const auto message = IoTHubMessage_CreateFromByteArray(
                reinterpret_cast<const unsigned char *>(data.constData()), data.size());

    const auto context = new MessageContext(this, id, message);

    const auto result = IoTHubDeviceClient_LL_SendEventAsync(mDeviceHandle, message, sendConfirmCallback, context);

    if (result != IOTHUB_CLIENT_OK) {
        IoTHubMessage_Destroy(message);
        delete context;
    }

    return result == IOTHUB_CLIENT_OK;
} 

Updating the device twin

Updating the reported object of the device twin is implemented in updateDeviceTwin(). IoTHubDeviceClient_LL_SendReportedState() takes a JSON object in text form and sends it to IoT Hub. The success of the operation is reported in the reported state callback.
bool IotHubClient::updateDeviceTwin(const QJsonObject &reported)
{
    const auto reportedDocument = QJsonDocument(reported).toJson();
    const auto result = IoTHubDeviceClient_LL_SendReportedState(
                mDeviceHandle,
                reinterpret_cast<const unsigned char *>(reportedDocument.constData()),
                reportedDocument.size(), reportedStateCallback, this);
    return result == IOTHUB_CLIENT_OK;
} 

Using IotHubClient

The main() function creates an IotHubClient object and connects all signals to lambdas.

Most of the lambdas just print reports, the interesting work happens in the desiredObjectChanged() handler. It extracts the sendInterval property from the desired object and starts a QTimer which triggers sending a message with a JSON body. The successful application of the new send interval is confirmed to the cloud by setting the sendInterval property in the device twin’s reported object to the new value.

int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);

    IotHubClient client(&a);

    const auto connectionString = QLatin1String("PUT_YOUR_CONNECTION_STRING_HERE");
    auto result = client.init(connectionString);

    qDebug() << "Init:" << (result ? "successful" : "failed");

    if (!result)
        return 1;

    QObject::connect(&client, &IotHubClient::connectedChanged, [](bool connected) {
        qDebug() << "Client is now" << (connected ? "connected" : "disconnected");
    });

    QObject::connect(&client, &IotHubClient::deviceTwinUpdated, [](int statusCode) {
        qDebug() << "Device twin update returned" << statusCode;
    });

    QObject::connect(&client, &IotHubClient::messageStatusChanged, [](int id, bool success) {
        qDebug() << "Message" << id << (success ? "successfully sent" : "failed to send");
    });

    QTimer sendMessageTimer;
    int messageId = 0;

    QObject::connect(&sendMessageTimer, &QTimer::timeout, &client, [&]() {
        qDebug() << "Message send timer triggered" << QDateTime::currentDateTime().toString(Qt::ISODate);
        QJsonObject body;
        body[QStringLiteral("messageNumber")] = ++messageId;
        const auto data = QJsonDocument(body).toJson(QJsonDocument::JsonFormat::Compact);
        qDebug() << "Send data:" << data;
        client.sendMessage(messageId, data);
    });

    QObject::connect(&client, &IotHubClient::desiredObjectChanged, [&](const QJsonObject &desired) {
        const auto sendIntervalKey = QLatin1String("sendInterval");

        auto jsonIterator = desired.constFind(sendIntervalKey);

        if (jsonIterator != desired.constEnd() && jsonIterator->isDouble() && jsonIterator->toInt(-1) > 0) {
            const auto sendInterval = jsonIterator->toInt();

            qDebug() << "Got send interval" << sendInterval << "from device twin";

            QJsonObject reported;
            reported[sendIntervalKey] = sendInterval;

            client.updateDeviceTwin(reported);

            // Initialize the send timer with the new interval
            sendMessageTimer.start(sendInterval * 1000);
        }
    });

    return a.exec();
}

#include "main.moc" 

Building the example

If azure-iot-sdk-c is installed to /opt/azure, the application can be built using Qt creator or qmake with the following qmake file
QT -= gui

CONFIG += c++11 console
CONFIG -= app_bundle

# Change /opt/azure/ to the location where azure-iot-sdk-c is installed
QMAKE_CXXFLAGS += -I/opt/azure/include/
QMAKE_CXXFLAGS += -I/opt/azure/include/azureiot
LIBS += -L/opt/azure/lib

SOURCES += \
        main.cpp

# Flags for the static version of azure-iot-sdk-c
LIBS += \
    -liothub_client \
    -liothub_client_amqp_transport \
    -laziotsharedutil \
    -liothub_service_client \
    -lparson \
    -lserializer \
    -luamqp \
    -lumock_c \
    -lcurl \
    -lssl \
    -lcrypto \
    -luuid 

Testing the example

To make the example send messages, the sendInterval property must be added to the device twin. This can be done on the Azure portal by opening the detailed information on the device in the IoT devices explorer and then switching to the Device twin tab. Just extend the desired object in the editor with the new property
"sendInterval": 60 
and you should see a device twin update on the console output of our example which will then begin to send a message every 60 seconds. Refreshing the device twin on the portal should show a new sendInterval property in the reported property corresponding to the value we just have set. All changes to sendInterval while the example is running will be applied directly and the interval of message sending will change. As an exercise for the reader, this example could be extended to react to other device twin properties. If other IoT Hub features are required, the examples in the azure-iot-sdk-c repository are a good starting point.

Conclusion

Azure IoT Hub fulfills all requirements to a central hub for IoT devices that were identified in the introduction.

The different mechanisms for device to cloud and cloud to device messages as well as the device twin can support various use cases and the availability of the Free tier makes it easy to evaluate if IoT Hub is suitable for your special use case.

As we have seen from the example, implementing an IoT Hub client using the official SDK is not too complicated, a prototype can be implemented and tested in a matter of hours

Leave a Reply

Your email address will not be published. Required fields are marked *

Jannis Völker

Jannis Völker

Jannis Völker is a software engineer at basysKom GmbH in Darmstadt. After joining basysKom in 2017, he has been working in connectivity projects for embedded devices, Azure based cloud projects and has made contributions to Qt OPC UA and open62541. He has a background in embedded Linux, Qt and OPC UA and holds a master's degree in computer science from the University of Applied Sciences in Darmstadt.
Share on facebook
Share on twitter
Share on linkedin
Share on reddit
Share on xing
Share on email
Share on stumbleupon
Share on whatsapp
Share on pocket

Read more

IoT Cloud
Azure
Heike Ziegler
IoT: Getting started with cloud and modern IoT and IIoT from scratch

IoT and IIoT applications are special compared to other kinds of cloud applications as they have to deal with devices existing “outside” of data centers.

The following series of articles provide an end-to-end overview of what Microsoft Azure offers to handle some of the challenges involved in connecting an IoT Device with the Cloud.

By working through this series you will learn about the major concepts involved in getting your IoT/IIoT device connected to Microsoft Azure. In our examples we will feature Qt, Node.Js, Protobuf from Google and much more to get you started.

Read More »
protobuf messages azure functions
Azure
Jannis Völker
How to consume Protobuf messages in Azure Functions (Part 3 of 4)

Azure Functions offer a way to execute custom functions in the Azure cloud.

Functions are invoked by a trigger or binding, for a example a cron style timer, an HTTP request or an event like incoming data on other Azure services, for example ServiceBus Queues and Topics, Event Hubs or Event Grids. 

The functions can be hosted on Windows or Linux. Both platforms support a wide range of programming languages like JavaScript, TypeScript, C#, F#, Java, Python and Powershell.

Depending on the selected service plan, resources can be scaled out dynamically to handle an increased load and will be scaled in automatically if the load drops.

Read More »
Toolbox
basysKom
Jeremias Bosch
Awesome :-) The basysKom Toolbox

We would like to introduce our basysKom Toolbox to you. It is a state-of-the-art collection of best practices in agile management and software development and a valuable tool for every kickoff meeting.

Come and visit us on the Embedded World 2020 and take your own copy of the printed card deck with you!

Read More »