Introduction

Just like the key-value store for homes, key-value pairs can be stored in the cloud for devices as well. We call it Device Data Proxy (DDP), because it serves not only as a data store, but also as a means of communication with devices.

At its core, Device Data Proxy is a key-value storage that applications can access at the following REST API endpoint:

/devices/_DEVICE_ID_/kv

What makes Device Data Proxy interesting is that it provides a mechanism for a device to watch the changes made by others to the data store, while the device itself can also update the data. Essentially, it acts as a proxy that allows bi-directional communication channel between the device and the app. It provides an alternative to the usual command-and-event method of communication.

Device Data Proxy is great for sending "state-ful" or persistent values. An obvious use is for holding configuration parameters for devices. Conversely, devices can use it to hold values that represent its "last-known" state.

Access from Applications

Accessing the key-value store of devices is quite similar to accessing the key-value store of homes via the REST API. For example, considering a device with ID 123, to set its key xyz to the value hello world, just make a PUT request like this:

$ curl -i -X PUT -H "Authorization: ModeCloud _YOUR_API_KEY_" -H "Content-Type: application/json" -d "{\"value\": \"hello world\"}" https://api.tinkermode.com/devices/123/kv/xyz

The types of value you can save to the key-value store are:

Similarly, you can use the REST API to read and delete key-value pairs.

Access from Devices

Obviously, devices are expected to interface with the Device Data Proxy so that key-value pairs can be propagated to them. In addition, devices can also push changes to key-value pairs back to DDP.

Data Synchronization

The key-value pairs are stored in the cloud and therefore can be read from and written to anytime by applications, even when the device itself is offline. Hence, this cloud-based data store acts as a “source of truth” for the key-value pairs associated with this device. However, this works only if the device implement the proper protocol to synchronize with the cloud storage.

There are two ways for a device to synchronize with the data storage:

Either way, the device should expect to receive “sync” messages from the connection. Each sync message is a JSON object. By applying the actions described in these messages, the device can maintain an up-to-date local mirror of the key-value pairs stored in the cloud.

There are three types of sync messages:

"Reload" Message

Device should expect a “reload” message immediately after it has established the websocket connection. It contains a full dump of all existing key-value pairs in the cloud storage. This allows the device to initialize its local mirror storage. Here is an example of a "reload" message:

{
  "action": "reload",
  "rev": 82,
  "numItems": 2,
  "items": [
    {
      "key": "xyz",
      "value": "hello world",
      "modificationTime": "2017-11-19T01:23:22Z"
    },
    {
      "key": "foo",
      "value": 22399,
      "modificationTime": "2017-11-21T11:09:18Z"
    }
  ]
}

The rev field is a revision number of the key-value store and is incremented every time a change happens (i.e. when a key-value is set or deleted). It is important for the device to keep track of the revision number with its local mirror storage, so that it can handle out-of-order sync messages.

"Set" Message

The device will receive a “set” message when a key-value pair has been created or updated. For example:

{
  "action": "set",
  "rev": 83,
  "key": "xyz",
  "value": "I have changed"
}

The device should apply this change to its local storage if and only if the rev field is greater than the revision number stored locally.

"Delete" Message

The device will receive a “delete” message when a key-value pair has been deleted. For example:

{
  "action": "delete",
  "rev": 85,
  "key": "xyz"
}

The proper way to handle a “delete” message is to mark the local mirror value as deleted, and record the new rev number. This way, an out-of-order “set” message to the same key can be identified and ignored.

Setting and Deleting Key-Value Pairs

DDP is also a mechanism for devices to report values or state changes back to the cloud. The REST API described earlier is available to device instances as well, so that they can update or delete key-value pairs in the data store.

The same feature is also available to devices that use MQTT to communicate with the MODE Cloud. As in data synchronization, it will be using the topic /devices/_DEVICE_ID_/kv. But instead of subscribing to the topic, a device will be publishing JSON messages that describe the desired actions.

For example, to set a key-value pair, a device should publish a JSON that looks like the following:

{
  "action": "set",
  "key": "xyz",
  "value": "this is changed"
}

And to delete a key-value pair, just publish a JSON like the following:

{
  "action": "delete",
  "key": "xyz"
}

System-Generated Events

Just like the Key-Value Store of a home, Device Data Proxy also generates some special events whenever there is a change to one of its key-value pairs. The following system-generated events are propagated to the home of the device whose Data Proxy has been modified:

As far as apps and Smart Modules are concerned, these system-generated events are no different from events triggered by devices or client programs. But as a general rule, you should expect events with type names that start and end with an underscore (_) to be generated by the MODE system.

Example Code

The official MODE device SDK in Go supports Device Data Proxy synchronization and mirroring. You may use it directly for your device software implementation, or use it as a reference for your own design.