Sending data to the cloud is great! You’ve followed our Getting Started on The Things Network Tutorial and you have data streaming out of your Pycom Lopy4 and into your TTN account. There is just one problem, it’s all bytes! How do we get all that sweet sweet data into a usable format? We decode it!
This tutorial is all about encoding and decoding your data. To effectively use a LoRaWAN device you need to keep the transmitted packets as small as possible. This means encoding your data into bytes and decoding it on the backend. In this tutorial, we will just scratch the surface and get you sending encoded messages from your LoRaWAN device and decoding them within The Things Network. The good news is there are a lot of available resources to get this done, and we’ve got some code to get you started with some useful examples. We’ve also compiled a great list of resources to help you take that next step farther with encoding and decoding.
Encoding with MicroPython
Since the LoRaWAN device we like to use most is the Pycom Lopy4, we are going to give encoding examples using MicroPython. If you aren’t connected to The Things Network yet, go back and follow our Getting Started on The Things Network Tutorial.
A very simple way to encode into bytes is by using the “struct” python library. It's too much to cover the whole thing here, so if you want all the details read the struct library documentation.
But wait, aren’t we programming in MicroPython? The struct library exists in MicroPython as ustruct, it is largely the same, and only has a few differences that are outlined in the ustruct documentation. ustruct is a little leaner than stuct, so most everything will work. If you run into a weird error verify that ustruct includes the operation you are performing.
What does ustruct do? It encodes your data into bytes. There are a lot of different shapes that this can take, a string takes more bytes than an integer or a float. Choose a format that is appropriate for your data that keeps it to as little bytes as possible.
Here is an example:
packet = ustruct.pack(format, value1, value2…)
You can unpack the bytes with:
ustruct.unpack(format, packet)
Decoding within The Things Network
The data that comes into The Things Network is all in bytes, so we need to decode it into a usable value again. From the application screen navigate to “Payload Formats”.
From here we can see where we create the decoder. The programming language that we need to use is JavaScript, and with it, we can easily create a JSON value. The encoder can be used to encode values downlinked to your Pycom, but for this tutorial, we will focus on uplink data.
Encoder and Decoder Examples
Short:
2 bytes, 0-65,536. We can the values from two bytes as two individual numbers 0-255 or a single number 0-65,536 (256^2).
To encode a short in MicroPython:
import ustruct # import ustruct library test = 270 # value to be packed testpack = ustruct.pack('h',test) # encode value s.send(testpack) # send it to TTN print('Sending: ', test) # print the value sent time.sleep(3) # take a break (you earned it!)
To decode a short in JavaScript:
function Decoder(bytes, port) { // Decode an uplink message from a buffer // (array) of bytes to an object of fields. var decoded = {}; // Decode bytes to int var testShort = (bytes[1] << 8) | bytes[0]; // Decode int decoded.short = testShort; return decoded; }
Float:
4 bytes, includes a decimal point and can be positive or negative
To encode a float in MicroPython:
# encode the packet, so that it's in BYTES (TTN friendly) # could be extended like this struct.pack('f',lipo_voltage) + struct.pack('c',"example text") packet = ustruct.pack('f',lipo_voltage) # send the prepared packet via LoRa s.send(packet) # example of unpacking a payload - unpack returns a sequence of #immutable objects (a list) and in this case the first object is the only object print ("Unpacked value is:", ustruct.unpack('f',packet)[0])
To Decode a Float in JavaScript:
(photo only, our website backend corrupts the javascript. See the comments for a text version)
String:
To encode a string in Micropython:
#strings are converted to bytes automatically (extremely inefficient) s.send("Hello World!")
To decode a string in JavaScript (not recommended as it eats up bandwidth!):
function Decoder(bytes, port) { // Decode plain text; for testing only return { receivedString: String.fromCharCode.apply(null, bytes) }; }
Multiple Bytes:
To encode multiple bytes in a single packet in MicroPython:
s.send(bytes([1, 2]))
To decode multiple bytes in a single packet in JavaScript:
function Decoder(bytes, port) { // Decode an uplink message from a buffer // Decode bytes to int var firstValue = (bytes[1] << 8) | bytes[0]; var secondValue = bytes[3]<
Converter and Validator and Encoder
The data that you have received and decoded might need a little logic to spice it up. That’s where the Converter and Validator come in. These are all programmed in javascript as well.
- The Converter is a JavaScript function that takes the object produced by the Decoder and returns an object containing the converted values.
- The Validator is a JavaScript function that can be set to check the value returned by the Converter and returns a Boolean value indicating the validity of the data.
- The Encoder is used for downlinks to encode data into bytes using JavaScript before it is sent to your device.
Further Resources
If you want to learn more about encoding and decoding your packets and getting the most out of your bytes. Here are some additional resources that might be helpful:
ustruct MicroPython Documentation
How to Customise Payload Functions on TTN Video Tutorial - this is very handy
Working with Bytes- TTN Tutorial on using Bytes
The Things Uno Workshop – Workshop on getting started with communicating using TTN and Decoding Payloads
Best Practices when Sending GPS Location Data
How to use the lora-serialization library
Decrypting Messages for Dummies – More on using bytes, sign extension and port numbers
Help a Poor Guy with this Payload – Decode 7 bytes into two 28 bit numbers
Simple Payload Conversion – Get two integers from a null-terminated string
Decode a Float Sent by Lopy as Node – How to Convert as True IEEE-754 Floating Point
How can I Correctly Cast My Value in Bytes Before Sending – Working with Struct, and decoding it in TTN
Nemeus NIS-UL Ultrasonic Sensor - Working with a variable number and variable type of measurements by using bit masks
Armed with these examples there is nothing standing between you and a working device using The Things Network! If this is one step too far along for you we have a Getting Started on The Things Network Tutorial that walks you through getting connected to TTN using LoRaWAN and the Pycom Lopy4. If you’ve decoded your packets and you are ready to send them to a dashboard, check out our The Things Network Tutorials section for more tutorials about how to utilize your data! If you're ready to send your data out to the great beyond, (starting with Adafruit.io) check out our TTN, IFTTT, and BEYOND Tutorial!