Wednesday, September 25, 2013

Data format required for streaming platform alpha launch

In response to several recent questions, here's a quick update on the data structure our system can handle for now.  

Input.  What the data should look like.  Each piece of data represents a point in time.  Your sensor is gathering data and then emitting (should use this term) it.  You don't specifically mark the time on each piece of data but as they are created they are sent to the Algorithms.io system. 

Each piece of data (or if you want to call it an event) has some properties associated with it.  It depends on what your sensor is designed to do.  For example for a gyroscope sensor.  It will emit rotation of the x, y, and z axis.  For every sample that the sensor emit it might look like gyro.x = -84.2, gyro.y = -132.2, gyro.z = -80.  All of that data represents 1 event.  You can send as many of these "events" to Algorithms.io as you want.

Most sensors can emit at a very high frequency.  You can try to send all that data into Algorithms.io (which it can handle) but is your internet connection good enough to send all that data?  Or do you need that high of a sample rate for your application?  What is the right sample rate?  Thats another topic for another blog.

Once you have all this time series data streamed in you can do some very cool stuff with it.  For our users, the most important thing to do with the data is classify it WITHOUT having to pre-define the device state.  What that means, is being able to tell from the raw data what's going on, not requiring pre-set parameters that define the events ("right now I am running, all data coming in is related to running") for a specific period of time.

We'll have more on this in our support FAQ.