Data Service Engine

Tom Schoellhammer, Mohammad Rahimi, Eric Osterweil, Mike Wimbrow

Executive Summary

It is the job of the Data Service Engine (DSE) to accept queries from the system bus (for which we plan to use Tiny Diffusion) and then to evaluate them, returning any query results to the queryer. A query can be one of three things: The DSE first evaluates whether there are enough resources to execute the query. If so, the DSE then executes the query and returns any desired results to the system bus.

The Query Engine interfaces with the system bus to provide data services to the system. The Query Engine needs to know how to interpret queries and measurement names, but need not know any details about the sampling hardware itself. For this reason, the combination of the Data Map and the Transform Manager abstract away the details of physical sensors. The Data Map takes a high-level measurement name and translates it into a physical address for the sensor (ADC channel/hardware address, Event Counter, Pulse rate counter, Mica2 battery voltage, SIB Temp, SIB Humidity, or virtual sensor). Once the sensor data is ready, the Transform Manager changes the raw sensor value into appropriate measurement units. This data is then used by the Query Engine to satisfy standing queries.

In the case of a configuration command, the message is directed to the Configuration Handler (CH). These messages are used to configure the DM so that it can use additional sensors. The CH interfaces with the EEPROM driver so that configuration information can be stored in a non-volatile manner. Rebooting the mote then does not neccessitate reconfiguring the mote.

The target platform for the DSE is TinyOS running on Mica2 motes. Each mote will be outfitted with a Mica2 Data Aquisition Board.

Component Decomposition

The DSE is broken into six pieces. The Query Engine (QE) interfaces with the system bus to accept queriess from it and provide data to it. The QE uses the Query Table (QT) to store each query and any associated state. The QT both allocates memory for each query and marshalls access to allocated memory. The QE interfaces with the Data Map (DM) in order to request data associated with a particular measurement name. The DM is responsible for keeping track of the parameters intrinsic to a particular sensor (detailed later). It also translates a meaurement name into an hardware address (ADC channel number). Once that translation is done the Sampler is tasked. When that data is ready, the Transform Manager (TM) converts the raw ADC value to a unitted data type. The TM uses DM to perform this conversion. The TM then returns the data to the QE.

Query Engine

The QE's job is to In order to perform these three tasks the QE interfaces with three components. The QE requests memory from the QT. The QT may deny memory to the QE if there is not enough. In the event that there is not enough memory the query is not executed.

Interfaces

Modules

Testing

A test driver will be written to test the QE in the following ways:

Data Map

The first job of the DM is to provide the QE with a list of available sensors. Its second job is to take handle sampling requests from the QE. The DM receives requests from the QE that specifies a measurement name and sampling period. The DM tranlates that measurement name into an hardware address (ADC channel). The hardware address and sample period are then passed to the Sampler. To do these jobs the DM maintains a table that correlates a measurement name with sensor parameters, hardware address, and transformation ID (detailed later). The DM also accepts requests to either update or add entries to this table

Interfaces

Modules

Testing

A test driver will be written to test the DM in the following ways:

Transform Manager

The TM keeps a list of transforms that take raw ADC values as input and convert them to meaningful unitted data types. In addition, it supports an "identity" transform that does nothing to the data. This identity transform can be used when a new sensor for which no prespecified transform exists.

Interfaces

Modules

Testing

A test driver will be written to test the Transform manager in the following ways:

Query Table

It is the job of the Query Table to manage the memory required to store a query as well as marshal access to query data structures. In order to use memory efficiently and to avoid fragmentation the Query Table performs memory compaction. To avoid inconsistancies caused by changing the location of various data blocks the Query Table also can grant or deny access to memory.

The Query Table needs an interface that will allow applications to request storage as well as to request access to data already stored. The QT accepts explicit requests to delete standing queries.

Interfaces

Modules

Testing

A test driver will be written to test the QueryTable in the following ways:

Configuration Handler

It is the job of the CH is to accept incoming configuration commands, store them in non-volatile memory (EEPROM) and then update the DM with the new configuration information. In addition, at startup, the CH reads an existing configuration information and uses it to configure the DM.

Although the diagram shows that the CH interfaces direclty with the system bus it has not been decided whether it will interface directly or some other multiplexing component will be used.

Interfaces

Modules

Testing

A test driver will be written to test the QueryTable in the following ways:

Interfaces

  1. QeAcceptQueryI

    2. The QeAcceptQueryI interface connects to the system bus to accept queries and return data.
    • passQuery( )
      This command is used to pass a buffer containing a query to the QE.
    • sendQueryResult( )
      This event is used to pass data to the system bus for transport.
    • returnBuffer( )
      This event is used to return a buffer that was borrowed from the system bus. It also conveys whether or not the query has been accepted or not.

  2. QeAcceptDataI

    3. This interface is presented to the lower layer (specifically, the Transform Manager) and is used to receive data that will be used to satisfy standing queries.
    • passData( )
      This command allows the QE to receive a unitted data type.

  3. QeStorageI

    4. This interface is presented to the QT and provides all functions associated with storage and retreival.
    • storeDone( )
      This command allows the QE to be notified when data is done being stored.

  4. DmAcceptMnAndTI

    5. This interface is presented to the QE so that the QE can request a particular measurement name of interest and the period that data should be returned.
    • passMnT( )
      This command is used to pass a measurement name and period. A sampling ID is returned.
    • cancelSamplingID( )
      This command is used to cancel a particular sampling task.
    • getRemainingSamplerJobs( )
      This command is used to get the number of sampler tasks that are unused.

  5. DmUpdateTableI

    6. This interface is used to update the Data Map's table that contains the parameters for all the sensors.
    • newSensorParams( )
      This command is used to specify the parameters for a new sensor or to update the parameters of an existing sensor
    • deleteSensor( )
      This command is used to delete a particular sensor from the Data Map's table.
    • newMnMapping( )
      This command takes a measurement name and hardware address as input. If an entry already exists for that measurement name then the hardware address is updated. If the measurement name is not in the table then a new entry is created.
    • deleteMnMapping( )
      This command is used to delete the table entry associated with the passed measurement name.
    • readSensorParams( )
      This command returns the parameters for a sensor based on measurement name.

  6. DmMappingI

    7. This interface is used to access the table that maps measurement names to hardware address and vice versa.
    • mn2hwAddr( )
      This command takes a measurement name as input and returns the associated hardware address.
    • hwAddr2mn( )
      This command takes an hardware address and returns the associated measurement name.

  7. TmTransformDataI

    8. This interface is presented to the Sampler so that data can be passed to the TM. The TM takes this data and applies a transform to convert a raw ADC value to a unitted data type. This data is then passed to the QE.
    • transformData( )
      This command takes a raw ADC value, measurement name, and sampling ID as input. It then applies the appropriate transform to the data (based on the measurement name).

  8. QueryTableI

    9. The QueryTableI interface defines the functions to insert, access, and expire queries. The commands for this interface are:
    • insert( )
      Used to request the storage of a query.
    • deleteQueryId( )
      Used to request the deletion of a query with matching ID.
    • request( )
      Used to get exclusive access to a query.
    • release( )
      Used to release access to a query.
    • insertionDone( )
      Used to notify the module that called insert that insertion is complete.

  9. ChAcceptCmdI

    10. This interface defines those commands required to interface with the system bus.
    • acceptCmd( )
      A configuration command is passed to this function.
    • acceptCmdDone( )
      This returns the buffer that arrived via the acceptCmd function.
    • readMNs( )
      Returns to the system bus the measurement names of all availabe sensors.
    • readMnConfig( )
      Returns configuration information about a particular measurement name/

Query Engine

  1. Interfaces

    2. This component implements the following interface:

  2. Modules

    3. QueryEngineM

    This module implements the QeAcceptQueryI, QeAcceptDataI, and QeStorage interfaces.

  3. Implementation

    4. The implementation of this component is as follows:

Data Map

  1. Interfaces

    2. This component implements the following interface:

  2. Modules

    3. DataMapM

    This component implements the following interfaces:

  3. Implementation

    4. The implementation of this component is as follows:

Transform Manager

  1. Interfaces

    2. This component implements the following interface:

  2. Modules

    3. TransformManagerM

    This module implements the TmTransformDataI interface.

  3. Implementation

    4. The implementation of this component is as follows:

Query Table

  1. Interfaces

    2. This compoent implements the following interface:

  2. Modules

    3. QueryTableM

    This module implements the QueryTableI interface.
    • compactMemory( )
      This method is internal and is used to post a task that will perform compaction of the memory pool that the Query Table manages.
    • finishInsertion( )
      This method is internal and is used to post a task that will finish the insertion of a buffer into memory.

    Testing

    The testing of this module will entail the insertion of several queries. These queries will then be deleted, refreshed, retrieved in order to validate functionality.

  3. Implementation

    4. The implementation of this component is as follows:

Configuration Handler

  1. Interfaces

    2. This component implements the following interface:

  2. Modules

    3. ConfigurationHandlerM

    This module implements the ChAcceptCmdI interface.

  3. Implementation

    4. The implementation of this component is as follows:

Internal Data Sturcutres

  1. Query Engine
    2. The QE interprets queries and data, which are either stored in temporary buffers or within the QT. Thus, the QE is a finite state machine and does not possess any large data structures.

  2. Query Table
    3. The QT implements a memory manager. As such it manages a large memory buffer in a linked list fashion.

  3. Data Map
    4. For each analog measurement name there are three parameters: power supply (3V or 5V), setup time (in ms), and the number of samples to average for noise reduction.

    For each digital measurement name there are also four parameters: suppression interval (time during which upper layers are not interrupted with events), triggering type (rising edge, falling edge, both edges), whether the sensor is a counter, and whether the sensor generates events.

    The DM maintains a table that has as many entiries as there are hardware addresses. For each hardware address there is an associated measurement name, along with all parameters.

  4. Transform Manager
    5. The TM keeps one transform per measurement name. In addition, it provides a default transform that does nothing to a raw ADC value, allowing this raw value to pass directly to the QE.

  5. Configuration Handler
    6. The CH reads and writes configuration information from the EEPROM. Because the configuration for a sensor could change, and because the ability to add new configuration information is needed the CH needs to implement a simple file system on the EEPROM. Due to the small amount of configuration information a linked list implementation is proposed, although this has not been explored throughly.

  1. Component Testing

    2. Create an application called 
    TEST_QueryTable_DRIVER
    This applications is expected to build and run without specific input, and is expected to produce very standard output. Upon successful completion, drivers should produce the string: 
    >>>SUCCESS<<<
    Upon a failure, drivers should produce the string: 
    >>>FAIL<<<
    These strings should be produced using DBG_USR3.

  2. Integration Testing

    3. The purpose of the regression test suite is to ensure that the Data Service Provider's functionality can be tested at any time. A system's regression test suite will be comprised of a list of operations that must be successfully performed in a simulated environment according to a pre-defined set of acceptance criteria.

  3. Regression Testing

    4. Forthcoming.