Difference between revisions of "Antelope(Database Management System) - Contiki"

From Contiki
Jump to: navigation, search
 
(25 intermediate revisions by the same user not shown)
Line 4: Line 4:
  
 
== Introduction ==
 
== Introduction ==
Antelope is a database management system for resource-constrained memory devices. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.
+
Antelope is a database management system which enables a sensor to act as a database server. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.
 +
 
 +
Antelope is the first DBMS for resource-constrained sensor nodes, provides energy-efficient querying, high-level data modelling, while being index independent. MaxHeap indexing enables fast insertions and queries over large data sets while having a low energy cost. Antelope hides the low-level I/O details of flash storage, which have thus far often been exposed to application developers. Moreover, Antelope uses LogicVM, a novel virtual machine architecture that analyses and executes propositional logic expressed in a byte-code format. By compiling queries and executing them in the virtual machine, the performance of relational selection queries increases by an order of magnitude compared to repeated parsing.
  
 
== You will learn ==
 
== You will learn ==
  
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience.
+
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience on some of the basic query operations of the antelope.
  
 
== Antelope architecture ==
 
== Antelope architecture ==
Line 17: Line 19:
  
  
*Query processor  : It parses AQL(query language of Antelope) queries.
+
*''Query processor''   : It parses AQL(query language for Antelope) queries.
*Privacy Control  : Checks whether the query is allowed or not.
+
*''Privacy Control''   : Checks whether the query is allowed or not.
*Logic VM          : It executes the queries.
+
*''Logic VM''         : It executes the queries.
*Database kernel  : It holds the database logic and coordinates query execution.
+
*''Database kernel''   : It holds the database logic and coordinates query execution.
*Index abstraction : It holds the indexing logic.
+
*''Index abstraction'' : It holds the indexing logic.
*Indexer process  : Builds indexes from existing data.
+
*''Indexer process''   : Builds indexes from existing data.
*Storage abstraction : Contains all storage logic.
+
*''Storage abstraction'' : Contains all storage logic.
*Result transformer : It presents the result of query in a way that makes it easy to use by programs.
+
*''Result transformer'' : It presents the result of query in a way that makes it easy to use by programs.
  
  
Line 42: Line 44:
 
[[File:Example Antelope terminology.png|jpg|500px|center]]
 
[[File:Example Antelope terminology.png|jpg|500px|center]]
  
== Source Code ==
+
== Source Code Directories ==
  
 
Navigate to the examples/antelope/netdb folder from the contiki home directory.
 
Navigate to the examples/antelope/netdb folder from the contiki home directory.
Line 53: Line 55:
 
There is one more file netdb.csc which we are going to use for our simulation on cooja.
 
There is one more file netdb.csc which we are going to use for our simulation on cooja.
  
Let's walk through the source files before we start the simulation.
+
The implementation of Antelope database in present in the directory:
 +
:: <code> ~/contiki-2.7/apps/antelope</code>
 +
 
 +
== Code building blocks ==
 +
In this section, source code for the example given in the <code> ~/contiki-2.7/examples/antelope/netdb/</code> is explored.
  
=== Code building blocks ===
+
=== netdb-client.c ===
==== netdb-client.c ====
+
This client program provides a serial line interface with the help of which we can enter the queries. The shell process of the client program process these serial messages(queries) and sends the queries to the server program(running on another mote). Once it receives the reply to the query, it prints out a confirmation message in the shell.
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.
+
  
 
Let's walk through the source file.
 
Let's walk through the source file.
Line 74: Line 79:
 
</source>
 
</source>
  
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.
+
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important data structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.
 
+
----
 
<source lang="c">
 
<source lang="c">
 
struct mesh_callbacks {
 
struct mesh_callbacks {
Line 92: Line 97:
 
AUTOSTART_PROCESSES(&netdb_process);
 
AUTOSTART_PROCESSES(&netdb_process);
 
</source>
 
</source>
The first two lines in the above source code creates two processes netdb_process(which is the database process), shell_process. "shell_process" is the process which provides serial interface at the client to input our queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread assosicated with this process will start running by the scheduler. Below is the code snipped of the thread for this process.
+
The first two lines in the above source code creates two processes "netdb_process"(which is the database process) and "shell_process". "shell_process" is the process which provides serial interface at the client to input the queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread associated with this process will start running by the scheduler. Below is the code snippet of the thread for this process.
 
+
----
 
<source lang="c">
 
<source lang="c">
 
PROCESS_THREAD(netdb_process, ev, data)
 
PROCESS_THREAD(netdb_process, ev, data)
Line 110: Line 115:
 
static const struct mesh_callbacks callbacks = {received, sent, timedout};
 
static const struct mesh_callbacks callbacks = {received, sent, timedout};
 
</source>
 
</source>
This will create a callabacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.
+
This will create a callbacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.
 +
 
 +
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow [https://github.com/contiki-os/contiki/wiki/Processes this] link.
 +
 
 +
"mesh_open" opens a new connection and here the created mesh connection operates on NETDB_CHANNEL(70) and the third argument "callbacks" contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another process, instead of using AUTOSTART_PROCESSES).
  
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow this link.
+
----
  
mesh_open opens a new connection and here this mesh connection operates on NETDB_CHANNEL(70) and the third argument callbacks contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another instead of using AUTOSTART_PROCESSES).
+
''' Callback handlers for the netdb_client process mesh connection: '''
  
===== Callback handlers for the netdb_client process mesh connection =====
 
 
<source lang="c">
 
<source lang="c">
 
static const struct mesh_callbacks callbacks = {received, sent, timedout};
 
static const struct mesh_callbacks callbacks = {received, sent, timedout};
Line 129: Line 137:
 
</source>
 
</source>
 
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.
 
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.
 +
 +
----
  
 
*Callback function - timedout
 
*Callback function - timedout
Line 140: Line 150:
  
 
This function gets called when we try to send a  packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".
 
This function gets called when we try to send a  packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".
 +
 +
----
  
 
*Callback function - received
 
*Callback function - received
Line 167: Line 179:
 
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.
 
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.
 
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.
 
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.
 +
 +
----
  
 
Let's look at the source code for the "shell_process".
 
Let's look at the source code for the "shell_process".
Line 197: Line 211:
 
</source>
 
</source>
  
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.
+
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite 'for loop' "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and '''mesh_send''' function is called to send the packet to the server using mesh routing protocol.
 +
----
  
 
+
=== netdb-server.c ===
==== netdb-server.c ====
+
Most of the source code for the ''netdb_server'' file is same as that of the above explained ''netdb_client'' source file like the '''mesh_conn''' data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.
+
  
 
<source lang="c">
 
<source lang="c">
Line 218: Line 232:
 
}
 
}
 
</source>
 
</source>
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.
+
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of '''netdb_process''' we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.
 
+
----
 
<source lang="c">
 
<source lang="c">
 
static void
 
static void
Line 246: Line 260:
 
}
 
}
 
</source>
 
</source>
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".
+
This function gets called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is passed as "serial_line_event_message".
 
+
----
Let's look at the query_process which plays an important role in processing the incoming queries.
+
 
<source lang="c">
 
<source lang="c">
 
PROCESS(query_process, "Query process");
 
PROCESS(query_process, "Query process");
Line 295: Line 308:
 
</source>
 
</source>
  
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.
+
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like shown in the cooja simulation section below.
 
+
[[File:Ready.png|left|400px]] [[File:NetdbClient.png|right|400px]]
+
 
+
  
 
<source lang="c">
 
<source lang="c">
 +
 
for(;;) {
 
for(;;) {
PROCESS_WAIT_EVENT();
+
    PROCESS_WAIT_EVENT();
if(ev == serial_line_event_message && data != NULL) {
+
 
printf("START %s\n", (char *)data);
+
    if(ev == serial_line_event_message && data != NULL) {
result = db_query(&handle, data);
+
      printf("START %s\n", (char *)data);
if(DB_ERROR(result)) {
+
      result = db_query(&handle, data);
buffer_db_data("Query error: %s\n", db_get_result_message(result));
+
      if(DB_ERROR(result)) {
stop_handler(NULL);
+
        buffer_db_data("Query error: %s\n", db_get_result_message(result));
db_free(&handle);
+
        stop_handler(NULL);
continue;
+
        db_free(&handle);
}
+
        continue;
if(!db_processing(&handle)) {
+
      }
buffer_db_data("OK\n");
+
 
send_buffered_data();
+
      if(!db_processing(&handle)) {
stop_handler(NULL);
+
        buffer_db_data("OK\n");
continue;
+
        send_buffered_data();
}
+
        stop_handler(NULL);
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,
+
        continue;
PACKETBUF_ATTR_PACKET_TYPE_STREAM);
+
      }
db_print_header(&handle);
+
 
matching = 0;
+
      packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,
processed = 0;
+
                        PACKETBUF_ATTR_PACKET_TYPE_STREAM);
while(db_processing(&handle)) {
+
 
PROCESS_PAUSE();
+
      db_print_header(&handle);
if(matching == RESPONSE_LIMIT) {
+
 
buffer_db_data("Response suppressed at %u tuples: limit reached\n",
+
      matching = 0;
RESPONSE_LIMIT);
+
      processed = 0;
stop_handler(NULL);
+
 
db_free(&handle);
+
      while(db_processing(&handle)) {
break;
+
        PROCESS_PAUSE();
}
+
 
result = db_process(&handle);
+
        if(matching == RESPONSE_LIMIT) {
if(result == DB_GOT_ROW) {
+
            buffer_db_data("Response suppressed at %u tuples: limit reached\n",
/* The processed tuple matched the condition in the query. */
+
                          RESPONSE_LIMIT);
matching++;
+
            stop_handler(NULL);
processed++;
+
            db_free(&handle);
db_print_tuple(&handle);
+
            break;
} else if(result == DB_OK) {
+
        }
/* A tuple was processed, but did not match the condition. */
+
 
processed++;
+
        result = db_process(&handle);
continue;
+
        if(result == DB_GOT_ROW) {
} else {
+
          /* The processed tuple matched the condition in the query. */
if(result == DB_FINISHED) {
+
          matching++;
/* The processing has finished. Wait for a new command. */
+
          processed++;
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",
+
          db_print_tuple(&handle);
(long)matching, (long)processed);
+
        } else if(result == DB_OK) {
buffer_db_data("OK\n");
+
          /* A tuple was processed, but did not match the condition. */
} else if(DB_ERROR(result)) {
+
          processed++;
buffer_db_data("Processing error: %s\n",
+
          continue;
db_get_result_message(result));
+
        } else {
}
+
          if(result == DB_FINISHED) {
stop_handler(NULL);
+
            /* The processing has finished. Wait for a new command. */
db_free(&handle);
+
            buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",
}
+
                          (long)matching, (long)processed);
}
+
            buffer_db_data("OK\n");
send_buffered_data();
+
          } else if(DB_ERROR(result)) {
}
+
            buffer_db_data("Processing error: %s\n",
 +
                          db_get_result_message(result));
 +
          }
 +
          stop_handler(NULL);
 +
          db_free(&handle);
 +
        }
 +
      }
 +
      send_buffered_data();
 +
    }
 
#if !PREPARE_DB
 
#if !PREPARE_DB
if(etimer_expired(&sampling_timer)) {
+
    if(etimer_expired(&sampling_timer)) {
take_sample();
+
      take_sample();
etimer_reset(&sampling_timer);
+
      etimer_reset(&sampling_timer);
}
+
    }
 
#endif
 
#endif
 
}
 
}
Line 399: Line 418:
 
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.
 
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.
  
 +
[[File:Ready.png|center|400px]]
 +
 +
[[File:NetdbClient.png|center|400px]]
  
 
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.
 
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.
Line 473: Line 495:
 
05:22.204    ID:4    END
 
05:22.204    ID:4    END
 
</source>
 
</source>
 +
 +
Below are the snapshots for Antelope client and server logs in cooja.
 +
 +
[[File:Client_Query.png|500px]]
 +
[[File:Server_Query.png|550px]]
 +
 +
Similarly, we could try some other examples like below:
 +
<source lang="c">
 +
CREATE RELATION sensor;
 +
 +
CREATE ATTRIBUTE id DOMAIN INT IN sensor;
 +
CREATE ATTRIBUTE name DOMAIN STRING(20) IN sensor;
 +
CREATE ATTRIBUTE position DOMAIN LONG IN sensor;
 +
 +
CREATE INDEX sensor.id TYPE INLINE;
 +
CREATE INDEX sensor.position TYPE MAXHEAP;
 +
</source>
 +
 +
 
== References ==
 
== References ==
 
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.
 
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.
  
Edited by Gopi.
+
[https://github.com/contiki-os/contiki/wiki 2] - Contiki Operating System Wiki page.
  
 +
----
 
[[Contiki_tutorials | Back to Contiki Tutorials]]
 
[[Contiki_tutorials | Back to Contiki Tutorials]]
 +
 +
Edited by Gopi Krishna.

Latest revision as of 00:01, 9 November 2014

Back to Contiki Tutorials

Introduction

Antelope is a database management system which enables a sensor to act as a database server. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.

Antelope is the first DBMS for resource-constrained sensor nodes, provides energy-efficient querying, high-level data modelling, while being index independent. MaxHeap indexing enables fast insertions and queries over large data sets while having a low energy cost. Antelope hides the low-level I/O details of flash storage, which have thus far often been exposed to application developers. Moreover, Antelope uses LogicVM, a novel virtual machine architecture that analyses and executes propositional logic expressed in a byte-code format. By compiling queries and executing them in the virtual machine, the performance of relational selection queries increases by an order of magnitude compared to repeated parsing.

You will learn

In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience on some of the basic query operations of the antelope.

Antelope architecture

Components

Antelope consists of eight components as shown in the below figure.

Antelope arch.jpg


  • Query processor  : It parses AQL(query language for Antelope) queries.
  • Privacy Control  : Checks whether the query is allowed or not.
  • Logic VM  : It executes the queries.
  • Database kernel  : It holds the database logic and coordinates query execution.
  • Index abstraction : It holds the indexing logic.
  • Indexer process  : Builds indexes from existing data.
  • Storage abstraction : Contains all storage logic.
  • Result transformer : It presents the result of query in a way that makes it easy to use by programs.



Terminology

Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:

  • Tuple: It is a set of attribute values.
  • Attribute: Different properties. E.g SensorID, Time.
  • Domain: Each attribute has got a domain that specifies the data type of the attribute values.
  • Relation: It is a collection of tuples that have the same set of attributes.
  • Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.
  • Cardinality: The number of tuples in a relation is called the cardinality of the relation.

To put the above terminology in easy terms, informally we can say :

  • A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.
jpg

Source Code Directories

Navigate to the examples/antelope/netdb folder from the contiki home directory.

cd /examples/antelope/netdb

There are two important files in this(netdb) subfolder.

  • netdb-server.c
  • netdb-client.c

There is one more file netdb.csc which we are going to use for our simulation on cooja.

The implementation of Antelope database in present in the directory:

~/contiki-2.7/apps/antelope

Code building blocks

In this section, source code for the example given in the ~/contiki-2.7/examples/antelope/netdb/ is explored.

netdb-client.c

This client program provides a serial line interface with the help of which we can enter the queries. The shell process of the client program process these serial messages(queries) and sends the queries to the server program(running on another mote). Once it receives the reply to the query, it prints out a confirmation message in the shell.

Let's walk through the source file.

#ifndef SERVER_ID
#define SERVER_ID 4
#endif

SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).

static unsigned server_id = SERVER_ID;
static struct mesh_conn mesh;

Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important data structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.


struct mesh_callbacks {
  /** Called when a packet is received. */
  void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);
  /** Called when a packet, sent with mesh_send(), is actually transmitted. */
  void (* sent)(struct mesh_conn *c);
  /** Called when a packet, sent with mesh_send(), times out and is dropped. */
  void (* timedout)(struct mesh_conn *c);
};
PROCESS(netdb_process, "NetDB");
PROCESS(shell_process, "Shell Process");
AUTOSTART_PROCESSES(&netdb_process);

The first two lines in the above source code creates two processes "netdb_process"(which is the database process) and "shell_process". "shell_process" is the process which provides serial interface at the client to input the queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread associated with this process will start running by the scheduler. Below is the code snippet of the thread for this process.


PROCESS_THREAD(netdb_process, ev, data)
{
  PROCESS_EXITHANDLER(mesh_close(&mesh));
  PROCESS_BEGIN();

  mesh_open(&mesh, NETDB_CHANNEL, &callbacks);
  process_start(&shell_process, NULL);

  PROCESS_END();
}
static const struct mesh_callbacks callbacks = {received, sent, timedout};

This will create a callbacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.

A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow this link.

"mesh_open" opens a new connection and here the created mesh connection operates on NETDB_CHANNEL(70) and the third argument "callbacks" contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another process, instead of using AUTOSTART_PROCESSES).


Callback handlers for the netdb_client process mesh connection:

static const struct mesh_callbacks callbacks = {received, sent, timedout};
  • Callback function - sent
static void
sent(struct mesh_conn *c)
{
}

This function gets called when we try send a packet with mesh_send() and it is actually transmitted.


  • Callback function - timedout
static void
timedout(struct mesh_conn *c)
{
  printf("Failed to send packet: time out\n");
}

This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".


  • Callback function - received
static void
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)
{
  char *data;
  unsigned len;
  static char reply[MAX_QUERY_SIZE + 1];

  data = (char *)packetbuf_dataptr();
  len = packetbuf_datalen();

  if(len > MAX_QUERY_SIZE) {
    printf("Too long query: %d bytes\n", len);
    return;
  }

  memcpy(reply, data, len);
  reply[len] = '\0';

  printf("%lu Reply received from %d.%d (%d hops): %s",
         clock_time(), from->u8[0], from->u8[1], (int)hops, reply);
}

This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server. Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.


Let's look at the source code for the "shell_process".

PROCESS_THREAD(shell_process, ev, data)
{
  rimeaddr_t addr;

  PROCESS_BEGIN();

  printf("NetDB client\n");

  for(;;) {
    PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);
    if(strncmp(data, "server ", 7) == 0) {
      server_id = atoi((char *)data + 7);
    } else {
      printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);
      packetbuf_copyfrom(data, strlen(data));
      addr.u8[0] = server_id;
      addr.u8[1] = 0;
      packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,
                         PACKETBUF_ATTR_PACKET_TYPE_STREAM);
      mesh_send(&mesh, &addr);
    }
  }

  PROCESS_END();
}

A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite 'for loop' "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.


netdb-server.c

Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.

PROCESS(netdb_process, "NetDB");
AUTOSTART_PROCESSES(&netdb_process);

PROCESS_THREAD(netdb_process, ev, data)
{
  PROCESS_EXITHANDLER(mesh_close(&mesh));
  PROCESS_BEGIN();

  mesh_open(&mesh, NETDB_CHANNEL, &callbacks);
  process_start(&query_process, NULL);

  PROCESS_END();
}

Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.


static void
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)
{
  char *data;
  unsigned len;
  static char query[MAX_BUFFER_SIZE + 1];

  data = (char *)packetbuf_dataptr();
  len = packetbuf_datalen();

  if(len > MAX_BUFFER_SIZE) {
    buffer_db_data("Too long query: %d bytes\n", len);
    return;
  }

  memcpy(query, data, len);
  query[len] = '\0';

  printf("Query received from %d.%d: %s (%d hops)\n",
         from->u8[0], from->u8[1], query, (int)hops);
  rimeaddr_copy(&reply_addr, from);

  process_post(&query_process, serial_line_event_message, query);
}

This function gets called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is passed as "serial_line_event_message".


PROCESS(query_process, "Query process");
PROCESS_THREAD(query_process, ev, data)
{
   PROCESS_BEGIN();
   ..
   ...
   ....
   PROCESS_END();
}

The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.

db_init();
db_set_output_function(buffer_db_data);
db_query(NULL, "REMOVE RELATION samples;");
db_query(NULL, "CREATE RELATION samples;");
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");

The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.

"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).

#if PREPARE_DB
printf("Preparing the DB with %d tuples...\n", CARDINALITY);
errors = 0;
for(i = 1; i <= CARDINALITY; i++) {
PROCESS_PAUSE();
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",
i, (unsigned)random_rand());
if(DB_ERROR(result)) {
errors++;
}
}
printf("Done. Insertion errors: %d\n", errors);
printf("Ready to process queries\n");
#else
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);
#endif

The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like shown in the cooja simulation section below.

for(;;) {
    PROCESS_WAIT_EVENT();

    if(ev == serial_line_event_message && data != NULL) {
      printf("START %s\n", (char *)data);
      result = db_query(&handle, data);
      if(DB_ERROR(result)) {
        buffer_db_data("Query error: %s\n", db_get_result_message(result));
        stop_handler(NULL);
        db_free(&handle);
        continue;
      }

      if(!db_processing(&handle)) {
        buffer_db_data("OK\n");
        send_buffered_data();
        stop_handler(NULL);
        continue;
      }

      packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,
                         PACKETBUF_ATTR_PACKET_TYPE_STREAM);

      db_print_header(&handle);

      matching = 0;
      processed = 0;

      while(db_processing(&handle)) {
        PROCESS_PAUSE();

        if(matching == RESPONSE_LIMIT) {
            buffer_db_data("Response suppressed at %u tuples: limit reached\n",
                           RESPONSE_LIMIT);
            stop_handler(NULL);
            db_free(&handle);
            break;
        }

        result = db_process(&handle);
        if(result == DB_GOT_ROW) {
          /* The processed tuple matched the condition in the query. */
          matching++;
          processed++;
          db_print_tuple(&handle);
        } else if(result == DB_OK) {
          /* A tuple was processed, but did not match the condition. */
          processed++;
          continue;
        } else {
          if(result == DB_FINISHED) {
            /* The processing has finished. Wait for a new command. */
            buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",
                           (long)matching, (long)processed);
            buffer_db_data("OK\n");
          } else if(DB_ERROR(result)) {
            buffer_db_data("Processing error: %s\n",
                           db_get_result_message(result));
          }
          stop_handler(NULL);
          db_free(&handle);
        }
      }
      send_buffered_data();
    }
#if !PREPARE_DB
    if(etimer_expired(&sampling_timer)) {
      take_sample();
      etimer_reset(&sampling_timer);
    }
#endif
}

The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.

Cooja Simulation-Antelope

In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes. Navigate to the examples/antelope/netdb folder from the contiki home directory.

cd /examples/antelope/netdb

In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.

Navigate to /tools/cooja/ from the contiki home directory.

cd /tools/cooja/

Type the following command.

sudo ant run

This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja). From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open. Your screen should look something like below.

  • Network Window.

Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.

Antelope network.png

  • Mote output.

Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).

Antelope ID4.png

  • Mote Interface Viewer(Sky 51).

Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies. Antelope interface.png

Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.

Ready.png
NetdbClient.png

Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.

  • Creating a relation

We can create a relation (like a table which holds the tuples) with the help of following command.

create relation myrelation1;


Now, create some attributes in this relation.

create attribute id domain int in myrelation1;

The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1. Similarly we can create other attributes such as "name" of type string etc.

Below is the log of some of the queries that are sent to the server.

Rime started with address 51.0
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.
CSMA nullrdc, channel check rate 128 Hz, radio channel 16
Starting 'NetDB'
NetDB client
> create relation sample;
11525 Transmitting query "create relation sample;" to node 4
11674 Reply received from 4.0 (2 hops): OK
> create attribute id domain int in sample;
18093 Transmitting query "create attribute id domain int in sample;" to node 4
18099 Reply received from 4.0 (2 hops): OK
> insert (4) into sample;
25141 Transmitting query "insert (4) into sample;" to node 4
25146 Reply received from 4.0 (2 hops): OK
> insert (23) into sample;
29965 Transmitting query "insert (23) into sample;" to node 4
29970 Reply received from 4.0 (2 hops): OK
> insert (31) into sample;
33323 Transmitting query "insert (31) into sample;" to node 4
33328 Reply received from 4.0 (2 hops): OK
> select max(id) from sample;
41179 Transmitting query "select max(id) from sample;" to node 4
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]
Row 1:31
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]
OK

Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.

00:00.607    ID:4    Rime started with address 4.0
00:00.616    ID:4    MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.
00:00.623    ID:4    CSMA nullrdc, channel check rate 128 Hz, radio channel 16
00:00.624    ID:4    Starting 'NetDB'
00:00.626    ID:4    NetDB host
00:00.701    ID:4    Preparing the DB with 1000 tuples...
00:08.273    ID:4    Done. Insertion errors: 0
00:08.274    ID:4    Ready to process queries
01:31.565    ID:4    Query received from 51.0: create relation sample; (2 hops)
01:31.567    ID:4    START create relation sample;
01:31.619    ID:4    END
02:21.804    ID:4    Query received from 51.0: create attribute id domain int in sample; (2 hops)
02:21.808    ID:4    START create attribute id domain int in sample;
02:21.817    ID:4    END
03:16.861    ID:4    Query received from 51.0: insert (4) into sample; (2 hops)
03:16.863    ID:4    START insert (4) into sample;
03:16.870    ID:4    END
03:54.549    ID:4    Query received from 51.0: insert (23) into sample; (2 hops)
03:54.551    ID:4    START insert (23) into sample;
03:54.558    ID:4    END
04:20.783    ID:4    Query received from 51.0: insert (31) into sample; (2 hops)
04:20.786    ID:4    START insert (31) into sample;
04:20.792    ID:4    END
05:22.159    ID:4    Query received from 51.0: select max(id) from sample; (2 hops)
05:22.162    ID:4    START select max(id) from sample;
05:22.204    ID:4    END

Below are the snapshots for Antelope client and server logs in cooja.

Client Query.png Server Query.png

Similarly, we could try some other examples like below:

CREATE RELATION sensor;

CREATE ATTRIBUTE id DOMAIN INT IN sensor;
CREATE ATTRIBUTE name DOMAIN STRING(20) IN sensor;
CREATE ATTRIBUTE position DOMAIN LONG IN sensor;

CREATE INDEX sensor.id TYPE INLINE;
CREATE INDEX sensor.position TYPE MAXHEAP;


References

1 - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.

2 - Contiki Operating System Wiki page.


Back to Contiki Tutorials

Edited by Gopi Krishna.