Difference between revisions of "Antelope(Database Management System) - Contiki"

From Contiki
Jump to: navigation, search
Line 304: Line 304:
 
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.
 
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.
 
Similarly we can create other attributes such as "name" of type string etc.
 
Similarly we can create other attributes such as "name" of type string etc.
+
 
 +
Below is the log of some of the queries that are sent to the server.
 +
:: <code>
 +
Rime started with address 51.0
 +
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.
 +
CSMA nullrdc, channel check rate 128 Hz, radio channel 16
 +
Starting 'NetDB'
 +
NetDB client
 +
> create relation sample;
 +
11525 Transmitting query "create relation sample;" to node 4
 +
11674 Reply received from 4.0 (2 hops): OK
 +
> create attribute id domain int in sample;
 +
18093 Transmitting query "create attribute id domain int in sample;" to node 4
 +
18099 Reply received from 4.0 (2 hops): OK
 +
> insert (4) into sample;
 +
25141 Transmitting query "insert (4) into sample;" to node 4
 +
25146 Reply received from 4.0 (2 hops): OK
 +
> insert (23) into sample;
 +
29965 Transmitting query "insert (23) into sample;" to node 4
 +
29970 Reply received from 4.0 (2 hops): OK
 +
> insert (31) into sample;
 +
33323 Transmitting query "insert (31) into sample;" to node 4
 +
33328 Reply received from 4.0 (2 hops): OK
 +
> select max(id) from sample;
 +
41179 Transmitting query "select max(id) from sample;" to node 4
 +
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]
 +
Row 1:31
 +
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]
 +
OK
 +
</code>
 
== References ==
 
== References ==
 
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.
 
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.

Revision as of 14:14, 8 November 2014

Back to Contiki Tutorials

Introduction

Antelope is a database management system for resource-constrained memory devices. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.

You will learn

In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience.

Antelope architecture

Components

Antelope consists of eight components as shown in the below figure.

Antelope arch.jpg


  • Query processor  : It parses AQL(query language of Antelope) queries.
  • Privacy Control  : Checks whether the query is allowed or not.
  • Logic VM  : It executes the queries.
  • Database kernel  : It holds the database logic and coordinates query execution.
  • Index abstraction : It holds the indexing logic.
  • Indexer process  : Builds indexes from existing data.
  • Storage abstraction : Contains all storage logic.
  • Result transformer : It presents the result of query in a way that makes it easy to use by programs.



Terminology

Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:

  • Tuple: It is a set of attribute values.
  • Attribute: Different properties. E.g SensorID, Time.
  • Domain: Each attribute has got a domain that specifies the data type of the attribute values.
  • Relation: It is a collection of tuples that have the same set of attributes.
  • Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.
  • Cardinality: The number of tuples in a relation is called the cardinality of the relation.

To put the above terminology in easy terms, informally we can say :

  • A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.
jpg

Source Code

Navigate to the examples/antelope/netdb folder from the contiki home directory.

cd /examples/antelope/netdb

There are two important files in this(netdb) subfolder.

  • netdb-server.c
  • netdb-client.c

There is one more file netdb.csc which we are going to use for our simulation on cooja.

Let's walk through the source files before we start the simulation.

Code building blocks

netdb-client.c

This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.

Let's walk through the source file.

#ifndef SERVER_ID
#define SERVER_ID 4
#endif

SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).

static unsigned server_id = SERVER_ID;
static struct mesh_conn mesh;

Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.

struct mesh_callbacks {
  /** Called when a packet is received. */
  void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);
  /** Called when a packet, sent with mesh_send(), is actually transmitted. */
  void (* sent)(struct mesh_conn *c);
  /** Called when a packet, sent with mesh_send(), times out and is dropped. */
  void (* timedout)(struct mesh_conn *c);
};
PROCESS(netdb_process, "NetDB");
PROCESS(shell_process, "Shell Process");
AUTOSTART_PROCESSES(&netdb_process);

The first two lines in the above source code creates two processes netdb_process(which is the database process), shell_process. "shell_process" is the process which provides serial interface at the client to input our queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread assosicated with this process will start running by the scheduler. Below is the code snipped of the thread for this process.

PROCESS_THREAD(netdb_process, ev, data)
{
  PROCESS_EXITHANDLER(mesh_close(&mesh));
  PROCESS_BEGIN();

  mesh_open(&mesh, NETDB_CHANNEL, &callbacks);
  process_start(&shell_process, NULL);

  PROCESS_END();
}
static const struct mesh_callbacks callbacks = {received, sent, timedout};

This will create a callabacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.

A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow this link.

mesh_open opens a new connection and here this mesh connection operates on NETDB_CHANNEL(70) and the third argument callbacks contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another instead of using AUTOSTART_PROCESSES).

Callback handlers for the netdb_client process mesh connection
static const struct mesh_callbacks callbacks = {received, sent, timedout};
  • Callback function - sent
static void
sent(struct mesh_conn *c)
{
}

This function gets called when we try send a packet with mesh_send() and it is actually transmitted.

  • Callback function - timedout
static void
timedout(struct mesh_conn *c)
{
  printf("Failed to send packet: time out\n");
}

This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".

  • Callback function - received
static void
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)
{
  char *data;
  unsigned len;
  static char reply[MAX_QUERY_SIZE + 1];

  data = (char *)packetbuf_dataptr();
  len = packetbuf_datalen();

  if(len > MAX_QUERY_SIZE) {
    printf("Too long query: %d bytes\n", len);
    return;
  }

  memcpy(reply, data, len);
  reply[len] = '\0';

  printf("%lu Reply received from %d.%d (%d hops): %s",
         clock_time(), from->u8[0], from->u8[1], (int)hops, reply);
}

This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server. Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.

Let's look at the source code for the "shell_process".

PROCESS_THREAD(shell_process, ev, data)
{
  rimeaddr_t addr;

  PROCESS_BEGIN();

  printf("NetDB client\n");

  for(;;) {
    PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);
    if(strncmp(data, "server ", 7) == 0) {
      server_id = atoi((char *)data + 7);
    } else {
      printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);
      packetbuf_copyfrom(data, strlen(data));
      addr.u8[0] = server_id;
      addr.u8[1] = 0;
      packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,
                         PACKETBUF_ATTR_PACKET_TYPE_STREAM);
      mesh_send(&mesh, &addr);
    }
  }

  PROCESS_END();
}

A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.


netdb-server.c

Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.

PROCESS(netdb_process, "NetDB");
AUTOSTART_PROCESSES(&netdb_process);

PROCESS_THREAD(netdb_process, ev, data)
{
  PROCESS_EXITHANDLER(mesh_close(&mesh));
  PROCESS_BEGIN();

  mesh_open(&mesh, NETDB_CHANNEL, &callbacks);
  process_start(&query_process, NULL);

  PROCESS_END();
}

Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.

static void
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)
{
  char *data;
  unsigned len;
  static char query[MAX_BUFFER_SIZE + 1];

  data = (char *)packetbuf_dataptr();
  len = packetbuf_datalen();

  if(len > MAX_BUFFER_SIZE) {
    buffer_db_data("Too long query: %d bytes\n", len);
    return;
  }

  memcpy(query, data, len);
  query[len] = '\0';

  printf("Query received from %d.%d: %s (%d hops)\n",
         from->u8[0], from->u8[1], query, (int)hops);
  rimeaddr_copy(&reply_addr, from);

  process_post(&query_process, serial_line_event_message, query);
}

This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".

Let's look at the query_process which plays an important role in processing the incoming queries.

PROCESS(query_process, "Query process");
PROCESS_THREAD(query_process, ev, data)
{
   ..
   ...
   ....
}


Cooja Simulation-Antelope

In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes. Navigate to the examples/antelope/netdb folder from the contiki home directory.

cd /examples/antelope/netdb

In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.

Navigate to /tools/cooja/ from the contiki home directory.

cd /tools/cooja/

Type the following command.

sudo ant run

This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja). From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open. Your screen should look something like below.

  • Network Window.

Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.

Antelope network.png

  • Mote output.

Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).

Antelope ID4.png

  • Mote Interface Viewer(Sky 51).

Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies. Antelope interface.png

Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.


Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.

  • Creating a relation

We can create a relation (like a table which holds the tuples) with the help of following command.

create relation myrelation1;


Now, create some attributes in this relation.

create attribute id domain int in myrelation1;

The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1. Similarly we can create other attributes such as "name" of type string etc.

Below is the log of some of the queries that are sent to the server.

Rime started with address 51.0 MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51. CSMA nullrdc, channel check rate 128 Hz, radio channel 16 Starting 'NetDB' NetDB client > create relation sample; 11525 Transmitting query "create relation sample;" to node 4 11674 Reply received from 4.0 (2 hops): OK > create attribute id domain int in sample; 18093 Transmitting query "create attribute id domain int in sample;" to node 4 18099 Reply received from 4.0 (2 hops): OK > insert (4) into sample; 25141 Transmitting query "insert (4) into sample;" to node 4 25146 Reply received from 4.0 (2 hops): OK > insert (23) into sample; 29965 Transmitting query "insert (23) into sample;" to node 4 29970 Reply received from 4.0 (2 hops): OK > insert (31) into sample; 33323 Transmitting query "insert (31) into sample;" to node 4 33328 Reply received from 4.0 (2 hops): OK > select max(id) from sample; 41179 Transmitting query "select max(id) from sample;" to node 4 41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)] Row 1:31 41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed] OK

References

1 - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.

Edited by Gopi.

Back to Contiki Tutorials