http://anrg.usc.edu/contiki/api.php?action=feedcontributions&user=Marella&feedformat=atomContiki - User contributions [en]2024-03-29T09:10:21ZUser contributionsMediaWiki 1.26.2http://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1440Antelope(Database Management System) - Contiki2014-11-09T07:01:08Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system which enables a sensor to act as a database server. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
Antelope is the first DBMS for resource-constrained sensor nodes, provides energy-efficient querying, high-level data modelling, while being index independent. MaxHeap indexing enables fast insertions and queries over large data sets while having a low energy cost. Antelope hides the low-level I/O details of flash storage, which have thus far often been exposed to application developers. Moreover, Antelope uses LogicVM, a novel virtual machine architecture that analyses and executes propositional logic expressed in a byte-code format. By compiling queries and executing them in the virtual machine, the performance of relational selection queries increases by an order of magnitude compared to repeated parsing.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience on some of the basic query operations of the antelope.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*''Query processor'' : It parses AQL(query language for Antelope) queries.<br />
*''Privacy Control'' : Checks whether the query is allowed or not.<br />
*''Logic VM'' : It executes the queries.<br />
*''Database kernel'' : It holds the database logic and coordinates query execution.<br />
*''Index abstraction'' : It holds the indexing logic.<br />
*''Indexer process'' : Builds indexes from existing data.<br />
*''Storage abstraction'' : Contains all storage logic.<br />
*''Result transformer'' : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code Directories ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
The implementation of Antelope database in present in the directory:<br />
:: <code> ~/contiki-2.7/apps/antelope</code><br />
<br />
== Code building blocks ==<br />
In this section, source code for the example given in the <code> ~/contiki-2.7/examples/antelope/netdb/</code> is explored.<br />
<br />
=== netdb-client.c ===<br />
This client program provides a serial line interface with the help of which we can enter the queries. The shell process of the client program process these serial messages(queries) and sends the queries to the server program(running on another mote). Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important data structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
----<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes "netdb_process"(which is the database process) and "shell_process". "shell_process" is the process which provides serial interface at the client to input the queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread associated with this process will start running by the scheduler. Below is the code snippet of the thread for this process.<br />
----<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callbacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow [https://github.com/contiki-os/contiki/wiki/Processes this] link.<br />
<br />
"mesh_open" opens a new connection and here the created mesh connection operates on NETDB_CHANNEL(70) and the third argument "callbacks" contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another process, instead of using AUTOSTART_PROCESSES).<br />
<br />
----<br />
<br />
''' Callback handlers for the netdb_client process mesh connection: '''<br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
----<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
----<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
----<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite 'for loop' "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and '''mesh_send''' function is called to send the packet to the server using mesh routing protocol.<br />
----<br />
<br />
=== netdb-server.c ===<br />
Most of the source code for the ''netdb_server'' file is same as that of the above explained ''netdb_client'' source file like the '''mesh_conn''' data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of '''netdb_process''' we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
----<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function gets called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is passed as "serial_line_event_message".<br />
----<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like shown in the cooja simulation section below.<br />
<br />
<source lang="c"><br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
<br />
db_print_header(&handle);<br />
<br />
matching = 0;<br />
processed = 0;<br />
<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
[[File:Ready.png|center|400px]] <br />
<br />
[[File:NetdbClient.png|center|400px]]<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
<br />
Below are the snapshots for Antelope client and server logs in cooja.<br />
<br />
[[File:Client_Query.png|500px]]<br />
[[File:Server_Query.png|550px]]<br />
<br />
Similarly, we could try some other examples like below:<br />
<source lang="c"><br />
CREATE RELATION sensor;<br />
<br />
CREATE ATTRIBUTE id DOMAIN INT IN sensor;<br />
CREATE ATTRIBUTE name DOMAIN STRING(20) IN sensor;<br />
CREATE ATTRIBUTE position DOMAIN LONG IN sensor;<br />
<br />
CREATE INDEX sensor.id TYPE INLINE;<br />
CREATE INDEX sensor.position TYPE MAXHEAP;<br />
</source><br />
<br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
[https://github.com/contiki-os/contiki/wiki 2] - Contiki Operating System Wiki page.<br />
<br />
----<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
Edited by Gopi Krishna.</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1436Antelope(Database Management System) - Contiki2014-11-09T06:56:41Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system which enables a sensor to act as a database server. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
Antelope is the first DBMS for resource-constrained sensor nodes, provides energy-efficient querying, high-level data modelling, while being index independent. MaxHeap indexing enables fast insertions and queries over large data sets while having a low energy cost. Antelope hides the low-level I/O details of flash storage, which have thus far often been exposed to application developers. Moreover, Antelope uses LogicVM, a novel virtual machine architecture that analyses and executes propositional logic expressed in a byte-code format. By compiling queries and executing them in the virtual machine, the performance of relational selection queries increases by an order of magnitude compared to repeated parsing.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience on some of the basic query operations of the antelope.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*''Query processor'' : It parses AQL(query language for Antelope) queries.<br />
*''Privacy Control'' : Checks whether the query is allowed or not.<br />
*''Logic VM'' : It executes the queries.<br />
*''Database kernel'' : It holds the database logic and coordinates query execution.<br />
*''Index abstraction'' : It holds the indexing logic.<br />
*''Indexer process'' : Builds indexes from existing data.<br />
*''Storage abstraction'' : Contains all storage logic.<br />
*''Result transformer'' : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code Directories ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
The implementation of Antelope database in present in the directory:<br />
:: <code> ~/contiki-2.7/apps/antelope</code><br />
<br />
== Code building blocks ==<br />
In this section, source code for the example given in the <code> ~/contiki-2.7/examples/antelope/netdb/</code> is explored.<br />
<br />
=== netdb-client.c ===<br />
This client program provides a serial line interface with the help of which we can enter the queries. The shell process of the client program process these serial messages(queries) and sends the queries to the server program(running on another mote). Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important data structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
----<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes "netdb_process"(which is the database process) and "shell_process". "shell_process" is the process which provides serial interface at the client to input the queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread associated with this process will start running by the scheduler. Below is the code snippet of the thread for this process.<br />
----<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callbacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow [https://github.com/contiki-os/contiki/wiki/Processes this] link.<br />
<br />
"mesh_open" opens a new connection and here the created mesh connection operates on NETDB_CHANNEL(70) and the third argument "callbacks" contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another process, instead of using AUTOSTART_PROCESSES).<br />
<br />
----<br />
<br />
''' Callback handlers for the netdb_client process mesh connection: '''<br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
----<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
----<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
----<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite 'for loop' "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and '''mesh_send''' function is called to send the packet to the server using mesh routing protocol.<br />
----<br />
<br />
=== netdb-server.c ===<br />
Most of the source code for the ''netdb_server'' file is same as that of the above explained ''netdb_client'' source file like the '''mesh_conn''' data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of '''netdb_process''' we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
----<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function gets called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is passed as "serial_line_event_message".<br />
----<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like shown in the cooja simulation section below.<br />
<br />
<source lang="c"><br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
<br />
db_print_header(&handle);<br />
<br />
matching = 0;<br />
processed = 0;<br />
<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
[[File:Ready.png|center|400px]] <br />
<br />
[[File:NetdbClient.png|center|400px]]<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
<br />
Below are the snapshots for Antelope client and server logs in cooja.<br />
<br />
[[File:Client_Query.png|500px]]<br />
[[File:Server_Query.png|550px]]<br />
<br />
Similarly, we could try some other examples like below:<br />
<source lang="c"><br />
CREATE RELATION sensor;<br />
<br />
CREATE ATTRIBUTE id DOMAIN INT IN sensor;<br />
CREATE ATTRIBUTE name DOMAIN STRING(20) IN sensor;<br />
CREATE ATTRIBUTE position DOMAIN LONG IN sensor;<br />
<br />
CREATE INDEX sensor.id TYPE INLINE;<br />
CREATE INDEX sensor.position TYPE MAXHEAP;<br />
</source><br />
<br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
[https://github.com/contiki-os/contiki/wiki 2] - Contiki Operating System Wiki page.<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
Edited by Gopi.</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1431Antelope(Database Management System) - Contiki2014-11-09T06:53:31Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system which enables a sensor to act as a database server. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
Antelope is the first DBMS for resource-constrained sensor nodes, provides energy-efficient querying, high-level data modelling, while being index independent. MaxHeap indexing enables fast insertions and queries over large data sets while having a low energy cost. Antelope hides the low-level I/O details of flash storage, which have thus far often been exposed to application developers. Moreover, Antelope uses LogicVM, a novel virtual machine architecture that analyses and executes propositional logic expressed in a byte-code format. By compiling queries and executing them in the virtual machine, the performance of relational selection queries increases by an order of magnitude compared to repeated parsing.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience on some of the basic query operations of the antelope.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*''Query processor'' : It parses AQL(query language for Antelope) queries.<br />
*''Privacy Control'' : Checks whether the query is allowed or not.<br />
*''Logic VM'' : It executes the queries.<br />
*''Database kernel'' : It holds the database logic and coordinates query execution.<br />
*''Index abstraction'' : It holds the indexing logic.<br />
*''Indexer process'' : Builds indexes from existing data.<br />
*''Storage abstraction'' : Contains all storage logic.<br />
*''Result transformer'' : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code Directories ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
The implementation of Antelope database in present in the directory:<br />
:: <code> ~/contiki-2.7/apps/antelope</code><br />
<br />
== Code building blocks ==<br />
In this section, source code for the example given in the <code> ~/contiki-2.7/examples/antelope/netdb/</code> is explored.<br />
<br />
=== netdb-client.c ===<br />
This client program provides a serial line interface with the help of which we can enter the queries. The shell process of the client program process these serial messages(queries) and sends the queries to the server program(running on another mote). Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important data structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
----<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes "netdb_process"(which is the database process) and "shell_process". "shell_process" is the process which provides serial interface at the client to input the queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread associated with this process will start running by the scheduler. Below is the code snippet of the thread for this process.<br />
----<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callbacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow [https://github.com/contiki-os/contiki/wiki/Processes this] link.<br />
<br />
"mesh_open" opens a new connection and here the created mesh connection operates on NETDB_CHANNEL(70) and the third argument "callbacks" contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another process, instead of using AUTOSTART_PROCESSES).<br />
<br />
----<br />
<br />
''' Callback handlers for the netdb_client process mesh connection: '''<br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
----<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
----<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
----<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite 'for loop' "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and '''mesh_send''' function is called to send the packet to the server using mesh routing protocol.<br />
----<br />
<br />
=== netdb-server.c ===<br />
Most of the source code for the ''netdb_server'' file is same as that of the above explained ''netdb_client'' source file like the '''mesh_conn''' data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of '''netdb_process''' we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
----<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function gets called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is passed as "serial_line_event_message".<br />
----<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready.png|center|400px]] <br />
<br />
[[File:NetdbClient.png|center|400px]]<br />
<br />
<source lang="c"><br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
<br />
db_print_header(&handle);<br />
<br />
matching = 0;<br />
processed = 0;<br />
<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
<br />
Below are the snapshots for Antelope client and server logs in cooja.<br />
<br />
[[File:Client_Query.png|500px]]<br />
[[File:Server_Query.png|550px]]<br />
<br />
Similarly, we could try some other examples like below:<br />
<source lang="c"><br />
CREATE RELATION sensor;<br />
<br />
CREATE ATTRIBUTE id DOMAIN INT IN sensor;<br />
CREATE ATTRIBUTE name DOMAIN STRING(20) IN sensor;<br />
CREATE ATTRIBUTE position DOMAIN LONG IN sensor;<br />
<br />
CREATE INDEX sensor.id TYPE INLINE;<br />
CREATE INDEX sensor.position TYPE MAXHEAP;<br />
</source><br />
<br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
[https://github.com/contiki-os/contiki/wiki 2] - Contiki Operating System Wiki page.<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
Edited by Gopi.</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1424Antelope(Database Management System) - Contiki2014-11-09T06:46:57Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system which enables a sensor to act as a database server. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
Antelope is the first DBMS for resource-constrained sensor nodes, provides energy-efficient querying, high-level data modelling, while being index independent. MaxHeap indexing enables fast insertions and queries over large data sets while having a low energy cost. Antelope hides the low-level I/O details of flash storage, which have thus far often been exposed to application developers. Moreover, Antelope uses LogicVM, a novel virtual machine architecture that analyses and executes propositional logic expressed in a byte-code format. By compiling queries and executing them in the virtual machine, the performance of relational selection queries increases by an order of magnitude compared to repeated parsing.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience on some of the basic query operations of the antelope.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*''Query processor'' : It parses AQL(query language for Antelope) queries.<br />
*''Privacy Control'' : Checks whether the query is allowed or not.<br />
*''Logic VM'' : It executes the queries.<br />
*''Database kernel'' : It holds the database logic and coordinates query execution.<br />
*''Index abstraction'' : It holds the indexing logic.<br />
*''Indexer process'' : Builds indexes from existing data.<br />
*''Storage abstraction'' : Contains all storage logic.<br />
*''Result transformer'' : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code Directories ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
The implementation of Antelope database in present in the directory:<br />
:: <code> ~/contiki-2.7/apps/antelope</code><br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
== Code building blocks ==<br />
In this section, source code for the example given in the :: <code> ~/contiki-2.7/examples/antelope/netdb/</code> is explored.<br />
<br />
=== netdb-client.c ===<br />
This client program provides a serial line interface with the help of which we can enter the queries. The shell process of the client program process these serial messages(queries) and sends the queries to the server program(running on another mote). Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important data structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes "netdb_process"(which is the database process) and "shell_process". "shell_process" is the process which provides serial interface at the client to input the queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread associated with this process will start running by the scheduler. Below is the code snippet of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callbacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow [https://github.com/contiki-os/contiki/wiki/Processes this] link.<br />
<br />
"mesh_open" opens a new connection and here the created mesh connection operates on NETDB_CHANNEL(70) and the third argument "callbacks" contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another process, instead of using AUTOSTART_PROCESSES).<br />
<br />
<br />
<br />
''' Callback handlers for the netdb_client process mesh connection: '''<br />
----<br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
----<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
----<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
----<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite 'for loop' "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and '''mesh_send''' function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
=== netdb-server.c ===<br />
Most of the source code for the ''netdb_server'' file is same as that of the above explained ''netdb_client'' source file like the '''mesh_conn''' data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of '''netdb_process''' we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function gets called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is passed as "serial_line_event_message".<br />
<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready.png|center|400px]] <br />
<br />
[[File:NetdbClient.png|center|400px]]<br />
<br />
<source lang="c"><br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
<br />
db_print_header(&handle);<br />
<br />
matching = 0;<br />
processed = 0;<br />
<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
<br />
Below are the snapshots for Antelope client and server logs in cooja.<br />
<br />
[[File:Client_Query.png|500px]]<br />
[[File:Server_Query.png|550px]]<br />
<br />
Similarly, we could try some other examples like below:<br />
<source lang="c"><br />
CREATE RELATION sensor;<br />
<br />
CREATE ATTRIBUTE id DOMAIN INT IN sensor;<br />
CREATE ATTRIBUTE name DOMAIN STRING(20) IN sensor;<br />
CREATE ATTRIBUTE position DOMAIN LONG IN sensor;<br />
<br />
CREATE INDEX sensor.id TYPE INLINE;<br />
CREATE INDEX sensor.position TYPE MAXHEAP;<br />
</source><br />
<br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
Edited by Gopi.</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1414Antelope(Database Management System) - Contiki2014-11-09T06:43:08Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system which enables a sensor to act as a database server. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
Antelope is the first DBMS for resource-constrained sensor nodes, provides energy-efficient querying, high-level data modelling, while being index independent. MaxHeap indexing enables fast insertions and queries over large data sets while having a low energy cost. Antelope hides the low-level I/O details of flash storage, which have thus far often been exposed to application developers. Moreover, Antelope uses LogicVM, a novel virtual machine architecture that analyses and executes propositional logic expressed in a byte-code format. By compiling queries and executing them in the virtual machine, the performance of relational selection queries increases by an order of magnitude compared to repeated parsing.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience on some of the basic query operations of the antelope.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*''Query processor'' : It parses AQL(query language for Antelope) queries.<br />
*''Privacy Control'' : Checks whether the query is allowed or not.<br />
*''Logic VM'' : It executes the queries.<br />
*''Database kernel'' : It holds the database logic and coordinates query execution.<br />
*''Index abstraction'' : It holds the indexing logic.<br />
*''Indexer process'' : Builds indexes from existing data.<br />
*''Storage abstraction'' : Contains all storage logic.<br />
*''Result transformer'' : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter the queries. The shell process of the client program process these serial messages(queries) and sends the queries to the server program(running on another mote). Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important data structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes "netdb_process"(which is the database process) and "shell_process". "shell_process" is the process which provides serial interface at the client to input the queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread associated with this process will start running by the scheduler. Below is the code snippet of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callbacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow [https://github.com/contiki-os/contiki/wiki/Processes this] link.<br />
<br />
"mesh_open" opens a new connection and here the created mesh connection operates on NETDB_CHANNEL(70) and the third argument "callbacks" contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another process, instead of using AUTOSTART_PROCESSES).<br />
<br />
<br />
<br />
''' Callback handlers for the netdb_client process mesh connection: '''<br />
----<br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
----<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
----<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
----<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite 'for loop' "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and '''mesh_send''' function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the ''netdb_server'' file is same as that of the above explained ''netdb_client'' source file like the '''mesh_conn''' data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of '''netdb_process''' we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function gets called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is passed as "serial_line_event_message".<br />
<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready.png|center|400px]] <br />
<br />
[[File:NetdbClient.png|center|400px]]<br />
<br />
<source lang="c"><br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
<br />
db_print_header(&handle);<br />
<br />
matching = 0;<br />
processed = 0;<br />
<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
<br />
Below are the snapshots for Antelope client and server logs in cooja.<br />
<br />
[[File:Client_Query.png|500px]]<br />
[[File:Server_Query.png|550px]]<br />
<br />
Similarly, we could try some other examples like below:<br />
<source lang="c"><br />
CREATE RELATION sensor;<br />
<br />
CREATE ATTRIBUTE id DOMAIN INT IN sensor;<br />
CREATE ATTRIBUTE name DOMAIN STRING(20) IN sensor;<br />
CREATE ATTRIBUTE position DOMAIN LONG IN sensor;<br />
<br />
CREATE INDEX sensor.id TYPE INLINE;<br />
CREATE INDEX sensor.position TYPE MAXHEAP;<br />
</source><br />
<br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
Edited by Gopi.</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1404Antelope(Database Management System) - Contiki2014-11-09T06:38:16Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system which enables a sensor to act as a database server. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
Antelope is the first DBMS for resource-constrained sensor nodes, provides energy-efficient querying, high-level data modelling, while being index independent. MaxHeap indexing enables fast insertions and queries over large data sets while having a low energy cost. Antelope hides the low-level I/O details of flash storage, which have thus far often been exposed to application developers. Moreover, Antelope uses LogicVM, a novel virtual machine architecture that analyses and executes propositional logic expressed in a byte-code format. By compiling queries and executing them in the virtual machine, the performance of relational selection queries increases by an order of magnitude compared to repeated parsing.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience on some of the basic query operations of the antelope.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*''Query processor'' : It parses AQL(query language for Antelope) queries.<br />
*''Privacy Control'' : Checks whether the query is allowed or not.<br />
*''Logic VM'' : It executes the queries.<br />
*''Database kernel'' : It holds the database logic and coordinates query execution.<br />
*''Index abstraction'' : It holds the indexing logic.<br />
*''Indexer process'' : Builds indexes from existing data.<br />
*''Storage abstraction'' : Contains all storage logic.<br />
*''Result transformer'' : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter the queries. The shell process of the client program process these serial messages(queries) and sends the queries to the server program(running on another mote). Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important data structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes "netdb_process"(which is the database process) and "shell_process". "shell_process" is the process which provides serial interface at the client to input the queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread associated with this process will start running by the scheduler. Below is the code snippet of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callbacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow [https://github.com/contiki-os/contiki/wiki/Processes this] link.<br />
<br />
"mesh_open" opens a new connection and here the created mesh connection operates on NETDB_CHANNEL(70) and the third argument "callbacks" contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another process, instead of using AUTOSTART_PROCESSES).<br />
<br />
<br />
<br />
''' Callback handlers for the netdb_client process mesh connection: '''<br />
----<br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
----<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
----<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
----<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite 'for loop' "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and '''mesh_send''' function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the ''netdb_server'' file is same as that of the above explained ''netdb_client'' source file like the '''mesh_conn''' data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of '''netdb_process''' we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function gets called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is passed as "serial_line_event_message".<br />
<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready.png|center|400px]] <br />
<br />
[[File:NetdbClient.png|center|400px]]<br />
<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
<br />
Below are the snapshots for Antelope client and server logs in cooja.<br />
<br />
[[File:Client_Query.png|500px]]<br />
[[File:Server_Query.png|550px]]<br />
<br />
Similarly, we could try some other examples like below:<br />
<source lang="c"><br />
CREATE RELATION sensor;<br />
<br />
CREATE ATTRIBUTE id DOMAIN INT IN sensor;<br />
CREATE ATTRIBUTE name DOMAIN STRING(20) IN sensor;<br />
CREATE ATTRIBUTE position DOMAIN LONG IN sensor;<br />
<br />
CREATE INDEX sensor.id TYPE INLINE;<br />
CREATE INDEX sensor.position TYPE MAXHEAP;<br />
</source><br />
<br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
Edited by Gopi.</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1291Antelope(Database Management System) - Contiki2014-11-09T03:18:07Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system which enables every sensor to act as a database server. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
Antelope is the first DBMS for resource-constrained sensor nodes, provides energy-efficient querying, high-level data modelling, while being index independent. MaxHeap indexing enables fast insertions and queries over large data sets while having a low energy cost. Antelope hides the low-level I/O details of flash storage, which have thus far often been exposed to application developers. Moreover, Antelope uses LogicVM, a novel virtual machine architecture that analyses and executes propositional logic expressed in a byte-code format. By compiling queries and executing them in the virtual machine, the performance of relational selection queries increases by an order of magnitude compared to repeated parsing.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience on some of the basic query operations of the antelope.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*''Query processor'' : It parses AQL(query language for Antelope) queries.<br />
*''Privacy Control'' : Checks whether the query is allowed or not.<br />
*''Logic VM'' : It executes the queries.<br />
*''Database kernel'' : It holds the database logic and coordinates query execution.<br />
*''Index abstraction'' : It holds the indexing logic.<br />
*''Indexer process'' : Builds indexes from existing data.<br />
*''Storage abstraction'' : Contains all storage logic.<br />
*''Result transformer'' : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes "netdb_process"(which is the database process) and "shell_process". "shell_process" is the process which provides serial interface at the client to input the queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread associated with this process will start running by the scheduler. Below is the code snippet of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callbacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow [https://github.com/contiki-os/contiki/wiki/Processes this] link.<br />
<br />
"mesh_open" opens a new connection and here the created mesh connection operates on NETDB_CHANNEL(70) and the third argument "callbacks" contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another process, instead of using AUTOSTART_PROCESSES).<br />
<br />
<br />
<br />
''' Callback handlers for the netdb_client process mesh connection:<br />
----<br />
'''<br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready.png|center|400px]] <br />
<br />
[[File:NetdbClient.png|center|400px]]<br />
<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
<br />
Below are the snapshots for Antelope client and server logs in cooja.<br />
<br />
[[File:Client_Query.png|500px]]<br />
[[File:Server_Query.png|550px]]<br />
<br />
Similarly, we could try some other examples like below:<br />
<source lang="c"><br />
CREATE RELATION sensor;<br />
<br />
CREATE ATTRIBUTE id DOMAIN INT IN sensor;<br />
CREATE ATTRIBUTE name DOMAIN STRING(20) IN sensor;<br />
CREATE ATTRIBUTE position DOMAIN LONG IN sensor;<br />
<br />
CREATE INDEX sensor.id TYPE INLINE;<br />
CREATE INDEX sensor.position TYPE MAXHEAP;<br />
</source><br />
<br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
Edited by Gopi.</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1289Antelope(Database Management System) - Contiki2014-11-09T03:16:57Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system which enables every sensor to act as a database server. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
Antelope is the first DBMS for resource-constrained sensor nodes, provides energy-efficient querying, high-level data modelling, while being index independent. MaxHeap indexing enables fast insertions and queries over large data sets while having a low energy cost. Antelope hides the low-level I/O details of flash storage, which have thus far often been exposed to application developers. Moreover, Antelope uses LogicVM, a novel virtual machine architecture that analyses and executes propositional logic expressed in a byte-code format. By compiling queries and executing them in the virtual machine, the performance of relational selection queries increases by an order of magnitude compared to repeated parsing.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience on some of the basic query operations of the antelope.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*''Query processor'' : It parses AQL(query language for Antelope) queries.<br />
*''Privacy Control'' : Checks whether the query is allowed or not.<br />
*''Logic VM'' : It executes the queries.<br />
*''Database kernel'' : It holds the database logic and coordinates query execution.<br />
*''Index abstraction'' : It holds the indexing logic.<br />
*''Indexer process'' : Builds indexes from existing data.<br />
*''Storage abstraction'' : Contains all storage logic.<br />
*''Result transformer'' : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes "netdb_process"(which is the database process) and "shell_process". "shell_process" is the process which provides serial interface at the client to input the queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread associated with this process will start running by the scheduler. Below is the code snippet of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callbacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow [https://github.com/contiki-os/contiki/wiki/Processes this] link.<br />
<br />
"mesh_open" opens a new connection and here the created mesh connection operates on NETDB_CHANNEL(70) and the third argument "callbacks" contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another process, instead of using AUTOSTART_PROCESSES).<br />
<br />
<br />
<br />
''' Callback handlers for the netdb_client process mesh connection:<br />
----<br />
'''<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready.png|center|400px]] <br />
<br />
[[File:NetdbClient.png|center|400px]]<br />
<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
<br />
Below are the snapshots for Antelope client and server logs in cooja.<br />
<br />
[[File:Client_Query.png|500px]]<br />
[[File:Server_Query.png|550px]]<br />
<br />
Similarly, we could try some other examples like below:<br />
<source lang="c"><br />
CREATE RELATION sensor;<br />
<br />
CREATE ATTRIBUTE id DOMAIN INT IN sensor;<br />
CREATE ATTRIBUTE name DOMAIN STRING(20) IN sensor;<br />
CREATE ATTRIBUTE position DOMAIN LONG IN sensor;<br />
<br />
CREATE INDEX sensor.id TYPE INLINE;<br />
CREATE INDEX sensor.position TYPE MAXHEAP;<br />
</source><br />
<br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
Edited by Gopi.</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1288Antelope(Database Management System) - Contiki2014-11-09T03:16:09Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system which enables every sensor to act as a database server. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
Antelope is the first DBMS for resource-constrained sensor nodes, provides energy-efficient querying, high-level data modelling, while being index independent. MaxHeap indexing enables fast insertions and queries over large data sets while having a low energy cost. Antelope hides the low-level I/O details of flash storage, which have thus far often been exposed to application developers. Moreover, Antelope uses LogicVM, a novel virtual machine architecture that analyses and executes propositional logic expressed in a byte-code format. By compiling queries and executing them in the virtual machine, the performance of relational selection queries increases by an order of magnitude compared to repeated parsing.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience on some of the basic query operations of the antelope.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*''Query processor'' : It parses AQL(query language for Antelope) queries.<br />
*''Privacy Control'' : Checks whether the query is allowed or not.<br />
*''Logic VM'' : It executes the queries.<br />
*''Database kernel'' : It holds the database logic and coordinates query execution.<br />
*''Index abstraction'' : It holds the indexing logic.<br />
*''Indexer process'' : Builds indexes from existing data.<br />
*''Storage abstraction'' : Contains all storage logic.<br />
*''Result transformer'' : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes "netdb_process"(which is the database process) and "shell_process". "shell_process" is the process which provides serial interface at the client to input the queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread associated with this process will start running by the scheduler. Below is the code snippet of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callbacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow [https://github.com/contiki-os/contiki/wiki/Processes this] link.<br />
<br />
"mesh_open" opens a new connection and here the created mesh connection operates on NETDB_CHANNEL(70) and the third argument "callbacks" contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another process, instead of using AUTOSTART_PROCESSES).<br />
<br />
''' Callback handlers for the netdb_client process mesh connection:<br />
----<br />
'''<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready.png|center|400px]] <br />
<br />
[[File:NetdbClient.png|center|400px]]<br />
<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
<br />
Below are the snapshots for Antelope client and server logs in cooja.<br />
<br />
[[File:Client_Query.png|500px]]<br />
[[File:Server_Query.png|550px]]<br />
<br />
Similarly, we could try some other examples like below:<br />
<source lang="c"><br />
CREATE RELATION sensor;<br />
<br />
CREATE ATTRIBUTE id DOMAIN INT IN sensor;<br />
CREATE ATTRIBUTE name DOMAIN STRING(20) IN sensor;<br />
CREATE ATTRIBUTE position DOMAIN LONG IN sensor;<br />
<br />
CREATE INDEX sensor.id TYPE INLINE;<br />
CREATE INDEX sensor.position TYPE MAXHEAP;<br />
</source><br />
<br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
Edited by Gopi.</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1287Antelope(Database Management System) - Contiki2014-11-09T03:14:44Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system which enables every sensor to act as a database server. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
Antelope is the first DBMS for resource-constrained sensor nodes, provides energy-efficient querying, high-level data modelling, while being index independent. MaxHeap indexing enables fast insertions and queries over large data sets while having a low energy cost. Antelope hides the low-level I/O details of flash storage, which have thus far often been exposed to application developers. Moreover, Antelope uses LogicVM, a novel virtual machine architecture that analyses and executes propositional logic expressed in a byte-code format. By compiling queries and executing them in the virtual machine, the performance of relational selection queries increases by an order of magnitude compared to repeated parsing.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience on some of the basic query operations of the antelope.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*''Query processor'' : It parses AQL(query language for Antelope) queries.<br />
*''Privacy Control'' : Checks whether the query is allowed or not.<br />
*''Logic VM'' : It executes the queries.<br />
*''Database kernel'' : It holds the database logic and coordinates query execution.<br />
*''Index abstraction'' : It holds the indexing logic.<br />
*''Indexer process'' : Builds indexes from existing data.<br />
*''Storage abstraction'' : Contains all storage logic.<br />
*''Result transformer'' : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes "netdb_process"(which is the database process) and "shell_process". "shell_process" is the process which provides serial interface at the client to input the queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread associated with this process will start running by the scheduler. Below is the code snippet of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callbacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow [https://github.com/contiki-os/contiki/wiki/Processes this] link.<br />
<br />
"mesh_open" opens a new connection and here the created mesh connection operates on NETDB_CHANNEL(70) and the third argument "callbacks" contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another process, instead of using AUTOSTART_PROCESSES).<br />
<br />
''' Callback handlers for the netdb_client process mesh connection '''<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready.png|center|400px]] <br />
<br />
[[File:NetdbClient.png|center|400px]]<br />
<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
<br />
Below are the snapshots for Antelope client and server logs in cooja.<br />
<br />
[[File:Client_Query.png|500px]]<br />
[[File:Server_Query.png|550px]]<br />
<br />
Similarly, we could try some other examples like below:<br />
<source lang="c"><br />
CREATE RELATION sensor;<br />
<br />
CREATE ATTRIBUTE id DOMAIN INT IN sensor;<br />
CREATE ATTRIBUTE name DOMAIN STRING(20) IN sensor;<br />
CREATE ATTRIBUTE position DOMAIN LONG IN sensor;<br />
<br />
CREATE INDEX sensor.id TYPE INLINE;<br />
CREATE INDEX sensor.position TYPE MAXHEAP;<br />
</source><br />
<br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
Edited by Gopi.</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1286Antelope(Database Management System) - Contiki2014-11-09T03:07:35Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system which enables every sensor to act as a database server. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
Antelope is the first DBMS for resource-constrained sensor nodes, provides energy-efficient querying, high-level data modelling, while being index independent. MaxHeap indexing enables fast insertions and queries over large data sets while having a low energy cost. Antelope hides the low-level I/O details of flash storage, which have thus far often been exposed to application developers. Moreover, Antelope uses LogicVM, a novel virtual machine architecture that analyses and executes propositional logic expressed in a byte-code format. By compiling queries and executing them in the virtual machine, the performance of relational selection queries increases by an order of magnitude compared to repeated parsing.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience on some of the basic query operations of the antelope.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*''Query processor'' : It parses AQL(query language for Antelope) queries.<br />
*''Privacy Control'' : Checks whether the query is allowed or not.<br />
*''Logic VM'' : It executes the queries.<br />
*''Database kernel'' : It holds the database logic and coordinates query execution.<br />
*''Index abstraction'' : It holds the indexing logic.<br />
*''Indexer process'' : Builds indexes from existing data.<br />
*''Storage abstraction'' : Contains all storage logic.<br />
*''Result transformer'' : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes "netdb_process"(which is the database process) and "shell_process". "shell_process" is the process which provides serial interface at the client to input the queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread associated with this process will start running by the scheduler. Below is the code snippet of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callbacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow [https://github.com/contiki-os/contiki/wiki/Processes this] link.<br />
<br />
"mesh_open" opens a new connection and here the created mesh connection operates on NETDB_CHANNEL(70) and the third argument "callbacks" contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another process, instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready.png|center|400px]] <br />
<br />
[[File:NetdbClient.png|center|400px]]<br />
<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
<br />
Below are the snapshots for Antelope client and server logs in cooja.<br />
<br />
[[File:Client_Query.png|500px]]<br />
[[File:Server_Query.png|550px]]<br />
<br />
Similarly, we could try some other examples like below:<br />
<source lang="c"><br />
CREATE RELATION sensor;<br />
<br />
CREATE ATTRIBUTE id DOMAIN INT IN sensor;<br />
CREATE ATTRIBUTE name DOMAIN STRING(20) IN sensor;<br />
CREATE ATTRIBUTE position DOMAIN LONG IN sensor;<br />
<br />
CREATE INDEX sensor.id TYPE INLINE;<br />
CREATE INDEX sensor.position TYPE MAXHEAP;<br />
</source><br />
<br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
Edited by Gopi.</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1285Antelope(Database Management System) - Contiki2014-11-09T03:06:54Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system which enables every sensor to act as a database server. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
Antelope is the first DBMS for resource-constrained sensor nodes, provides energy-efficient querying, high-level data modelling, while being index independent. MaxHeap indexing enables fast insertions and queries over large data sets while having a low energy cost. Antelope hides the low-level I/O details of flash storage, which have thus far often been exposed to application developers. Moreover, Antelope uses LogicVM, a novel virtual machine architecture that analyses and executes propositional logic expressed in a byte-code format. By compiling queries and executing them in the virtual machine, the performance of relational selection queries increases by an order of magnitude compared to repeated parsing.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience on some of the basic query operations of the antelope.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*''Query processor'' : It parses AQL(query language for Antelope) queries.<br />
*''Privacy Control'' : Checks whether the query is allowed or not.<br />
*''Logic VM'' : It executes the queries.<br />
*''Database kernel'' : It holds the database logic and coordinates query execution.<br />
*''Index abstraction'' : It holds the indexing logic.<br />
*''Indexer process'' : Builds indexes from existing data.<br />
*''Storage abstraction'' : Contains all storage logic.<br />
*''Result transformer'' : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes "netdb_process"(which is the database process) and "shell_process". "shell_process" is the process which provides serial interface at the client to input the queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread associated with this process will start running by the scheduler. Below is the code snippet of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callbacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow [https://github.com/contiki-os/contiki/wiki/Processes this] link.<br />
<br />
"mesh_open" opens a new connection and here the created mesh connection operates on NETDB_CHANNEL(70) and the third argument "callbacks" contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another process, instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready.png|center|400px]] <br />
<br />
[[File:NetdbClient.png|center|400px]]<br />
<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
<br />
Below are the snapshots for Antelope client and server logs in cooja.<br />
<br />
[[File:Client_Query.png|400px]]<br />
[[File:Server_Query.png|450px]]<br />
<br />
Similarly, we could try some other examples like below:<br />
<source lang="c"><br />
CREATE RELATION sensor;<br />
<br />
CREATE ATTRIBUTE id DOMAIN INT IN sensor;<br />
CREATE ATTRIBUTE name DOMAIN STRING(20) IN sensor;<br />
CREATE ATTRIBUTE position DOMAIN LONG IN sensor;<br />
<br />
CREATE INDEX sensor.id TYPE INLINE;<br />
CREATE INDEX sensor.position TYPE MAXHEAP;<br />
</source><br />
<br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
Edited by Gopi.</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1284Antelope(Database Management System) - Contiki2014-11-09T03:06:16Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system which enables every sensor to act as a database server. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
Antelope is the first DBMS for resource-constrained sensor nodes, provides energy-efficient querying, high-level data modelling, while being index independent. MaxHeap indexing enables fast insertions and queries over large data sets while having a low energy cost. Antelope hides the low-level I/O details of flash storage, which have thus far often been exposed to application developers. Moreover, Antelope uses LogicVM, a novel virtual machine architecture that analyses and executes propositional logic expressed in a byte-code format. By compiling queries and executing them in the virtual machine, the performance of relational selection queries increases by an order of magnitude compared to repeated parsing.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience on some of the basic query operations of the antelope.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*''Query processor'' : It parses AQL(query language for Antelope) queries.<br />
*''Privacy Control'' : Checks whether the query is allowed or not.<br />
*''Logic VM'' : It executes the queries.<br />
*''Database kernel'' : It holds the database logic and coordinates query execution.<br />
*''Index abstraction'' : It holds the indexing logic.<br />
*''Indexer process'' : Builds indexes from existing data.<br />
*''Storage abstraction'' : Contains all storage logic.<br />
*''Result transformer'' : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes "netdb_process"(which is the database process) and "shell_process". "shell_process" is the process which provides serial interface at the client to input the queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread associated with this process will start running by the scheduler. Below is the code snippet of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callbacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow [https://github.com/contiki-os/contiki/wiki/Processes this] link.<br />
<br />
"mesh_open" opens a new connection and here the created mesh connection operates on NETDB_CHANNEL(70) and the third argument "callbacks" contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another process, instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready.png|center|400px]] <br />
<br />
[[File:NetdbClient.png|center|400px]]<br />
<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
<br />
Below are the snapshots for Antelope client and server logs in cooja.<br />
<br />
[[File:Client_Query.png|400px]]<br />
[[File:Server_Query.png|400px]]<br />
<br />
Similarly, we could try some other examples like below:<br />
<source lang="c"><br />
CREATE RELATION sensor;<br />
<br />
CREATE ATTRIBUTE id DOMAIN INT IN sensor;<br />
CREATE ATTRIBUTE name DOMAIN STRING(20) IN sensor;<br />
CREATE ATTRIBUTE position DOMAIN LONG IN sensor;<br />
<br />
CREATE INDEX sensor.id TYPE INLINE;<br />
CREATE INDEX sensor.position TYPE MAXHEAP;<br />
</source><br />
<br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
Edited by Gopi.</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1283Antelope(Database Management System) - Contiki2014-11-09T03:05:23Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system which enables every sensor to act as a database server. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
Antelope is the first DBMS for resource-constrained sensor nodes, provides energy-efficient querying, high-level data modelling, while being index independent. MaxHeap indexing enables fast insertions and queries over large data sets while having a low energy cost. Antelope hides the low-level I/O details of flash storage, which have thus far often been exposed to application developers. Moreover, Antelope uses LogicVM, a novel virtual machine architecture that analyses and executes propositional logic expressed in a byte-code format. By compiling queries and executing them in the virtual machine, the performance of relational selection queries increases by an order of magnitude compared to repeated parsing.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience on some of the basic query operations of the antelope.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*''Query processor'' : It parses AQL(query language for Antelope) queries.<br />
*''Privacy Control'' : Checks whether the query is allowed or not.<br />
*''Logic VM'' : It executes the queries.<br />
*''Database kernel'' : It holds the database logic and coordinates query execution.<br />
*''Index abstraction'' : It holds the indexing logic.<br />
*''Indexer process'' : Builds indexes from existing data.<br />
*''Storage abstraction'' : Contains all storage logic.<br />
*''Result transformer'' : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes "netdb_process"(which is the database process) and "shell_process". "shell_process" is the process which provides serial interface at the client to input the queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread associated with this process will start running by the scheduler. Below is the code snippet of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callbacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow [https://github.com/contiki-os/contiki/wiki/Processes this] link.<br />
<br />
"mesh_open" opens a new connection and here the created mesh connection operates on NETDB_CHANNEL(70) and the third argument "callbacks" contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another process, instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready.png|center|400px]] <br />
<br />
[[File:NetdbClient.png|center|400px]]<br />
<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
<br />
Below are the snapshots for Antelope client and server logs in cooja.<br />
<br />
[[File:Client_Query.png|400px]]<br />
[[File:Server_Query.png|500px]]<br />
<br />
Similarly, we could try some other examples like below:<br />
<source lang="c"><br />
CREATE RELATION sensor;<br />
<br />
CREATE ATTRIBUTE id DOMAIN INT IN sensor;<br />
CREATE ATTRIBUTE name DOMAIN STRING(20) IN sensor;<br />
CREATE ATTRIBUTE position DOMAIN LONG IN sensor;<br />
<br />
CREATE INDEX sensor.id TYPE INLINE;<br />
CREATE INDEX sensor.position TYPE MAXHEAP;<br />
</source><br />
<br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
Edited by Gopi.</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1280Antelope(Database Management System) - Contiki2014-11-09T03:04:30Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system which enables every sensor to act as a database server. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
Antelope is the first DBMS for resource-constrained sensor nodes, provides energy-efficient querying, high-level data modelling, while being index independent. MaxHeap indexing enables fast insertions and queries over large data sets while having a low energy cost. Antelope hides the low-level I/O details of flash storage, which have thus far often been exposed to application developers. Moreover, Antelope uses LogicVM, a novel virtual machine architecture that analyses and executes propositional logic expressed in a byte-code format. By compiling queries and executing them in the virtual machine, the performance of relational selection queries increases by an order of magnitude compared to repeated parsing.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience on some of the basic query operations of the antelope.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*''Query processor'' : It parses AQL(query language for Antelope) queries.<br />
*''Privacy Control'' : Checks whether the query is allowed or not.<br />
*''Logic VM'' : It executes the queries.<br />
*''Database kernel'' : It holds the database logic and coordinates query execution.<br />
*''Index abstraction'' : It holds the indexing logic.<br />
*''Indexer process'' : Builds indexes from existing data.<br />
*''Storage abstraction'' : Contains all storage logic.<br />
*''Result transformer'' : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes "netdb_process"(which is the database process) and "shell_process". "shell_process" is the process which provides serial interface at the client to input the queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread associated with this process will start running by the scheduler. Below is the code snippet of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callbacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow [https://github.com/contiki-os/contiki/wiki/Processes this] link.<br />
<br />
"mesh_open" opens a new connection and here the created mesh connection operates on NETDB_CHANNEL(70) and the third argument "callbacks" contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another process, instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready.png|center|400px]] <br />
<br />
[[File:NetdbClient.png|center|400px]]<br />
<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
<br />
Below are the snapshots for Antelope client and server logs in cooja.<br />
<br />
[[File:Client_Query.png|500px]]<br />
[[File:Server_Query.png|500px]]<br />
<br />
Similarly, we could try some other examples like below:<br />
<source lang="c"><br />
CREATE RELATION sensor;<br />
<br />
CREATE ATTRIBUTE id DOMAIN INT IN sensor;<br />
CREATE ATTRIBUTE name DOMAIN STRING(20) IN sensor;<br />
CREATE ATTRIBUTE position DOMAIN LONG IN sensor;<br />
<br />
CREATE INDEX sensor.id TYPE INLINE;<br />
CREATE INDEX sensor.position TYPE MAXHEAP;<br />
</source><br />
<br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
Edited by Gopi.</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1279Antelope(Database Management System) - Contiki2014-11-09T03:03:50Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system which enables every sensor to act as a database server. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
Antelope is the first DBMS for resource-constrained sensor nodes, provides energy-efficient querying, high-level data modelling, while being index independent. MaxHeap indexing enables fast insertions and queries over large data sets while having a low energy cost. Antelope hides the low-level I/O details of flash storage, which have thus far often been exposed to application developers. Moreover, Antelope uses LogicVM, a novel virtual machine architecture that analyses and executes propositional logic expressed in a byte-code format. By compiling queries and executing them in the virtual machine, the performance of relational selection queries increases by an order of magnitude compared to repeated parsing.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience on some of the basic query operations of the antelope.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*''Query processor'' : It parses AQL(query language for Antelope) queries.<br />
*''Privacy Control'' : Checks whether the query is allowed or not.<br />
*''Logic VM'' : It executes the queries.<br />
*''Database kernel'' : It holds the database logic and coordinates query execution.<br />
*''Index abstraction'' : It holds the indexing logic.<br />
*''Indexer process'' : Builds indexes from existing data.<br />
*''Storage abstraction'' : Contains all storage logic.<br />
*''Result transformer'' : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes "netdb_process"(which is the database process) and "shell_process". "shell_process" is the process which provides serial interface at the client to input the queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread associated with this process will start running by the scheduler. Below is the code snippet of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callbacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow [https://github.com/contiki-os/contiki/wiki/Processes this] link.<br />
<br />
"mesh_open" opens a new connection and here the created mesh connection operates on NETDB_CHANNEL(70) and the third argument "callbacks" contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another process, instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready.png|center|400px]] <br />
<br />
[[File:NetdbClient.png|center|400px]]<br />
<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
<br />
Below are the snapshots for Antelope client and server logs in cooja.<br />
<br />
[[File:Client_Query.png|500px]]<br />
[[File:Server_Query.png|500px]]<br />
<br />
Similarly, we could try some other examples like below:<br />
<source lang="c"><br />
CREATE RELATION sensor;<br />
<br />
CREATE ATTRIBUTE id DOMAIN INT IN sensor;<br />
CREATE ATTRIBUTE name DOMAIN STRING(20) IN sensor;<br />
CREATE ATTRIBUTE position DOMAIN LONG IN sensor;<br />
<br />
CREATE INDEX sensor.id TYPE INLINE;<br />
CREATE INDEX sensor.position TYPE MAXHEAP;<br />
</source><br />
<br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
<br />
Edited by Gopi.</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1277Antelope(Database Management System) - Contiki2014-11-09T02:57:37Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system which enables every sensor to act as a database server. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
Antelope is the first DBMS for resource-constrained sensor nodes, provides energy-efficient querying, high-level data modelling, while being index independent. MaxHeap indexing enables fast insertions and queries over large data sets while having a low energy cost. Antelope hides the low-level I/O details of flash storage, which have thus far often been exposed to application developers. Moreover, Antelope uses LogicVM, a novel virtual machine architecture that analyses and executes propositional logic expressed in a byte-code format. By compiling queries and executing them in the virtual machine, the performance of relational selection queries increases by an order of magnitude compared to repeated parsing.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience on some of the basic query operations of the antelope.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*''Query processor'' : It parses AQL(query language for Antelope) queries.<br />
*''Privacy Control'' : Checks whether the query is allowed or not.<br />
*''Logic VM'' : It executes the queries.<br />
*''Database kernel'' : It holds the database logic and coordinates query execution.<br />
*''Index abstraction'' : It holds the indexing logic.<br />
*''Indexer process'' : Builds indexes from existing data.<br />
*''Storage abstraction'' : Contains all storage logic.<br />
*''Result transformer'' : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes "netdb_process"(which is the database process) and "shell_process". "shell_process" is the process which provides serial interface at the client to input the queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread associated with this process will start running by the scheduler. Below is the code snippet of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callbacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow [https://github.com/contiki-os/contiki/wiki/Processes this] link.<br />
<br />
"mesh_open" opens a new connection and here the created mesh connection operates on NETDB_CHANNEL(70) and the third argument "callbacks" contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another process, instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready.png|center|400px]] <br />
<br />
[[File:NetdbClient.png|center|400px]]<br />
<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
<br />
Below are the snapshots for Antelope client and server logs in cooja.<br />
<br />
[[File:Client_Query.png|500px]]<br />
[[File:Server_Query.png|500px]]<br />
<br />
Similarly, we could try some other examples like below:<br />
<source lang="c"><br />
CREATE RELATION sensor;<br />
<br />
CREATE ATTRIBUTE id DOMAIN INT IN sensor;<br />
CREATE ATTRIBUTE name DOMAIN STRING(20) IN sensor;<br />
CREATE ATTRIBUTE position DOMAIN LONG IN sensor;<br />
<br />
CREATE INDEX sensor.id TYPE INLINE;<br />
CREATE INDEX sensor.position TYPE MAXHEAP;<br />
</source><br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
Edited by Gopi.<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1276Antelope(Database Management System) - Contiki2014-11-09T02:55:43Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system which enables every sensor to act as a database server. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
Antelope is the first DBMS for resource-constrained sensor nodes, provides energy-efficient querying, high-level data modelling, while being index independent. MaxHeap indexing enables fast insertions and queries over large data sets while having a low energy cost. Antelope hides the low-level I/O details of flash storage, which have thus far often been exposed to application developers. Moreover, Antelope uses LogicVM, a novel virtual machine architecture that analyses and executes propositional logic expressed in a byte-code format. By compiling queries and executing them in the virtual machine, the performance of relational selection queries increases by an order of magnitude compared to repeated parsing.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience on some of the basic query operations of the antelope.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*''Query processor'' : It parses AQL(query language for Antelope) queries.<br />
*''Privacy Control'' : Checks whether the query is allowed or not.<br />
*''Logic VM'' : It executes the queries.<br />
*''Database kernel'' : It holds the database logic and coordinates query execution.<br />
*''Index abstraction'' : It holds the indexing logic.<br />
*''Indexer process'' : Builds indexes from existing data.<br />
*''Storage abstraction'' : Contains all storage logic.<br />
*''Result transformer'' : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes "netdb_process"(which is the database process) and "shell_process". "shell_process" is the process which provides serial interface at the client to input the queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread associated with this process will start running by the scheduler. Below is the code snippet of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callbacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow [https://github.com/contiki-os/contiki/wiki/Processes this] link.<br />
<br />
"mesh_open" opens a new connection and here the created mesh connection operates on NETDB_CHANNEL(70) and the third argument "callbacks" contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another process, instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready.png|center|400px]] <br />
<br />
[[File:NetdbClient.png|center|400px]]<br />
<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
<br />
Below are the snapshots for Antelope client and server logs in cooja.<br />
<br />
[[File:Client_Query.png|500px]]<br />
[[File:Server_Query.png|500px]]<br />
<br />
Similarly, we could try some other examples like below:<br />
<soource lang="c"><br />
CREATE RELATION sensor;<br />
<br />
CREATE ATTRIBUTE id DOMAIN INT IN sensor;<br />
CREATE ATTRIBUTE name DOMAIN STRING(20) IN sensor;<br />
CREATE ATTRIBUTE position DOMAIN LONG IN sensor;<br />
<br />
CREATE INDEX sensor.id TYPE INLINE;<br />
CREATE INDEX sensor.position TYPE MAXHEAP;<br />
</soruce><br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
Edited by Gopi.<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1273Antelope(Database Management System) - Contiki2014-11-09T02:49:34Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system which enables every sensor to act as a database server. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
Antelope is the first DBMS for resource-constrained sensor nodes, provides energy-efficient querying, high-level data modelling, while being index independent. MaxHeap indexing enables fast insertions and queries over large data sets while having a low energy cost. Antelope hides the low-level I/O details of flash storage, which have thus far often been exposed to application developers. Moreover, Antelope uses LogicVM, a novel virtual machine architecture that analyses and executes propositional logic expressed in a byte-code format. By compiling queries and executing them in the virtual machine, the performance of relational selection queries increases by an order of magnitude compared to repeated parsing.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience on some of the basic query operations of the antelope.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*''Query processor'' : It parses AQL(query language for Antelope) queries.<br />
*''Privacy Control'' : Checks whether the query is allowed or not.<br />
*''Logic VM'' : It executes the queries.<br />
*''Database kernel'' : It holds the database logic and coordinates query execution.<br />
*''Index abstraction'' : It holds the indexing logic.<br />
*''Indexer process'' : Builds indexes from existing data.<br />
*''Storage abstraction'' : Contains all storage logic.<br />
*''Result transformer'' : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes "netdb_process"(which is the database process) and "shell_process". "shell_process" is the process which provides serial interface at the client to input the queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread associated with this process will start running by the scheduler. Below is the code snippet of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callbacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow [https://github.com/contiki-os/contiki/wiki/Processes this] link.<br />
<br />
"mesh_open" opens a new connection and here the created mesh connection operates on NETDB_CHANNEL(70) and the third argument "callbacks" contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready.png|center|400px]] <br />
<br />
[[File:NetdbClient.png|center|400px]]<br />
<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
<br />
Below are the snapshots for Antelope client and server logs in cooja.<br />
<br />
[[File:Client_Query.png|500px]]<br />
[[File:Server_Query.png|500px]]<br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
Edited by Gopi.<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1272Antelope(Database Management System) - Contiki2014-11-09T02:47:27Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system which enables every sensor to act as a database server. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
Antelope is the first DBMS for resource-constrained sensor nodes, provides energy-efficient querying, high-level data modelling, while being index independent. MaxHeap indexing enables fast insertions and queries over large data sets while having a low energy cost. Antelope hides the low-level I/O details of flash storage, which have thus far often been exposed to application developers. Moreover, Antelope uses LogicVM, a novel virtual machine architecture that analyses and executes propositional logic expressed in a byte-code format. By compiling queries and executing them in the virtual machine, the performance of relational selection queries increases by an order of magnitude compared to repeated parsing.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience on some of the basic query operations of the antelope.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*''Query processor'' : It parses AQL(query language for Antelope) queries.<br />
*''Privacy Control'' : Checks whether the query is allowed or not.<br />
*''Logic VM'' : It executes the queries.<br />
*''Database kernel'' : It holds the database logic and coordinates query execution.<br />
*''Index abstraction'' : It holds the indexing logic.<br />
*''Indexer process'' : Builds indexes from existing data.<br />
*''Storage abstraction'' : Contains all storage logic.<br />
*''Result transformer'' : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes "netdb_process"(which is the database process) and "shell_process". "shell_process" is the process which provides serial interface at the client to input the queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread associated with this process will start running by the scheduler. Below is the code snippet of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callbacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow [https://github.com/contiki-os/contiki/wiki/Processes this] link.<br />
<br />
mesh_open opens a new connection and here this mesh connection operates on NETDB_CHANNEL(70) and the third argument callbacks contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready.png|center|400px]] <br />
<br />
[[File:NetdbClient.png|center|400px]]<br />
<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
<br />
Below are the snapshots for Antelope client and server logs in cooja.<br />
<br />
[[File:Client_Query.png|500px]]<br />
[[File:Server_Query.png|500px]]<br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
Edited by Gopi.<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1268Antelope(Database Management System) - Contiki2014-11-09T02:41:00Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system which enables every sensor to act as a database server. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
Antelope is the first DBMS for resource-constrained sensor nodes, provides energy-efficient querying, high-level data modelling, while being index independent. MaxHeap indexing enables fast insertions and queries over large data sets while having a low energy cost. Antelope hides the low-level I/O details of flash storage, which have thus far often been exposed to application developers. Moreover, Antelope uses LogicVM, a novel virtual machine architecture that analyses and executes propositional logic expressed in a byte-code format. By compiling queries and executing them in the virtual machine, the performance of relational selection queries increases by an order of magnitude compared to repeated parsing.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience on some of the basic query operations of the antelope.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*''Query processor'' : It parses AQL(query language of Antelope) queries.<br />
*''Privacy Control'' : Checks whether the query is allowed or not.<br />
*''Logic VM'' : It executes the queries.<br />
*''Database kernel'' : It holds the database logic and coordinates query execution.<br />
*''Index abstraction'' : It holds the indexing logic.<br />
*''Indexer process'' : Builds indexes from existing data.<br />
*''Storage abstraction'' : Contains all storage logic.<br />
*''Result transformer'' : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes netdb_process(which is the database process), shell_process. "shell_process" is the process which provides serial interface at the client to input our queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread assosicated with this process will start running by the scheduler. Below is the code snipped of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callabacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow this link.<br />
<br />
mesh_open opens a new connection and here this mesh connection operates on NETDB_CHANNEL(70) and the third argument callbacks contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready.png|center|400px]] <br />
<br />
[[File:NetdbClient.png|center|400px]]<br />
<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
<br />
Below are the snapshots for Antelope client and server logs in cooja.<br />
<br />
[[File:Client_Query.png|500px]]<br />
[[File:Server_Query.png|500px]]<br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
Edited by Gopi.<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1267Antelope(Database Management System) - Contiki2014-11-09T02:38:48Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system which enables every sensor to act as a database server. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
Antelope is the first DBMS for resource-constrained sensor nodes, provides energy-efficient querying, high-level data modelling, while being index independent. MaxHeap indexing enables fast insertions and queries over large data sets while having a low energy cost. Antelope hides the low-level I/O details of flash storage, which have thus far often been exposed to application developers. Moreover, Antelope uses LogicVM, a novel virtual machine architecture that analyses and executes propositional logic expressed in a byte-code format. By compiling queries and executing them in the virtual machine, the performance of relational selection queries increases by an order of magnitude compared to repeated parsing.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience on some of the basic query operations of the antelope.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*Query processor : It parses AQL(query language of Antelope) queries.<br />
*Privacy Control : Checks whether the query is allowed or not.<br />
*Logic VM : It executes the queries.<br />
*Database kernel : It holds the database logic and coordinates query execution.<br />
*Index abstraction : It holds the indexing logic.<br />
*Indexer process : Builds indexes from existing data.<br />
*Storage abstraction : Contains all storage logic.<br />
*Result transformer : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes netdb_process(which is the database process), shell_process. "shell_process" is the process which provides serial interface at the client to input our queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread assosicated with this process will start running by the scheduler. Below is the code snipped of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callabacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow this link.<br />
<br />
mesh_open opens a new connection and here this mesh connection operates on NETDB_CHANNEL(70) and the third argument callbacks contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready.png|center|400px]] <br />
<br />
[[File:NetdbClient.png|center|400px]]<br />
<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
<br />
Below are the snapshots for Antelope client and server logs in cooja.<br />
<br />
[[File:Client_Query.png|500px]]<br />
[[File:Server_Query.png|500px]]<br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
Edited by Gopi.<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1252Antelope(Database Management System) - Contiki2014-11-09T02:28:00Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system for resource-constrained memory devices. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*Query processor : It parses AQL(query language of Antelope) queries.<br />
*Privacy Control : Checks whether the query is allowed or not.<br />
*Logic VM : It executes the queries.<br />
*Database kernel : It holds the database logic and coordinates query execution.<br />
*Index abstraction : It holds the indexing logic.<br />
*Indexer process : Builds indexes from existing data.<br />
*Storage abstraction : Contains all storage logic.<br />
*Result transformer : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes netdb_process(which is the database process), shell_process. "shell_process" is the process which provides serial interface at the client to input our queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread assosicated with this process will start running by the scheduler. Below is the code snipped of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callabacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow this link.<br />
<br />
mesh_open opens a new connection and here this mesh connection operates on NETDB_CHANNEL(70) and the third argument callbacks contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready.png|center|400px]] <br />
<br />
[[File:NetdbClient.png|center|400px]]<br />
<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
<br />
Below are the snapshots for Antelope client and server logs in cooja.<br />
<br />
[[File:Client_Query.png|500px]]<br />
[[File:Server_Query.png|500px]]<br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
Edited by Gopi.<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=File:Client_Query.png&diff=1250File:Client Query.png2014-11-09T02:24:48Z<p>Marella: </p>
<hr />
<div></div>Marellahttp://anrg.usc.edu/contiki/index.php?title=File:Server_Query.png&diff=1249File:Server Query.png2014-11-09T02:24:02Z<p>Marella: </p>
<hr />
<div></div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1247Antelope(Database Management System) - Contiki2014-11-09T02:20:55Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system for resource-constrained memory devices. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*Query processor : It parses AQL(query language of Antelope) queries.<br />
*Privacy Control : Checks whether the query is allowed or not.<br />
*Logic VM : It executes the queries.<br />
*Database kernel : It holds the database logic and coordinates query execution.<br />
*Index abstraction : It holds the indexing logic.<br />
*Indexer process : Builds indexes from existing data.<br />
*Storage abstraction : Contains all storage logic.<br />
*Result transformer : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes netdb_process(which is the database process), shell_process. "shell_process" is the process which provides serial interface at the client to input our queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread assosicated with this process will start running by the scheduler. Below is the code snipped of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callabacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow this link.<br />
<br />
mesh_open opens a new connection and here this mesh connection operates on NETDB_CHANNEL(70) and the third argument callbacks contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready.png|center|400px]] <br />
<br />
[[File:NetdbClient.png|center|400px]]<br />
<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
Edited by Gopi.<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1244Antelope(Database Management System) - Contiki2014-11-09T02:19:46Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system for resource-constrained memory devices. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*Query processor : It parses AQL(query language of Antelope) queries.<br />
*Privacy Control : Checks whether the query is allowed or not.<br />
*Logic VM : It executes the queries.<br />
*Database kernel : It holds the database logic and coordinates query execution.<br />
*Index abstraction : It holds the indexing logic.<br />
*Indexer process : Builds indexes from existing data.<br />
*Storage abstraction : Contains all storage logic.<br />
*Result transformer : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes netdb_process(which is the database process), shell_process. "shell_process" is the process which provides serial interface at the client to input our queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread assosicated with this process will start running by the scheduler. Below is the code snipped of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callabacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow this link.<br />
<br />
mesh_open opens a new connection and here this mesh connection operates on NETDB_CHANNEL(70) and the third argument callbacks contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready.png|center|400px]] <br />
[[File:NetdbClient.png|center|400px]]<br />
<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
Edited by Gopi.<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1242Antelope(Database Management System) - Contiki2014-11-09T02:19:04Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system for resource-constrained memory devices. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*Query processor : It parses AQL(query language of Antelope) queries.<br />
*Privacy Control : Checks whether the query is allowed or not.<br />
*Logic VM : It executes the queries.<br />
*Database kernel : It holds the database logic and coordinates query execution.<br />
*Index abstraction : It holds the indexing logic.<br />
*Indexer process : Builds indexes from existing data.<br />
*Storage abstraction : Contains all storage logic.<br />
*Result transformer : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes netdb_process(which is the database process), shell_process. "shell_process" is the process which provides serial interface at the client to input our queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread assosicated with this process will start running by the scheduler. Below is the code snipped of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callabacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow this link.<br />
<br />
mesh_open opens a new connection and here this mesh connection operates on NETDB_CHANNEL(70) and the third argument callbacks contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready.png|none|400px]] [[File:NetdbClient.png|right|400px]]<br />
<br />
<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
Edited by Gopi.<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1237Antelope(Database Management System) - Contiki2014-11-09T02:14:17Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system for resource-constrained memory devices. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*Query processor : It parses AQL(query language of Antelope) queries.<br />
*Privacy Control : Checks whether the query is allowed or not.<br />
*Logic VM : It executes the queries.<br />
*Database kernel : It holds the database logic and coordinates query execution.<br />
*Index abstraction : It holds the indexing logic.<br />
*Indexer process : Builds indexes from existing data.<br />
*Storage abstraction : Contains all storage logic.<br />
*Result transformer : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes netdb_process(which is the database process), shell_process. "shell_process" is the process which provides serial interface at the client to input our queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread assosicated with this process will start running by the scheduler. Below is the code snipped of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callabacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow this link.<br />
<br />
mesh_open opens a new connection and here this mesh connection operates on NETDB_CHANNEL(70) and the third argument callbacks contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready.png|left|400px]] [[File:NetdbClient.png|right|400px]]<br />
<br />
<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
Edited by Gopi.<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1235Antelope(Database Management System) - Contiki2014-11-09T02:13:27Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system for resource-constrained memory devices. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*Query processor : It parses AQL(query language of Antelope) queries.<br />
*Privacy Control : Checks whether the query is allowed or not.<br />
*Logic VM : It executes the queries.<br />
*Database kernel : It holds the database logic and coordinates query execution.<br />
*Index abstraction : It holds the indexing logic.<br />
*Indexer process : Builds indexes from existing data.<br />
*Storage abstraction : Contains all storage logic.<br />
*Result transformer : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes netdb_process(which is the database process), shell_process. "shell_process" is the process which provides serial interface at the client to input our queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread assosicated with this process will start running by the scheduler. Below is the code snipped of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callabacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow this link.<br />
<br />
mesh_open opens a new connection and here this mesh connection operates on NETDB_CHANNEL(70) and the third argument callbacks contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready.png|left|400px]] [[File:NetdbClient.png|left|400px]]<br />
<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
Edited by Gopi.<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1233Antelope(Database Management System) - Contiki2014-11-09T02:12:17Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system for resource-constrained memory devices. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*Query processor : It parses AQL(query language of Antelope) queries.<br />
*Privacy Control : Checks whether the query is allowed or not.<br />
*Logic VM : It executes the queries.<br />
*Database kernel : It holds the database logic and coordinates query execution.<br />
*Index abstraction : It holds the indexing logic.<br />
*Indexer process : Builds indexes from existing data.<br />
*Storage abstraction : Contains all storage logic.<br />
*Result transformer : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes netdb_process(which is the database process), shell_process. "shell_process" is the process which provides serial interface at the client to input our queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread assosicated with this process will start running by the scheduler. Below is the code snipped of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callabacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow this link.<br />
<br />
mesh_open opens a new connection and here this mesh connection operates on NETDB_CHANNEL(70) and the third argument callbacks contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready.png|left|500px]] [[File:NetdbClient.png|right|500px]]<br />
<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
Edited by Gopi.<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=File:NetdbClient.png&diff=1230File:NetdbClient.png2014-11-09T02:10:43Z<p>Marella: </p>
<hr />
<div></div>Marellahttp://anrg.usc.edu/contiki/index.php?title=File:Ready.png&diff=1229File:Ready.png2014-11-09T02:10:11Z<p>Marella: </p>
<hr />
<div></div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1222Antelope(Database Management System) - Contiki2014-11-09T01:55:29Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system for resource-constrained memory devices. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*Query processor : It parses AQL(query language of Antelope) queries.<br />
*Privacy Control : Checks whether the query is allowed or not.<br />
*Logic VM : It executes the queries.<br />
*Database kernel : It holds the database logic and coordinates query execution.<br />
*Index abstraction : It holds the indexing logic.<br />
*Indexer process : Builds indexes from existing data.<br />
*Storage abstraction : Contains all storage logic.<br />
*Result transformer : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes netdb_process(which is the database process), shell_process. "shell_process" is the process which provides serial interface at the client to input our queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread assosicated with this process will start running by the scheduler. Below is the code snipped of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callabacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow this link.<br />
<br />
mesh_open opens a new connection and here this mesh connection operates on NETDB_CHANNEL(70) and the third argument callbacks contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes well, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready_to_process_queries.png|center|500px]]<br />
<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
Edited by Gopi.<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1219Antelope(Database Management System) - Contiki2014-11-09T01:54:21Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system for resource-constrained memory devices. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*Query processor : It parses AQL(query language of Antelope) queries.<br />
*Privacy Control : Checks whether the query is allowed or not.<br />
*Logic VM : It executes the queries.<br />
*Database kernel : It holds the database logic and coordinates query execution.<br />
*Index abstraction : It holds the indexing logic.<br />
*Indexer process : Builds indexes from existing data.<br />
*Storage abstraction : Contains all storage logic.<br />
*Result transformer : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes netdb_process(which is the database process), shell_process. "shell_process" is the process which provides serial interface at the client to input our queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread assosicated with this process will start running by the scheduler. Below is the code snipped of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callabacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow this link.<br />
<br />
mesh_open opens a new connection and here this mesh connection operates on NETDB_CHANNEL(70) and the third argument callbacks contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes will, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready_to_process_queries.png|center|400px]]<br />
<br />
/TODO: Insert diagram/<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
Edited by Gopi.<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1214Antelope(Database Management System) - Contiki2014-11-09T01:52:18Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system for resource-constrained memory devices. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*Query processor : It parses AQL(query language of Antelope) queries.<br />
*Privacy Control : Checks whether the query is allowed or not.<br />
*Logic VM : It executes the queries.<br />
*Database kernel : It holds the database logic and coordinates query execution.<br />
*Index abstraction : It holds the indexing logic.<br />
*Indexer process : Builds indexes from existing data.<br />
*Storage abstraction : Contains all storage logic.<br />
*Result transformer : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes netdb_process(which is the database process), shell_process. "shell_process" is the process which provides serial interface at the client to input our queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread assosicated with this process will start running by the scheduler. Below is the code snipped of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callabacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow this link.<br />
<br />
mesh_open opens a new connection and here this mesh connection operates on NETDB_CHANNEL(70) and the third argument callbacks contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<source lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes will, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
[[File:Ready_to_process_queries.png]|center|400px]<br />
<br />
/TODO: Insert diagram/<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
Edited by Gopi.<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=File:Ready_to_process_queries.png&diff=1213File:Ready to process queries.png2014-11-09T01:50:09Z<p>Marella: </p>
<hr />
<div></div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1212Antelope(Database Management System) - Contiki2014-11-09T01:49:21Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system for resource-constrained memory devices. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*Query processor : It parses AQL(query language of Antelope) queries.<br />
*Privacy Control : Checks whether the query is allowed or not.<br />
*Logic VM : It executes the queries.<br />
*Database kernel : It holds the database logic and coordinates query execution.<br />
*Index abstraction : It holds the indexing logic.<br />
*Indexer process : Builds indexes from existing data.<br />
*Storage abstraction : Contains all storage logic.<br />
*Result transformer : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes netdb_process(which is the database process), shell_process. "shell_process" is the process which provides serial interface at the client to input our queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread assosicated with this process will start running by the scheduler. Below is the code snipped of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callabacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow this link.<br />
<br />
mesh_open opens a new connection and here this mesh connection operates on NETDB_CHANNEL(70) and the third argument callbacks contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
PROCESS_BEGIN();<br />
..<br />
...<br />
....<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
The above code block creates a process named "query_process" and a protothread for the process "query_process". Let's walk through the code of query_process.<br />
<source lang="c"><br />
db_init();<br />
db_set_output_function(buffer_db_data);<br />
db_query(NULL, "REMOVE RELATION samples;");<br />
db_query(NULL, "CREATE RELATION samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE time DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE ATTRIBUTE hum DOMAIN INT IN samples;");<br />
db_query(NULL, "CREATE INDEX samples.time TYPE INLINE;");<br />
</source><br />
<br />
The above source code initializes the database. "db_init" function sets up the memory needed for relation, attributes, indexes and everything needed for the antelope database to operate.<br />
<br />
"db_query" executes the query passed as argument by parsing it and calling aql_execute(remember AQL is the query language for Antelope database).<br />
<br />
<soruce lang="c"><br />
#if PREPARE_DB<br />
printf("Preparing the DB with %d tuples...\n", CARDINALITY);<br />
errors = 0;<br />
for(i = 1; i <= CARDINALITY; i++) {<br />
PROCESS_PAUSE();<br />
result = db_query(NULL, "INSERT (%u, %u) INTO samples;",<br />
i, (unsigned)random_rand());<br />
if(DB_ERROR(result)) {<br />
errors++;<br />
}<br />
}<br />
printf("Done. Insertion errors: %d\n", errors);<br />
printf("Ready to process queries\n");<br />
#else<br />
etimer_set(&sampling_timer, SAMPLING_INTERVAL * CLOCK_SECOND);<br />
#endif<br />
</source><br />
<br />
The above source code prepares the database with CARDINALITY(1000) number of tuples based on the flag PREPARE_DB. It inserts CARDINALITY number of random integer samples into the database. If everything goes will, you should see the printf statement "Ready to process Queries" in your mote output like below.<br />
<br />
/TODO: Insert diagram/<br />
<source lang="c"><br />
for(;;) {<br />
PROCESS_WAIT_EVENT();<br />
if(ev == serial_line_event_message && data != NULL) {<br />
printf("START %s\n", (char *)data);<br />
result = db_query(&handle, data);<br />
if(DB_ERROR(result)) {<br />
buffer_db_data("Query error: %s\n", db_get_result_message(result));<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
continue;<br />
}<br />
if(!db_processing(&handle)) {<br />
buffer_db_data("OK\n");<br />
send_buffered_data();<br />
stop_handler(NULL);<br />
continue;<br />
}<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
db_print_header(&handle);<br />
matching = 0;<br />
processed = 0;<br />
while(db_processing(&handle)) {<br />
PROCESS_PAUSE();<br />
if(matching == RESPONSE_LIMIT) {<br />
buffer_db_data("Response suppressed at %u tuples: limit reached\n",<br />
RESPONSE_LIMIT);<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
break;<br />
}<br />
result = db_process(&handle);<br />
if(result == DB_GOT_ROW) {<br />
/* The processed tuple matched the condition in the query. */<br />
matching++;<br />
processed++;<br />
db_print_tuple(&handle);<br />
} else if(result == DB_OK) {<br />
/* A tuple was processed, but did not match the condition. */<br />
processed++;<br />
continue;<br />
} else {<br />
if(result == DB_FINISHED) {<br />
/* The processing has finished. Wait for a new command. */<br />
buffer_db_data("[%ld tuples returned; %ld tuples processed]\n",<br />
(long)matching, (long)processed);<br />
buffer_db_data("OK\n");<br />
} else if(DB_ERROR(result)) {<br />
buffer_db_data("Processing error: %s\n",<br />
db_get_result_message(result));<br />
}<br />
stop_handler(NULL);<br />
db_free(&handle);<br />
}<br />
}<br />
send_buffered_data();<br />
}<br />
#if !PREPARE_DB<br />
if(etimer_expired(&sampling_timer)) {<br />
take_sample();<br />
etimer_reset(&sampling_timer);<br />
}<br />
#endif<br />
}<br />
</source><br />
<br />
The above source code runs an infinite for loop which waits for an event to be posted to the process "query_process", data contains the query to be processed and db_query is called to execute the query and error conditions are processed after executing the query. This is the meat of the "query_process" process.<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
Edited by Gopi.<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1133Antelope(Database Management System) - Contiki2014-11-08T22:19:46Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system for resource-constrained memory devices. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*Query processor : It parses AQL(query language of Antelope) queries.<br />
*Privacy Control : Checks whether the query is allowed or not.<br />
*Logic VM : It executes the queries.<br />
*Database kernel : It holds the database logic and coordinates query execution.<br />
*Index abstraction : It holds the indexing logic.<br />
*Indexer process : Builds indexes from existing data.<br />
*Storage abstraction : Contains all storage logic.<br />
*Result transformer : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes netdb_process(which is the database process), shell_process. "shell_process" is the process which provides serial interface at the client to input our queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread assosicated with this process will start running by the scheduler. Below is the code snipped of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callabacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow this link.<br />
<br />
mesh_open opens a new connection and here this mesh connection operates on NETDB_CHANNEL(70) and the third argument callbacks contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
..<br />
...<br />
....<br />
}<br />
</source><br />
<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
<br />
Below is the log file at the server node (Mote ID:4) which shows the processing of the queries we sent from the Mote:51.<br />
<source lang="c"><br />
00:00.607 ID:4 Rime started with address 4.0<br />
00:00.616 ID:4 MAC 04:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 4.<br />
00:00.623 ID:4 CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
00:00.624 ID:4 Starting 'NetDB'<br />
00:00.626 ID:4 NetDB host<br />
00:00.701 ID:4 Preparing the DB with 1000 tuples...<br />
00:08.273 ID:4 Done. Insertion errors: 0<br />
00:08.274 ID:4 Ready to process queries<br />
01:31.565 ID:4 Query received from 51.0: create relation sample; (2 hops)<br />
01:31.567 ID:4 START create relation sample;<br />
01:31.619 ID:4 END<br />
02:21.804 ID:4 Query received from 51.0: create attribute id domain int in sample; (2 hops)<br />
02:21.808 ID:4 START create attribute id domain int in sample;<br />
02:21.817 ID:4 END<br />
03:16.861 ID:4 Query received from 51.0: insert (4) into sample; (2 hops)<br />
03:16.863 ID:4 START insert (4) into sample;<br />
03:16.870 ID:4 END<br />
03:54.549 ID:4 Query received from 51.0: insert (23) into sample; (2 hops)<br />
03:54.551 ID:4 START insert (23) into sample;<br />
03:54.558 ID:4 END<br />
04:20.783 ID:4 Query received from 51.0: insert (31) into sample; (2 hops)<br />
04:20.786 ID:4 START insert (31) into sample;<br />
04:20.792 ID:4 END<br />
05:22.159 ID:4 Query received from 51.0: select max(id) from sample; (2 hops)<br />
05:22.162 ID:4 START select max(id) from sample;<br />
05:22.204 ID:4 END<br />
</source><br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
Edited by Gopi.<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1132Antelope(Database Management System) - Contiki2014-11-08T22:16:58Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system for resource-constrained memory devices. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*Query processor : It parses AQL(query language of Antelope) queries.<br />
*Privacy Control : Checks whether the query is allowed or not.<br />
*Logic VM : It executes the queries.<br />
*Database kernel : It holds the database logic and coordinates query execution.<br />
*Index abstraction : It holds the indexing logic.<br />
*Indexer process : Builds indexes from existing data.<br />
*Storage abstraction : Contains all storage logic.<br />
*Result transformer : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes netdb_process(which is the database process), shell_process. "shell_process" is the process which provides serial interface at the client to input our queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread assosicated with this process will start running by the scheduler. Below is the code snipped of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callabacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow this link.<br />
<br />
mesh_open opens a new connection and here this mesh connection operates on NETDB_CHANNEL(70) and the third argument callbacks contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
..<br />
...<br />
....<br />
}<br />
</source><br />
<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
<source lang="c"><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</source><br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
Edited by Gopi.<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1131Antelope(Database Management System) - Contiki2014-11-08T22:14:00Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system for resource-constrained memory devices. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*Query processor : It parses AQL(query language of Antelope) queries.<br />
*Privacy Control : Checks whether the query is allowed or not.<br />
*Logic VM : It executes the queries.<br />
*Database kernel : It holds the database logic and coordinates query execution.<br />
*Index abstraction : It holds the indexing logic.<br />
*Indexer process : Builds indexes from existing data.<br />
*Storage abstraction : Contains all storage logic.<br />
*Result transformer : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes netdb_process(which is the database process), shell_process. "shell_process" is the process which provides serial interface at the client to input our queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread assosicated with this process will start running by the scheduler. Below is the code snipped of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callabacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow this link.<br />
<br />
mesh_open opens a new connection and here this mesh connection operates on NETDB_CHANNEL(70) and the third argument callbacks contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
..<br />
...<br />
....<br />
}<br />
</source><br />
<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
Below is the log of some of the queries that are sent to the server.<br />
:: <code><br />
Rime started with address 51.0<br />
MAC 33:00:00:00:00:00:00:00 Contiki 2.7 started. Node id is set to 51.<br />
CSMA nullrdc, channel check rate 128 Hz, radio channel 16<br />
Starting 'NetDB'<br />
NetDB client<br />
> create relation sample;<br />
11525 Transmitting query "create relation sample;" to node 4<br />
11674 Reply received from 4.0 (2 hops): OK<br />
> create attribute id domain int in sample;<br />
18093 Transmitting query "create attribute id domain int in sample;" to node 4<br />
18099 Reply received from 4.0 (2 hops): OK<br />
> insert (4) into sample;<br />
25141 Transmitting query "insert (4) into sample;" to node 4<br />
25146 Reply received from 4.0 (2 hops): OK<br />
> insert (23) into sample;<br />
29965 Transmitting query "insert (23) into sample;" to node 4<br />
29970 Reply received from 4.0 (2 hops): OK<br />
> insert (31) into sample;<br />
33323 Transmitting query "insert (31) into sample;" to node 4<br />
33328 Reply received from 4.0 (2 hops): OK<br />
> select max(id) from sample;<br />
41179 Transmitting query "select max(id) from sample;" to node 4<br />
41189 Reply received from 4.0 (2 hops): [relation = db-result, attributes = (id)]<br />
Row 1:31<br />
41191 Reply received from 4.0 (2 hops): [1 tuples returned; 4 tuples processed]<br />
OK<br />
</code><br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
Edited by Gopi.<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=1056Antelope(Database Management System) - Contiki2014-11-08T06:38:30Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system for resource-constrained memory devices. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*Query processor : It parses AQL(query language of Antelope) queries.<br />
*Privacy Control : Checks whether the query is allowed or not.<br />
*Logic VM : It executes the queries.<br />
*Database kernel : It holds the database logic and coordinates query execution.<br />
*Index abstraction : It holds the indexing logic.<br />
*Indexer process : Builds indexes from existing data.<br />
*Storage abstraction : Contains all storage logic.<br />
*Result transformer : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes netdb_process(which is the database process), shell_process. "shell_process" is the process which provides serial interface at the client to input our queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread assosicated with this process will start running by the scheduler. Below is the code snipped of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callabacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow this link.<br />
<br />
mesh_open opens a new connection and here this mesh connection operates on NETDB_CHANNEL(70) and the third argument callbacks contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
..<br />
...<br />
....<br />
}<br />
</source><br />
<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
Click on "start" in the simulation control window. Below are the screen shots for the Mote interface viewer and Mote output for Mote ID:4.<br />
<br />
<br />
Now, let's try to create a relation(table), attributes(Columns), inserting a tuple into the relation.<br />
<br />
*Creating a relation<br />
We can create a relation (like a table which holds the tuples) with the help of following command.<br />
<br />
:: <code>create relation myrelation1;</code><br />
<br />
<br />
Now, create some attributes in this relation.<br />
<br />
:: <code> create attribute id domain int in myrelation1;</code><br />
The above command creates an attribute named "id" which is of type integer(int) in the relation myrelation1.<br />
Similarly we can create other attributes such as "name" of type string etc.<br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
Edited by Gopi.<br />
<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=946Antelope(Database Management System) - Contiki2014-11-07T12:26:06Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system for resource-constrained memory devices. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*Query processor : It parses AQL(query language of Antelope) queries.<br />
*Privacy Control : Checks whether the query is allowed or not.<br />
*Logic VM : It executes the queries.<br />
*Database kernel : It holds the database logic and coordinates query execution.<br />
*Index abstraction : It holds the indexing logic.<br />
*Indexer process : Builds indexes from existing data.<br />
*Storage abstraction : Contains all storage logic.<br />
*Result transformer : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes netdb_process(which is the database process), shell_process. "shell_process" is the process which provides serial interface at the client to input our queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread assosicated with this process will start running by the scheduler. Below is the code snipped of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callabacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow this link.<br />
<br />
mesh_open opens a new connection and here this mesh connection operates on NETDB_CHANNEL(70) and the third argument callbacks contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
..<br />
...<br />
....<br />
}<br />
</source><br />
<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
Below is the network window for the simulation. In the below network, Mote 51 is the antelope netdb client and Mote 4 is the server.<br />
<br />
[[File:Antelope_network.png]]<br />
<br />
*Mote output.<br />
Below is the mote output window. A filter "ID:4$" is being employed at the output window so that only mote 4 information is displayed in the mote output window(selective filtering).<br />
<br />
[[File:Antelope_ID4.png]]<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
Below is the interface window for the antelope database client(Mote 51). We use this window to send queries to server and print the query replies.<br />
[[File:Antelope_interface.png]]<br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.<br />
<br />
Edited by Gopi.<br />
[[Contiki_tutorials | Back to Contiki Tutorials]]</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=945Antelope(Database Management System) - Contiki2014-11-07T12:19:03Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system for resource-constrained memory devices. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*Query processor : It parses AQL(query language of Antelope) queries.<br />
*Privacy Control : Checks whether the query is allowed or not.<br />
*Logic VM : It executes the queries.<br />
*Database kernel : It holds the database logic and coordinates query execution.<br />
*Index abstraction : It holds the indexing logic.<br />
*Indexer process : Builds indexes from existing data.<br />
*Storage abstraction : Contains all storage logic.<br />
*Result transformer : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes netdb_process(which is the database process), shell_process. "shell_process" is the process which provides serial interface at the client to input our queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread assosicated with this process will start running by the scheduler. Below is the code snipped of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callabacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow this link.<br />
<br />
mesh_open opens a new connection and here this mesh connection operates on NETDB_CHANNEL(70) and the third argument callbacks contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
..<br />
...<br />
....<br />
}<br />
</source><br />
<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
[[File:Antelope_network.png]]<br />
*Mote output.<br />
[[File:Antelope_ID4.png]]<br />
*Mote Interface Viewer(Sky 51).<br />
[[File:Antelope_interface.png]]<br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=File:Antelope_network.png&diff=944File:Antelope network.png2014-11-07T12:07:35Z<p>Marella: </p>
<hr />
<div></div>Marellahttp://anrg.usc.edu/contiki/index.php?title=File:Antelope_interface.png&diff=943File:Antelope interface.png2014-11-07T12:07:24Z<p>Marella: </p>
<hr />
<div></div>Marellahttp://anrg.usc.edu/contiki/index.php?title=File:Antelope_ID4.png&diff=942File:Antelope ID4.png2014-11-07T12:07:02Z<p>Marella: </p>
<hr />
<div></div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=940Antelope(Database Management System) - Contiki2014-11-07T12:05:25Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system for resource-constrained memory devices. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*Query processor : It parses AQL(query language of Antelope) queries.<br />
*Privacy Control : Checks whether the query is allowed or not.<br />
*Logic VM : It executes the queries.<br />
*Database kernel : It holds the database logic and coordinates query execution.<br />
*Index abstraction : It holds the indexing logic.<br />
*Indexer process : Builds indexes from existing data.<br />
*Storage abstraction : Contains all storage logic.<br />
*Result transformer : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes netdb_process(which is the database process), shell_process. "shell_process" is the process which provides serial interface at the client to input our queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread assosicated with this process will start running by the scheduler. Below is the code snipped of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callabacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow this link.<br />
<br />
mesh_open opens a new connection and here this mesh connection operates on NETDB_CHANNEL(70) and the third argument callbacks contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
..<br />
...<br />
....<br />
}<br />
</source><br />
<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
In this folder, there exists a simulation file "netdb.csc" which we are going to use to simulate antelope in cooja simulator.<br />
<br />
Navigate to /tools/cooja/ from the contiki home directory.<br />
:: <code>cd /tools/cooja/</code><br />
Type the following command.<br />
:: <code>sudo ant run</code><br />
This command will bring up the cooja simulator (Follow this tutorial to learn more about cooja).<br />
From the cooja simulator, click File->open simulation -> Browse. Navigate to /examples/antelope/netdb and select "netdb.csc" file and click on open.<br />
Your screen should look something like below.<br />
<br />
*Network Window.<br />
<br />
*Mote output.<br />
<br />
*Mote Interface Viewer(Sky 51).<br />
<br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=932Antelope(Database Management System) - Contiki2014-11-07T11:46:03Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system for resource-constrained memory devices. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*Query processor : It parses AQL(query language of Antelope) queries.<br />
*Privacy Control : Checks whether the query is allowed or not.<br />
*Logic VM : It executes the queries.<br />
*Database kernel : It holds the database logic and coordinates query execution.<br />
*Index abstraction : It holds the indexing logic.<br />
*Indexer process : Builds indexes from existing data.<br />
*Storage abstraction : Contains all storage logic.<br />
*Result transformer : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes netdb_process(which is the database process), shell_process. "shell_process" is the process which provides serial interface at the client to input our queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread assosicated with this process will start running by the scheduler. Below is the code snipped of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callabacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow this link.<br />
<br />
mesh_open opens a new connection and here this mesh connection operates on NETDB_CHANNEL(70) and the third argument callbacks contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data)<br />
{<br />
..<br />
...<br />
....<br />
}<br />
</source><br />
<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.</div>Marellahttp://anrg.usc.edu/contiki/index.php?title=Antelope(Database_Management_System)_-_Contiki&diff=931Antelope(Database Management System) - Contiki2014-11-07T11:44:54Z<p>Marella: </p>
<hr />
<div>[[Contiki_tutorials | Back to Contiki Tutorials]]<br />
<br />
__TOC__<br />
<br />
== Introduction ==<br />
Antelope is a database management system for resource-constrained memory devices. It is a relational database management system(provides a set of relational database operations) which enables dynamic creation of databases and complex data querying.<br />
<br />
== You will learn ==<br />
<br />
In this tutorial, we briefly study how antelope is implemented in Contiki operating system by going through some of the basic and important code blocks. We will also study how to run database client and server in cooja and get some hands on experience.<br />
<br />
== Antelope architecture ==<br />
=== Components ===<br />
Antelope consists of eight components as shown in the below figure.<br />
<br />
[[File:Antelope arch.jpg|500px|left]]<br />
<br />
<br />
*Query processor : It parses AQL(query language of Antelope) queries.<br />
*Privacy Control : Checks whether the query is allowed or not.<br />
*Logic VM : It executes the queries.<br />
*Database kernel : It holds the database logic and coordinates query execution.<br />
*Index abstraction : It holds the indexing logic.<br />
*Indexer process : Builds indexes from existing data.<br />
*Storage abstraction : Contains all storage logic.<br />
*Result transformer : It presents the result of query in a way that makes it easy to use by programs.<br />
<br />
<br />
<br />
<br />
=== Terminology ===<br />
Antelope uses the standard terminology from the relational database management system. Below are some of the important terms:<br />
*Tuple: It is a set of attribute values.<br />
*Attribute: Different properties. E.g SensorID, Time.<br />
*Domain: Each attribute has got a domain that specifies the data type of the attribute values.<br />
*Relation: It is a collection of tuples that have the same set of attributes.<br />
*Primary Key: One of the attribues, which uniquely identifies the tuple in a relation.<br />
*Cardinality: The number of tuples in a relation is called the cardinality of the relation.<br />
<br />
To put the above terminology in easy terms, informally we can say :<br />
* A relation is like a table, attribute is like a column and a tuple is like a row. Below is an example.<br />
[[File:Example Antelope terminology.png|jpg|500px|center]]<br />
<br />
== Source Code ==<br />
<br />
Navigate to the examples/antelope/netdb folder from the contiki home directory.<br />
:: <code>cd /examples/antelope/netdb</code><br />
<br />
There are two important files in this(netdb) subfolder.<br />
*netdb-server.c<br />
*netdb-client.c<br />
<br />
There is one more file netdb.csc which we are going to use for our simulation on cooja.<br />
<br />
Let's walk through the source files before we start the simulation.<br />
<br />
=== Code building blocks ===<br />
==== netdb-client.c ====<br />
This client program provides a serial line interface with the help of which we can enter our queries. The shell process of the client program thse serial messages and sends the queries to the server program. Once it receives the reply to the query, it prints out a confirmation message in the shell.<br />
<br />
Let's walk through the source file.<br />
<br />
<source lang="c"><br />
#ifndef SERVER_ID<br />
#define SERVER_ID 4<br />
#endif<br />
</source><br />
<br />
SERVER_ID is used to send queries to server. Here, it is set to 4 because netdb-server.c file is uploaded onto the mote with id 4 in the netdb.csc(cooja simulation file given in the examples/antelope/netdb folder).<br />
<br />
<source lang="c"><br />
static unsigned server_id = SERVER_ID;<br />
static struct mesh_conn mesh;<br />
</source><br />
<br />
Last line in the above source code snippet creates a mesh_conn structure to store the details about the connection between the client and server. Antelope uses the Rime mesh routing protocol to exchange the messages between the client and the server. Below is another important structure "mesh_callbacks" which contains call back functions which we are implemented in netdb_client program file.<br />
<br />
<source lang="c"><br />
struct mesh_callbacks {<br />
/** Called when a packet is received. */<br />
void (* recv)(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops);<br />
/** Called when a packet, sent with mesh_send(), is actually transmitted. */<br />
void (* sent)(struct mesh_conn *c);<br />
/** Called when a packet, sent with mesh_send(), times out and is dropped. */<br />
void (* timedout)(struct mesh_conn *c);<br />
};<br />
</source><br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
PROCESS(shell_process, "Shell Process");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
</source><br />
The first two lines in the above source code creates two processes netdb_process(which is the database process), shell_process. "shell_process" is the process which provides serial interface at the client to input our queries. AUTOSTART_PROCESSES(&netdb_process) starts the netdb_process, means thread assosicated with this process will start running by the scheduler. Below is the code snipped of the thread for this process.<br />
<br />
<source lang="c"><br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&shell_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
This will create a callabacks structure which contains the function pointers to the callback functions which are used upon a packet arrival.<br />
<br />
A protothread is created for netdb_process. An exit handler is declared for the netdb_process. PROCESS_BEGIN() and PROCESS_END() are required inside a proto thread declaration. To know more about the Contiki Processes and Protothreads, please follow this link.<br />
<br />
mesh_open opens a new connection and here this mesh connection operates on NETDB_CHANNEL(70) and the third argument callbacks contains the function pointers to functions which will be called when a packet arrives on the channel. process_start(&shell_process, NULL) starts the shell process to provide the serial line interface(This is an example of starting one process from another instead of using AUTOSTART_PROCESSES).<br />
<br />
===== Callback handlers for the netdb_client process mesh connection =====<br />
<source lang="c"><br />
static const struct mesh_callbacks callbacks = {received, sent, timedout};<br />
</source><br />
<br />
*Callback function - sent<br />
<source lang="c"><br />
static void<br />
sent(struct mesh_conn *c)<br />
{<br />
}<br />
</source><br />
This function gets called when we try send a packet with mesh_send() and it is actually transmitted.<br />
<br />
*Callback function - timedout<br />
<source lang="c"><br />
static void<br />
timedout(struct mesh_conn *c)<br />
{<br />
printf("Failed to send packet: time out\n");<br />
}<br />
</source><br />
<br />
This function gets called when we try to send a packet with mesh_send() and the packet is dropped because of timeout. Here we are just printing "Failed to send the packet: timeout".<br />
<br />
*Callback function - received<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char reply[MAX_QUERY_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_QUERY_SIZE) {<br />
printf("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(reply, data, len);<br />
reply[len] = '\0';<br />
<br />
printf("%lu Reply received from %d.%d (%d hops): %s",<br />
clock_time(), from->u8[0], from->u8[1], (int)hops, reply);<br />
}<br />
</source><br />
This function is called when a packet is received. In the function handler "received" in netdb_client process, "c" denotes the mesh connection for the channel, "from" is the address of the server from which the packet is received and "hops" gives us the hop count between the client and server.<br />
Inside this function, one error condition is checked (len > MAX_QUERY_SIZE) and prints out an error for too long query if the condition evaluates to true. If there is nothing wrong with the query reply, reply is printed to the serial line interface.<br />
<br />
Let's look at the source code for the "shell_process".<br />
<source lang="c"><br />
PROCESS_THREAD(shell_process, ev, data)<br />
{<br />
rimeaddr_t addr;<br />
<br />
PROCESS_BEGIN();<br />
<br />
printf("NetDB client\n");<br />
<br />
for(;;) {<br />
PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);<br />
if(strncmp(data, "server ", 7) == 0) {<br />
server_id = atoi((char *)data + 7);<br />
} else {<br />
printf("%lu Transmitting query \"%s\" to node %u\n", clock_time(), (char *)data, server_id);<br />
packetbuf_copyfrom(data, strlen(data));<br />
addr.u8[0] = server_id;<br />
addr.u8[1] = 0;<br />
packetbuf_set_attr(PACKETBUF_ATTR_PACKET_TYPE,<br />
PACKETBUF_ATTR_PACKET_TYPE_STREAM);<br />
mesh_send(&mesh, &addr);<br />
}<br />
}<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
<br />
A proto thread is created for the process shell_start. Inside an infinite loop, we wait for an event to occur. There is a macro in the infinite for loop "PROCESS_WAIT_EVENT_UNTIL(ev == serial_line_event_message && data != NULL);" which makes the thread to wait for serial line event message to occur. Serial line event message here defines user entering a some information(data should not be NULL) from the serial line interface. If there is a server in the data(user entered data from the shell serial line interface), server id is copied from the data and "server_id" variable is set to this value, else the data is considered as a query message and the addr is set to the server's address and mesh_send function is called to send the packet to the server using mesh routing protocol.<br />
<br />
<br />
==== netdb-server.c ====<br />
Most of the source code for the netdb_server file is same as that of the above explained netdb_client source file like the mesh_conn data structure, callback functions and PROCESS and PROTO_THREADS. Below are some of the important code blocks which we will go into details and try to understand.<br />
<br />
<source lang="c"><br />
PROCESS(netdb_process, "NetDB");<br />
AUTOSTART_PROCESSES(&netdb_process);<br />
<br />
PROCESS_THREAD(netdb_process, ev, data)<br />
{<br />
PROCESS_EXITHANDLER(mesh_close(&mesh));<br />
PROCESS_BEGIN();<br />
<br />
mesh_open(&mesh, NETDB_CHANNEL, &callbacks);<br />
process_start(&query_process, NULL);<br />
<br />
PROCESS_END();<br />
}<br />
</source><br />
Creation of NetDB process and the protothread for this process is shown in the above code snippet. Inside the protothread of netdb_process we start another process "query_process". Callback functions "sent" and "timedout" for the server program file are implemented exactly like client's callback functions. Only "received" callback function differs in the implementation. Below is the implementation of the "received" call back function at the server side.<br />
<br />
<source lang="c"><br />
static void<br />
received(struct mesh_conn *c, const rimeaddr_t *from, uint8_t hops)<br />
{<br />
char *data;<br />
unsigned len;<br />
static char query[MAX_BUFFER_SIZE + 1];<br />
<br />
data = (char *)packetbuf_dataptr();<br />
len = packetbuf_datalen();<br />
<br />
if(len > MAX_BUFFER_SIZE) {<br />
buffer_db_data("Too long query: %d bytes\n", len);<br />
return;<br />
}<br />
<br />
memcpy(query, data, len);<br />
query[len] = '\0';<br />
<br />
printf("Query received from %d.%d: %s (%d hops)\n",<br />
from->u8[0], from->u8[1], query, (int)hops);<br />
rimeaddr_copy(&reply_addr, from);<br />
<br />
process_post(&query_process, serial_line_event_message, query);<br />
}<br />
</source><br />
This function get called when a packet is received on the channel. Here, sanity check(like length of the incoming query crossing the maximum buffer size) on incoming query takes place and "query_process" is called using "process_post" method to process the query and event type is being passed as "serial_line_event_message".<br />
<br />
Let's look at the query_process which plays an important role in processing the incoming queries.<br />
<source lang="c"><br />
PROCESS(query_process, "Query process");<br />
PROCESS_THREAD(query_process, ev, data);<br />
</soruce><br />
<br />
<br />
== Cooja Simulation-Antelope ==<br />
In this section, we will go over how to simulate Antelope database behavior in Cooja using sky motes.<br />
<br />
== References ==<br />
[https://www.sics.se/~nvt/papers/tsiftes11database.pdf 1] - A database in every sensor network, Nicolas Tsiftes ,Adam Dunkels Swedish Institute of Computer Science, Kista, Sweden.</div>Marella