Difference between revisions of "CFS-Coffee"

Revision as of 20:16, 27 February 2014

Permanent Storage of Contiki on Tmote Sky (CFS-Coffee)

Contiki provides File System to achieve the goal of permanent storage. The Contiki File System(CFS) works as a Virtual File System and provides some interfaces for different file systems. CFS-POSIX and Coffee are two with full functionalities. CFS-POSIX is used in Contiki platforms that runs in native mode. Coffee, on the other hand, is primarily aimed at sensor devices that equiped with Flash Memories or EEPROM.

The CFS Programming Interface

<code>int cfs_open(const char* name, int flags)</code>

Open a file with specific flags.

name: The name of the file want to be opened.

flags: The way the file will be opened: CFS_READ, CFS_WRITE, CFS_APPEND or their combiantion.

return value: A file descriptor, if the file could be opened, or -1 if the file could not be opened.

e.g: fd = cfs_open(“Contiki_Group”, CFS_READ | CFS_WRITE | CFS_APPEND)

Open a file called “Contiki_Group” which allow both to read and to write. Notice the “CFS_WRITE | CFS_APPEND” means append to the file. Only make use of CFS_APPEND will not achieve this goal. CFS_WRITE has to be included.

<code>void cfs_close(int fd)</code>

Close a open file.

fd: The file descriptor of the open file.

e.g: cfs_close(fd)

Close the file with file descriptor fd.

<code>int cfs_read(int fd, void* buf, unsigned int len)</code>

Read data from a open file and fills buf with at most len bytes, starting from the current position in the file descriptor.

fd: The file descriptor of the open file.

buf: The buffer in which data should be read from the file.

len: The number of bytes that should be read.

Return Value: The number of bytes that was actually read from the file or -1 if an error happens

e.g: ret = cfs_read(fd, buf, sizeof(buf))

Read from file with file descriptor fd and fill buf with at most its size. Here buf is a pointer. ret contains the numbers of bytes that read from the file or -1 if an error happens.

<code>int cfs_write(int fd, const void *buf, unsigned int len)</code>

Write len bytes from the memory buffer buf into the file, starting from the current position in the file descriptor.

fd: The file descriptor of the open file.

buf: The buffer from which data should be written to the file.

len: The number of bytes that should be written.

Return Value: The number of bytes that was actually written to the file.

e.g: ret = cfs_write(fd, buf, sizeof(buf))

Write to file with file descriptor fd from buffer buf with at most its size. Here buf is a pointer. ret contains the numbers of bytes that was actually written to the file.

<code>cfs_offset_t cfs_seek(int fd, cfs_offset_t offset, int whence)</code>

Move the current file position to the position determined by the combination of the offset and the whence.

fd: The file descriptor of the open file.

offset: A position, either relative or absolute, in the file.

whence: Determines how to interpret the offset parameter. CFS_SEEK_SET computes the offset from the beginning of the file; CFS_SEEK_CUR computes the offset from the current position of the file position; CFS_SEEK_END computes the offset in relation to the end of the file. Negative offset values are accepted by both CFS_SEEK_CUR and CFS_SEEK_END.

Return Value: The new absolute file position upon success, or -1 if the file pointer could not be moved to the requested position.

e.g: ret = cfs_seek(fd, 0, CFS_SEEK_SET)

Move the file position to the beginning of the file.

<code>int cfs_remove(const char *name)</code>

Remove a file with name name.

name: The name of the file.

Return Value: 0 if the file was removed or -1 if the file could not be removed or if it doesn’t exist.

e.g: ret = sfc_remove(name)

Remove the file with name pointed by name.

<code>int cfs_opendir(struct cfs_dir *dirp, const char *name)</code>

Open the directory name and fills in an opaque handle pointed to by dirp. The contents of this handle is unspecified and is only for use internally by CFS implementation. struct cfs_dir { char dummy_space[32]; } dirp: A pointer to a struct cfs_dir that is filled in by the function. name: The name of the directory. Return Value: 0 or -1 if the directory could not be opened. e.g: ret = cfs_opendir(&dir, "/") Open the root directory.

int cfs_readdir(struct cfs_dir *dirp, struct cfs_dirent *dirent) Read one entry of directroy dirp at a time. Write the directory entry into the space pointed to by dirent. dirent should be preallocated. struct cfs_dirent { char name[32]; cfs_offset_t size; } dirp: A pointer to a struct cfs_dir that has been opened with cfs_opendir(). dirent: A pointer to a struct cfs_dirent that is filled in by cfs_readdir(). Return Value: 0 or -1 if no more directory entries can be read. e.g: if (cfs_opendir(&dir, “/”) == 0) { while(cfs_readdir(&dir, &dirent) != -1) { printf(“File: %s (%ld bytes)\n”, dirent.name, (long)dirent.size); } cfs_closedir(&dir); } The first line opens the root directory. If it successes, the while() in the second line reads all the entries in root directory and prints the name and size of those entries. After that, close the root directory.

void cfs_closedir(struct cfs_dir *dirp) Close the dirctory dirp opened with cfs_opendir(). dirp: A pointer to a struct cfs_dir that has been opened with cfs_opendir().

Introduction to Coffee

Coffee is a fully functional file system designed specifically for the characteristics of flash memories and EEPROM. Coffee is usually used in sensor devices with flash (e.g Tmote Sky), so two principles are pretty important. First, the metadata stored in RAM for each file should be small because sensor devices have very limited RAM space. Second, bits in flash memories can be toggled from 1 to 0, but not toggled back from 0 to 1 without doing an expensive erase. Thus, when modifying a file, Coffee creates an invisible micro log which is linked to the original file. The micro log contains the lastest data of the file. For user, the micro log and the original file is just one logical file, as if they modify the original file. When the micro log eventually fills up, Coffee transparently merges the content of the original file and the micro log into a new file, and deletes the two former files. Coffee has a flat directory structure. It only has the root directory. Thus, cfs_opendir() only accept “/” or “.” as the second argument. When removing a file from a Coffee, there are two steps. First, external user calls cfs_remove(). The file mentioned by user will be marked as obsolete. Obsolete files cannot be seen by external users. Coffee will actually delete files only when a new file reservation request cannot be granted. The implementation of Coffee in core/cfs/cfs-coffee.c is totally platform independent. The specific configuration of Coffee for different platforms is written in csf-coffee-arch.c. We can see many csf-coffee-arch.c files in plarform/./csf-coffee-arch.c, e.g plarform/z1/csf-coffee-arch.c, plarform/esb/csf-coffee-arch.c, plarform/sky/csf-coffee-arch.c. When we decide the platform, such as TARGET=sky, we compile plarform/sky/csf-coffee-arch.c and ignore all other csf-coffee-arch.c files. Macro definitions in plarform/sky/csf-coffee-arch.c configure the details of coffee in Sky Mote. Macro definition contains parameters like COFFEE_SECTOR_SIZE, COFFEE_PAGE_SIZE, COFFEE_FD_SET_SIZE, COFFEE_MICRO_LOGS and so on. Also, it defines COFFEE_WRITE, COFFEE_READ and COFFEE_ERASE to point to the device drivers I/O function.

CFS-Coffee Interface Extensions

int cfs_coffee_format(void) Coffee formats the underlying storage by setting all bits to zero. Formatting must be done before using Coffee for the first time in a mote. Return Value: 0 on success, -1 on failure.

int cfs_coffee_reserve(const char *name, cfs_offset_t size) Reserve space for a file of name with size. Return Value: 0 on success, -1 on failure.

int cfs_coffee_configure_log(const char *file, unsigned log_size, unsigned log_entry_size) Configure the on-demand log file. Return Value: 0 on success, -1 on failure.

cfs_coffee_format() may take a few second because all sectors must be erased. cfs_coffee_reserve() and int cfs_coffee_configure_log() can optimize Coffee’s handling of the file.

Here is an example of how everything works. Suppose we want to run Contiki on Sky Mote. First make sure TARGET=sky. Then every file about sky is compiled and you can also see a new folder which is created called sky_obj. Inside contains everything you need to run on Sky Mote. Suppose you want to use Coffee as the File System. Write cfs_coffee_format() before the first time you try to operate on Coffee. This function notice the system that you want to use Coffee and the system will do some formatting. Let’s say then you want to read something from an existing file with the function call cfs_read(). This function is defined in core/cfs/cfs.h. Because we make use of Coffee, the implementation of this function will be in core/cfs/cfs-coffee.c. The function cfs_read() actually calls the macro definition COFFEE_READ(). The definition of COFFEE_READ() is in cfs-coffee-arch.c. Since we make TARGET=sky, the one that was compiled and used by us is platform/sky/cfs-coffee-arch.c. This file defines COFFEE_READ() to point to Sky Mote drivers I/O function. Finally, the external users’ cfs_read() gets to the Sky Driver I/O function.

An Example of Running Coffee on Tmote Sky This example shows the basic way to open, read, write and append files in Coffee. The platform we choose here is Tmote Sky. First, open an example provided by Contiki (/home/user/contiki-2.7/example/sky/example- coffee.c). Here, it defines two test functions: static int file_test(const char *filename, char *msg) opens a file named FILENAME with CFS_WRITE | CFS_APPEND | CFS_READ. It appends the opening file with records and prints out everything in the file. static int dir_test(void) opens the root directory and prints out all entries of root. The Process calls file_test() twice and dir_test() once. Then the process ends. Now, we do some modifications based on this example. First we add a function static int read_test(void). The implementation of this function is shown as below: static int read_test(void) { struct cfs_dir dir; struct cfs_dirent dirent;

int fd; int r;

struct record {

   		char message[16];
   		uint8_t sequence;
 	} record;

static int i = 0; if (cfs_opendir(&dir, "/") != 0) { printf("Fail to open the root directory\n"); return -1; } while(cfs_readdir(&dir, &dirent) == 0) { i ++; printf("\nFile name: %s\n", dirent.name); if ((fd =cfs_open(dirent.name, CFS_READ)) != -1) { cfs_seek(fd, 0, CFS_SEEK_SET); for(;;) {

   				r = cfs_read(fd, &record, sizeof(record));
   				if(r == 0) {
     					break;
   				} else if(r > sizeof(record)) {
     					printf("failed to read %d bytes from %s, got %d\n",(int)sizeof(record), dirent.name, r);
     					cfs_close(fd);
     					return -1;
   				}

   				printf("Read message \"%s\", sequence %u\n",record.message, record.sequence);
 			}

} }

if (i == 0) { printf("\nNo file exists.\n"); return 0; } else { return 1; }

}

The function of read_test() is quite simple. It opens the root file system and reads all the data in the file system. In the while() loop, it reads an entry of a file. Then it makes use of the entry to open the file and read all the data out.

Also, in the process, we add an etimer at the beginning. The process can only move on either when the timer expired or someone push a button. If the timer expire, the process will run cfs_coffee_format(), which will set all the flash to 1. Otherwise, we don’t do this. Then we call read_test(), which read all the data in the file system. After that, we check whether there are files exist in the system. If there is, we give up wirting. Otherwise, we create two files and write some data in. The process then looks like:

PROCESS_THREAD(example_coffee_process, ev, data) {

 PROCESS_BEGIN();

static int flag = 0; button_sensor.configure(SENSORS_ACTIVE, 1);

static struct etimer ak; etimer_set(&ak, CLOCK_SECOND * 6);

PROCESS_WAIT_EVENT_UNTIL(ev == PROCESS_EVENT_TIMER || ev == sensors_event && data == &button_sensor);

if (ev == PROCESS_EVENT_TIMER) cfs_coffee_format();

if ((flag = read_test()) == -1) PROCESS_EXIT();

static struct etimer et; etimer_set(&et, CLOCK_SECOND * 2); PROCESS_WAIT_EVENT_UNTIL(ev == PROCESS_EVENT_TIMER);

if (flag == 1) printf("\nData exists...No written again...\n"); else { printf("\nData begin to write...\n"); if(file_test(FILENAME_1, "The first test") == 0) {

   			printf("file test 1 failed\n");
 		}

if(file_test(FILENAME_1, "The second test") == 0) {

   			printf("file test 2 failed\n");

} if(file_test(FILENAME_2, "The third test") == 0) {

   			printf("file test 3 failed\n");

} if(dir_test() == 0) {

   			printf("dir test failed\n");
 		}

}

 PROCESS_END();

}

Now, connect your mote to your PC’s USB port and change directory. cd contiki-2.7/examples/sky

Compile this Coffee example to Sky Mote. sudo make TARGET=sky example-coffee.upload login

Once login, wait for 6 seconds patiently for the first time out.If it success, we should see terminal has the output like this: Here you can see, we create two files and write some data in. Then, disconnect your mote and do whatever you want. Maybe one day later, you come back. Now it’s the time to keep moving. Welcome back. Now connect your mote with USB port again and do: sudo make TARGET=sky login You are in the process again and please follow the following two steps: 1. push “RESET” button; 2. push “USER” button within 6 seconds. Now, the program will not call cfs_coffee_format(). If succeed, you will see: As you can see, we read out all the data that you wrote in one day ago. Everything is right there without any change.