ECMusic

An Android App

The aim of this project is to build a mediaPlayer application on Android Phone. Apart from the basic mediaplayer functions including play, pause, continue, stop, previous and next song, it can automatically play a song according to user’s emotion. It could also display it’s spectrum both on screen and on WigWag filaments.

Overview

Details

ECG & Body Temp Sensors + Mathematics Modeling

The first part of this project is emotion detection.
We selected one of the mathematics models for emotion detection, which retrieves heart rate and body temperature from sensors and return possible emotion.
Emotions are defined in a multi-dimensional space of emotion attributes. A popular conception uses a valence-arousal plane. Valence defines whether the emotion is positive or negative, and to what degree. Arousal defined the intensity of the emotion, ranging from calm(lowest value)to excited(highest value).
Input data is simulated by a simulator and retrieved through shimmer sensor.


Android App

We build an application on Android studio. The main page lists all song titles within SDK card in android phone. Users are allowed to play any of them by clicking on the title. If we click the detect emotion button at the bottom, the bluetooth connection and data transmitting process between shimmer sensor and android phone would start.
When emotion is detected, the app would jump to a new page showing the detected emotion and randomly pick one of the songs from the corresponding music folder. For example, here we detected the emotion of user was happy, so we choose a song from folder happy. After 3 seconds, it jumps to the player page witch would then automatically play the selected song.


Database

Since one android phone can contact with merely one bluetooth device at the same time, and we have already had shimmer sensor connected to the phone. So we need to find another way to communicate between the phone and the WigWag Device in Linux System. So we build a database and two php file for saving and retrieving data. As the song plays, frequency info of the song would be constantly stored to database by calling a php file. Then in the Linux System, we retrieve frequency info by calling another php file.


Filament

The main task in Linux system is to retrieve the data as crystal said, and perform the spectrum on 6 filament. These 6 filaments in the picture are arranged from low frequency to high frequency. In order to show the value on spectrum changing obviously, thresholds are not even distributed.

References

1. GOUIZI, K., BEREKSI REGUIG, F., & MAAOUI, C. (2011). Emotion recognition from physiological signals. Journal of Medical Engineering & Technology, 35(6-7), 300-307. doi:10.3109/03091902.2011.601784

2. Koelstra, S., Muhl, C., Soleymani, M., Lee, J., Yazdani, A., Ebrahimi, T., et al. (2012). DEAP: A database for emotion analysis ;using physiological signals. IEEE Transactions on Affective Computing, 3(1), 18-31. doi:10.1109/T-AFFC.2011.15

3. Lee, C., Yoo, S. K., Park, Y., Kim, N., Jeong, K., & Lee, B. (2005). Using neural network to recognize human emotions from heart rate variability and skin resistance. Conference Proceedings : ...Annual International Conference of the IEEE Engineering in Medicine and Biology Society.IEEE Engineering in Medicine and Biology Society.Annual Conference, 5, 5523. Lisetti, C. L., & Nasoz, F. (2004). Using noninvasive wearable computers to recognize human emotions from physiological signals. EURASIP Journal on Applied Signal Processing, 2004(11), 1672-1687. doi:10.1155/S1110865704406192

4. Quintana, D. S., Guastella, A. J., Outhred, T., Hickie, I. B., & Kemp, A. H. (2012). Heart rate variability is associated with emotion recognition: Direct evidence for a relationship between the autonomic nervous system and social cognition. International Journal of Psychophysiology : Official Journal of the International Organization of Psychophysiology, 86(2), 168-172. doi:10.1016/j.ijpsycho.2012.08.012

5. Valderas, M. T., Bolea, J., Laguna, P., Vallverdu, M., & Bailon, R. (2015). Human emotion recognition using heart rate variability analysis with spectral bands based on respiration. Paper presented at the , 2015. pp. 6134-6137. doi:10.1109/EMBC.2015.7319792

6. Valenza, G., Citi, L., Lanatá, A., Scilingo, E. P., & Barbieri, R. (2014). Revealing real-time emotional responses: A personalized assessment based on heartbeat dynamics. Scientific Reports, 4, 4998. doi:10.1038/srep04998

7. Wen, W., Liu, G., Cheng, N., Wei, J., Shangguan, P., & Huang, W. (2014). Emotion recognition based on multi-variant correlation of physiological signals. IEEE Transactions on Affective Computing, 5(2), 126-140. doi:10.1109/TAFFC.2014.2327617

Contact Me

Rui Zhu: crystaltrojans@gmail.com
MengQian Huang: mengqiah@usc.edu