Online Learning for Networks

Description

The arrival of cognitive computing has provided the networking community with a new design dimension to meet the exponentially increasing needs of data traffic. The essence is to allow networks and communication systems to adapt their settings by learning from operating environment. In this project, we focus on both algorithm design and system implementation by applying online learning to different networking problems.

Many problems in the context of cognitive computing filed, such as dynamic multi-channel access, can be formulated as a Partially Observable Markov Decision Problem (POMDP) (with known or unknown system dynamics) that is PSAPCE-hard. To overcome the challenge of computationally prohibition, we investigate the idea from Deep Reinforcement Learning field and experiment with Deep Q Networks that are promising to perform well in large and complicated systems.

People

PI/Director: Bhaskar Krishnamachari

Ph.D. Students:
Pranav Sakulkar
Shangxing Wang
Yi-Hsuan Kao
Pedro Henrique Gomes
Parisa Mansourifard
Yi Gai
Yanting Wu

Collaborators:
Tara Javidi (UCSD)
Mingyan Liu (U. Michigan)
Qing Zhao (Cornell)

Publications

1. Shangxing Wang, Hanpeng Liu, Pedro Henrique Gomes and Bhaskar Krishnamachari, “Deep Reinforcement Learning for Dynamic Multichannel Access”, under submission.
2. Pranav Sakulkar and Bhaskar Krishnamachari, “Online Learning of Power Allocation Policies in Energy Harvesting Communications“, International Conference on Signal Processing and Communications (SPCOM), 2016.
3. Wenhan Dai, Yi Gai, Bhaskar Krishnamachari “Online Learning for Multi-Channel Opportunistic Access over Unknown Markovian Channels“, IEEE International Conference on Sensing, Communication, and Networking (SECON), 2014.
4. Parisa Mansourifard, Farrokh Jazizadeh, Bhaskar Krishnamachari, Burcin Becerik-Gerber “Online Learning for Personalized Room-Level Thermal Control: A Multi-Armed Bandit Framework“, BuildSys 2013 Workshop, Rome, Italy.
5. Yanting Wu, Bhaskar Krishnamachari, “Online Learning to Optimize Transmission over an Unknown Gilbert-Elliott Channel“, 10th Intl. Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks, WiOpt 2012.
6. Yi Gai, Bhaskar Krishnamachari and Mingyan Liu, “Online Learning for Combinatorial Network Optimization with Restless Markovian Rewards“, the 9th Annual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks (SECON), Seoul, Korea, June, 2012.
7. Wenhan Dai, Yi Gai and Bhaskar Krishnamachari, “Efficient Online Learning for Opportunistic Spectrum Access“, IEEE International Conference on Computer Communications (IEEE INFOCOM Mini-conference 2012), Orlando, FL, USA, March, 2012.
8. Yi Gai and Bhaskar Krishnamachari, “Online Learning Algorithms for Stochastic Water-Filling“, Information Theory and Applications Workshop (ITA 2012), San Diego, USA, February, 2012.
9. Yi Gai and Bhaskar Krishnamachari, “Decentralized Online Learning Algorithms for Opportunistic Spectrum Access“, the IEEE Global Communications Conference (GLOBECOM 2011), Houston, USA, December, 2011.