List of Contributors xv Preface xxi Part I Spectrum Intelligence and Adaptive Resource Management 1 1 Machine Learning for Spectrum Access and Sharing 3 Kobi Cohen 1.1 Introduction 3 1.2 Online Learning Algorithms for Opportunistic Spectrum Access 4 1.2.1 The Network Model 4 1.2.2 Performance Measures of the Online Learning Algorithms 5 1.2.

3 The Objective 6 1.2.4 Random and Deterministic Approaches 6 1.2.5 The Adaptive Sequencing Rules Approach 7 1.2.5.1 Structure of Transmission Epochs 7 1.

2.5.2 Selection Rule under the ASR Algorithm 8 1.2.5.3 High-Level Pseudocode and Implementation Discussion 9 1.3 Learning Algorithms for Channel Allocation 9 1.3.

1 The Network Model 10 1.3.2 Distributed Learning, Game-Theoretic, and Matching Approaches 11 1.3.3 Deep Reinforcement Learning for DSA 13 1.3.3.1 Background on Q-learning and Deep Reinforcement Learning (DRL): 13 1.

3.4 Existing DRL-Based Methods for DSA 14 1.3.5 Deep Q-Learning for Spectrum Access (DQSA) Algorithm 15 1.3.5.1 Architecture of the DQN Used in the DQSA Algorithm 15 1.3.

5.2 Training the DQN and Online Spectrum Access 16 1.3.5.3 Simulation Results 17 1.4 Conclusions 19 Acknowledgments 20 Bibliography 20 2 Reinforcement Learning for Resource Allocation in Cognitive Radio Networks 27 Andres Kwasinski, Wenbo Wang, and Fatemeh Shah Mohammadi 2.1 Use of Q-Learning for Cross-layer Resource Allocation 29 2.2 Deep Q-Learning and Resource Allocation 33 2.

3 Cooperative Learning and Resource Allocation 36 2.4 Conclusions 42 Bibliography 43 3 Machine Learning for Spectrum Sharing in Millimeter-Wave Cellular Networks 45 Hadi Ghauch, Hossein Shokri-Ghadikolaei, Gabor Fodor, Carlo Fischione, and Mikael Skoglund 3.1 Background and Motivation 45 3.1.1 Review of Cellular Network Evolution 45 3.1.2 Millimeter-Wave and Large-Scale Antenna Systems 46 3.1.

3 Review of Spectrum Sharing 47 3.1.4 Model-Based vs. Data-Driven Approaches 48 3.2 System Model and Problem Formulation 49 3.2.1 Models 49 3.2.

1.1 Network Model 49 3.2.1.2 Association Model 49 3.2.1.3 Antenna and Channel Model 49 3.

2.1.4 Beamforming and Coordination Models 50 3.2.1.5 Coordination Model 50 3.2.2 Problem Formulation 51 3.

2.2.1 Rate Models 52 3.2.3 Model-based Approach 52 3.2.4 Data-driven Approach 53 3.3 Hybrid Solution Approach 54 3.

3.1 Data-Driven Component 55 3.3.2 Model-Based Component 56 3.3.2.1 Illustrative Numerical Results 58 3.3.

3 Practical Considerations 58 3.3.3.1 Implementing Training Frames 58 3.3.3.2 Initializations 59 3.3.

3.3 Choice of the Penalty Matrix 59 3.4 Conclusions and Discussions 59 Appendix A Appendix for Chapter 3 61 A.1 Overview of Reinforcement Learning 61 Bibliography 61 4 Deep Learning-Based Coverage and Capacity Optimization 63 Andrei Marinescu, Zhiyuan Jiang, Sheng Zhou, Luiz A. DaSilva, and Zhisheng Niu 4.1 Introduction 63 4.2 Related Machine Learning Techniques for Autonomous Network Management 64 4.2.

1 Reinforcement Learning and Neural Networks 64 4.2.2 Application to Mobile Networks 66 4.3 Data-Driven Base-Station Sleeping Operations by Deep Reinforcement Learning 67 4.3.1 Deep Reinforcement Learning Architecture 67 4.3.2 Deep Q-Learning Preliminary 68 4.

3.3 Applications to BS Sleeping Control 68 4.3.3.1 Action-Wise Experience Replay 69 4.3.3.2 Adaptive Reward Scaling 70 4.

3.3.3 Environment Models and Dyna Integration 70 4.3.3.4 DeepNap Algorithm Description 71 4.3.4 Experiments 71 4.

3.4.1 Algorithm Comparisons 71 4.3.5 Summary 72 4.4 Dynamic Frequency Reuse through a Multi-Agent Neural Network Approach 72 4.4.1 Multi-Agent System Architecture 73 4.

4.1.1 Cell Agent Architecture 75 4.4.2 Application to Fractional Frequency Reuse 75 4.4.3 Scenario Implementation 76 4.4.

3.1 Cell Agent Neural Network 76 4.4.4 Evaluation 78 4.4.4.1 Neural Network Performance 78 4.4.

4.2 Multi-Agent System Performance 79 4.4.5 Summary 81 4.5 Conclusions 81 Bibliography 82 5 Machine Learning for Optimal Resource Allocation 85 Marius Pesavento and Florian Bahlke 5.1 Introduction and Motivation 85 5.1.1 Network Capacity and Densification 86 5.

1.2 Decentralized Resource Minimization 87 5.1.3 Overview 88 5.2 System Model 88 5.2.1 Heterogeneous Wireless Networks 88 5.2.

2 Load Balancing 89 5.3 Resource Minimization Approaches 90 5.3.1 Optimized Allocation 91 5.3.2 Feature Selection and Training 91 5.3.3 Range Expansion Optimization 93 5.

3.4 Range Expansion Classifier Training 94 5.3.5 Multi-Class Classification 94 5.4 Numerical Results 96 5.5 Concluding Remarks 99 Bibliography 100 6 Machine Learning in Energy Efficiency Optimization 105 Muhammad Ali Imran, Ana Flávia dos Reis, Glauber Brante, Paulo Valente Klaine, and Richard Demo Souza 6.1 Self-Organizing Wireless Networks 106 6.2 Traffic Prediction and Machine Learning 110 6.

3 Cognitive Radio and Machine Learning 111 6.4 Future Trends and Challenges 112 6.4.1 Deep Learning 112 6.4.2 Positioning of Unmanned Aerial Vehicles 113 6.4.3 Learn-to-Optimize Approaches 113 6.

4.4 Some Challenges 114 6.5 Conclusions 114 Bibliography 114 7 Deep Learning Based Traffic and Mobility Prediction 119 Honggang Zhang, Yuxiu Hua, Chujie Wang, Rongpeng Li, and Zhifeng Zhao 7.1 Introduction 119 7.2 Related Work 120 7.2.1 Traffic Prediction 120 7.2.

2 Mobility Prediction 121 7.3 Mathematical Background 122 7.4 ANN-Based Models for Traffic and Mobility Prediction 124 7.4.1 ANN for Traffic Prediction 124 7.4.1.1 Long Short-Term Memory Network Solution 124 7.

4.1.2 Random Connectivity Long Short-Term Memory Network Solution 125 7.4.2 ANN for Mobility Prediction 128 7.4.2.1 Basic LSTM Network for Mobility Prediction 128 7.

4.2.2 Spatial-Information-Assisted LSTM-Based Framework of Individual Mobility Prediction 130 7.4.2.3 Spatial-Information-Assisted LSTM-Based Framework of Group Mobility Prediction 131 7.5 Conclusion 133 Bibliography 134 8 Machine Learning for Resource-Efficient Data Transfer in Mobile Crowdsensing 137 Benjamin Sliwa, Robert Falkenberg, and Christian Wietfeld 8.1 Mobile Crowdsensing 137 8.

1.1 Applications and Requirements 138 8.1.2 Anticipatory Data Transmission 139 8.2 ML-Based Context-Aware Data Transmission 140 8.2.1 Groundwork: Channel-aware Transmission 140 8.2.

2 Groundwork: Predictive CAT 142 8.2.3 ML-based CAT 144 8.2.4 ML-based pCAT 146 8.3 Methodology for Real-World Performance Evaluation 148 8.3.1 Evaluation Scenario 148 8.

3.2 Power Consumption Analysis 148 8.4 Results of the Real-World Performance Evaluation 149 8.4.1 Statistical Properties of the Network Quality Indicators 149 8.4.2 Comparison of the Transmission Schemes 149 8.4.

3 Summary 151 8.5 Conclusion 152 Acknowledgments 154 Bibliography 154 Part II Transmission Intelligence and Adaptive Baseband Processing 157 9 Machine Learning-Based Adaptive Modulation and Coding Design 159 Lin Zhang and Zhiqiang Wu 9.1 Introduction and Motivation 159 9.1.1 Overview of ML-Assisted AMC 160 9.1.2 MCS Schemes Specified by IEEE 802.11n 161 9.

2 SL-Assisted AMC 162 9.2.1 k -NN-Assisted AMC 162 9.2.1.1 Algorithm for k -NN-Assisted AMC 163 9.2.2 Performance Analysis of k -NN-Assisted AMC System 164 9.

2.3 SVM-Assisted AMC 166 9.2.3.1 SVM Algorithm 166 9.2.3.2 Simulation and Results 170 9.

3 RL-Assisted AMC 172 9.3.1 Markov Decision Process 172 9.3.2 Solution for the Markov Decision 173 9.3.3 Actions, States, and Rewards 174 9.3.

4 Performance Analysis and Simulations 175 9.4 Further Discussion and Conclusions 178 Bibliography 178 10 Machine Learning-Based Nonlinear MIMO Detector 181 Song-Nam Hong and Seonho Kim 10.1 Introduction 181 10.2 A Multihop MIMO Channel Model 182 10.3 Supervised-Learning-based MIMO Detector 184 10.3.1 Non-Parametric Learning 184 10.3.

2 Parametric Learning 185 10.4 Low-Complexity SL (LCSL) Detector 188 10.5 Numerical Results 191 10.6 Conclusions 193 Bibliography 193 11 Adaptive Learning for Symbol Detection: A Reproducing Kernel Hilbert Space Approach 197 Daniyal Amir Awan, Renato Luis Garrido Cavalcante, Masahario Yukawa, and Slawomir Stanczak 11.1 Introduction 197 11.2 Preliminaries 198 11.2.1 Reproducing Kernel Hilbert Spaces 198 11.

2.2 Sum Spaces of Reproducing Kernel Hilbert Spaces 199 11.3 System Model 200 11.3.1 Symbol Detection in Multiuser Environments 201 11.3.2 Detection of Complex-Valued Symbols in Real Hilbert Spaces 202 11.4 The Proposed Learning Algorithm 203 11.

4.1 The Canonical Iteration 203 11.4.2 Practical Issues 2.