TY - CONF

T1 - A Reinforcement Learning QoS negotiation Model for IoT middleware

T2 - 5th International Conference on Internet of Things: Systems, Management and Security, IoTSMS 2018

AU - Udoh, I.S.

AU - Kotonya, G.

AU - Institute for Systems and Technologies of Information, Control and Communication (INSTICC)

A2 - G., Wills

A2 - P., Kacsuk

A2 - V., Chang

N1 - Conference code: 160387 Export Date: 1 September 2020 References: Patel, K. K., Patel, S. M., Internet of Things-IoT: Definition, characteristics, architecture, enabling technologies, application & future challenges (2016) International Journal of Engineering Science and Computing, 6 (5), pp. 6122-6131; Duan, R., Chen, X., Xing, T., A QoS architecture for IoT (2011) 2011 International Conference on Internet of Things and 4th International Conference on Cyber, Physical and Social Computing, pp. 717-720; White, G., Nallur, V., Clarke, S., Quality of service approaches in IoT: A systematic mapping (2017) Journal of Systems and Software, pp. 186-203; Thoma, M., Meyer, S., Sperner, K., Meissner, S., Braun, T., On IoT-services: Survey, Classification and Enterprise Integration (2012) 2012 IEEE International Conference on Green Computing and Communications, pp. 257-260; Razzaque, M. A., Milojevic-Jevric, M., Palade, A., Clarke, S., Middleware for Internet of Things: A Survey (2015) IEEE Internet of Things Journal, 3 (1), pp. 70-95; Issarny, V., Bouloukakis, G., Georgantas, N., Billet, B., Revisiting Service-Oriented Architecture for the IoT: A Middleware Perspective (2016) International Conference on Service-Oriented Computing, pp. 1-16; Bala, M. I., Chishti, M. A., A model to incorporate automated negotiation in IoT (2017) 2017 IEEE International Conference on Advanced Networks and Telecommunications Systems (ANTS), pp. 1-4; Ghumman, W. A., Schill, A., The Flip-Flop SLA Negotiation Strategy Using Concession Extrapolation and 3D Utility Function (2016) 2016 IEEE 2nd International Conference on Collaboration and Internet Computing (CIC), pp. 159-168. , Lï¿1/2ssig, J; Zheng, X., Martin, P., Brohman, K., Da Xu, L., Cloud Service Negotiation in Internet of Things Environment: A Mixed Approach (2014) IEEE Transactions on Industrial Informatics, 10 (2), pp. 1506-1515; Alanezi, K., Mishra, S., A privacy negotiation mechanism for IoT (2018) 2018 IEEE 16th Intl Conf on Dependable, Autonomic and Secure Computing, 16th Intl Conf on Pervasive Intelligence and Computing, 4th Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress(DASC/PiCom/DataCom/CyberSciTech), pp. 512-519; Li, F., Clarke, S., A Context-Based Strategy for SLA Negotiation in the IoT Environment (2019) 2019 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), pp. 208-213; Zulkernine, F. H., Martin, P., An adaptive and intelligent SLA negotiation system for web services (2011) IEEE Transactions on Services Computing, 4 (1), p. 31ï1243; Al-Aaidroos, M., Jailani, N., Mukhtar, M., Agent-based negotiation framework for web services SLA (2011) 2011 7th International Conference on Information Technology in Asia, pp. 1-7; Pouyllau, H., Carofiglio, G., Inter-carrier SLA negotiation using Q-learning (2013) Telecommunication Systems, 52 (2), pp. 611-622; Besanko, D., Braeutigam, R. R., (2010) Microeconomics, , and Wiley. New Jersey, USA, 4th edition; Parkin, M., Kuo, D., Brooke, J., A Framework and Negotiation Protocol for Service Contracts (2006) 2006 IEEE International Conference on Services Computing (SCC'06), pp. 253-256; Aydoan, R., Festen, D., Hindriks, K. V., Jonker, C. M., Alternating Offers Protocols for Multilateral Negotiation (2017) Modern Approaches to Agent-based Complex Automated Negotiation. Studies in Computational Intelligence, pp. 153-167. , Springer, Switzerland; Zheng, X., Martin, P., Brohman, K., Cloud service negotiation: Concession vs. tradeoff approaches (2012) Proc. 12th IEEE/ACM Int. Symp. Cluster, Cloud Grid Comput. (CCGrid 2012), , 515ï¿1/2 522. Puterman, M. L. (2005); Markov Decision Processes: Discrete Stochastic Dynamic Programming, , Wiley & Sons. New Jersey, USA; Schwartz, H. M., (2014) Multi-Agent Machine Learning: A Reinforcement Approach, , Wiley & Sons. New Jersey, USA. Alpaydin, E. (2014). Introduction to Machine Learning, MIT Press. Cambridge, MA, USA, 3rd edition; Russell, S., Norvig, P., (2014) Artificial Intelligence: A Modern Approach, , Pearson; Essex, England, Bellifemine, F. L., Caire, G., Greenwood, D., (2007) Developing Multi-Agent Systems with JADE, , 3rd edition and, Wiley & Sons. England; Villalonga, C., Bauer, M., Aguilar, F. L., Huang, V. A., Strohbach, M., A resource model for the real world internet (2010) European Conference on Smart Sensing and Context, , 163ï¿1/2176

PY - 2020/5/9

Y1 - 2020/5/9

N2 - A large number of heterogeneous and mobile devices interacting with each other, leading to the execution of tasks with little human interference, characterizes the Internet of Things (IoT) ecosystem. This interaction typically occurs in a service-oriented manner facilitated by an IoT middleware. The service provision paradigm in the IoT dynamic environment requires a negotiation process to resolve Quality of Service (QoS) contentions between heterogeneous devices with conflicting preferences. This paper proposes a negotiation model that allows negotiating agents to dynamically adapt their strategies using a model-based reinforcement learning as the QoS preferences evolve and the negotiation resources changes due to the changes in the physical world. We use a simulated environment to illustrate the improvements that our proposed negotiation model brings to the QoS negotiation process in a dynamic IoT environment. Copyright © 2020 by SCITEPRESS - Science and Technology Publications, Lda. All rights reserved.

AB - A large number of heterogeneous and mobile devices interacting with each other, leading to the execution of tasks with little human interference, characterizes the Internet of Things (IoT) ecosystem. This interaction typically occurs in a service-oriented manner facilitated by an IoT middleware. The service provision paradigm in the IoT dynamic environment requires a negotiation process to resolve Quality of Service (QoS) contentions between heterogeneous devices with conflicting preferences. This paper proposes a negotiation model that allows negotiating agents to dynamically adapt their strategies using a model-based reinforcement learning as the QoS preferences evolve and the negotiation resources changes due to the changes in the physical world. We use a simulated environment to illustrate the improvements that our proposed negotiation model brings to the QoS negotiation process in a dynamic IoT environment. Copyright © 2020 by SCITEPRESS - Science and Technology Publications, Lda. All rights reserved.

KW - Internet of Things

KW - IoT Services

KW - Negotiation

KW - QoS

KW - Reinforcement Learning

KW - Big data

KW - Learning systems

KW - Middleware

KW - Quality of service

KW - Reinforcement learning

KW - Conflicting preferences

KW - Dynamic environments

KW - Heterogeneous devices

KW - Internet of thing (IOT)

KW - Model-based reinforcement learning

KW - Negotiation process

KW - Service provisions

KW - Simulated environment

KW - Internet of things

M3 - Conference paper

SP - 205

EP - 212

Y2 - 15 October 2018 through 18 October 2018

ER -