Context aware Q-Learning-based model for decision support in the negotiation of energy contracts. (January 2019)

• Record Type: Journal Article
• Authors: Rodriguez-Fernandez, J.; Pinto, T.; Silva, F.; Praça, I.; Vale, Z.; Corchado, J.M.
• Is Part Of: International journal of electrical power & energy systems. Volume 104(2019)
• Page Start: 489
• Page End: 501
• Publication Date: 2019-01
• Subjects: Automated negotiation -- Bilateral contracts -- Context awareness -- Decision support -- Electricity markets -- Reinforcement learning algorithm; Electrical engineering -- Periodicals; Electric power systems -- Periodicals; Électrotechnique -- Périodiques; Réseaux électriques (Énergie) -- Périodiques; Electric power systems; Electrical engineering; Periodicals; 621.3
• DOI: 10.1016/j.ijepes.2018.06.050
• Languages: English
• ISSNs: 0142-0615

Abstract

Highlights: New methodology based on Q-Learning, adapted to learn under different contexts. A new dimension, representing the context, is included in Q-Learning to complement the standard action vs state model. Proposed method enables learning the most likely scenario to be met in bilateral energy contracts negotiations. Integration in decision support system for energy contracts negotiation. Results show that context aware learning outperforms standard reinforcement learning. Abstract: Automated negotiation plays a crucial role in the decision support for bilateral energy transactions. In fact, an adequate analysis of past actions of opposing negotiators can improve the decision-making process of market players, allowing them to choose the most appropriate parties to negotiate with in order to increase their outcomes. This paper proposes a new model to estimate the expected prices that can be achieved in bilateral contracts under a specific context, enabling adequate risk management in the negotiation process. The proposed approach is based on an adaptation of the Q-Learning reinforcement learning algorithm to choose the best scenario (set of forecast contract prices) from a set of possible scenarios that are determined using several forecasting and estimation methods. The learning process assesses the probability of occurrence of each scenario, by comparing each expected scenario with the real scenario. The final chosen scenario is the one that presents the higher expected utility value. Besides, the learning method can determine which is the best scenario for each context, since the behaviour of players can change according to the negotiation environment. Consequently, these conditions influence the final contract price of negotiations. This approach allows the supported player to be prepared for the negotiation scenario that is the most probable to represent a reliable approximation of the actual negotiation environment.