Blockchain-Enabled Computing Resource Trading: A Deep Reinforcement Learning Approach

Driven by the vision of the Internet of Things (IoT) under the fifth-generation (5G) wireless network, computing resource trading attracts numerous attention from both academia and industry. Prior works mainly focus on the design of auction mechanisms to implement pricing and resource allocation. However, it is still a challenging problem because of the following three aspects: 1) How to ensure that the auction mechanism runs fairly? An auction mechanism is vulnerable and questionable since the auctioneer may fail the orders matching operation or collude with a few peers. 2) It's hard to assign the computing resources of providers to customers and guarantee reasonable rewards for each participator. 3) How to make bidding strategies for each participator? Each participator has its willingness to sell/buy, which are time-variant and private. To address the above issues, we build a blockchain-enabled computing resource trading system that takes both pricing and bidding strategies into consideration, on which providers and customers can trade computing resources securely, safely and willingly. Next, we formulate a decision-making problem in the continuous double auction (CDA) to maximize their payoffs. Then, we propose a universal model-free Deep Reinforcement Learning (DRL) framework for both computing resource providers and customers. We conduct extensive experiments to evaluate the performance of our DRL framework. Simulation results show that our solution outperforms others in both static and dynamic scenarios. Our DRL framework can achieve higher rewards than others by at least 35%. Furthermore, the average trading price from our DRL framework is less volatile than that from the compared methods. The DRL framework promotes trading and brings larger trading quantities, thus resulting in higher social welfare by at least 25% than the compared schemes.