Task Offloading for Edge Metaverse: A Joint BSUM and Reinforcement Learning Approach

Abstract

A metaverse can bring many benefits (i.e., self-sustaining and proactive analytics (e.g., analysis before user requests)) to wireless applications; however, its deployment is very challenging due to simultaneous quality of service (QoS) and quality of physical experience (QoE) constraints. Furthermore, the computing and communication resources of end-nodes are limited. Therefore, this paper proposes a novel task offloading framework for metaverse-empowered wireless systems. Our formulated problem aims at minimizing the cost of task offloading in the metaverse while considering both QoS and QoE constraints by optimizing the task offloading, resource allocation, and transmit power allocation variables. To optimize the formulated problem, we use a decomposition-based scheme that further uses modified block-successive upper-bound minimization (BSUM), convex optimization, and multi-agent reinforcement learning (MARL) for transmit power allocation, resource allocation, and task offloading, respectively. Our solution of using convex optimization-assisted MARL for joint resource allocation and task offloading significantly improves the performance of learning in terms of reward. Furthermore, BSUM significantly improves transmit power allocation when used in conjunction with a convex optimizer and MARL. Other than that, we also use dueling to further improve the performance of MARL. Our analyses show that convex optimization, BSUM, and dueling help in significantly improving the performance of MARL. Compared to traditional MARL, our proposal results in significant improvement in terms of reward and cost, as illustrated by the results.

Keywords: convex optimization, deep reinforcement learning, digital twins, Internet of Things, Metaverse