Towards an AI-Enabled Metaverse: Architecture, Resource Allocation, and Multimodal Benchmarking

Abstract

The metaverse, envisioned as a paradigm for next-generation digital environments, remains fundamentally constrained by isolated, pre-defined, and reactive virtual systems. Artificial intelligence (AI) provides a conceptual and technical foundation for overcoming these limitations by enabling persistent perception, autonomous decision-making, and adaptive coordination. This thesis investigates how AI can be systematically embedded into metaverse systems from a system-level perspective. It begins with a comprehensive analysis of metaverse network traffic characteristics, examining the necessity, performance implications, and trade-offs between remote rendering and local rendering. Building on these empirical insights, the thesis proposes a unified thing–edge–cloud architecture that coordinates heterogeneous intelligent agents operating across network layers. Within this architecture, the thesis addresses two core challenges in AI-enabled metaverse systems: adaptive resource allocation and cognitive scene understanding. At the edge layer, adaptive bandwidth allocation for immersive streaming is formulated as a cooperative multi-agent decision-making problem under dynamic network conditions. A Multi-Agent Soft Actor Critic (MASAC)–based strategy is developed and yields consistent improvements in user Quality of Experience (QoE) of at least 14\% compared with representative streaming baselines. At the cloud layer, the thesis introduces a metaverse-oriented benchmarking framework for evaluating the scene understanding capabilities of Multimodal Large Language Models (MLLMs). Thirteen representative MLLMs are assessed through a pairwise comparison protocol under an MLLM-as-a-judge paradigm guided by metaverse-specific semantic criteria. Human expert validation confirms the robustness of the proposed benchmark, achieving an agreement rate of 87.6\% with human consensus. Overall, this thesis establishes an integrated methodological and architectural foundation for the design, optimization, and evaluation of AI-enabled metaverse systems.