Semantic communication is not obsessed with improving the accuracy of transmitted symbols, but is concerned with expressing the desired meaning that the symbol sequence exactly carried. However, the generation and measurement of semantic messages are still an open problem. Expansion combines simple things into complex systems and even generates intelligence, which is consistent with the evolution of the human language system. We apply this idea to semantic communication system, quantifying and transmitting semantics by symbol sequences, and investigate the semantic information system in a similar way as Shannon did for digital communication systems. This work was the first to propose the concept of semantic expansion and knowledge collision, which may provide a new paradigm for semantic communications. We believe that expansion and collision will be the cornerstone of semantic information theory.

In recent years, semantic communication has received significant attention from both academia and industry, driven by the growing demands for ultra-low latency and high-throughput capabilities in emerging intelligent services. Nonetheless, a comprehensive and effective theoretical framework for semantic communication has yet to be established. In particular, finding the fundamental limits of semantic communication, exploring the capabilities of semantic-aware networks, or utilizing theoretical guidance for deep learning in semantic communication are very important yet still unresolved issues. In general, the mathematical theory of semantic communication and the mathematical representation of semantics are referred to as semantic information theory. In this paper, we introduce the pertinent advancements in semantic information theory. Grounded in the foundational work of Claude Shannon, we present the latest developments in semantic entropy, semantic rate-distortion, and semantic channel capacity. Additionally, we analyze some open problems in semantic information measurement and semantic coding, providing a theoretical basis for the design of a semantic communication system. Furthermore, we carefully review several mathematical theories and tools and evaluate their applicability in the context of semantic communication. Finally, we shed light on the challenges encountered in both semantic communication and semantic information theory.