Multimodal remote sensing big model: Unlocking the earth observation capabilities of remote sensing big data

Recently, Aerospace Information Research Institute (AIR) under the Chinese Academy of Sciences (CAS) published an article in The Innovation Geoscience titled "Multimodal Artificial Intelligence Foundation Models: Unleashing the Power of Remote Sensing Big Data in Earth Observation". The article was selected by the editorial department as the only Editorial publication in the first issue of 2024. The article establishes a bridge between artificial intelligence (AI) and multimodal remote sensing big data, innovates a new paradigm for multimodal AI big models for earth observation, and unlocks the earth observation capabilities of remote sensing big data.

With the explosive growth in the amount and types of remote sensing data, traditional analysis methods can no longer meet the needs of fine ground perception, and there is an urgent need to improve the processing and analysis capabilities of multimodal remote sensing big data that match it. At present, large model technology with artificial intelligence as the core is developing rapidly. Large models can fully mine and extract information from remote sensing big data, realize the "dry squeeze" of information, and provide new possibilities for solving the shortcomings of remote sensing big data in information extraction and analysis. Therefore, developing a new paradigm for earth observation based on multimodal remote sensing AI big models has become a consensus in the industry and a research frontier.

In order to solve the limitations of information extraction and analysis of existing models, improve the ability to mine multimodal remote sensing big data, and the problems of insufficient information depth mining and application homogeneity, the research team developed a high-precision, full-link multimodal remote sensing big data intelligent interpretation system (as shown in the figure) with multimodal AI big models as the core.

Multimodal remote sensing big data intelligent interpretation system enabled by multimodal AI big model

The system uses different observation platforms to obtain multimodal remote sensing big data, develop multimodal AI big models, make full use of various modal data, adapt to various geoscientific applications, and further apply them to the actual user end. The user's use results can be fed back to the verification and design of the payload and platform, realizing a recyclable full-link feedback mechanism.

The establishment of the system depends on several key elements: one is the integration of a large amount of multimodal remote sensing big data, the second is high-performance computing capabilities, and the third is the integration of remote sensing AI big models. The first two elements are currently easier to achieve, but there is a lack of customized multimodal remote sensing AI big models, which cannot reduce the gap between remote sensing big data and high-performance computing capabilities.

After going through the era of statistical models, physical models, and big data, we are moving towards the era of big models. Multimodal AI big models represent the future of intelligent analysis of remote sensing big data. Their design and proposal unlock the huge potential and value of multimodal remote sensing big data in geoscientific applications. These models can effectively utilize the richness and diversity of multimodal remote sensing big data, providing a powerful framework for solving the complexity of earth observation applications. The shift toward multimodal big models will achieve breakthrough progress in optimizing remote sensing big data for various Earth observation objectives, marking a transformative era in the field.