RT-DETR-Based Multimodal Detection with Modality Attention and Feature Alignment
- Authors
-
-
Yujia Lou
-
- Keywords:
- Array
- Abstract
-
Infrared-visible fusion object detection plays a vital role in visual perception under complex environments. However, existing methods still face challenges in feature alignment, modality complementarity, and detection accuracy. To address these issues, this paper proposes a multimodal object detection method based on an improved RT-DETR. A dual-branch feature extraction network is designed to process infrared and visible images separately, and a modality attention mechanism is introduced to enhance cross-modal information interaction. In addition, a feature alignment loss is employed to optimize the fusion process and improve the model's adaptability to different modalities. Experimental results show that the proposed method achieves superior performance on multiple benchmark datasets. Compared to traditional single-modal approaches, the improved RT-DETR shows higher mAP@50 and mAP@95 scores and demonstrates greater robustness under challenging lighting conditions. Compared with existing multimodal detection methods, the proposed model maintains high detection accuracy while improving class discrimination and reducing false positives and missed detections, validating its effectiveness in multimodal visual perception tasks.
- References
- Downloads
- Published
- 2024-08-30
- Issue
- Vol. 3 No. 5 (2024)
- Section
- Articles
- License
-

This work is licensed under a Creative Commons Attribution 4.0 International License.
Academic Scholar Hub Press also operates under the Creative Commons Licence CC-BY 4.0. This allows for copy and redistribute the material in any medium or format for any purpose, even commercially. The premise is that you must provide appropriate citation information.
How to Cite
Similar Articles
- Ting Xu, Yanlin Xiang, Junliang Du, Hanchao Zhang, Cross-Scale Attention and Multi-Layer Feature Fusion YOLOv8 for Skin Disease Target Detection in Medical Images , Journal of Computer Technology and Software: Vol. 4 No. 2 (2025)
- Zhimin Qiu, Time Series and Graph Structure Fusion for AI-Based Anomaly Detection in Microservice Environments , Journal of Computer Technology and Software: Vol. 3 No. 7 (2024)
- Yaokun Ren, Deep Learning for Root Cause Detection in Distributed Systems with Structural Encoding and Multi-modal Attention , Journal of Computer Technology and Software: Vol. 3 No. 5 (2024)
- Carter Evans, Enhanced Lung Lesion Detection with Improved Attention Mechanisms , Journal of Computer Technology and Software: Vol. 3 No. 4 (2024)
- Xiaopei Zhang, Qingquan Wang, EEG Anomaly Detection Using Temporal Graph Attention for Clinical Applications , Journal of Computer Technology and Software: Vol. 4 No. 7 (2025)
- Jianlin Lai, Graph State Detection for Identifying Fictitious and Related Transaction Chains in Financial Networks , Journal of Computer Technology and Software: Vol. 5 No. 2 (2026)
- Nia Qi, Deep Learning-Based Multimodal Fusion and Semantic Embedding for Medical ETL Data Integration , Journal of Computer Technology and Software: Vol. 3 No. 4 (2024)
- Samy Bengio, Deep Nonlinear Regression via Bidirectional Self-Attention and Residual Fusion , Journal of Computer Technology and Software: Vol. 5 No. 9 (2026)
- Yinghao Zhao, Cross-Timescale Transformer with One-Dimensional Convolution for Integrated Financial Risk Anomaly Detection and Discrimination , Journal of Computer Technology and Software: Vol. 3 No. 8 (2024)
- Yun Zi, Xiaoxiao Deng, Joint Modeling of Medical Images and Clinical Text for Early Diabetes Risk Detection , Journal of Computer Technology and Software: Vol. 4 No. 7 (2025)
You may also start an advanced similarity search for this article.
