Cross-Modal Vision Representation Learning for Real-World Visual Understanding
- Authors
-
-
Corwin Thatch
-
Liora Bramwell
-
- Abstract
-
Recent advances in vision-language pretraining have significantly improved performance across a wide range of visual understanding tasks, including image captioning, visual question answering (VQA), and open-world object detection. However, existing models often suffer from domain sensitivity, shallow cross-modal alignment, and limited adaptability to real-world multimodal scenes. In this work, we propose a unified cross-modal representation learning framework that integrates image, text, and depth modalities through a multi-stream transformer architecture. Our approach emphasizes three design principles: modality-specific feature enhancement, global alignment via contrastive learning, and adaptive fine-tuning with dynamic negative sampling. We demonstrate the effectiveness of our framework on four benchmark datasets spanning open-vocabulary detection, cross-modal retrieval, and zero-shot classification. Extensive experiments show consistent performance gains over state-of-the-art baselines, including CLIP and BLIP-2, with up to +6.3% improvement in retrieval recall and +4.8 mAP in detection tasks. Qualitative analysis further confirms the model’s ability to capture high-level semantic associations across modalities. This work provides new insights into robust cross-modal vision systems and offers a scalable solution for real-world multimodal reasoning applications.
- References
- Downloads
- Published
- 2025-04-30
- Issue
- Vol. 4 No. 4 (2025)
- Section
- Articles
- License
-

This work is licensed under a Creative Commons Attribution 4.0 International License.
Academic Scholar Hub Press also operates under the Creative Commons Licence CC-BY 4.0. This allows for copy and redistribute the material in any medium or format for any purpose, even commercially. The premise is that you must provide appropriate citation information.
