Cross-Modal Vision Representation Learning for Real-World Visual Understanding

Authors
  • Corwin Thatch

  • Liora Bramwell

Abstract

Recent advances in vision-language pretraining have significantly improved performance across a wide range of visual understanding tasks, including image captioning, visual question answering (VQA), and open-world object detection. However, existing models often suffer from domain sensitivity, shallow cross-modal alignment, and limited adaptability to real-world multimodal scenes. In this work, we propose a unified cross-modal representation learning framework that integrates image, text, and depth modalities through a multi-stream transformer architecture. Our approach emphasizes three design principles: modality-specific feature enhancement, global alignment via contrastive learning, and adaptive fine-tuning with dynamic negative sampling. We demonstrate the effectiveness of our framework on four benchmark datasets spanning open-vocabulary detection, cross-modal retrieval, and zero-shot classification. Extensive experiments show consistent performance gains over state-of-the-art baselines, including CLIP and BLIP-2, with up to +6.3% improvement in retrieval recall and +4.8 mAP in detection tasks. Qualitative analysis further confirms the model’s ability to capture high-level semantic associations across modalities. This work provides new insights into robust cross-modal vision systems and offers a scalable solution for real-world multimodal reasoning applications.

References
Cover Image
Downloads
Published
2025-04-30
Section
Articles
License
Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Academic Scholar Hub Press also operates under the Creative Commons Licence CC-BY 4.0. This allows for copy and redistribute the material in any medium or format for any purpose, even commercially. The premise is that you must provide appropriate citation information.

How to Cite

Thatch, C., & Bramwell, L. (2025). Cross-Modal Vision Representation Learning for Real-World Visual Understanding. Journal of Computer Technology and Software, 4(4). https://doi.org/10.5281/zenodo.15340705