-
GenAI – Multimodal CLIP Model .
GenAI – Multimodal CLIP Model Table Of Contents: Introduction To CLIP Model. What is CLIP? Why CLIP was developed (motivation) Use cases and real-world applications Comparison with previous approaches (e.g., ImageNet classification) CLIP Architecture Dual-encoder design: Image encoder and Text encoder ViT (Vision Transformer) or ResNet for images Transformer for text encoding Embedding spaces and joint alignment Contrastive Pretraining in CLIP How CLIP learns from (image, text) pairs Contrastive loss function (InfoNCE) Positive vs negative sampling Similarity computation using cosine similarity CLIP Training Dataset & Setup Dataset used: 400M (image, text) pairs Uncurated, noisy internet data Zero-shot learning setup CLIP
