ruclip-vit-base-patch32-384

RuCLIP ( Ru ssian C ontrastive L anguage– I mage P retraining) is a multimodal model for obtaining images and text similarities and rearranging captions and pictures. RuCLIP builds on a large body of work on zero-shot transfer, computer vision, natural language processing and multimodal learning.

Model was trained by Sber AI and SberDevices teams.

• Task: text ranking ; image ranking ; zero-shot image classification ;
• Type: encoder
• Num Parameters: 150M
• Training Data Volume: 240 million text-image pairs
• Language: Russian
• Context Length: 77
• Transformer Layers: 12
• Transformer Width: 512
• Transformer Heads: 8
• Image Size: 384
• Vision Layers: 12
• Vision Width: 768
• Vision Patch Size: 32

Usage Github

pip install ruclip

clip, processor = ruclip.load("ruclip-vit-base-patch32-384", device="cuda")

Performance

We have evaluated the performance on the following datasets:

Authors

• Alex Shonenkov: Github , Kaggle GM
• Daniil Chesakov: Github
• Denis Dimitrov: Github
• Igor Pavlov: Github