[MIEB] Change Flickr30k to test split (#1449)

* wip: start adding BLIP models

* add other blip variants

* wip: add blip2_models.py

* make lint

* wip: implement blip2 wrapper

* feat: add blip2 models, still mismatched names

* fix: remove projections from image and text embeddings

* make lint

* wip: add coco BLIP2

* fix: BLIP2 better zero-shot classification without text_proj and vision_proj

* tidy blip2

* add imagenet-dog-15 dataset

* tidy and lint

* remove unused import

* add cluster_accuracy, ari and nmi to Image.ClusteringEvaluator

* add imagenet-10 clustering task

* add SOPI2IRetrieval

* add results forclip on ImageNet10Clustering and ImageNetDog15Clustering

* add SOPI2IRetrieval results for clip 32

* add results for clip vit 32/SOPI2IRetrieval

* resolve conflict

* add RP2kI2IRetrieval dataset

* add RP2kI2IRetrieval results with clip-vit-base-patch32

* update image retrieval __init__.py

* fix ImageTextPair dataloading for large datasets; more compositionality evaluation datasets

* add RP2kI2IRetrieval and METI2IRetrieval

* add METI2IRetreival

* add SOP results

* make lign

* new revision for METI2IRetrieval

* make lint

* reset corpus chunk size

* remove wrong classification import

* add Flickr30k T2I and I2T

* add Flickr30k T2I retriebal

* reduced-size MET revision

* fix: add Flickr30k T2I

* make lint

* add two landmark datasets and results

* add Sketchy i2i retrieval

* add task metadata

* add BLINKIT2IRetrieval dataset

* add BLINKIT2TRetrieval

* add ImageCoDeT2IRetrieval

* make lint

* add vizwiz retrieval and results

* fix vizwiz duplicate texts

* add new vizwiz results

* add VQA2 results

* add GLD v2 I2T retrieval

* add gld v2 i2i retrieval

* make lint

* add AbsTaskAny2AnyMultiChoice

* make lint

* remove GLDv2I2IRetrieval

* exclude AbsTaskAny2AnyMultiChoice from test_load_data

* fix e5v&vista

* remove duplicate corpus entries from BLINKIT2TRetreival dataset

* task type fix for running tasks

* update BLINKIT2T metadata

* fix wrong meta

* run mieb script

* split ROxford, RParis into easy, medium and hard

* make lint

* add BLINK as multi choice tasks

* fix: license metadata in wrong format

* remove null examples from corpus of ROxford and RParis

* fix: add/remove subtasks from BLINKIT2IMultiChoice and BLINKIT2TMultiChoice

* update blink metadata

* add updated BLINK results

* merge upstream mieb

* change Flickr30k to test split

* change flickr to test split

---------

Co-authored-by: gowitheflow-1998 <[email protected]>

mteb/tasks/Image/Any2AnyRetrieval/eng/Flickr30kI2TRetrieval.py

@@ -11,7 +11,7 @@ class Flickr30kI2TRetrieval(AbsTaskAny2AnyRetrieval):
         reference="https://www.semanticscholar.org/paper/From-image-descriptions-to-visual-denotations%3A-New-Young-Lai/44040913380206991b1991daf1192942e038fe31",
         dataset={
             "path": "JamieSJS/flickr30k",
-            "revision": "a4cf34ac79215f9e2cd6a10342d84f606fc41cc3",
+            "revision": "24acb2d0b72e18b03388eb20a6225983c0e3f629",
         },
         type="Any2AnyRetrieval",
         category="i2t",

mteb/tasks/Image/Any2AnyRetrieval/eng/Flickr30kT2IRetrieval.py

@@ -11,7 +11,7 @@ class Flickr30kT2IRetrieval(AbsTaskAny2AnyRetrieval):
         reference="https://www.semanticscholar.org/paper/From-image-descriptions-to-visual-denotations%3A-New-Young-Lai/44040913380206991b1991daf1192942e038fe31",
         dataset={
             "path": "JamieSJS/flickr30k",
-            "revision": "a4cf34ac79215f9e2cd6a10342d84f606fc41cc3",
+            "revision": "24acb2d0b72e18b03388eb20a6225983c0e3f629",
         },
         type="Any2AnyRetrieval",
         category="t2i",