Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[MIEB] Change Flickr30k to test split #1449

Merged
merged 94 commits into from
Nov 15, 2024
Merged

[MIEB] Change Flickr30k to test split #1449

merged 94 commits into from
Nov 15, 2024

Commits on Sep 6, 2024

  1. wip: start adding BLIP models

    @Jamie-Stirling
    Jamie-Stirling committed Sep 6, 2024
    Configuration menu
    Copy the full SHA
    1e50557 View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2024

  1. add other blip variants

    @Jamie-Stirling
    Jamie-Stirling committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    8f8e05c View commit details
    Browse the repository at this point in the history

  2. Merge branch 'mieb' of https://github.com/embeddings-benchmark/mteb i… 

    …nto mieb-blip

Merge changes from main
    @Jamie-Stirling
    Jamie-Stirling committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    3ea4d3e View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2024

  1. wip: add blip2_models.py

    @Jamie-Stirling
    Jamie-Stirling committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    be8b4bb View commit details
    Browse the repository at this point in the history

  2. make lint

    @Jamie-Stirling
    Jamie-Stirling committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    b57a395 View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2024

  1. wip: implement blip2 wrapper

    @Jamie-Stirling
    Jamie-Stirling committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    236a94f View commit details
    Browse the repository at this point in the history

Commits on Sep 15, 2024

  1. feat: add blip2 models, still mismatched names

    @Jamie-Stirling
    Jamie-Stirling committed Sep 15, 2024
    Configuration menu
    Copy the full SHA
    1f2f8c3 View commit details
    Browse the repository at this point in the history

  2. fix: remove projections from image and text embeddings

    @Jamie-Stirling
    Jamie-Stirling committed Sep 15, 2024
    Configuration menu
    Copy the full SHA
    8c64860 View commit details
    Browse the repository at this point in the history

  3. make lint

    @Jamie-Stirling
    Jamie-Stirling committed Sep 15, 2024
    Configuration menu
    Copy the full SHA
    20839ca View commit details
    Browse the repository at this point in the history

  4. wip: add coco BLIP2

    @Jamie-Stirling
    Jamie-Stirling committed Sep 15, 2024
    Configuration menu
    Copy the full SHA
    ec47c69 View commit details
    Browse the repository at this point in the history

Commits on Sep 16, 2024

  1. fix: BLIP2 better zero-shot classification without text_proj and visi… 

    …on_proj
    @Jamie-Stirling
    Jamie-Stirling committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    e8f4ae1 View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2024

  1. tidy blip2

    @Jamie-Stirling
    Jamie-Stirling committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    57bc3b8 View commit details
    Browse the repository at this point in the history

  2. add imagenet-dog-15 dataset

    @Jamie-Stirling
    Jamie-Stirling committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    4cbec1b View commit details
    Browse the repository at this point in the history

  3. Merge branch 'mieb' of https://github.com/embeddings-benchmark/mteb i… 

    …nto mieb-imagenet
    @Jamie-Stirling
    Jamie-Stirling committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    6da26dc View commit details
    Browse the repository at this point in the history

  4. tidy and lint

    @Jamie-Stirling
    Jamie-Stirling committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    35be38d View commit details
    Browse the repository at this point in the history

  5. remove unused import

    @Jamie-Stirling
    Jamie-Stirling committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    83d0f45 View commit details
    Browse the repository at this point in the history

  6. add cluster_accuracy, ari and nmi to Image.ClusteringEvaluator

    @Jamie-Stirling
    Jamie-Stirling committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    a309de5 View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2024

  1. add imagenet-10 clustering task

    @Jamie-Stirling
    Jamie-Stirling committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    02c1f81 View commit details
    Browse the repository at this point in the history

  2. add SOPI2IRetrieval

    @Jamie-Stirling
    Jamie-Stirling committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    cede108 View commit details
    Browse the repository at this point in the history

  3. add results forclip on ImageNet10Clustering and ImageNetDog15Clustering

    @Jamie-Stirling
    Jamie-Stirling committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    d226748 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'mieb-imagenet' of https://github.com/Jamie-Stirling/mteb … 

    …into mieb-sop
    @Jamie-Stirling
    Jamie-Stirling committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    2086904 View commit details
    Browse the repository at this point in the history

  5. add SOPI2IRetrieval results for clip 32

    @Jamie-Stirling
    Jamie-Stirling committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    c7c6893 View commit details
    Browse the repository at this point in the history

Commits on Sep 23, 2024

  1. add results for clip vit 32/SOPI2IRetrieval

    @Jamie-Stirling
    Jamie-Stirling committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    5bf66fd View commit details
    Browse the repository at this point in the history

  2. merge incoming change from mieb

    @Jamie-Stirling
    Jamie-Stirling committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    3866647 View commit details
    Browse the repository at this point in the history

  3. resolve conflict

    @Jamie-Stirling
    Jamie-Stirling committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    b1759dc View commit details
    Browse the repository at this point in the history

  4. add RP2kI2IRetrieval dataset

    @Jamie-Stirling
    Jamie-Stirling committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    a890171 View commit details
    Browse the repository at this point in the history

  5. add RP2kI2IRetrieval results with clip-vit-base-patch32

    @Jamie-Stirling
    Jamie-Stirling committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    d3396fa View commit details
    Browse the repository at this point in the history

  6. update image retrieval __init__.py

    @Jamie-Stirling
    Jamie-Stirling committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    aa58556 View commit details
    Browse the repository at this point in the history

  7. fix ImageTextPair dataloading for large datasets; more compositionali… 

    …ty evaluation datasets
    @gowitheflow-1998
    gowitheflow-1998 committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    91ad565 View commit details
    Browse the repository at this point in the history

Commits on Sep 24, 2024

  1. add RP2kI2IRetrieval and METI2IRetrieval

    @Jamie-Stirling
    Jamie-Stirling committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    fc4d1ee View commit details
    Browse the repository at this point in the history

  2. add METI2IRetreival

    @Jamie-Stirling
    Jamie-Stirling committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    7d4f332 View commit details
    Browse the repository at this point in the history

  3. add SOP results

    @Jamie-Stirling
    Jamie-Stirling committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    57593eb View commit details
    Browse the repository at this point in the history

  4. make lign

    @Jamie-Stirling
    Jamie-Stirling committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    62e6e9f View commit details
    Browse the repository at this point in the history

  5. merge upstream mieb

    @Jamie-Stirling
    Jamie-Stirling committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    5d84ace View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2024

  1. new revision for METI2IRetrieval

    @Jamie-Stirling
    Jamie-Stirling committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    094a654 View commit details
    Browse the repository at this point in the history

  2. make lint

    @Jamie-Stirling
    Jamie-Stirling committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    2cc7ffe View commit details
    Browse the repository at this point in the history

  3. reset corpus chunk size

    @Jamie-Stirling
    Jamie-Stirling committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    d3f9db2 View commit details
    Browse the repository at this point in the history

Commits on Sep 26, 2024

  1. remove wrong classification import

    @Jamie-Stirling
    Jamie-Stirling committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    b94e4e2 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'mieb' of https://github.com/embeddings-benchmark/mteb i… 

    …nto mieb-datasets
    @Jamie-Stirling
    Jamie-Stirling committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    a037081 View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2024

  1. add Flickr30k T2I and I2T

    @Jamie-Stirling
    Jamie-Stirling committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    b54c227 View commit details
    Browse the repository at this point in the history

  2. add Flickr30k T2I retriebal

    @Jamie-Stirling
    Jamie-Stirling committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    08ce105 View commit details
    Browse the repository at this point in the history

  3. reduced-size MET revision

    @Jamie-Stirling
    Jamie-Stirling committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    5a45bc4 View commit details
    Browse the repository at this point in the history

Commits on Sep 29, 2024

  1. fix: add Flickr30k T2I

    @Jamie-Stirling
    Jamie-Stirling committed Sep 29, 2024
    Configuration menu
    Copy the full SHA
    9b8c46d View commit details
    Browse the repository at this point in the history

  2. make lint

    @Jamie-Stirling
    Jamie-Stirling committed Sep 29, 2024
    Configuration menu
    Copy the full SHA
    65c2bfb View commit details
    Browse the repository at this point in the history

Commits on Oct 1, 2024

  1. add two landmark datasets and results

    @Jamie-Stirling
    Jamie-Stirling committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    8d79c5b View commit details
    Browse the repository at this point in the history

Commits on Oct 2, 2024

  1. add Sketchy i2i retrieval

    @Jamie-Stirling
    Jamie-Stirling committed Oct 2, 2024
    Configuration menu
    Copy the full SHA
    fe0d053 View commit details
    Browse the repository at this point in the history

Commits on Oct 8, 2024

  1. resolve conflicts

    @Jamie-Stirling
    Jamie-Stirling committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    999d680 View commit details
    Browse the repository at this point in the history

Commits on Oct 12, 2024

  1. Merge branch 'mieb' of https://github.com/embeddings-benchmark/mteb i… 

    …nto mieb
    @gowitheflow-1998
    gowitheflow-1998 committed Oct 12, 2024
    Configuration menu
    Copy the full SHA
    a5011e3 View commit details
    Browse the repository at this point in the history

Commits on Oct 14, 2024

  1. add task metadata

    @Jamie-Stirling
    Jamie-Stirling committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    72ace6e View commit details
    Browse the repository at this point in the history

  2. add BLINKIT2IRetrieval dataset

    @Jamie-Stirling
    Jamie-Stirling committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    5afb2aa View commit details
    Browse the repository at this point in the history

  3. add BLINKIT2TRetrieval

    @Jamie-Stirling
    Jamie-Stirling committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    4ba56b1 View commit details
    Browse the repository at this point in the history

  4. add ImageCoDeT2IRetrieval

    @Jamie-Stirling
    Jamie-Stirling committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    dd99ddc View commit details
    Browse the repository at this point in the history

  5. make lint

    @Jamie-Stirling
    Jamie-Stirling committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    411e313 View commit details
    Browse the repository at this point in the history

Commits on Oct 15, 2024

  1. add vizwiz retrieval and results

    @Jamie-Stirling
    Jamie-Stirling committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    74a2400 View commit details
    Browse the repository at this point in the history

  2. fix vizwiz duplicate texts

    @Jamie-Stirling
    Jamie-Stirling committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    8067b17 View commit details
    Browse the repository at this point in the history

  3. add new vizwiz results

    @Jamie-Stirling
    Jamie-Stirling committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    a80f3bb View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2024

  1. add VQA2 results

    @Jamie-Stirling
    Jamie-Stirling committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    4e1155c View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2024

  1. add GLD v2 I2T retrieval

    @Jamie-Stirling
    Jamie-Stirling committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    e91faf6 View commit details
    Browse the repository at this point in the history

  2. add gld v2 i2i retrieval

    @Jamie-Stirling
    Jamie-Stirling committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    f2721c0 View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2024

  1. make lint

    @Jamie-Stirling
    Jamie-Stirling committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    c9df7ac View commit details
    Browse the repository at this point in the history

  2. add AbsTaskAny2AnyMultiChoice

    @Jamie-Stirling
    Jamie-Stirling committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    a730ecc View commit details
    Browse the repository at this point in the history

  3. make lint

    @Jamie-Stirling
    Jamie-Stirling committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    dab5913 View commit details
    Browse the repository at this point in the history

  4. remove GLDv2I2IRetrieval

    @Jamie-Stirling
    Jamie-Stirling committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    1123d53 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'mieb-datasets' into mieb-any2image-multi

    @Jamie-Stirling
    Jamie-Stirling committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    e053546 View commit details
    Browse the repository at this point in the history

Commits on Oct 20, 2024

  1. Merge branch 'mieb' of https://github.com/embeddings-benchmark/mteb i… 

    …nto mieb
    @gowitheflow-1998
    gowitheflow-1998 committed Oct 20, 2024
    Configuration menu
    Copy the full SHA
    d7a1e25 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'mieb' of https://github.com/embeddings-benchmark/mteb i… 

    …nto mieb-any2image-multi
    @Jamie-Stirling
    Jamie-Stirling committed Oct 20, 2024
    Configuration menu
    Copy the full SHA
    6d22297 View commit details
    Browse the repository at this point in the history

  3. exclude AbsTaskAny2AnyMultiChoice from test_load_data

    @Jamie-Stirling
    Jamie-Stirling committed Oct 20, 2024
    Configuration menu
    Copy the full SHA
    d0cddd4 View commit details
    Browse the repository at this point in the history

Commits on Oct 21, 2024

  1. fix e5v&vista

    @gowitheflow-1998
    gowitheflow-1998 committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    bc8aa01 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'mieb' of https://github.com/embeddings-benchmark/mteb i… 

    …nto mieb-run-tasks-fix
    @gowitheflow-1998
    gowitheflow-1998 committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    7aa5bd2 View commit details
    Browse the repository at this point in the history

  3. remove duplicate corpus entries from BLINKIT2TRetreival dataset

    @Jamie-Stirling
    Jamie-Stirling committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    6d8ed9b View commit details
    Browse the repository at this point in the history

  4. task type fix for running tasks

    @gowitheflow-1998
    gowitheflow-1998 committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    53065b8 View commit details
    Browse the repository at this point in the history

  5. update BLINKIT2T metadata

    @Jamie-Stirling
    Jamie-Stirling committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    3385584 View commit details
    Browse the repository at this point in the history

  6. fix wrong meta

    @gowitheflow-1998
    gowitheflow-1998 committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    3b93440 View commit details
    Browse the repository at this point in the history

  7. run mieb script

    @gowitheflow-1998
    gowitheflow-1998 committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    3998ccd View commit details
    Browse the repository at this point in the history

  8. split ROxford, RParis into easy, medium and hard

    @Jamie-Stirling
    Jamie-Stirling committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    a3649a7 View commit details
    Browse the repository at this point in the history

  9. Merge branch 'mieb' of https://github.com/embeddings-benchmark/mteb i… 

    …nto mieb-datasets
    @Jamie-Stirling
    Jamie-Stirling committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    59a19a1 View commit details
    Browse the repository at this point in the history

  10. make lint

    @Jamie-Stirling
    Jamie-Stirling committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    de9c573 View commit details
    Browse the repository at this point in the history

  11. Merge branch 'mieb-run-tasks-fix' of https://github.com/embeddings-be… 

    …nchmark/mteb into mieb-datasets
    @Jamie-Stirling
    Jamie-Stirling committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    4c345a7 View commit details
    Browse the repository at this point in the history

Commits on Oct 22, 2024

  1. resolve conflict with mieb

    @Jamie-Stirling
    Jamie-Stirling committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    2f7e2d9 View commit details
    Browse the repository at this point in the history

Commits on Oct 28, 2024

  1. add BLINK as multi choice tasks

    @Jamie-Stirling
    Jamie-Stirling committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    50af768 View commit details
    Browse the repository at this point in the history

  2. resolve conflicts with mieb

    @Jamie-Stirling
    Jamie-Stirling committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    baf9674 View commit details
    Browse the repository at this point in the history

  3. fix conflicts with mieb

    @Jamie-Stirling
    Jamie-Stirling committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    e646368 View commit details
    Browse the repository at this point in the history

  4. resolve conflict

    @Jamie-Stirling
    Jamie-Stirling committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    016c0d0 View commit details
    Browse the repository at this point in the history

Commits on Oct 29, 2024

  1. fix: license metadata in wrong format

    @Jamie-Stirling
    Jamie-Stirling committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    27444fd View commit details
    Browse the repository at this point in the history

Commits on Oct 31, 2024

  1. Merge branch 'mieb' of https://github.com/embeddings-benchmark/mteb i… 

    …nto mieb-datasets
    @Jamie-Stirling
    Jamie-Stirling committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    3a3f812 View commit details
    Browse the repository at this point in the history

  2. remove null examples from corpus of ROxford and RParis

    @Jamie-Stirling
    Jamie-Stirling committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    1abce84 View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2024

  1. Merge branch 'mieb' of https://github.com/embeddings-benchmark/mteb i… 

    …nto mieb-datasets
    @Jamie-Stirling
    Jamie-Stirling committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    a0e8caa View commit details
    Browse the repository at this point in the history

  2. fix: add/remove subtasks from BLINKIT2IMultiChoice and BLINKIT2TMulti… 

    …Choice
    @Jamie-Stirling
    Jamie-Stirling committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    ac51be8 View commit details
    Browse the repository at this point in the history

  3. update blink metadata

    @Jamie-Stirling
    Jamie-Stirling committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    dca764a View commit details
    Browse the repository at this point in the history

  4. add updated BLINK results

    @Jamie-Stirling
    Jamie-Stirling committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    2d371c7 View commit details
    Browse the repository at this point in the history

Commits on Nov 13, 2024

  1. Merge branch 'mieb' of https://github.com/embeddings-benchmark/mteb i… 

    …nto mieb-datasets
    @Jamie-Stirling
    Jamie-Stirling committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    220362d View commit details
    Browse the repository at this point in the history

  2. merge upstream mieb

    @Jamie-Stirling
    Jamie-Stirling committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    7e69e16 View commit details
    Browse the repository at this point in the history

Commits on Nov 14, 2024

  1. change Flickr30k to test split

    @Jamie-Stirling
    Jamie-Stirling committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    460a2b8 View commit details
    Browse the repository at this point in the history

  2. change flickr to test split

    @Jamie-Stirling
    Jamie-Stirling committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    bada500 View commit details
    Browse the repository at this point in the history