Definition of minCountLabel

[budiryan]

What is the definition of minCountLabel? Isn't this algorithm supposed to be unsupervised?

Thanks!

[kaushikacharya]

@budiryan
As per my understanding,
minCountLabel is the min word count threshold.

Have a look at its usage:
https://github.com/epfml/sent2vec/blob/master/src/fasttext.cc#L473

    if (uniform(model.rng) > dict_->getPDiscard(line[i]) || dict_->getTokenCount(line[i]) < args_->minCountLabel)
      continue;

Here line represents the vector of words( to be specific its hash index).

Following code snippet shows how the word index is extracted:
https://github.com/epfml/sent2vec/blob/master/src/dictionary.cc#L30

int32_t Dictionary::find(const std::string& w) const {
  int32_t h = hash(w) % MAX_VOCAB_SIZE;

getTokenCount is defined in
https://github.com/epfml/sent2vec/blob/master/src/dictionary.cc#L133

int64_t Dictionary::getTokenCount(int32_t id) const {
    assert(id >= 0);
    assert(id < size_);
    return words_[id].count;
}

[martinjaggi]

yes, thanks @kaushikacharya

and yes the algorithm is unsupervised (contrastive or self-supervised learning).

can i close this?

[budiryan]

Thank you @kaushikacharya and @martinjaggi for replying!

I have further question:

Then how will it differ from minCount?

It seems that minCount is only used here:
threshold(args_->minCount, args_->minCountLabel);

What is the purpose of that function? I am having trouble understanding it

[kaushikacharya]

@budiryan
Seems you have raised a valid doubt.

sent2vec source code is developed over the existing C++ code of fastText.
Hence it has several sections which are not used in sent2vec e.g. supervised training.

facebookresearch/fastText#530 (comment)
Edouard Grave has mentioned that minCountLabel is to be used for supervised mode only.

Based on this observation, my understanding is that
minCountLabel should be replaced by minCount
in the functions cbowCWNgrams and sent2vec

i.e. in the following lines
https://github.com/epfml/sent2vec/blob/master/src/fasttext.cc#L439
if (uniformSub(model.rng) > dict_->getPDiscard(line[i]) || dict_->getTokenCount(line[i]) < args_->minCountLabel) {

https://github.com/epfml/sent2vec/blob/master/src/fasttext.cc#L473
if (uniform(model.rng) > dict_->getPDiscard(line[i]) || dict_->getTokenCount(line[i]) < args_->minCountLabel)

In threshold(args_->minCount, args_->minCountLabel)
minCount acts as threshold for word token and
minCountLabel acts as threshold for labels (supervised mode).

https://github.com/epfml/sent2vec/blob/master/src/dictionary.cc#L276
void Dictionary::threshold(int64_t t, int64_t tl) {