Question about the knowledge matching strategies

[jihwanp]

Hi I have a small question about the various knowledge matching strategies.

In the experiment, bottom-up strategy is way better than top-down one, and why do you think so?
In my knowledge, upper layer may predict more precise logits, which can be more beneficial to guide lower layers to mimic precise logits, so that can lead to top-down strategy surpass bottom-up.

By the way, thanks for providing such a wonderful work! Im very impressed from your research.