receive: port expanded postings cache from Cortex

[GiedriusS]

Port expanded postings cache from Cortex. Huge kudos to @alanprot for the implementation. I added a TODO item to convert our whole internal caching infra to be promise based.

[alanprot]

Nice.

Maybe if we have this code here, i can make cortex use it and we don't have the code duplicated across both repos. :D

[GiedriusS]

👍 good point. I see a panic in iter.Pull2 from time to time so I am keeping this a draft until I figure out what's wrong.

[alanprot]

can you share the go routine trace when the panic happens?

[GiedriusS]

can you share the go routine trace when the panic happens?

I think I somehow ported some older version of the code which didn't properly check for nil before popping an item from the container. Now it works well.

Also, the coroutines in Go "eat" the original stack trace if there's a panic. This is because it just panic()s again from a different place if seq() panic. So, I added the following code to capture the stack trace:

	defer func() {
			if r := recover(); r != nil {
				st := debug.Stack()
				panic(fmt.Sprintf("panic %v in server iterator: %s", r, st))
			}
		}()