[Feature Request] Expose the index LSN (x-pinecone-max-indexed-lsn) for robust freshness verification

[Amayas29]

What motivated you to submit this feature request?
We need a reliable way to verify whether recent write operations are visible to queries. The use of vector count comparison fails when insert/delete operations offset each other (e.g., inserting 5 and deleting 5 documents leaves the count unchanged). An LSN in index statistics would provide more robust verification of freshness.

Describe the solution you'd like
Expose the index lsn (x-pinecone-max-indexed-lsn) in describe_index_stats()

Describe alternatives you've considered
We currently use vector count comparison to detect ingestion freshness

write operations (upsert/delete) code:

# Store the count before write operation
initial_count = (
  index.describe_index_stats()
  .get("namespaces", {})
  .get(namespace, {})
  .get("vector_count", None)
)

index.upsert(...)  

Search code:

count = (
  index.describe_index_stats()
  .get("namespaces", {})
  .get(namespace, {})
  .get("vector_count", None)
)

if count == initial_count:
  LOGGER.warning(
      "Recently ingested documents may not yet be visible to queries "
      "due to Pinecone's indexing delay."
  )
  # Potentially followed by a sleep to await visibility.

[anawishnoff]

Hey @Amayas29 thank you for opening. We know that eventual consistency can pose this problem for users and are investigating internally to figure out the best way to add more flexibility and visibility into this workflow. While I don't have an update to share yet, I'll circle back if we decide to go with the route of exposing the LSN.