You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I want to inform you that the search with the partition key does not work
Let's first look at the documentation here, what's notable is that you don't call the load collection or load partition method into memory. Apparently it is assumed that the presence of a partitioning key will cause the partition to be loaded from the filter expression automatically inside Milvus, but it doesnt work, because Milvus throw the errors: <MilvusException: (code=65535, message=not support manually specifying the partition names if partition key mode is used)>, <MilvusException: (code=65535, message=disable load partitions if partition key mode is used)>
Workaround
For example, here is my collection, which has a partition key - category field:
I made my code by analogy with the code in the documentation: this means that I call the search method without calling the load_collection or load_partitions, while in the filter expression I have a filter on the field which is the partition key. In this case Milvus throw the next error: <MilvusException: (code=101, message=failed to search: collection not loaded[collection=452934647721745216])>
Ok... if milvus asks to load a collection, then knowing that the collection has a partition key, I don’t want to load the entire collection, but only the required partition. Hence I will call the load_partitions method, but Milvus throw error <MilvusException: (code=65535, message=not support manually specifying the partition names if partition key mode is used)>. In this situation Milvus literally leaves no choice and forces me to unload the entire collection into memory
I have two questions:
Is the example code in the documentation wrong?
How can I load only the required partition into memory if I have a partitioning key?
I make a full code example where I demonstrated cases of how I tried to do a search by partition
Expected Behavior
I see two ways to fix the situation:
I want to be able to load partitions using the load_partitions method in the current configuration (when we have a partition key in the collection);
Milvus automatically loads partitions into memory if we pass the corresponding field names in the filter expression
How can I load only the required partition into memory if I have a partitioning key?
Enable Mmap might be your answer. We introduced partition key to support millions of partition keys in Milvus, it's really impossible to managa all those partition keys. And to avoid mix-usage of partition and partitionkey, we banned each other when whichever one is enabled. So when you enabled partitionkey, you gained the possiblity to speed up search by filtering on millions of partitionkey, but lose the control over partitions entirely.
Since you're a user of ZillizCloud, it's recommanded to open a ticket on ZillizCloud and describe your requirements. We values our customs needs, and perhaps features like "LazyLoad" could be on our roadmaps.
Is there an existing issue for this?
Describe the bug
hi!
I want to inform you that the search with the partition key does not work
Let's first look at the documentation here, what's notable is that you don't call the load collection or load partition method into memory. Apparently it is assumed that the presence of a partitioning key will cause the partition to be loaded from the filter expression automatically inside Milvus, but it doesnt work, because Milvus throw the errors:
<MilvusException: (code=65535, message=not support manually specifying the partition names if partition key mode is used)>
,<MilvusException: (code=65535, message=disable load partitions if partition key mode is used)>
Workaround
For example, here is my collection, which has a partition key -
category
field:I made my code by analogy with the code in the documentation: this means that I call the
search
method without calling theload_collection
orload_partitions
, while in the filter expression I have a filter on the field which is the partition key. In this case Milvus throw the next error:<MilvusException: (code=101, message=failed to search: collection not loaded[collection=452934647721745216])>
Ok... if milvus asks to load a collection, then knowing that the collection has a partition key, I don’t want to load the entire collection, but only the required partition. Hence I will call the
load_partitions
method, but Milvus throw error<MilvusException: (code=65535, message=not support manually specifying the partition names if partition key mode is used)>
. In this situation Milvus literally leaves no choice and forces me to unload the entire collection into memoryI have two questions:
I make a full code example where I demonstrated cases of how I tried to do a search by partition
Expected Behavior
I see two ways to fix the situation:
load_partitions
method in the current configuration (when we have a partition key in the collection);Steps/Code To Reproduce behavior
Environment details
Anything else?
No response
The text was updated successfully, but these errors were encountered: