intent ranking added #1656
intent ranking added #1656
Conversation
WalkthroughThe changes involve modifications to the Changes
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Outside diff range and nitpick comments (3)
kairon/nlu/classifiers/llm.py (3)
Line range hint
182-187: Enhance error handling in predict methodThe current error handling is too broad and lacks context. Consider:
- Adding specific exception handling
- Including request context in error logs
- Implementing proper fallback behavior
Example implementation:
- except Exception as e: - logger.error(e) + except json.JSONDecodeError as e: + logger.error(f"Failed to parse LLM response for text '{text}': {str(e)}") + except litellm.exceptions.OpenAIError as e: + logger.error(f"LLM API error for text '{text}': {str(e)}") + except Exception as e: + logger.error(f"Unexpected error processing text '{text}': {str(e)}", exc_info=True)
Line range hint
173-174: Security: Remove sensitive data from debug logsThe debug logging of the complete API response could expose sensitive information.
Consider sanitizing the logged response:
- logger.debug(response) + logger.debug(f"Received response for text length: {len(text)}, status: {response.choices[0]['finish_reason']}")
Line range hint
134-159: Optimize context preparation performanceThe context preparation could be more efficient by:
- Pre-formatting static parts of the message
- Using list comprehension for entity extraction
- Reducing redundant set operations
Consider this optimization:
- entities = set([entity[ENTITY_ATTRIBUTE_TYPE] for entity in entities]) + entities.update(entity[ENTITY_ATTRIBUTE_TYPE] for entity in self.data[i].get(ENTITIES, []))
| @@ -213,7 +213,7 @@ def process(self, messages: List[Message]) -> List[Message]: | |||
| else: | |||
| label, reason, entities = self.predict(message.get(TEXT)) | |||
| intent = {"name": label, "confidence": 1, "reason": reason} | |||
| intent_ranking = [] | |||
| intent_ranking = [intent.copy()] | |||
There was a problem hiding this comment.
🛠️ Refactor suggestion
Consider implementing proper confidence scoring for intent ranking
While copying the intent is better than an empty list, the current implementation doesn't provide true intent ranking as it:
- Always sets confidence to 1
- Only includes a single intent in the ranking
Consider implementing proper confidence scoring and multiple intents in the ranking. Example approach:
def process(self, messages: List[Message]) -> List[Message]:
for message in messages:
if not self.vector and not self.data:
intent = None
intent_ranking = []
entities = []
else:
label, reason, entities = self.predict(message.get(TEXT))
# Get top N intents with confidence scores
intent = {"name": label, "confidence": self._calculate_confidence(label), "reason": reason}
intent_ranking = self._get_intent_ranking(label) # Return top N intents with scores
entities = self.add_extractor_name(entities)
Summary by CodeRabbit
New Features
Bug Fixes