Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mail channel filter #1651

Merged
merged 4 commits into from
Dec 12, 2024
Merged

Conversation

hasinaxp
Copy link
Contributor

@hasinaxp hasinaxp commented Dec 12, 2024

Summary by CodeRabbit

  • New Features

    • Introduced a new endpoint for stopping mail reading associated with a bot.
    • Added functionality for generating IMAP criteria to filter emails based on various parameters.
    • Enhanced mail integration configuration with new optional fields for better email handling.
  • Bug Fixes

    • Improved error handling in methods related to stopping mail reading.
  • Tests

    • Added new test cases for validating mail reading stop functionality and criteria generation.
  • Documentation

    • Updated integration configuration documentation to reflect new optional fields.

Copy link

coderabbitai bot commented Dec 12, 2024

Walkthrough

The changes introduce a new endpoint in the FastAPI application for stopping mail reading associated with a bot. This includes the addition of a stop_mail_reading method in the server, a corresponding static method in EventUtility, and enhancements to the MailProcessor for generating email criteria. Additionally, the MailScheduler class gains a request_stop method, and error handling is improved in the ChatDataProcessor. The metadata/integrations.yml file is updated with new optional fields for mail integration, and various tests are added to ensure functionality.

Changes

File Change Summary
kairon/events/server.py - Added stop_mail_reading(bot: Text) method.
- Updated request_epoch(bot: Text) method for formatting.
kairon/events/utility.py - Added stop_channel_mail_reading(bot: str) method.
- Removed unnecessary imports.
kairon/shared/channels/mail/processor.py - Added generate_criteria(...) method.
- Updated get_mail_channel_state_data(bot) to specify str type.
- Refactored read_mails to use new criteria generation.
kairon/shared/channels/mail/scheduler.py - Added request_stop(bot: str) method for stopping email channel reading.
kairon/shared/chat/processor.py - Enhanced error handling in delete_channel_config to log errors from MailScheduler.request_stop(bot).
kairon/shared/utils.py - Added string_to_list(comma_sep_string: str, delimiter: str) method for converting strings to lists.
metadata/integrations.yml - Added optional fields to mail integration: subjects, ignore_subjects, from_emails, ignore_from_emails, seen_status.
tests/unit_test/channels/mail_channel_test.py - Added test_generate_criteria to validate generate_criteria method.
tests/unit_test/channels/mail_scheduler_test.py - Added tests for request_stop and stop_channel_mail_reading methods.
tests/unit_test/utility_test.py - Added test_comma_sep_string_to_list to validate string_to_list function.

Possibly related PRs

  • mail channel fix #1637: This PR is related as it addresses changes in the mail channel functionality, including the removal of the MailProcessEvent, which is directly connected to the mail reading process that the main PR introduces an endpoint for managing.

Suggested reviewers

  • hiteshghuge
  • sushantpatade

🐰 In the meadow, the changes bloom,
New endpoints and methods, dispelling the gloom.
With criteria to filter, and tests that ensure,
Our mail reading's robust, of that we are sure!
So hop with joy, for the code is now bright,
A rabbit's delight in the soft moonlight! 🌙✨


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Experiment)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (10)
kairon/shared/channels/mail/processor.py (1)

310-317: Inconsistent Variable Naming: Use Plural Form for Lists

Variables holding lists should be named in the plural form for clarity. Recommend renaming subject to subjects, ignore_subject to ignore_subjects, and ignore_from to ignore_from_addresses for consistency with from_addresses.

Apply this diff to improve variable naming consistency:

-    subject = mp.config.get('subjects', "")
-    subject = Utility.string_to_list(subject)
-    ignore_subject = mp.config.get('ignore_subjects', "")
-    ignore_subject = Utility.string_to_list(ignore_subject)
+    subjects = mp.config.get('subjects', "")
+    subjects = Utility.string_to_list(subjects)
+    ignore_subjects = mp.config.get('ignore_subjects', "")
+    ignore_subjects = Utility.string_to_list(ignore_subjects)

-    ignore_from = mp.config.get('ignore_from_emails', "")
-    ignore_from = Utility.string_to_list(ignore_from)
+    ignore_from_addresses = mp.config.get('ignore_from_emails', "")
+    ignore_from_addresses = Utility.string_to_list(ignore_from_addresses)
kairon/shared/channels/mail/scheduler.py (1)

34-46: Handle Non-Existent Mail Channel Gracefully

In the request_stop method, when the mail channel does not exist, an AppException is raised. Consider handling this scenario more gracefully by logging a warning instead of raising an exception, allowing the method to exit without error if the channel is already stopped.

Apply this diff to adjust the exception handling:

             else:
-                raise AppException("Mail channel does not exist")
+                logger.warning("Mail channel does not exist for bot: %s", bot)
metadata/integrations.yml (1)

88-92: Consider Providing Default Values or Validation for New Fields

To enhance robustness, consider providing default values or input validation for the new optional fields to handle cases where users may provide incorrect configurations.

For example, ensure that seen_status accepts only valid options like 'all', 'seen', or 'unseen'.

kairon/events/utility.py (1)

73-74: Chain Exceptions for Clearer Error Traceback

When raising an exception inside an except block, it's best practice to use raise ... from e to maintain the original traceback, aiding in debugging.

Apply this diff to chain the exception:

            except Exception as e:
-                raise AppException(f"Failed to stop mail reading for bot {bot}. Error: {str(e)}")
+                raise AppException(f"Failed to stop mail reading for bot {bot}. Error: {str(e)}") from e
🧰 Tools
🪛 Ruff (0.8.2)

74-74: Within an except clause, raise exceptions with raise ... from err or raise ... from None to distinguish them from errors in exception handling

(B904)

kairon/events/server.py (1)

155-158: Consider adding error response documentation.

The endpoint implementation looks good, but consider documenting possible error responses in the FastAPI path operation decorator.

-@app.get('/api/mail/stop/{bot}', response_model=Response)
+@app.get(
+    '/api/mail/stop/{bot}',
+    response_model=Response,
+    responses={
+        404: {"description": "Mail channel not found"},
+        500: {"description": "Internal server error while stopping mail scheduler"}
+    }
+)
🧰 Tools
🪛 Ruff (0.8.2)

156-156: Do not perform function call Path in argument defaults; instead, perform the call within the function, or read the default from a module-level singleton variable

(B008)

tests/unit_test/channels/mail_scheduler_test.py (1)

111-126: Add test case for when event_id is None.

The test only covers the case when event_id exists. Consider adding a test case for when event_id is None to ensure complete coverage.

# Add this test case
mock_mail_processor_instance.state.event_id = None
EventUtility.stop_channel_mail_reading(bot)
mock_delete_job.assert_not_called()
kairon/shared/chat/processor.py (1)

73-77: Consider selective exception handling.

The current implementation catches all exceptions. Consider catching specific exceptions that might occur during the mail scheduler stop operation.

 try:
     from kairon.shared.channels.mail.scheduler import MailScheduler
     MailScheduler.request_stop(bot)
-except Exception as e:
+except (ConnectionError, AppException) as e:
     logger.error(f"Error while stopping mail scheduler for bot {bot}. Error: {str(e)}")
tests/unit_test/channels/mail_channel_test.py (2)

280-323: Consider parameterizing test cases and adding negative scenarios.

The test provides good coverage of various email filtering criteria combinations. However, consider these improvements:

  1. Parameterize test cases using @pytest.mark.parametrize to improve maintainability
  2. Add negative test cases (e.g., invalid read_status, empty lists)

Example parameterization:

@pytest.mark.parametrize("test_input,expected", [
    ({"read_status": "seen"}, "((SEEN) (UID 124:*))"),
    ({"read_status": "unseen"}, "((UNSEEN) (UID 124:*))"),
    ({"subjects": ["Test Subject"]}, '((OR SUBJECT "Test Subject") (UID 124:*))'),
    # Add more test cases...
])
async def test_generate_criteria(self, test_input, expected, mock_get_channel_config):
    mp = MailProcessor(bot=pytest.mail_test_bot)
    mp.state.last_email_uid = 123
    criteria = mp.generate_criteria(**test_input)
    assert criteria == expected

588-591: Clean up extra newlines at the end of the file.

Maintain a single newline at the end of the file instead of multiple blank lines.

 Channels.objects(connector_type=ChannelTypes.MAIL.value).delete()
-

-

-

+
kairon/shared/utils.py (1)

2226-2234: LGTM! Consider enhancing the docstring.

The implementation is clean and handles edge cases well. Consider adding parameter descriptions and return type to the docstring for better documentation.

    @staticmethod
    def string_to_list(comma_sep_string: str, delimilter: str = ",") -> List[str]:
        """
-        Convert comma separated string to list
+        Convert a string to list by splitting on delimiter.
+        
+        Args:
+            comma_sep_string: Input string to split
+            delimilter: Character to split on, defaults to comma
+        
+        Returns:
+            List[str]: List of non-empty strings after splitting and stripping whitespace
        """
        if not comma_sep_string:
            return []
        return [item.strip() for item in comma_sep_string.split(delimilter) if item.strip()]
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 3586fe9 and 57b595f.

📒 Files selected for processing (10)
  • kairon/events/server.py (1 hunks)
  • kairon/events/utility.py (2 hunks)
  • kairon/shared/channels/mail/processor.py (4 hunks)
  • kairon/shared/channels/mail/scheduler.py (2 hunks)
  • kairon/shared/chat/processor.py (1 hunks)
  • kairon/shared/utils.py (2 hunks)
  • metadata/integrations.yml (1 hunks)
  • tests/unit_test/channels/mail_channel_test.py (3 hunks)
  • tests/unit_test/channels/mail_scheduler_test.py (1 hunks)
  • tests/unit_test/utility_test.py (1 hunks)
🧰 Additional context used
🪛 Ruff (0.8.2)
kairon/events/utility.py

74-74: Within an except clause, raise exceptions with raise ... from err or raise ... from None to distinguish them from errors in exception handling

(B904)

kairon/events/server.py

156-156: Do not perform function call Path in argument defaults; instead, perform the call within the function, or read the default from a module-level singleton variable

(B008)

tests/unit_test/channels/mail_channel_test.py

6-6: imap_tools.AND imported but unused

Remove unused import: imap_tools.AND

(F401)

🔇 Additional comments (10)
kairon/shared/channels/mail/processor.py (3)

42-42: Type Hint Added to get_mail_channel_state_data

Adding the type annotation bot: str enhances code clarity and facilitates static type checking.


233-283: New generate_criteria Method Implemented Correctly

The generate_criteria method effectively constructs IMAP criteria based on provided parameters, improving the flexibility and specificity of email fetching.


269-278: Ensure Correct Handling of Initial Email Fetch

When last_processed_uid is 0, the base criteria uses date_gte with the current date minus the interval. Verify that this approach correctly fetches emails from the intended timeframe and that time zone differences do not cause any discrepancies.

kairon/shared/channels/mail/scheduler.py (1)

31-47: New request_stop Method Added

The request_stop method correctly implements the functionality to stop email channel reading for a bot. It checks for the existence of the mail channel and handles HTTP requests appropriately.

metadata/integrations.yml (1)

88-92: New Optional Fields Added to Mail Integration

The addition of subjects, ignore_subjects, from_emails, ignore_from_emails, and seen_status expands the configurability of the mail integration, allowing for finer-grained email filtering.

kairon/events/utility.py (1)

62-74: New Method stop_channel_mail_reading Implemented

The stop_channel_mail_reading method correctly stops the mail reading process by updating the event ID and deleting the scheduled job.

🧰 Tools
🪛 Ruff (0.8.2)

74-74: Within an except clause, raise exceptions with raise ... from err or raise ... from None to distinguish them from errors in exception handling

(B904)

kairon/events/server.py (1)

152-153: LGTM!

The formatting change maintains consistency with the rest of the file.

tests/unit_test/channels/mail_scheduler_test.py (1)

128-162: LGTM! Comprehensive test coverage.

The test cases effectively cover:

  • Successful stop request
  • Failed response handling
  • Non-existent channel scenario
tests/unit_test/channels/mail_channel_test.py (1)

6-6: LGTM!

While AND is not directly used, it provides context about the underlying implementation being tested in the test_generate_criteria method.

🧰 Tools
🪛 Ruff (0.8.2)

6-6: imap_tools.AND imported but unused

Remove unused import: imap_tools.AND

(F401)

tests/unit_test/utility_test.py (1)

3311-3335: LGTM! Comprehensive test coverage.

The test cases thoroughly validate the functionality including edge cases and various input scenarios. The assertions are clear and verify the expected behavior.

Copy link
Collaborator

@hiteshghuge hiteshghuge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reviewed

@hiteshghuge hiteshghuge merged commit fc6a5e9 into digiteinfotech:master Dec 12, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants