Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

community: Enhance Atlassian Integration with New API Wrapper and Toolkit #24994

Closed
wants to merge 16 commits into from

Conversation

danielgines
Copy link

@danielgines danielgines commented Aug 2, 2024

Enhance Atlassian Integration with New API Wrapper and Toolkit

Description

This pull request introduces significant enhancements to the Langchain project by integrating the new AtlassianAPIWrapper and AtlassianAction libraries. These updates replace the outdated jira library, providing a more robust and comprehensive solution for interacting with Atlassian's services, including Jira and Confluence.

Key Enhancements

  1. Enhanced Jira Integration:

    • Full support for Jira functions as provided by the latest Atlassian Python API.
    • Advanced error handling mechanisms to ensure reliable operation.
    • Fixes authentication issues for both Atlassian Cloud and Data Center.
  2. Comprehensive Confluence Integration:

    • Full support for Confluence functions as provided by the latest Atlassian Python API.
    • Supports CQL queries and detailed function retrieval and execution.
  3. Improved Configuration and Inputs:

    • Inputs for instance URLs, usernames, API tokens, and cloud configurations.
    • Support for filtering response keys to streamline the data handling process.
  4. Migration to Pydantic v2:

    • Updated to use the latest version of Pydantic for data validation and management.
    • Ensures compatibility with the most recent developments in the Python ecosystem.
  5. Full Coverage of Atlassian Python API Functionalities:

    • Provides coverage of functionalities for both Jira and Confluence, ensuring feature parity with the official Atlassian Python API.
  6. Error Handling and Authentication Improvements:

    • Advanced error handling for better resilience and reliability.
    • Resolves previous issues with authentication for cloud and data center deployments.

Additional Enhancements

  1. Docstrings and Documentation:

    • Added detailed docstrings to all major functions in the AtlassianAPIWrapper module.
    • Improved code documentation to facilitate easier understanding and future development.
  2. Integration Tests:

    • Comprehensive test suite to ensure the reliability and correctness of the AtlassianAPIWrapper.
    • Tests cover both Jira and Confluence functionalities provided by the Atlassian Python API.
  3. Formatting and Environment Checks:

    • Applied lint formatting corrections to ensure compliance with standards.
    • Added checks for required environment variables before running tests.
    • Improved code readability and maintainability.
  4. Jupyter Notebook Guide:

    • Added a comprehensive Jupyter notebook that serves as a guide for using the newly developed Atlassian Toolkit.
    • The notebook includes detailed examples for interacting with Jira and Confluence APIs, showcasing how to:
      • Set up necessary environment variables
      • Initialize the AtlassianAPIWrapper
      • Execute JQL queries in Jira
      • Retrieve and use Jira and Confluence functions
      • Apply filters to responses
      • Extend the module to support future Atlassian products
    • This addition aims to provide a helpful resource for developers to understand and utilize the module effectively in their projects.

For more details, users are directed to the Atlassian Python API documentation.

Issue:

#20084

Dependencies:

No additional dependencies required.

Twitter handle:

Twitter handle: dgines


  • Added tests and docs for the new integration.
  • Ensured all changes are backward compatible.
  • Ran make format, make lint, and make test to ensure code quality.

## Description:
This commit introduces the new `AtlassianAPIWrapper` and `AtlassianAction` libraries, including prompts and enhanced functionalities for integration with Jira and Confluence. These new libraries are designed to replace the outdated `jira` library in Langchain, providing a more robust and comprehensive solution for interacting with Atlassian's services.

## Functionalities:
1. **Enhanced Jira Integration:**
   - Full support for Jira functions as provided by the latest Atlassian Python API.
   - Advanced error handling mechanisms to ensure reliable operation.
   - Fixes authentication issues for both Atlassian Cloud and Data Center.

2. **Comprehensive Confluence Integration:**
   - Full support for Confluence functions as provided by the latest Atlassian Python API.
   - Supports CQL queries and detailed function retrieval and execution.

3. **Improved Configuration and Inputs:**
   - Inputs for instance URLs, usernames, API tokens, and cloud configurations.
   - Support for filtering response keys to streamline the data handling process.

4. **Migration to Pydantic v2:**
   - Updated to use the latest version of Pydantic for data validation and management.
   - Ensures compatibility with the most recent developments in the Python ecosystem.

5. **Full Coverage of Atlassian Python API Functionalities:**
   - Provides coverage of functionalities for both Jira and Confluence, ensuring feature parity with the official Atlassian Python API.

6. **Error Handling and Authentication Improvements:**
   - Advanced error handling for better resilience and reliability.
   - Resolves previous issues with authentication for cloud and data center deployments.

## To-Do:
- Create the necessary tests and documentation on how to use these new libraries.
## Description:
This commit enhances the `AtlassianAPIWrapper` module by adding comprehensive docstrings to all primary functions, ensuring better readability and maintainability of the code. Additionally, it introduces a new integration test for the `AtlassianAPIWrapper`, verifying the correctness and reliability of the library's functionalities.

## Enhancements:
1. **Docstrings for Primary Functions:**
   - Detailed descriptions for each function within the `AtlassianAPIWrapper` module.
   - Explanation of parameters, return values, and examples where applicable.
   - Improved code documentation to facilitate easier understanding and future development.

2. **Integration Test for AtlassianAPIWrapper:**
   - A new test suite to validate the initialization and operations of the `AtlassianAPIWrapper`.
   - Tests for Jira functionalities, including JQL queries, function retrieval, and issue creation.
   - Tests for Confluence functionalities, including function retrieval, CQL queries, and page operations.
   - Verification of environment variable configuration and application within tests.

## Summary of Changes:
- **Docstrings Added:**
  - Added detailed docstrings to all major functions in the `AtlassianAPIWrapper` module.

- **Integration Test Created:**
  - Comprehensive test suite to ensure the reliability and correctness of the `AtlassianAPIWrapper`.
  - Tests cover both Jira and Confluence functionalities provided by the Atlassian Python API.

## To-Do:
- Continue refining and expanding the test suite to cover edge cases and additional functionalities.
- Ensure that the documentation is consistently updated with any new features or changes to the library.
## Enhancements:
1. **Docstrings for Primary Functions:**
   - Detailed descriptions for each function
   - Explanation of parameters, return values, and examples where applicable.
   - Improved code documentation to facilitate easier understanding and future development.
- Applied lint formatting corrections to ensure compliance with standards.
- Added checks for required environment variables before running tests.
- Improved code readability and maintainability.
- Ensured tests run correctly only when the environment is properly configured.
This commit adds a comprehensive Jupyter notebook that serves as a guide for using the newly developed Atlassian Toolkit. The notebook includes detailed examples for interacting with Jira and Confluence APIs, showcasing how to:

- Set up necessary environment variables
- Initialize the AtlassianAPIWrapper
- Execute JQL queries in Jira
- Retrieve and use Jira and Confluence functions
- Apply filters to responses
- Extend the module to support future Atlassian products

The notebook ensures users can leverage the full capabilities of the Atlassian Toolkit, supporting all functions currently available in the `atlassian-python-api` library. For more details, users are directed to the [Atlassian Python API documentation](https://atlassian-python-api.readthedocs.io/).

This addition aims to provide a helpful resource for developers to understand and utilize the module effectively in their projects.
Copy link

vercel bot commented Aug 2, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
langchain ✅ Ready (Inspect) Visit Preview 💬 Add feedback Dec 14, 2024 2:15am

@dosubot dosubot bot added size:XXL This PR changes 1000+ lines, ignoring generated files. community Related to langchain-community 🤖:enhancement A large net-new component, integration, or chain. Use sparingly. The largest features labels Aug 2, 2024
Comment on lines +38 to +47
self.atlassian_username = get_env_var(
"atlassian_username", "ATLASSIAN_USERNAME", ""
)
self.atlassian_api_token = get_env_var(
"atlassian_api_token", "ATLASSIAN_API_TOKEN"
)
self.atlassian_instance_url = get_env_var(
"atlassian_instance_url", "ATLASSIAN_INSTANCE_URL"
)
self.atlassian_cloud = bool(get_env_var("atlassian_cloud", "ATLASSIAN_CLOUD"))
Copy link

@jeffWild jeffWild Aug 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @danielgines, How to handle if jira and confluence are not hosted under the same url. Could we have separate variable for both jira and confluence ? I try with the current master branch, and I need to declare OS env variable JIRA_USERNAME, JIRA_API_TOKEN, JIRA_INSTANCE_URL to get it to work. Passing the variables in the JiraAPIWrapper does not handle it right (causing errors in the agent workflow). So I can't declare on the same host Jira and Confluence because it uses the same env variable. thanks

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jeffWild Thank you for your feedback! It makes perfect sense to have separate variables for Jira and Confluence, especially since it's common to host them separately in self-hosted environments. I will work on this improvement to allow for distinct environment variables for each service (like CONFLUENCE_USERNAME, CONFLUENCE_API_TOKEN, CONFLUENCE_INSTANCE_URL), and I'll publish an update soon.

Thanks for pointing this out!

@jeffWild
Copy link

Hello @danielgines , I'm very interested by this work, do you know when will it be merged ?

@efriis
Copy link
Member

efriis commented Dec 14, 2024

@danielgines instead of adding these to the community package, would you be interested in publishing your own oss package following the new integration process with these two? then we don't have to be in the loop for reviews, and you'll be able to property integration test the toolkit! In order to get this one in, you would have to get CI passing. I'll take another look next week and close if it's still failing!

Future PRs against langchain would just be {docs updates, as well as registering your package in libs/packages.yml, deprecating this community integration in favor of your integration package}

Here's the guide, and if you have questions, feel free to leave them in the comments on those pages so others can see them! https://python.langchain.com/docs/contributing/how_to/integrations/

@efriis efriis self-assigned this Dec 14, 2024
@ccurme
Copy link
Collaborator

ccurme commented Dec 18, 2024

Closing with recommendation of implementing via a separate integration package as @efriis described. Let me know if you have feedback on the walkthrough. I think langchain-atlassian would be a great addition!

@ccurme ccurme closed this Dec 18, 2024
@krammnic
Copy link

There is no separate package, so what is the temporal solution?

@ccurme
Copy link
Collaborator

ccurme commented Jan 2, 2025

There is no separate package, so what is the temporal solution?

The package would need to be created. We've written a walkthrough to help make the process quick.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
community Related to langchain-community 🤖:enhancement A large net-new component, integration, or chain. Use sparingly. The largest features size:XXL This PR changes 1000+ lines, ignoring generated files.
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

5 participants