Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: review and normalize haystack.components.websearch #7236

Merged
merged 7 commits into from
Feb 28, 2024

Conversation

wochinge
Copy link
Contributor

@wochinge wochinge commented Feb 28, 2024

Related Issues

Proposed Changes:

  • add searchapi to the indexed modules
  • normalize / review docstrings
  • fix error in test

How did you test it?

Notes for the reviewer

Checklist

@wochinge wochinge requested a review from a team as a code owner February 28, 2024 09:40
@wochinge wochinge requested review from silvanocerza and removed request for a team February 28, 2024 09:40
@github-actions github-actions bot added topic:tests 2.x Related to Haystack v2.0 labels Feb 28, 2024
@@ -1,7 +1,7 @@
loaders:
- type: haystack_pydoc_tools.loaders.CustomPythonLoader
search_path: [../../../haystack/components/websearch]
modules: ["serper_dev"]
modules: ["serper_dev", "searchapi"]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was missing 😱

@github-actions github-actions bot added the type:documentation Improvements on the docs label Feb 28, 2024
@@ -174,7 +174,7 @@ def test_web_search(self):
ws = SerperDevWebSearch(top_k=10)
results = ws.run(query="Who is the boyfriend of Olivia Wilde?")
documents = results["documents"]
links = results["documents"]
links = results["links"]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test was wrong 🙈

@wochinge
Copy link
Contributor Author


title: Websearch
excerpt: Web search engine for Haystack.
category: placeholder-haystack-api
slug: websearch-api
parentDoc:
order: 170
hidden: false

Module serper_dev

SerperDevWebSearch

@component
class SerperDevWebSearch()

Uses Serper to search the web for relevant documents.

See the Serper Dev website for more details.

Usage example:

from haystack.components.websearch import SerperDevWebSearch
from haystack.utils import Secret

websearch = SerperDevWebSearch(top_k=10, api_key=Secret.from_token("test-api-key"))
results = websearch.run(query="Who is the boyfriend of Olivia Wilde?")

assert results["documents"]
assert results["links"]

SerperDevWebSearch.__init__

def __init__(api_key: Secret = Secret.from_env_var("SERPERDEV_API_KEY"),
             top_k: Optional[int] = 10,
             allowed_domains: Optional[List[str]] = None,
             search_params: Optional[Dict[str, Any]] = None)

Arguments:

  • api_key: API key for the Serper API.
  • top_k: Number of documents to return.
  • allowed_domains: List of domains to limit the search to.
  • search_params: Additional parameters passed to the Serper API.
    For example, you can set 'num' to 20 to increase the number of search results.
    See the Serper website for more details.

SerperDevWebSearch.to_dict

def to_dict() -> Dict[str, Any]

Serializes the component to a dictionary.

Returns:

Dictionary with serialized data.

SerperDevWebSearch.from_dict

@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "SerperDevWebSearch"

Serializes the component to a dictionary.

Returns:

Dictionary with serialized data.

SerperDevWebSearch.run

@component.output_types(documents=List[Document],
                        links=List[Union[Document, str]])
def run(query: str) -> Dict[str, List[Union[Document, str]]]

Use Serper to search the web.

Arguments:

  • query: Search query.

Raises:

  • SerperDevError: If an error occurs while querying the SerperDev API.
  • TimeoutError: If the request to the SerperDev API times out.

Returns:

The documents with the content of the search results and the links to the search results.

Module searchapi

SearchApiWebSearch

@component
class SearchApiWebSearch()

Uses SearchApi to search the web for relevant documents.

See the SearchApi website for more details.

Usage example:

from haystack.components.websearch import SearchApiWebSearch
from haystack.utils import Secret

websearch = SearchApiWebSearch(top_k=10, api_key=Secret.from_token("test-api-key"))
results = websearch.run(query="Who is the boyfriend of Olivia Wilde?")

assert results["documents"]
assert results["links"]

SearchApiWebSearch.__init__

def __init__(api_key: Secret = Secret.from_env_var("SEARCHAPI_API_KEY"),
             top_k: Optional[int] = 10,
             allowed_domains: Optional[List[str]] = None,
             search_params: Optional[Dict[str, Any]] = None)

Arguments:

  • api_key: API key for the SearchApi API
  • top_k: Number of documents to return.
  • allowed_domains: List of domains to limit the search to.
  • search_params: Additional parameters passed to the SearchApi API.
    For example, you can set 'num' to 100 to increase the number of search results.
    See the SearchApi website for more details.

SearchApiWebSearch.to_dict

def to_dict() -> Dict[str, Any]

Serializes the component to a dictionary.

Returns:

Dictionary with serialized data.

SearchApiWebSearch.from_dict

@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "SearchApiWebSearch"

Deserializes the component from a dictionary.

Arguments:

  • data: The dictionary to deserialize from.

Returns:

The deserialized component.

SearchApiWebSearch.run

@component.output_types(documents=List[Document],
                        links=List[Union[Document, str]])
def run(query: str) -> Dict[str, List[Union[Document, str]]]

Uses SearchApi to search the web.

Arguments:

  • query: Search query.

Raises:

  • TimeoutError: If the request to the SearchApi API times out.
  • SearchApiError: If an error occurs while querying the SearchApi API.

Returns:

The documents with the content of the search results and the links to the search results.

@coveralls
Copy link
Collaborator

coveralls commented Feb 28, 2024

Pull Request Test Coverage Report for Build 8081642480

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 19 unchanged lines in 2 files lost coverage.
  • Overall coverage remained the same at 90.091%

Files with Coverage Reduction New Missed Lines %
components/websearch/searchapi.py 2 96.36%
components/websearch/serper_dev.py 17 69.84%
Totals Coverage Status
Change from base Build 8081267643: 0.0%
Covered Lines: 5264
Relevant Lines: 5843

💛 - Coveralls

Copy link
Contributor

@silvanocerza silvanocerza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments, there's a couple of things to change.

@wochinge wochinge requested a review from a team as a code owner February 28, 2024 14:04
@wochinge wochinge requested review from dfokina and silvanocerza and removed request for a team February 28, 2024 14:04
@wochinge wochinge merged commit f22d499 into main Feb 28, 2024
25 checks passed
@wochinge wochinge deleted the docs/review-haystack.components.websearch branch February 28, 2024 15:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.x Related to Haystack v2.0 topic:build/distribution topic:tests type:documentation Improvements on the docs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

API Docs - haystack.components.websearch Docstrings - haystack.components.websearch
3 participants