Skip to content
This repository has been archived by the owner on Sep 5, 2024. It is now read-only.

Commit

Permalink
renaming download method
Browse files Browse the repository at this point in the history
  • Loading branch information
michaelharms committed Jan 19, 2020
1 parent a7c4d96 commit 2e99a23
Show file tree
Hide file tree
Showing 3 changed files with 7 additions and 7 deletions.
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ from comcrawl import IndexClient
client = IndexClient()

client.search("reddit.com/r/MachineLearning/*")
client.download_pages()
client.download()

results = client.results
```
Expand All @@ -59,7 +59,7 @@ from comcrawl import IndexClient
client = IndexClient()

client.search("reddit.com/r/MachineLearning/*", threads=4)
client.download_pages()
client.download()

results = client.results
```
Expand All @@ -80,7 +80,7 @@ sorted_df = df.sort_values(by="timestamp")
filtered_df = sorted_df.drop_duplicates("urlkey", keep="last")

client.results = filtered_df.to_dict("records")
client.download_pages()
client.download()

pd.DataFrame(client.results).to_csv("results.csv")
```
Expand All @@ -94,7 +94,7 @@ from comcrawl import IndexClient

client = IndexClient(["2019-51", "2019-47"])
client.search("reddit.com/r/MachineLearning/*")
client.download_pages()
client.download()

results = client.results
```
Expand All @@ -108,7 +108,7 @@ from comcrawl import IndexClient

client = IndexClient(verbose=True)
client.search("reddit.com/r/MachineLearning/*")
client.download_pages()
client.download()

results = client.results
```
Expand Down
2 changes: 1 addition & 1 deletion comcrawl/core/index_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ def search(self, url: str, threads: int = None) -> None:
"""
self.results = search_multiple_indexes(url, self.indexes, threads)

def download_pages(self, threads: int = None) -> None:
def download(self, threads: int = None) -> None:
"""Download
Downloads the HTML for every result in the
Expand Down
2 changes: 1 addition & 1 deletion tests/test_comcrawl.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,6 @@ def test_comcrawl(snapshot):

assert len(client.results) == 2

client.download_pages()
client.download()

snapshot.assert_match(client.results[1])

0 comments on commit 2e99a23

Please sign in to comment.