metagoofil cant find any files

[jaoh]

Hi all

i build the container and run it using

sudo docker run -v $PWD/data:/data metagoofil -d github.com -f -n 10 -r 4 -t pdf -w

[] Downloaded files will be saved here: /data
[] Searching for 100 .pdf files and waiting 30.0 seconds between searches
[+] Total download: 0 bytes / 0.00 KB / 0.00 MB
[+] Done!

no files are found looks unrealistic

I tried multiple domains . Is it possible to increase loging ?

Thanks Jaoh

[opsdisk]

Hi @jaoh - Thanks for bringing this to my attention. I got 0 results as well when using the container and the Python virtual environment. I also tried a couple of domains. This code is showing it's age and only has print statements instead of proper logging levels...so what you see is what you get.

I suspect the underlying google library may be the issue or Google is changing how results are returned (which has happened in the past). When I set a breakpoint here, there are no results. I also wrote a google search library called yagooglesearch that I've wanted to incorporate, but not had the time.

Unfortunately, making that migration and maintaining this repo are not at the top of my TODO list. I'd be more than happy to review a PR using the yagooglesearch library if you have the skillset.

[jaoh]

i could narrow it down to the user agent and created a pr

[opsdisk]

EDITED - Oops, didn't see #40 before posting this.

Hi @jaoh

1. Try with the -u switch

python metagoofil.py -d github.com -f -n 10 -r 4 -t pdf -w -u

2. I created this PR to swap out the Google libraries being used.

You should be able to test it out using these commands. Mind taking it for a spin, but using the Python virtual environment instead? The Docker container is giving me some issues.

# Fetch the latest branch
git checkout master
git pull origin master
git fetch
git checkout v2-using-yagooglesearch

# Delete the old Python virtual environment and create a new one
rm -rf .venv
virtualenv -p python3 .venv  # If using a virtual environment.
source .venv/bin/activate  # If using a virtual environment.
pip install -r requirements.txt

# Run metagoofil
python metagoofil.py -d github.com -f -n 10 -r 4 -t pdf -w

[opsdisk]

@jaoh Feel free to disregard the above. I made a smaller fix and merged it into v.1.3.0.

# Fetch the latest branch
git checkout master
git pull origin master