Nonetype Error on Update Players

[JessePresnell]

Ran into #316 earlier and also needed to update my init.py file to add the LA Chargers line per BurntSushi/nfldb#252 and now I'm getting the following error when trying to run

>>> python nflgame-update-players

Loading games for POST 2016 week 5
Downloading team rosters...
1/33 complete. (3.03%)Traceback (most recent call last):
File "nflgame-update-players", line 4, in
nflgame.update_players.run()
File "C:\Python27\lib\site-packages\nflgame\update_players.py", line 415, in run
tbodys = soup.find(id='result').find_all('tbody')
AttributeError: 'NoneType' object has no attribute 'find_all'

So far I've:

• Uninstalled nflgame via pip and reinstalled directly from the git file per Player Position Error #325
• Ran nfldb-update successfully EDIT: nfldb-update throws the error below

Loading games for POST 2016 week 5
Downloading team rosters...
1/33 complete. (3.03%)Traceback (most recent call last):
File "C:\Python27\lib\runpy.py", line 162, in _run_module_as_main
"main", fname, loader, pkg_name)
File "C:\Python27\lib\runpy.py", line 72, in _run_code
exec code in run_globals
File "C:\Python27\lib\site-packages\nflgame\update_players.py", line 501, in
run()
File "C:\Python27\lib\site-packages\nflgame\update_players.py", line 415, in run
tbodys = soup.find(id='result').find_all('tbody')
AttributeError: 'NoneType' object has no attribute 'find_all'
Exception in thread Thread-3 (most likely raised during interpreter shutdown):C:\Python27\python.exe -m nflgame.update_players --no-block failed (exit status 1)
done.
Locking player table...
Updating 7827 players... done.
Locking write access to tables... done.
Updating season phase, year and week... done.
Bulk inserting data for 3 games...
Sending batch of data to database.
done.
Updating schedule JSON database...
Last updated: 2017-09-13 04:39:00.186000
done.
Updating schedule for (Postseason, 2016, 5)
done.
Closing database connection... done.
FINISHED NFLDB UPDATE AT 2017-09-13 18:43:31.060000

Any suggestions on what may be the culprit?

[ochawkeye]

You mention that nfldb-update ran successfully, but nfldb-update invokes nflgame-update-players.

Is this an issue you continue to see when attempting to re-run nflgame-update-players?

[JessePresnell]

@ochawkeye that's correct. This is when specifically invoking nflgame-update-players

[JessePresnell]

@ochawkeye actually, it looks like nfldb-update is now failing with the same error.

[colemani45]

I'm also having the same problem

[JessePresnell]

Hey @ochawkeye any idea why changing the PARSER object in line 129 of nflgame/update_players.py makes things work? Seems lxml parsing throws out all the HTML from the content object?

From this

    resp, content = new_http().request(urls['roster'] % team, 'GET')
    eprint(content)
    if resp['status'] != '200':
        return None
    return BeautifulSoup(content, PARSER)

To this:

def roster_soup(team):
    resp, content = new_http().request(urls['roster'] % team, 'GET')
    eprint(content)
    if resp['status'] != '200':
        return None
    return BeautifulSoup(content, 'html.parser')

[ochawkeye]

Certainly helps explain why I'm not seeing the issue. From update_players.py in order the parser nflgame tries to use is lxml.html, html5lib, and html.parser.

D:\>python
Python 2.7.8 (default, Jun 30 2014, 16:03:49) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import lxml.html
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named lxml.html
>>> import html5lib
>>>

I don't have lxml.html installed so presumably am using html5lib for the parser.

[tzane]

I just wanted to add that I am also having this same problem. I tried making the same modification to update_players.py as JessePresnell did above but still appear to have the same problem with soup as before only my traceback is slightly different:

.
.
.
.
Could not get roster for team TB

Could not get roster for team TEN

Could not get roster for team WAS
Exception in thread Thread-4 (most likely raised during interpreter shutdown):
Traceback (most recent call last):
File "C:\Python27\lib\threading.py", line 530, in __bootstrap_inner
File "C:\Python27\lib\threading.py", line 483, in run
File "C:\Python27\lib\multiprocessing\pool.py", line 272, in _handle_workers
<type 'exceptions.TypeError'>: 'NoneType' object is not callable

Would hugely appreciate any guidance if someone has made further progress on on this -- Thanks!

[ochawkeye]

That really doesn't look like the same error...
Your traceback is clipped, but it looks to me like update_players.py went through and failed to access the roster URL for all 32 teams (ie. http://www.nfl.com/teams/roster?team=TB).

I suspect it was not anticipated that update_players.py wouldn't be able to get to any of the URLs. Exception handling aside, are you able to get to a link like http://www.nfl.com/teams/roster?team=TB from that PC?

[tzane]

Hi @ochawkeye , yes I can definitely access the url.

[ochawkeye]

Maybe I'm barking up the wrong tree on this, but I'm still leaning toward you not being able to get to the urls.

Can you do this for me? Open up a python interpreter in a terminal window / command prompt.

>>> from nflgame import update_players
>>> soup = update_players.roster_soup('TB')
>>> print len(soup)
7
>>> soup = update_players.roster_soup('FAKE')
>>> print len(soup)
2

[tzane]

My output:

from nflgame import update_players
soup = update_players.roster_soup('TB')
print len(soup)
15
soup = update_players.roster_soup('FAKE')
print len(soup)
3

[ochawkeye]

Is the error you pasted above something that happens every time you run it? Can you just try to run update_players.py again?