How to avoid nan exception?

[auroua]

I am using ShangHaiTech Part A to train model, but only a few iteration it get the nan excpetion.

Traceback (most recent call last):
  File "Counting-with-Focus-for-Free/code/main.py", line 50, in <module>
    tf.app.run()
  File "Softwares/anaconda3/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "Counting-with-Focus-for-Free/code/main.py", line 44, in main
    model.train()
  File "Counting-with-Focus-for-Free/code/model.py", line 310, in train
    self.inputI_size, self.batch_size)
  File "Counting-with-Focus-for-Free/code/utils.py", line 120, in get_batch_patches
    batch_num[k] = dmap_temp.sum()/L
ValueError: cannot convert float NaN to integer

In this code

    dmap_temp = copy.deepcopy(rand_dmap[pos[0]:pos[0]+patch_width, pos[1]:pos[1]+patch_heigh])

the rand_dmap contains Nan. I have tried to modify the learning rate, but it have no effect.

[shizenglin]

Hi, for my experience, some of your density map ground truth are generated incorrectly, their values are Nan, please note some annotations of shanghai partA are not correct, such as the coordinates of some annotated points are labeled with a negative value or bigger than the size of image. So you need make sure your data is good before you use this code.

[auroua]

Could you share the code used to correct the shanghai partA training data?

[shizenglin]

Sorry, no code, you can do that.