Experimental image support

[pjbull]

Runs the existing video models on images. This is not the long-term-support version since it is pretty inefficient, but it's a useful test.

Needs:

• tests
• docs

This can be tested with:

1. Install in fresh Python 3.9 environment with pip install git+https://github.com/drivendataorg/zamba@exp-image-to-vid
2. Running commands prefixed with PREDICT_ON_IMAGES=True

For example, see sample logs and output below

> PREDICT_ON_IMAGES=True zamba predict --data-dir test-images/ --output-class-names --overwrite

2024-04-10 12:36:07.789 | INFO     | zamba.models.config:get_filepaths:221 - Getting files in test-images.
2024-04-10 12:36:07.789 | INFO     | zamba.models.config:get_filepaths:244 - Found 6 videos in test-images.
2024-04-10 12:36:07.791 | INFO     | zamba.models.config:check_files_exist_and_load:114 - Checking all 6 filepaths exist. Trying fast file checking...
QUEUEING TASKS | : 100%|████████████████████████████████████████████████████████████████████████████████████████████| 6/6 [00:00<00:00, 7906.32it/s]
PROCESSING TASKS | : 100%|█████████████████████████████████████████████████████████████████████████████████████████| 6/6 [00:00<00:00, 61832.49it/s]
COLLECTING RESULTS | : 100%|██████████████████████████████████████████████████████████████████████████████████████| 6/6 [00:00<00:00, 120410.64it/s]
2024-04-10 12:36:07.802 | INFO     | zamba.models.config:check_files_exist_and_load:151 - Checking that all videos can be loaded. If you're very confident all your videos can be loaded, you can skip this with `skip_load_validation`, but it's not recommended.
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 6/6 [00:00<00:00, 46.74it/s]
The following configuration will be used for inference:

    Config file: None
    Data directory: test-images
    Filepath csv: None
    Model: time_distributed
    Checkpoint: None
    Batch size: 2
    Number of workers: 3
    GPUs: 0
    Dry run: False
    Save directory: /Users/bull/code/zamba
    Proba threshold: None
    Output class names: True
    Weight download region: us


Is this correct? [Y/n]: y
2024-04-10 12:36:12.070 | INFO     | zamba.models.model_manager:instantiate_model:77 - Instantiating model: TimeDistributedEfficientNet
2024-04-10 12:36:12.070 | INFO     | zamba.models.model_manager:instantiate_model:81 - Loading from checkpoint.
Lightning automatically upgraded your loaded checkpoint from v1.7.5 to v2.0.9.post0. To apply the upgrade to your files permanently, run `python -m pytorch_lightning.utilities.upgrade_checkpoint --file ../../Library/Caches/zamba/time_distributed_1d483fc723.ckpt`
2024-04-10 12:36:13.394 | INFO     | zamba.models.model_manager:predict_model:385 - No cache dir is specified. Videos will not be cached.
GPU available: True (mps), used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/Users/bull/miniconda3/envs/zamba/lib/python3.9/site-packages/pytorch_lightning/trainer/setup.py:201: UserWarning: MPS available but not used. Set `accelerator` and `devices` using `Trainer(accelerator='mps', devices=1)`.
  rank_zero_warn(
2024-04-10 12:36:13.476 | INFO     | zamba.models.model_manager:predict_model:408 - Writing out full configuration to /Users/bull/code/zamba/predict_configuration.yaml.
2024-04-10 12:36:13.478 | INFO     | zamba.models.model_manager:predict_model:413 - Starting prediction...
Predicting: 0it [00:00, ?it/s]/Users/bull/code/zamba/zamba/pytorch/dataloaders.py:148: FutureWarning: Series.__getitem__ treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior). To access a value by position, use `ser.iloc[pos]`
  target = torch.tensor(target).float()
/Users/bull/code/zamba/zamba/pytorch/dataloaders.py:148: FutureWarning: Series.__getitem__ treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior). To access a value by position, use `ser.iloc[pos]`
  target = torch.tensor(target).float()
/Users/bull/code/zamba/zamba/pytorch/dataloaders.py:148: FutureWarning: Series.__getitem__ treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior). To access a value by position, use `ser.iloc[pos]`
  target = torch.tensor(target).float()
/Users/bull/code/zamba/zamba/pytorch/dataloaders.py:148: FutureWarning: Series.__getitem__ treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior). To access a value by position, use `ser.iloc[pos]`
  target = torch.tensor(target).float()
/Users/bull/code/zamba/zamba/pytorch/dataloaders.py:148: FutureWarning: Series.__getitem__ treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior). To access a value by position, use `ser.iloc[pos]`
  target = torch.tensor(target).float()
/Users/bull/code/zamba/zamba/pytorch/dataloaders.py:148: FutureWarning: Series.__getitem__ treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior). To access a value by position, use `ser.iloc[pos]`
  target = torch.tensor(target).float()
Predicting DataLoader 0: 100%|████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:34<00:00, 11.50s/it]
2024-04-10 12:36:50.918 | INFO     | zamba.models.model_manager:predict_model:432 - Saving out predictions to /Users/bull/code/zamba/zamba_predictions.csv.

Output:

cat zamba_predictions.csv
filepath,0
/Users/bull/code/zamba/test-images/chimps.jpeg,chimpanzee_bonobo
/Users/bull/code/zamba/test-images/wild-dog.jpg,wild_dog_jackal
/Users/bull/code/zamba/test-images/elephant.jpg,elephant
/Users/bull/code/zamba/test-images/aardvark.jpg,badger
/Users/bull/code/zamba/test-images/serval.jpg,small_cat
/Users/bull/code/zamba/test-images/zebra.webp,equid

[netlify]

✅ Deploy Preview for silly-keller-664934 ready!

Name	Link
🔨 Latest commit	249ab3c
🔍 Latest deploy log	https://app.netlify.com/sites/silly-keller-664934/deploys/661b1751b778f7000892c975
😎 Deploy Preview	https://deploy-preview-314--silly-keller-664934.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[github-actions]

🚀 Deployed on https://deploy-preview-314--silly-keller-664934.netlify.app

[codecov]

Codecov Report

Attention: Patch coverage is 60.00000% with 6 lines in your changes are missing coverage. Please review.

Project coverage is 87.4%. Comparing base (f11fa5a) to head (249ab3c).

Additional details and impacted files

@@           Coverage Diff            @@
##           master    #314     +/-   ##
========================================
- Coverage    87.6%   87.4%   -0.3%     
========================================
  Files          26      26             
  Lines        2178    2191     +13     
========================================
+ Hits         1908    1915      +7     
- Misses        270     276      +6     

Files	Coverage Δ
zamba/models/config.py	97.5% <100.0%> (+<0.1%)	⬆️
zamba/settings.py	87.5% <100.0%> (+4.1%)	⬆️
zamba/data/video.py	80.8% <33.3%> (-1.7%)	⬇️

[pjbull]

Ready for review. Added docs here:
https://deploy-preview-314--silly-keller-664934.netlify.app/predict-tutorial/#predicting-species-from-images

[ejm714]

LGTM! 📷