The Repository contains the First Place Vesuvius Grand Prize solution. This repository is part of the First Place Grand Prize Submission to the Vesuvius Challenge 2023 from Youssef Nader, Luke Farritor and Julian Schilliger.
Check out the automatic segmentation pipeline ThaumatoAnakalyptor of our winning Grand Prize submission by Julian Schilliger. ThaumatoAnakalyptor performs in full 3D and is also capable of segmenting in very mushy and twisted scroll regions.
Our final canonical model was a timesformer small architecture with divided space-time attention. The dataset underwent expansion and cleaning rounds to increase accuracy of the labels and become as accurate as possible, approximately 15 rounds were performed between the first letters and final solution. Our solution also consisted of 2 other architectures, Resnet3D-101 with pretrained weights, I3D with non-local block and maxpooling.
Our implementation uses torch, torch-lightning,the timesformer-pytorch and 3D-ResNets-PyTorch.
EASY: build the docker image:
docker build -t youssef_gp .
docker run --gpus all --shm-size=150g -it -v </your-path-to-train-scrolls>:/workspace/train_scrolls youssef_gpThen to train:
python train_timesformer_og.pyOr to run inference with the already trained model:
python inference_timesformer.py --model_path timesformer_weights.ckpt --segment_path train_scrolls --segment_id 20231005123336Important note: to install the ink labels and training data inside the docker image, run:
#to download the segments from the server
./download.sh
#propagates the inklabels into the respective segment folders for training
python prepare.pyYou can find the weights of the canonical timesformer uploaded here
To run inference of timesformer:
python inference_timesformer.py --segment_id 20231210121321 20231221180251 --segment_path $(pwd)/train_scrolls --model_path timesformer_weights.ckptThe optional parameter --out_path can be used to specify the output path of the predictions.