[COPD-MICCAI 2018] Apply the method to all 10K subjects

[kayhan-batmanghelich]

• Extract 32 x 32 x 32 overlapping patches from all 10k subjects

• Create a private repo with the code.

• Get new COPD table from Kayhan

• Rerun MICCAI experiment without decoder (Fully discriminative)

• Rerun MICCAI experiment with lambda_1 = 0.01, 0.1 , 1, 10, 100

• Get the R-square and spectral property of latent space plot.

Exploratory Analysis

• Identify multiple modes in t-SNE plot. Does it still appears in experiments with 10k subjects? If yes, then try to identify other trends that might be responsible for these modes.

[sumedhasingla]

• Extract 32 x 32 x 32 overlapping patches from all 10k subjects
  Location: /pghbio/dbmi/batmanlab/singla/COPD_MICCAI_2018/COPDGene_PatchData/OL_32_RF_20_Max_Num_of_patch_1000/

[kayhan-batmanghelich]

@sumedhasingla would you please make a git repo (private) for this?

[sumedhasingla]

https://github.com/sumedhasingla/COPD_Project

[sumedhasingla]

The new COPD dataset is at location: /pghbio/dbmi/batmanlab/Data/COPDGene/ClinicalData

Tracking its progress in a separate issue: #45

[sumedhasingla]

Completed first round of experiments with varying lambda_1. The results are on one run of the experiment. **These are not cross validation results

In Lambda_1 =100 the FEV1/FVC ratio and % gold accuracy is poor. Need to validate this trend by cross validation.

t-SNE plot is still showing multi modal distribution

Lambda_1 = 10 and preplexity = 80
We consider only Gold stage 3 and 4 and created 2 groups, for the 2 modes.

Next we tried to find any significant difference between the 2 groups. We compared

• the Change in FEV1 over 2 phases
  For group-1, FEV1 becomes better for 89 subjects and worse for 179 subjects.
  For group-2, FEV1 becomes better for 97 subjects and worse for 146 subjects.
  

• the Change in FEV1/FVC over 2 phases
  

• FEV1 Vs FEV1/FVC - Phase 1 colored by gold score
  

• FEV1 Vs FEV1/FVC - Phase 2 colored by gold score
  

Both the groups have subjects which becomes better as we go from phase 1 to phase 2.

Conclusion: Couldn't find much difference.

Github: https://github.com/sumedhasingla/COPD_Project/blob/master/Experiments_Results_10k.ipynb

[sumedhasingla]

The spectral properties of the latent space. The lambda_1 regularization, made latent space more informative.