Plot biomass index #39

chantelwetzel-noaa · 2022-04-12T21:31:23Z

A few updates were needed to align with the latest version of VAST. The branch fixed the following items:

Correct names and function arguments to read and create index plots
Added the specifications for the tweedie distribution in the summary_nwfsc function
Modified the ss_plot function to use red for CA, purple for OR, and blue for WA in order to align better with temperature gradients along the West Coast.

kellijohnson-NOAA · 2022-04-12T21:34:54Z

Thanks @chantelwetzel-noaa for these edits! Super helpful. I peaked at some of them yesterday so I have an initial question ... do we care that these are breaking changes and now the plotting won't work with older saved output?

chantelwetzel-noaa · 2022-04-12T21:42:57Z

That is a great question and was going to leave it up to you decide. I created the pull request to ensure that you saw these changes and did not go through the work to figure out them on your own. Personally, I suspect the frequency of people wanting to replot old runs is fairly low.

kellijohnson-NOAA · 2022-04-12T21:45:27Z

Yeah, I do not want to open any old runs that is for sure. And, if we were too open an old run, I am guessing that the csv of model output is going to be the only thing that is of real interest.

kellijohnson-NOAA

Chantel - thank you so much for all of these proposed changes. There are a few that are absolutely necessary and a few that I think need modifications. Let me know how you want to proceed from here and if any of my comments do not make sense.

kellijohnson-NOAA · 2022-04-12T22:38:23Z

R/summary_nwfsc.R

@@ -42,7 +42,8 @@ summary_nwfsc <- function(obj, parameter_estimates, savedir = NULL) {
    "Poisson-link", 
    switch(as.character(obj$env$data$ObsModel[1]),
      "1" = "Lognormal",
-      "2" = "Gamma")))
+      "2" = "Gamma",
+      "10" = "Tweedie")))


I think we should also add this distribution to obs_model in VAST_spp.

I agree. I only changed it where the code broke. In addition to adding this to the VAST_spp function the other setting that is required with tweedie is:

rho_config = c(Beta1 = 3, Beta2 = 0, Epsilon1 = 0, Epsilon2 = 0)
field = c(Omega1 = 0, Epsilon1 = 0, Omega2 = "IID", Epsilon2 = "IID")

which is then passed into the settings. I think adding a check or setting these two items to match these settings by default when using the tweedie might be helpful for users.

kellijohnson-NOAA · 2022-04-14T16:34:52Z

R/VAST_do.R

    FieldConfig = settings[["FieldConfig"]],
    RhoConfig = settings[["RhoConfig"]],
    OverdispersionConfig = settings[["overdispersion"]],
    ObsModel = settings[["ObsModelcondition"]],
    bias.correct = TRUE,
    #calculate derived quantities of interest, no harm
    #Options = ,#default is more than original VASTWestCoast
-    #use_anisotropy = TRUE, #default
+    #use_anisotropy = settings[["use_anistrophy"]], #default


Did you mean to leave this commented out? I was using comments with default values listed for VAST just so that I knew what the default was. Also needs to be added to the default settings list in get_settings().

I think I missed that this was still in there. I did this while trying to troubleshoot why my tweedie model would not run so I was exploring controlling whether the anisotropy was on or off.

kellijohnson-NOAA · 2022-04-14T17:28:44Z

R/VAST_do.R

@@ -73,15 +73,15 @@ VAST_do <- function(Database, settings, conditiondir, compiledir,
    purpose = "index2",
    fine_scale = settings[["fine_scale"]],
    strata.limits = settings[["strata"]],
-    #zone = NA, #default
+    zone = settings[["zone"]], #default


This setting needs to be added to the default setting list in get_settings().

kellijohnson-NOAA · 2022-04-14T17:33:04Z

R/VAST_do.R

@@ -135,28 +135,32 @@ VAST_do <- function(Database, settings, conditiondir, compiledir,
    save(list = ls(all.names = TRUE), file = file.path(conditiondir, "Save.RData"))
    return(out)
  }
+  save(list = ls(all.names = TRUE), file = file.path(conditiondir, "Save.RData"))


The objects are saved at the end of the function. I am assuming that you added this here just because the errors in plots were causing things to not be saved. I think that it is better to just fix the plots and save things at the end rather than save all present objects and save them again. So, I recommend removing this change.

We could change the if statement above to include a check to see if the estimates of standard error are present as well and if not then exit early.
if ("simpleError" %in% class(out) | is.null(out$parameter_estimates$SD) ) {

kellijohnson-NOAA · 2022-04-14T17:36:51Z

R/VAST_do.R


  maps <- tryCatch(suppressWarnings(FishStatsUtils::plot_results(settings = info, fit = out,
    working_dir = file.path(conditiondir, .Platform$file.sep),
    check_residuals = TRUE)),
    error = function(e) e)

  index <- suppressWarnings(FishStatsUtils::plot_biomass_index(
+    fit = out,


Great, this should fix #41.

kellijohnson-NOAA · 2022-04-14T18:11:03Z

R/plot_index.R

@@ -66,11 +66,11 @@ plot_index <- function(dir, recursive = TRUE, area = NULL, filter = NULL,
    basename(dirname(files)), SIMPLIFY = FALSE))

  #subset data and calculate confidence intervals
-  if (is.null(keepyears)) keepyears <- unique(data[, "Year"])
-  data <- data[data[, "Year"] %in% keepyears, ]
+  if (is.null(keepyears)) keepyears <- unique(data[, "Time"])


Do you really want to use "Time_1", etc. for plotting?

kellijohnson-NOAA · 2022-04-14T18:13:00Z

R/plot_index.R

@@ -52,7 +52,7 @@ plot_index <- function(dir, recursive = TRUE, area = NULL, filter = NULL,

  #get the index information that is saved to the disk
  dir <- normalizePath(dir, mustWork = TRUE)
-  files <- dir(dir, pattern = "Table_for_SS3", full.names = TRUE,
+  files <- dir(dir, pattern = "Index", full.names = TRUE,


Seems like there would be less changes if we just used "our" Table_for_SS3" rather than the automated Index.csv. That way only the csv would need to be saved and all downstream code would work b/c the csv would have the previously utilized column names. This is different than what I said in our previous chat, but I think rather than changing given that Jim doesn't use years that we should just use what we had before.

kellijohnson-NOAA · 2022-04-15T13:38:25Z

I have partitioned out of the required changes into individual commits along with some other needed changes in the branch fix_2022update.
Can you create a single commit on the master branch or in a new branch (with a pull request) for the color changes? Then we can just close this pull request?

@chantelwetzel-noaa

Changes to VAST and FishStatsUtils that create the file Index rather than Table_for_SS3 led to column name changes. These legacy columns are easily recreated and saved similar to the old table in Table_for_SS3.csv to allow downstream code to work. This change was discussed in #39 with great changes made by @chantelwetzel-noaa but we decided to go with the simpler change here rather than changing the column names of everything in VASTWestCoast which would also lead to code changes that would be needed for assessment purposes.

chantelwetzel-noaa · 2022-04-15T14:51:32Z

As always, I am not sure if I followed instructions correctly (I am a git problem). I made a color change commit to the new "fix_2022update" branch. After re-reading your instructions again, I don't think this is exactly what you were looking for, but hopefully the way I did it was ok. I am going to now close this pull request.

kellijohnson-NOAA · 2022-04-15T14:57:00Z

No worries @chantelwetzel-noaa committing to the fix_2022update is just fine. I am also going to delete the plot_biomass_index branch now.

@chantelwetzel-noaa

Changes to VAST and FishStatsUtils that create the file Index rather than Table_for_SS3 led to column name changes. These legacy columns are easily recreated and saved similar to the old table in Table_for_SS3.csv to allow downstream code to work. This change was discussed in #39 with great changes made by @chantelwetzel-noaa but we decided to go with the simpler change here rather than changing the column names of everything in VASTWestCoast which would also lead to code changes that would be needed for assessment purposes.

Chantel Wetzel added 4 commits April 11, 2022 13:54

fix: correct to match current vast csv file and column names

9fd08d2

add: tweedie distributioon

fd378cf

refactor: change colors where red starts in CA

fce310e

fix: correct index plot inputs to match current vast

0986fbd

chantelwetzel-noaa requested a review from kellijohnson-NOAA April 12, 2022 21:31

kellijohnson-NOAA requested changes Apr 14, 2022

View reviewed changes

chantelwetzel-noaa closed this Apr 15, 2022

kellijohnson-NOAA deleted the plot_biomass_index branch April 15, 2022 14:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Plot biomass index #39

Plot biomass index #39

chantelwetzel-noaa commented Apr 12, 2022

kellijohnson-NOAA commented Apr 12, 2022

chantelwetzel-noaa commented Apr 12, 2022

kellijohnson-NOAA commented Apr 12, 2022

kellijohnson-NOAA left a comment

kellijohnson-NOAA Apr 12, 2022

chantelwetzel-noaa Apr 14, 2022

kellijohnson-NOAA Apr 14, 2022

chantelwetzel-noaa Apr 14, 2022

kellijohnson-NOAA Apr 14, 2022

kellijohnson-NOAA Apr 14, 2022

kellijohnson-NOAA Apr 14, 2022

kellijohnson-NOAA Apr 14, 2022

kellijohnson-NOAA Apr 14, 2022

kellijohnson-NOAA Apr 14, 2022

kellijohnson-NOAA commented Apr 15, 2022

chantelwetzel-noaa commented Apr 15, 2022

kellijohnson-NOAA commented Apr 15, 2022

Plot biomass index #39

Plot biomass index #39

Conversation

chantelwetzel-noaa commented Apr 12, 2022

kellijohnson-NOAA commented Apr 12, 2022

chantelwetzel-noaa commented Apr 12, 2022

kellijohnson-NOAA commented Apr 12, 2022

kellijohnson-NOAA left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kellijohnson-NOAA commented Apr 15, 2022

chantelwetzel-noaa commented Apr 15, 2022

kellijohnson-NOAA commented Apr 15, 2022