04-post-model.qmd

# So you ran an assessment model, more notes and best practices

## Retrospectives

1.  Retrospective analysis is a way to assess whether something is inconsistent in data or model assumptions. A retrospective pattern can arise from changes in catch, M, q, selectivity, or growth from that which is in the model. It is primarily an exploratory/diagnostic tool, though see point 4 below.

2.  See the 11/17/2021 [PEP team meeting presentation](https://docs.google.com/presentation/d/1ZfW1-d6x-lzhxLZpYQhlIAFuAW8ij_e5oEEV0ujwP5A/edit#slide=id.gf83411298e_0_18) for background information and past publications on retrospective analysis

3.  PEP's approach is to apply:

    a.  5 years of peels (based on findings from Miller and Legault 2017),

    b.  using the alternative Mohn's rho (Mohn's rho averaged over peels - the AFSC_Hurtado term from `r4ss::SSmohnsrho`),

    c.  for depletion, biomass, fishing mortality, and recruitment. Often biomass, depletion, and recruitment are provided in assessment reports

<!-- -->

4.  Although the east coast uses mohn's rho to "correct" status indicators or when setting quotas, our practice is not to. This is based on past precedent, and that our fishing history is not as long as it is on the east coast.

5.  Hurtado-Ferro et al. (2015) provide a rule of thumb on the significance of mohn's rho (average over peels) dependent on life history. They suggest a retrospective pattern is not meaningful if $\rho \in (-0.15, 0.2)$ for long lived species and $\rho \in (-0.22, 0.3)$ for short lived species, and note that magnitude of Mohn's rho not related to true bias in assessment. Miller and Legault (2017) argue that variance of Mohn's rho is truly needed to ascertain whether an effect is significant.

## Conducting a rebuilding analysis (aka Puntalyzer)

A rebuilding plan will need to be developed for any species (or stock) that has an GFSC SSC approved stock assessment that estimates the relative spawning biomass to be below the corresponding minimum stock size threshold (0.10 for flatfish and 0.25 for all other groundfish species). A rebuilding analysis is developed using a software program called the "rebuilder" (aka the Puntalyzer) developed and maintained by Andre Punt ([aepunt\@uw.edu](aepunt@uw.edu)). The program is designed to work with an input rebuilding data file created by Stock Synthesis called "rebuild.dat". The rebuilder software executable then develops numerous future projections and calculates the probability of rebuilding based on alternative harvest strategies. The RES.CSV created by the rebuilder executable contains all of the resulting estimates of rebuilding (although many quantities are not labeled within the CSV file).

The rebuilder github repository ([pfmc-assessment/rebuilder](https://github.com/pfmc-assessments/rebuilder)) contains the most up-to-date rebuilder executable, the user manual, code tools for processing output, and code examples. This repository should have all the pertinent information to conduct a rebuilding analysis for a species or stock managed by the Pacific Fishery Management Council

## `r4ss`

1.  The {r4ss} package is available on CRAN but the most up-to-date version can be installed from GitHub. Installation instructions are available [here](https://github.com/r4ss/r4ss).

2.  See the intro vignette [here](https://r4ss.github.io/r4ss/) for info on more advanced topics like scripting Stock Synthesis workflows with r4ss and using the `r4ss::tune_comps()` function.

3.  It's good idea to reinstall it on a regular basis to keep up. You can also watch the github repository if you want to get notifications of changes.

4.  Look through all the figures that get produced by `r4ss::SS_plots()`. There have been cases where models were put forth with incorrect assumptions about biology, ageing error, etc. that could have been caught if the assessment authors had paid attention to all the plots.

5.  Remember that the figures can be modified to look better. For instance, you can replace the fleet names in the model with a less abbreviated set of names that will go in the plot labels.

6.  If something is not working right, complain about it on the [`r4ss` issues list](https://github.com/r4ss/r4ss/issues).

## Writing assessment documents

1.  Read the [terms of reference](https://www.pcouncil.org/documents/2020/05/f-2-attachment-3-draft-revised-terms-of-reference-for-the-groundfish-and-coastal-pelagic-species-stock-assessment-process-for-2021-and-2022.pdf/) and actually follow them.

2.  See the [{sa4ss}](https://github.com/pfmc-assessments/sa4ss) package.

3.  If you're using Word, talk to power users of Word about formatting of section headings, links to figures or tables, etc.. These links save much time when document is updated with sections or elements added or removed.

<!-- -->

a.  See the Assessment Document Template for a pre-made setup that reflects 2017-18 TORs: `\\nwcfile\fram\Assessments\Archives` and shared on the [Google Drive](https://drive.google.com/a/noaa.gov/file/d/0B57aOEaukI73UDlNVUVNZTFScWs/view?usp=sharing)

<!-- -->

4.  Use proper dashes and hyphens for readability: [Hyphen, En dash, and Em dash](http://cutewriting.blogspot.com/2008/06/en-dash-em-dash-and-hyphen.html)

5.  Jason says: "It would be really easy to compare results of every assessment we do with the results of the data-poor methods." He did this for Cabezon in 2009 and thinks we could do it for all our assessments.

6.  For Word, consider using Endnote or Zotero to find/organize/insert references. The assessment endnote library is at `\\Nwcfile\fram\Assessment\References`, but please export references you need and do not link a Word document directly to that library.

7.  Make sure that labels on Figures are readable. The best way to get figures produced in R into word is to save them as PNG files. The plotting functions in the R4SS package have an input for resolution that can be increased as needed. The default resolution is higher than what you get if you save the images one by one from the RGUI graphics window.

8.  Crop out non-informative headings and labels from figures imported from R4SS or other packages.

9.  The council website has a [useful dashboard](https://reports.psmfc.org/pacfin/f?p=501:5302:5212201589304:INITIAL::) for information on recent fishery performance, OFLs, ACLs, etc. Go to the "GMT016 - Stock Summary" table and filter by species and area.