05-Using.Rmd

# How to use R and Pmetrics

In this section, we suggest a workflow to help you maintain organized
modeling projects.

## Setting up a Pmetrics project
<span class="legacy">Legacy</span></br><span class="r6">R6</span>

When beginning a new modeling project, it is convenient to use the
command `PMtree`. This command will set up a new directory
in the current working directory named whatever you have included as the
"project name". 

```{r echo=T, eval=F}
PMtree("DrugX")
```

In the above example, a directory called "DrugX" will be created
in the current working directory in R, which you can check with the `getwd` function. Beneath the new DrugX directory, several subdirectories will
be also created.

* **Rscript** contains a skeleton R script to begin Pmetrics runs in the new project.
* **Runs** should contain all files required for a run (described next) and it will also contain the resulting numerically ordered run directories created after each Pmetrics NPAG or IT2B run.
* **Sim** can contain any files related to simulations
* **src** is a repository for original and manipulated source data files

You are free to edit this directory tree structure as you please, or make your
own entirely.

## Getting the required inputs to run Pmetrics

<span class="legacy">Legacy</span>

When you wish to execute a Pmetrics run, you must ensure that
appropriate Pmetrics model.txt and data.csv files are in the working
directory, i.e. the Runs subdirectory of the project directory. R can be
used to help prepare the data.csv file by importing and manipulating
spreadsheets (e.g. `read.csv`). The Pmetrics function `PMcheck` can be
used to check a .csv file or an R dataframe that is to be saved as a
Pmetrics data.csv file for errors. It can also check a model file for
errors in the context of a datafile, e.g. covariates that do not match.
`PMcheck(...,fix=T)` attempts to automatically rid data files of errors.
The function `PMwriteMatrix` can be used to write the R data object in
the correct format for use by IT2B, NPAG, or the Simulator.

<!-- ### R6 -->
<span class="r6">R6</span>

There is a full tutorial encoded inside Pmetrics to teach new users the basic functionality of the whole package. To start that tutorial in R type:
```{r echo=T, eval=F}
library(Pmetrics)
PM_tutorial()
```
And follow the instructions prompted in the terminal.

To setup a R6 Pmetrics run, you must provide a `PM_data` and a `PM_model` objects.
It does not matter where the files (if there are any) that represent those files are located on your system.

Both objects provide a `$check` method to validate the consistency of each one:
```{r echo=T, eval=F}
my_model <- PM_model$new(...)
my_data <- PM_data$new(...)

# Scan both model and data for errors
my_model$check()
my_data$check()

```

The next step is to define a fitting problem, this is an object that represents a fitting proccess that will be solved by Pmetrics. On its minimum a fitting problem consists of a data object and a model object. This object is represented by the `PM_fit` class, let's see a couple of examples:
```{r echo=T, eval=F}
my_data <- PM_data$new(...)
my_simple_model <- PM_model$new(...)
my_complex_model <- PM_model$new(...)

#Here we're defining a fitting problem using the drug's data and a simple model:
simple_fit <- PM_fit$new(my_data, my_simple_model)

#Let's do the same but using the other model
complex_fit <- PM_fit$new(my_data, my_complex_model)
```

The fit object also contains a cumple of methods attached to it, i.e:

```{r echo=T, eval=F}
simple_fit$check() #This will check the consistency between the model and the data
complex_fit$save("complex_fit.rds") #This will save the fit to a file in your working directory

my_complex_fit_again <- PM_fit$load("complex_fit.rds") #This will load the fit again from disk
```

To see the full list of methods available see `?PM_fit`.



## Fitting models to data

<span class="legacy">Legacy</span>

As you will see in the skeleton R script made by `PMtree` and placed in
the Rscript subdirectory, if this is a first-time run, the R commands to
run IT2B or NPAG are as follows. Recall that the "\#" character is a
comment character.

``` {r echo=T, eval=F}

library(Pmetrics)
#Run 1 - add your run description here
setwd("working directory")
NPrun() #for NPAG or ITrun() for IT2B
```


The first line will load the Pmetrics library of functions. The second
line sets the working directory to the specified path. The third line
generates the batch file to run NPAG or IT2B and saves it to the working
directory.

**NOTE**: On Mac systems, the batch file will be automatically launched
in a Terminal window. On Windows systems prior to version 1.9, the batch
file must be launched manually by double clicking the *npscript.bat* or
*itscript.bat* file in the working directory. As of version 1.9, Windows
users no longer need to do this.

`ITrun` and `NPrun` are described in full detail via their help commands in R and later in this manual. At minimum, they require a data file and a model file. If the default names of "data.csv" and "model.txt" are used, they may be called with no arguments. Again, the data and model files must be in the current working directory, usually the Runs folder. 

Both functions return the full path of the output directory to
the clipboard. By default, runs are placed in folders numbered
sequentially, beginning with "1".

When you wish to execute a Pmetrics run, you must ensure that 
both of the appropriate Pmetrics data.csv and model.txt files are in the working
directory, i.e. the Runs subdirectory of the project directory. The names are supplied as arguments to `NPrun`, `ITrun`, and `ERRrun`. A shorthand notation is to supply the  number of a previous run for either the data, model or both files so that you do not have to manually copy them into the working directory.

``` {r echo=T, eval=F}
#Using default names data.csv and model.txt
NPrun()

#Using custom names
ITrun(model = "model1.txt", data = "mydata.csv")

#Grab data from run 1 and use default model.txt
NPrun(data=1)

#Use model and data from run 2 and continue where run 2 ended
NPrun(data=2, model=2, prior=2, cycles=1000)
```

See the sections on runing NPAG and IT2B later in the manual for more details on arguments available to modify run behavior.


You can also download sample data and scripts from the [Pmetrics
downloads](http://lapk.org/pmetrics.php) section of our
website. Edit prior versions of model files to make new model files.

<span class="r6">R6</span>

We have discussed the creation of model objects with `PM_model`.

To bring these together, use the `PM_fit` object creator. It only needs two arguments: the name of the data file in the working directory or
<span class="update"> Update</span> in memory loaded via `PMreadMatrix`and a model object. `PM_fit` will accept a model object created by `PM_model` or the name of a model file in Legacy format and in the working directory.

``` {r echo=T, eval=F}

#Example 1 - data file and PM_model object
fit1 <- PM_fit$new(data = "data.csv", model = mod1)

#Example 2 - data object and model file
PMdata <- PMreadMatrix("data.csv")
fit2 <- PM_fit$new(data = PMdata, model = "model.txt")

#Example 3 - data and model objects
PMdata <- PM_data$new("data.csv")
mod1 <- PM_model$new(list(...))
fit3 <- PM_fit$new(PMdata, mod1)

```

Once the `PM_fit` object is created it has only one function defined for it: `$run()`. Arguments for this function can be found in the help for `PM_fit` and later in this manual. 

``` {r echo=T, eval=F}
#default run parameters
fit3$run()

#change the cycle number from default 100
fit3$run(cycles=500)

#change the engine from default NPAG
fit3$run(engine = "IT2B")
```

See the sections on running NPAG and IT2B later in the manual for more details on arguments available to modify run behavior.

## Loading results after a completed run
<span class="legacy">Legacy</span>

Now the output of IT2B or NPAG needs to be loaded into R, so the next
command does this.

`PMload(run_number)`

Details of these commands and what is loaded are described in the R
documentation (?PMload) and in the following section. The run\_number
should be included within the parentheses to be appended to the names of
loaded R objects, allowing for comparison between runs, e.g. `PMload(1)`.
Finally, at this point other Pmetrics commands can be added to the
script to process the data, such as the following.

``` {r echo=T, eval=F}
plot(final.1)
plot(cycle.1)
plot(op.1,type="pop") #or plot(op.1$pop1)
plot(op.1) #default is to plot posterior predictions for output 1
plot(op.1,type="pop",resid=T)
```

Of course, the full power of R can be used in scripts to analyze data,
but these simple statements serve as examples.

If you do not use the `PMtree` structure, we suggest that the R script
for a particular project be saved into a folder called "Rscript" or some
other meaningful name in the working directory. Folders are not be moved
by the batch file. Within the script, number runs sequentially and use
comments liberally to distinguish runs, as shown below.

``` {r echo=T, eval=F}
library(Pmetrics)
#Run 1 - Ka, Kel, V, all subjects
setwd("working directory")
NPrun() #assumes model="model.txt" and data="data.csv"
PMload(1)
```

Remember in R that the command `example(function)` will provide examples
for the specified function. Most Pmetrics functions have examples.

<span class="r6">R6</span>

After the execution is done, you can load the output into R using `my_run<-PM_load(run_number)`, where `my_run` is the variable where the information will be stored in.

Detailed information about the different elements contained in the result object can be accessed via `?PM_result` or by typing the result object into the terminal.

After that, that object can be used to access the different elements of the results, for example:

``` {r echo=T, eval=F}
exRes <- PM_load(1)
plot(exRes$final) # or exRes$final$plot() will plot the marginal distribution of the support points. See ?plot.PM_final.

exRes$op$plot(type="pop") # see ?plot.PM_op,
exRes$data$plot(overlay = F, 
                pred = list(exRes$post, color = "green"), 
                marker = list(symbol = "diamond-open", color = "blue", opacity = 0.8), 
                line = F, log = T) # see ?plot.PM_data
```

<!-- ## Using Shiny to control Pmetrics -->

<!-- Shiny is an R package made by the same group who produces Rstudio. While -->
<!-- you are learning to use Pmetrics, you can use a graphical user interface -->
<!-- that teaches you how to build code. Currently we have implemented Shiny -->
<!-- interfaces for NPAG, IT2B and many plots in Pmetrics. -->

<!-- To use these Shiny interfaces, type `PMcode("x")`, where x is the function -->
<!-- you wish. Currently you have two choices: "run" and "plot". -->

<!-- `PMcode("run")` will launch the following dialog window in your default -->
<!-- browser. As you make your selections, you will see R code generated -->
<!-- which you can copy back to your script and execute. Simply close the -->
<!-- browser window when you are finished. -->

<!-- You can select from an NPAG or IT2B run, and then use the Data, Model, -->
<!-- and Run tabs to navigate to different options. With a valid Pmetrics -->
<!-- model and data file loaded, you will see a summary of the model in the -->
<!-- Current Model window. -->

<!-- If you use `PMcode("plot")` you will see the dialog window below. It will -->
<!-- allow you to select various Pmetrics objects that have been already -->
<!-- loaded with `PMload`, and offer you many plot options, as well as the -->
<!-- code to generate you selected plot. -->