forked from ltorgo/performanceEstimation
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathCHANGES
67 lines (51 loc) · 7.24 KB
/
CHANGES
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
VERSION 1.1.1 (2017-08-11)
- Corrected a bug on the code of timeseriesWF that was introduced before. The bug was causing the "slide" strategy to actually using always the same training set, in effect corresponding to the standardWF approach! Note that the bug was introduced in a previous update motivated by make the package compatible with gbm, which means that on other previous versions this was working correctly.
- The test folds being used in cross validation estimates were wrong (thanks Paula Branco for noticing). There were some situations where not all cases were part of one of the test paritions as it should by definition of k-fold CV.
- The code for stratification had some errors (thanks Paula Branco for noticing). Introduced a new function provided by Paula Branco that solves these issues.
VERSION 1.1.0 (2016-10-12)
- Updated man page of standardPRE
- Small bug corrections on knnImp (what to do when called without NAs)
- Small bug correction on bootEstimates
- The help page of classificationMetrics was updated to mention the structure that is assumed on cost/benefit matrices
- The way the learner is called on both standard workflows was slightly changed because of gbm, that does not use the data in the second argument as "all" other modeling functions. Because of this now the training data is passed as a named (name "data") argument to the learner. This may be a problem if there is some modeling function that does not use that name for the parameter that receives the training set. Still, I've not found one yet... For those cases (if they exist) a user-defined workflow is the only solution. I think this is better because gbm is an important modeling tool.
- Added a new function (results2table) that produces a dplyr data frame table from the results of an experiment for easier querying using dplyr-style facilities
- Changed slightly the show method of class Holdout
- Changed several parameter names in several functions that were referring to the metrics as "stats". All these parameters are now named "metrics"
- Corrected EstimationTask man page that was still referring to things of the previous versions
- corrections to the clustering parts(thanks to Andre Mikulec)
- corrected small confusion in notation used in the help page of classificationMetrics (thanks to Andre Mikulec for noting this).
- show method of workflows changed to not truncate the parameters of learners. Now it always shows all parameters spread over different text lines.
- Calls to .loadedPackages replaced by calls to the base function .packages that does the same
- Changed the parallel computation backend. The package now is built upon the infra-structure of parallelMap
- Corrected a small typo on the example of the help page of mcEstimates
- Added a few new two-class metrics.
- Corrected serious mistakes (giving incorrect values) on some two-class metrics (thanks Kohji Muraoka!).
- Added a new parameter to functions rankWorkflows, topPerformers and topPerformer, which allows the user to specify which of the summary statistics shown in the summary, is to be used (defaults to "avg")
- Corrected a problem with the metric names in the results objects. They were not properly filled in when using user-supplied evaluator functions.
- Changed the way some functions where being called internally (suggestion of Andre Mikulec). The new version allows you to supply, for instance, a learner equal to "e1071::svm" thus avoiding having to load the e1071 package before running the experiments.
VERSION 1.0.2 (2015-09-08)
- WFouput was removed from the framework. Now workflows should simply return a list with the information necessary for the calculation of the required performance metrics. Our provided evaluation functions (classificationMetrics and regressionMetrics) required some information from the result of the workflows (at least a component named "preds" containing a named vector of predicted values and another component named "trues" containing the corresponding vector of true values of the test cases). If you are using some of our provided standard workflows you do not need to worry about this because they return a list with such components. However, if using your own user-defined workflows and you want to use our evaluation functions then you need to worry abou this. In summary, if your code was using standard workflows and evaluation metrics provided by this package, then no changes should be needed to your code as a consequence of this removal of the class WFoutput. However, if that is not the case then you may need to carry out some small changes to your workflows or your evaluation functions.
- The removal of WFoutput had many changes in several functions namely: .scoresIts (experiments.R), runWorkflow (workflows.R) and classificationMetrics and regressionMetrics (evaluationMetrics.R). There was also a change in the EstimationResults class. All these changes have no impact in any code that was using the standard workflows provided by the package - they should work as before with no change necessary. Code using user-defined workflows will have an impact as well as user-defined evaluator functions.
- With the disappearance of WFoutput also disappear the workflowInformation and workflowPredictions functions
- Functions getIterationInfo and getIterationPreds where renamed into getIterationsInfo and getIterationsPreds, respectively. They now have an additional functionality of allowing the user to obtain a list with the results of all iterations instead of only of a specific one, as before. Moreover, the results (either of a specific iteration or all of them) are of a different type. They use to be of class WFouput but now are simply lists, which is now the result of running any workflow.
- There were some problems in the timeseriesWF that were corrected
- The metrics parameter of the EstimationTask constructor is now optional and it defaults to all metrics of the selected evaluation function.
VERSION 1.0.1 (2015-06-17)
- EstimationTask class changed to include a parameter that specifies whether the training data is required for the evaluator function to calculate the metrics.
- Example of user-defined function in EstimationTask documentation was corrected.
- Documentation of the subset method of ComparisonResults objects was corrected to refer the parameter partial.
- Corrected bug on the way mcEstimates accessed the user suplied dataSplits
- The post-processing method that maximizes the utility of predictions for classification tasks was wrongly associated with the MetaCost work of Pedro Domingos. Although strongly related this was incorrect. Renamed everything to 'maxutil' and referred the work of Charles Elkan as the major reference on this approach.
- Some changes and additions to the package vignette
VERSION 1.0.0 (2014-12-02)
- Very radical change on the package introducing incompatibilities with
the version 0.x.y versions. A kind of fresh new re-start for the
package.
- Far too many changes to be written here. You are strongly advised to
check the included package vignette to check the new way the package
works.
VERSION 0.1.1 (2014-03-11)
- corrections to Citation and minor stuff on package vignette
- corrected bug on subset method for class ComparisonResults
- corrected bug on topPerformers() function
- corrected but on workflowVariants() when using a benefit matrix