data_mining_static_code_attributes_to_learn_defect_predictors
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision | Next revision Both sides next revision | ||
|
data_mining_static_code_attributes_to_learn_defect_predictors [2014/02/15 12:23] yann |
data_mining_static_code_attributes_to_learn_defect_predictors [2014/02/15 12:33] yann |
||
|---|---|---|---|
| Line 13: | Line 13: | ||
| The authors start by justifying the need for such predictors: "These potential defect-prone trouble spots can then be examined in more detail by, say, modelchecking, intensive testing, etc." Then, they justify the use of static code metrics, against Fenton and Pfleeger's "insightful example where the same functionality is achieved using different programming language constructs resulting in different static measurements for that module" by stating that static code metrics "are useful as probabilistic statements that the frequency of faults tends to increase in code modules that trigger the | The authors start by justifying the need for such predictors: "These potential defect-prone trouble spots can then be examined in more detail by, say, modelchecking, intensive testing, etc." Then, they justify the use of static code metrics, against Fenton and Pfleeger's "insightful example where the same functionality is achieved using different programming language constructs resulting in different static measurements for that module" by stating that static code metrics "are useful as probabilistic statements that the frequency of faults tends to increase in code modules that trigger the | ||
| predictor" and that "[w]e [should] actively [research] better code metrics which, potentially, [would] yield "better" predictors." But, before finding "better" code metrics, the authors argue that we need a baseline for prediction models. They propose such a baseline in their paper. | predictor" and that "[w]e [should] actively [research] better code metrics which, potentially, [would] yield "better" predictors." But, before finding "better" code metrics, the authors argue that we need a baseline for prediction models. They propose such a baseline in their paper. | ||
| + | |||
| + | The authors build a baseline using the following data: | ||
| + | * Independent variables: the three learners OneR, J48, and naïve Bayes; | ||
| + | * Studied objects: 8 systems whose metric values are available in the NASA MDP dataset; | ||
| + | * Input data to the independent variables: 38 code metrics available in the NASA MDP dataset; | ||
| + | * Dependent variables: the binary variable //defective? = (error_count ≥ 1)// from the NASA MDP dataset; | ||
| + | * Measures: the probability of detection, //pd//, and of false alarm, //pf//; | ||
| + | |||
| + | The authors explain their choice of the measures by explaining that "accuracy is a poor measure of a learner's performance. For example, a learner could score 90 percent accuracy on a data set with 10 percent defective modules, even if it predicts that all defective modules were defect-free". They also avoids using self-tests that "can grossly overestimate performance". A self-test is a test on part of the object used to build the predictor, typically when doing 10-fold validation. The authors favour using "//holdout// modules not used in the generation of that predictor". Finally, the authors apply a logarithmic filter "on all numeric values [to] improve predictor performance". | ||
data_mining_static_code_attributes_to_learn_defect_predictors.txt · Last modified: 2019/10/06 20:37 (external edit)
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
