Metric parity with Understand
Target: F1 1.000 for every metric. This page is the gap between
that and where the metrics stand. Generated by
scripts/compare/09_metric_f1.py; regenerate it after changing a
metric rather than editing the table.
Measured on the JSON fixture against Understand, over
1158 methods, constructors, classes, interfaces and packages
that both tools name identically. An entity/metric pair is a hit when
this project reports a value and it equals Understand's.
precision = hits / pairs this project answers
recall = hits / pairs Understand answers
100 long names are shared by overloads and are skipped: this project merges those into one entity and Understand does not, so a comparison keyed on the name says nothing.
| macro | micro | |
|---|---|---|
| F1 | 0.991 | 0.990 |
| F1 over pairs Understand answers | 0.991 | 0.990 |
The two rows used to be far apart, because Ent.metric() answered 0
for every entity while Understand answers only for the kinds a metric
is defined on -- which capped every class-level metric near 0.17
however right its values were. Ent.metrics() is scoped to the
entity's kind now, so the strict and the restricted score agree and
what is left is genuine disagreement about values.
| metric | F1 | F1 where Understand answers | precision | recall | hits | ours | und |
|---|---|---|---|---|---|---|---|
AvgCountLine |
1.000 | 1.000 | 1.00 | 1.00 | 110 | 110 | 110 |
AvgCountLineBlank |
1.000 | 1.000 | 1.00 | 1.00 | 110 | 110 | 110 |
AvgCountLineCode |
1.000 | 1.000 | 1.00 | 1.00 | 110 | 110 | 110 |
CCViolDensityCode |
1.000 | 1.000 | 1.00 | 1.00 | 1048 | 1048 | 1048 |
CCViolDensityLine |
1.000 | 1.000 | 1.00 | 1.00 | 1048 | 1048 | 1048 |
CountCCViol |
1.000 | 1.000 | 1.00 | 1.00 | 1048 | 1048 | 1048 |
CountCCViolType |
1.000 | 1.000 | 1.00 | 1.00 | 1048 | 1048 | 1048 |
CountClassBase |
1.000 | 1.000 | 1.00 | 1.00 | 106 | 106 | 106 |
CountClassDerived |
1.000 | 1.000 | 1.00 | 1.00 | 106 | 106 | 106 |
CountDeclClass |
1.000 | 1.000 | 1.00 | 1.00 | 4 | 4 | 4 |
CountDeclClassMethod |
1.000 | 1.000 | 1.00 | 1.00 | 110 | 110 | 110 |
CountDeclClassVariable |
1.000 | 1.000 | 1.00 | 1.00 | 110 | 110 | 110 |
CountDeclFile |
1.000 | 1.000 | 1.00 | 1.00 | 4 | 4 | 4 |
CountDeclInstanceMethod |
1.000 | 1.000 | 1.00 | 1.00 | 110 | 110 | 110 |
CountDeclInstanceVariable |
1.000 | 1.000 | 1.00 | 1.00 | 110 | 110 | 110 |
CountDeclMethod |
1.000 | 1.000 | 1.00 | 1.00 | 110 | 110 | 110 |
CountDeclMethodAll |
1.000 | 1.000 | 1.00 | 1.00 | 106 | 106 | 106 |
CountDeclMethodDefault |
1.000 | 1.000 | 1.00 | 1.00 | 110 | 110 | 110 |
CountDeclMethodPrivate |
1.000 | 1.000 | 1.00 | 1.00 | 110 | 110 | 110 |
CountDeclMethodProtected |
1.000 | 1.000 | 1.00 | 1.00 | 110 | 110 | 110 |
CountDeclMethodPublic |
1.000 | 1.000 | 1.00 | 1.00 | 110 | 110 | 110 |
MaxInheritanceTree |
1.000 | 1.000 | 1.00 | 1.00 | 106 | 106 | 106 |
CountLineCode |
0.999 | 0.999 | 1.00 | 1.00 | 1149 | 1150 | 1150 |
CountSemicolon |
0.999 | 0.999 | 1.00 | 1.00 | 1149 | 1150 | 1150 |
Cyclomatic |
0.999 | 0.999 | 1.00 | 1.00 | 1039 | 1040 | 1040 |
CyclomaticModified |
0.999 | 0.999 | 1.00 | 1.00 | 1039 | 1040 | 1040 |
CyclomaticStrict |
0.998 | 0.998 | 1.00 | 1.00 | 1038 | 1040 | 1040 |
SumCyclomatic |
0.997 | 0.997 | 1.00 | 1.00 | 1155 | 1158 | 1158 |
SumCyclomaticModified |
0.997 | 0.997 | 1.00 | 1.00 | 1155 | 1158 | 1158 |
CountLineBlank |
0.997 | 0.997 | 1.00 | 1.00 | 1147 | 1150 | 1150 |
CountStmt |
0.997 | 0.997 | 1.00 | 1.00 | 1147 | 1150 | 1150 |
CountStmtDecl |
0.997 | 0.997 | 1.00 | 1.00 | 1147 | 1150 | 1150 |
MaxNesting |
0.997 | 0.997 | 1.00 | 1.00 | 1147 | 1150 | 1150 |
SumCyclomaticStrict |
0.997 | 0.997 | 1.00 | 1.00 | 1154 | 1158 | 1158 |
CountStmtExe |
0.996 | 0.996 | 1.00 | 1.00 | 1145 | 1150 | 1150 |
Essential |
0.994 | 0.994 | 0.99 | 0.99 | 1034 | 1040 | 1040 |
RatioCommentToCode |
0.994 | 0.994 | 0.99 | 0.99 | 1151 | 1158 | 1158 |
CountLineComment |
0.994 | 0.994 | 0.99 | 0.99 | 1143 | 1150 | 1150 |
SumEssential |
0.992 | 0.992 | 0.99 | 0.99 | 1149 | 1158 | 1158 |
CountLine |
0.991 | 0.991 | 0.99 | 0.99 | 1140 | 1150 | 1150 |
AvgCountLineComment |
0.991 | 0.991 | 0.99 | 0.99 | 109 | 110 | 110 |
MaxCyclomatic |
0.991 | 0.991 | 0.99 | 0.99 | 109 | 110 | 110 |
MaxCyclomaticModified |
0.991 | 0.991 | 0.99 | 0.99 | 109 | 110 | 110 |
MaxCyclomaticStrict |
0.991 | 0.991 | 0.99 | 0.99 | 109 | 110 | 110 |
CountPathLog |
0.990 | 0.990 | 0.99 | 0.99 | 1030 | 1040 | 1040 |
PercentLackOfCohesionModified |
0.990 | 0.990 | 0.99 | 0.99 | 101 | 102 | 102 |
CountLineCodeExe |
0.990 | 0.990 | 0.99 | 0.99 | 1138 | 1150 | 1150 |
CountLineCodeDecl |
0.985 | 0.985 | 0.99 | 0.99 | 1133 | 1150 | 1150 |
AvgCyclomatic |
0.982 | 0.982 | 0.98 | 0.98 | 108 | 110 | 110 |
AvgCyclomaticModified |
0.982 | 0.982 | 0.98 | 0.98 | 108 | 110 | 110 |
AvgCyclomaticStrict |
0.982 | 0.982 | 0.98 | 0.98 | 108 | 110 | 110 |
AvgEssential |
0.982 | 0.982 | 0.98 | 0.98 | 108 | 110 | 110 |
CountClassCoupledModified |
0.981 | 0.981 | 0.98 | 0.98 | 104 | 106 | 106 |
PercentLackOfCohesion |
0.980 | 0.980 | 0.98 | 0.98 | 100 | 102 | 102 |
MaxEssentialKnots |
0.973 | 0.973 | 0.97 | 0.97 | 1012 | 1040 | 1040 |
MaxEssential |
0.973 | 0.973 | 0.97 | 0.97 | 107 | 110 | 110 |
MinEssentialKnots |
0.972 | 0.972 | 0.97 | 0.97 | 1011 | 1040 | 1040 |
CountInput |
0.969 | 0.969 | 0.97 | 0.97 | 1008 | 1040 | 1040 |
CountPath |
0.968 | 0.968 | 0.97 | 0.97 | 1007 | 1040 | 1040 |
CountOutput |
0.963 | 0.963 | 0.96 | 0.96 | 1002 | 1040 | 1040 |
CountClassCoupled |
0.953 | 0.953 | 0.95 | 0.95 | 101 | 106 | 106 |
Knots |
0.944 | 0.944 | 0.94 | 0.94 | 982 | 1040 | 1040 |
No Understand value on this fixture
F1 is undefined for these: Understand reports nothing to compare against, while this project answers for every entity.
CountDeclExecutableUnit-- answered for 0 entitiesCountDeclFunction-- answered for 0 entitiesCountDeclInstanceVariablePrivate-- answered for 0 entitiesCountDeclInstanceVariableProtected-- answered for 0 entitiesCountDeclInstanceVariablePublic-- answered for 0 entities