Skip to content

Parity with Understand

How close OpenUnderstand's output is to SciTools Understand's on the same source. Generated by scripts/gen_docs.py from the comparison output; it needs a licensed Understand install, so it lives outside this repository.

Both tools analyse the same fixture. Entities are matched on (kind family, normalised long name); references on (kind, entity, scope, file, line, column).

105_freemind

Understand OpenUnderstand
Entities 22975 23695
  • Entities matched: 22055 of 22975
  • Entities with no Understand counterpart: 1640
  • Open findings: 14

Entities by kind family

Family Understand OpenUnderstand Δ
variable 8583 9292 +709
parameter 6680 6143 -537
method 5901 5903 +2
class 984 1044 +60
constructor 684 684 +0
interface 90 90 +0
package 53 53 +0
file 0 486 +486

Largest reference gaps

Kind Understand OpenUnderstand Missing
Java Coupleby 4460 3038 +1422
Java Callby 12011 10794 +1217
Java Endby 7659 6841 +818
Java End 7659 6841 +818
Java Begin 7653 6841 +812
Java Beginby 7653 6841 +812
Java Callby Nondynamic 2881 2357 +524
Java Useby 18350 17893 +457
Java Define 22446 22294 +152
Java DotRefby 1671 1559 +112

Open findings

Score Severity Finding
20.6 wrong-data 5568 of 27643 OpenUnderstand entities have no Understand counterpart
16.4 wrong-data 337 duplicate entity rows across 253 logical entities
15.9 wrong-data 281 references written as 'Java Extendby Coupleby External', which Understand never emits here
15.9 wrong-data 281 references written as 'Java Extend Couple External', which Understand never emits here
15.7 wrong-data 255 references written as 'Java Extendby Coupleby Implicit External', which Understand never emits here
15.7 wrong-data 255 references written as 'Java Extend Couple Implicit External', which Understand never emits here
14.5 missing-data 24118 references Understand finds are absent, ignoring position (recall 89%)
9.8 missing-data 900 of 22975 entities Understand finds are absent from OpenUnderstand
9.6 wrong-data 29 references written as 'Java Import Demand', which Understand never emits here
8.3 wrong-data 32 references have the right line but the wrong column (204062 of 204094 match once the column is included)
6.8 wrong-data 10 references match only once ent and scope are swapped
6.5 wrong-data 14 entity long names are structurally malformed

JSON

Understand OpenUnderstand
Entities 4928 4702
  • Entities matched: 4567 of 4928
  • Entities with no Understand counterpart: 135
  • Open findings: 56

Entities by kind family

Family Understand OpenUnderstand Δ
variable 2402 2378 -24
method 1243 1243 +0
parameter 1057 766 -291
constructor 100 100 +0
class 99 105 +6
unknown 16 7 -9
interface 5 5 +0
package 4 4 +0
annotation 2 2 +0
file 0 92 +92

Largest reference gaps

Kind Understand OpenUnderstand Missing
Java Callby 4651 4504 +147
Java DotRefby 1019 901 +118
Java Callby Nondynamic 1043 975 +68
Java Endby 1441 1388 +53
Java End 1441 1388 +53
Java Useby 4886 4834 +52
Java Useby Return 205 154 +51
Java Define 4827 4784 +43
Java Createby 1087 1045 +42
Java Typedby 1380 1347 +33

Open findings

Score Severity Finding
15.4 wrong-data 637 of 5204 OpenUnderstand entities have no Understand counterpart
15.4 wrong-data 233 duplicate entity rows across 127 logical entities
12.1 missing-data 4534 references Understand finds are absent, ignoring position (recall 92%)
12.0 wrong-data 69 references written as 'Java Extend Couple Implicit External', which Understand never emits here
12.0 wrong-data 69 references written as 'Java Extendby Coupleby Implicit External', which Understand never emits here
10.9 wrong-data 47 references written as 'Java Importby', which Understand never emits here
10.5 wrong-data 40 referencemodel._scope_id values are text, not integer
10.5 wrong-data 40 referencemodel._file_id values are text, not integer
10.5 wrong-data 40 references written as 'Java Import Demand', which Understand never emits here
9.9 wrong-data CountOutput disagrees with Understand on 95 of 119 entities (20% agreement)
9.8 missing-data 4 reference pairs have unequal forward/inverse row counts
9.4 wrong-data CountInput disagrees with Understand on 74 of 119 entities (38% agreement)

SalaryCalculator

Understand OpenUnderstand
Entities 50 55
  • Entities matched: 50 of 50
  • Entities with no Understand counterpart: 5
  • Open findings: 12

Entities by kind family

Family Understand OpenUnderstand Δ
method 22 22 +0
variable 14 14 +0
parameter 8 8 +0
class 5 5 +0
package 1 1 +0
file 0 5 +5

Largest reference gaps

Kind Understand OpenUnderstand Missing
Java Setby Init 9 5 +4
Java Set Init 9 5 +4
Java Setby 7 4 +3
Java Useby 17 17 +0
Java Use Return 11 11 +0
Java Callby Nondynamic 16 16 +0
Java Extend Couple 3 3 +0
Java Call Nondynamic 16 16 +0
Java Endby 27 27 +0
Java Useby Deref Partial 14 14 +0

Open findings

Score Severity Finding
7.2 wrong-data 19 of 69 OpenUnderstand entities have no Understand counterpart
5.7 wrong-data 10 references have the right line but the wrong column (350 of 360 match once the column is included)
5.1 wrong-data 5 references written as 'Java Declarein', which Understand never emits here
5.1 wrong-data 5 references written as 'Java Callby', which Understand never emits here
5.1 wrong-data 5 references written as 'Java Call', which Understand never emits here
4.8 missing-data 28 references Understand finds are absent, ignoring position (recall 93%)
3.1 wrong-data 2 references written as 'Java Useby Annotation', which Understand never emits here
3.1 wrong-data 2 references written as 'Java Overrides', which Understand never emits here
3.1 wrong-data 2 references written as 'Java Use Annotation', which Understand never emits here
3.1 wrong-data 2 references written as 'Java Overriddenby', which Understand never emits here
3.1 wrong-data 2 references written as 'Java Extendby Coupleby Implicit External', which Understand never emits here
3.1 wrong-data 2 references written as 'Java Extend Couple Implicit External', which Understand never emits here

TheAlgorithms

Understand OpenUnderstand
Entities 4626 4846
  • Entities matched: 4487 of 4626
  • Entities with no Understand counterpart: 359
  • Open findings: 61

Entities by kind family

Family Understand OpenUnderstand Δ
variable 2050 2023 -27
parameter 1150 1116 -34
method 941 942 +1
class 293 356 +63
constructor 106 106 +0
unknown 56 38 -18
package 27 28 +1
interface 3 3 +0
file 0 234 +234

Largest reference gaps

Kind Understand OpenUnderstand Missing
Java Callby Nondynamic 799 657 +142
Java Setby Init 1656 1523 +133
Java Set Init 1656 1543 +113
Java Set 1094 1014 +80
Java Setby 1094 1014 +80
Java Typedby 572 494 +78
Java Useby GenericArgument 65 33 +32
Java End 1343 1311 +32
Java Endby 1343 1311 +32
Java Useby 6021 5991 +30

Open findings

Score Severity Finding
16.0 wrong-data 799 of 5286 OpenUnderstand entities have no Understand counterpart
15.9 wrong-data 279 references written as 'Java Extend Couple Implicit External', which Understand never emits here
15.9 wrong-data 279 references written as 'Java Extendby Coupleby Implicit External', which Understand never emits here
12.1 wrong-data 72 duplicate entity rows across 54 logical entities
11.6 wrong-data 59 references written as 'Java Use Annotation', which Understand never emits here
11.6 wrong-data 59 references written as 'Java Useby Annotation', which Understand never emits here
11.4 wrong-data CountInput disagrees with Understand on 190 of 325 entities (42% agreement)
11.2 missing-data 4 reference pairs have unequal forward/inverse row counts
11.1 missing-data 2230 references Understand finds are absent, ignoring position (recall 95%)
10.7 wrong-data 43 references written as 'Java Useby Constrains Coupleby', which Understand never emits here
10.7 wrong-data 43 references written as 'Java Use Constrains Couple', which Understand never emits here
10.2 wrong-data CountOutput disagrees with Understand on 108 of 325 entities (67% agreement)

calculator_app

Understand OpenUnderstand
Entities 72 78
  • Entities matched: 67 of 72
  • Entities with no Understand counterpart: 11
  • Open findings: 33

Entities by kind family

Family Understand OpenUnderstand Δ
parameter 26 21 -5
method 17 17 +0
variable 15 18 +3
class 8 8 +0
package 6 6 +0
file 0 8 +8

Largest reference gaps

Kind Understand OpenUnderstand Missing
Java Use Return 4 1 +3
Java Useby Return 4 1 +3
Java Callby Nondynamic 8 7 +1
Java Contain 8 8 +0
Java Modifyby 5 5 +0
Java Overriddenby 1 1 +0
Java Set Init 15 15 +0
Java Useby Deref Partial 5 5 +0
Java Beginby 25 25 +0
Java Use Deref Partial 5 5 +0

Open findings

Score Severity Finding
8.1 wrong-data 29 of 96 OpenUnderstand entities have no Understand counterpart
5.5 missing-data 2 reference pairs have unequal forward/inverse row counts
5.1 wrong-data 5 references written as 'Java Extend Couple Implicit External', which Understand never emits here
5.1 wrong-data 5 references written as 'Java Extendby Coupleby Implicit External', which Understand never emits here
5.0 wrong-data SumCyclomatic disagrees with Understand on 9 of 28 entities (68% agreement)
4.8 wrong-data CountStmt disagrees with Understand on 8 of 28 entities (71% agreement)
4.2 wrong-data CountInput disagrees with Understand on 6 of 14 entities (57% agreement)
4.2 wrong-data CountStmtExe disagrees with Understand on 6 of 28 entities (79% agreement)
3.9 wrong-data 3 duplicate entity rows across 3 logical entities
3.9 wrong-data CountLineCodeDecl disagrees with Understand on 5 of 28 entities (82% agreement)
3.9 wrong-data CountOutput disagrees with Understand on 5 of 14 entities (64% agreement)
3.7 missing-data 12 references Understand finds are absent, ignoring position (recall 98%)

ganttproject

Understand OpenUnderstand
Entities 26076 26425
  • Entities matched: 24663 of 26076
  • Entities with no Understand counterpart: 1762
  • Open findings: 19

Entities by kind family

Family Understand OpenUnderstand Δ
variable 9016 9659 +643
method 7413 7428 +15
parameter 7058 6437 -621
class 1486 1066 -420
constructor 756 756 +0
interface 180 180 +0
package 102 106 +4
unknown 65 53 -12
file 0 740 +740

Largest reference gaps

Kind Understand OpenUnderstand Missing
Java Callby 15157 12862 +2295
Java Coupleby 6581 4492 +2089
Java Endby 9775 8150 +1625
Java End 9775 8150 +1625
Java Beginby 9703 8150 +1553
Java Begin 9703 8150 +1553
Java Define 25163 24564 +599
Java Typedby 8094 7559 +535
Java Typedby GenericArgument 1400 976 +424
Java DotRefby 2792 2420 +372

Open findings

Score Severity Finding
22.6 wrong-data 3054 references written as 'Java Useby Annotation', which Understand never emits here
22.6 wrong-data 3054 references written as 'Java Use Annotation', which Understand never emits here
20.7 wrong-data 5736 of 30429 OpenUnderstand entities have no Understand counterpart
17.9 wrong-data 563 duplicate entity rows across 423 logical entities
17.0 wrong-data 417 references written as 'Java Extend Couple Implicit External', which Understand never emits here
17.0 wrong-data 417 references written as 'Java Extendby Coupleby Implicit External', which Understand never emits here
15.3 wrong-data 229 references written as 'Java Import Demand', which Understand never emits here
15.0 missing-data 35726 references Understand finds are absent, ignoring position (recall 87%)
13.4 wrong-data 115 references written as 'Java Extendby Coupleby External', which Understand never emits here
13.4 wrong-data 115 references written as 'Java Extend Couple External', which Understand never emits here
11.0 wrong-data 98 references have the right line but the wrong column (236492 of 236590 match once the column is included)
10.4 missing-data 1383 of 26076 entities Understand finds are absent from OpenUnderstand

jfreechart

Understand OpenUnderstand
Entities 44277 42746
  • Entities matched: 41263 of 44277
  • Entities with no Understand counterpart: 1483
  • Open findings: 25

Entities by kind family

Family Understand OpenUnderstand Δ
variable 19674 19142 -532
parameter 11972 9877 -2095
method 10496 10549 +53
class 929 990 +61
constructor 840 840 +0
unknown 207 164 -43
interface 114 114 +0
package 44 47 +3
module 1 0 -1
file 0 1023 +1023

Largest reference gaps

Kind Understand OpenUnderstand Missing
Java Callby 34733 31722 +3011
Java Useby Return 2714 1263 +1451
Java Use Return 2714 1271 +1443
Java Endby 12297 11660 +637
Java End 12297 11660 +637
Java Createby 7634 7066 +568
Java Callby Nondynamic 5321 4775 +546
Java Beginby 12205 11660 +545
Java Begin 12205 11660 +545
Java Coupleby 5444 5076 +368

Open findings

Score Severity Finding
23.9 wrong-data 4787 references written as 'Java Use Annotation', which Understand never emits here
23.9 wrong-data 4787 references written as 'Java Useby Annotation', which Understand never emits here
23.6 wrong-data 4341 duplicate entity rows across 2668 logical entities
19.8 wrong-data 1094 references written as 'Java Importby', which Understand never emits here
19.4 wrong-data 3417 of 47640 OpenUnderstand entities have no Understand counterpart
19.2 wrong-data 895 references written as 'Java Throwby', which Understand never emits here
19.2 wrong-data 895 references written as 'Java Throw', which Understand never emits here
17.8 wrong-data 550 references written as 'Java Extendby Coupleby Implicit External', which Understand never emits here
17.8 wrong-data 550 references written as 'Java Extend Couple Implicit External', which Understand never emits here
16.0 missing-data 71364 references Understand finds are absent, ignoring position (recall 87%)
15.0 wrong-data 201 references written as 'Java Useby Constrains Coupleby', which Understand never emits here
15.0 wrong-data 201 references written as 'Java Use Constrains Couple', which Understand never emits here

jhotdraw-develop

Understand OpenUnderstand
Entities 27694 27010
  • Entities matched: 25315 of 27694
  • Entities with no Understand counterpart: 1695
  • Open findings: 23

Entities by kind family

Family Understand OpenUnderstand Δ
variable 10514 10882 +368
parameter 8317 6763 -1554
method 6843 6843 +0
class 1027 832 -195
constructor 791 791 +0
package 73 74 +1
unknown 72 43 -29
interface 57 57 +0
file 0 725 +725

Largest reference gaps

Kind Understand OpenUnderstand Missing
Java Callby 13428 11895 +1533
Java Coupleby 4330 3171 +1159
Java Endby 8678 7740 +938
Java End 8678 7740 +938
Java Beginby 8677 7740 +937
Java Begin 8677 7740 +937
Java Useby 28771 27863 +908
Java Callby Nondynamic 3583 2912 +671
Java Typedby 5323 4986 +337
Java Createby 2018 1744 +274

Open findings

Score Severity Finding
23.1 wrong-data 3621 references written as 'Java Use Annotation', which Understand never emits here
23.1 wrong-data 3621 references written as 'Java Useby Annotation', which Understand never emits here
21.0 wrong-data 6612 of 31957 OpenUnderstand entities have no Understand counterpart
20.9 wrong-data 1628 references written as 'Java Import Demand', which Understand never emits here
18.7 wrong-data 754 duplicate entity rows across 474 logical entities
18.0 wrong-data 584 references written as 'Java Throw', which Understand never emits here
18.0 wrong-data 584 references written as 'Java Throwby', which Understand never emits here
14.7 missing-data 29050 references Understand finds are absent, ignoring position (recall 90%)
14.7 wrong-data 182 references written as 'Java Extend Couple External', which Understand never emits here
14.7 wrong-data 182 references written as 'Java Extendby Coupleby External', which Understand never emits here
14.7 wrong-data 179 references written as 'Java Extend Couple Implicit External', which Understand never emits here
14.7 wrong-data 179 references written as 'Java Extendby Coupleby Implicit External', which Understand never emits here

jvlt-1.3.2

Understand OpenUnderstand
Entities 7945 8861
  • Entities matched: 7633 of 7945
  • Entities with no Understand counterpart: 1228
  • Open findings: 17

Entities by kind family

Family Understand OpenUnderstand Δ
variable 3666 4479 +813
parameter 1779 1615 -164
method 1706 1706 +0
class 405 451 +46
constructor 344 344 +0
package 25 25 +0
interface 15 15 +0
unknown 5 5 +0
file 0 221 +221

Largest reference gaps

Kind Understand OpenUnderstand Missing
Java Callby 4068 3648 +420
Java Coupleby 1753 1440 +313
Java Useby 7170 6911 +259
Java DotRefby 1339 1103 +236
Java Typedby 1798 1699 +99
Java End 2456 2371 +85
Java Begin 2456 2371 +85
Java Beginby 2456 2371 +85
Java Endby 2456 2371 +85
Java Use Deref Partial 6912 6834 +78

Open findings

Score Severity Finding
18.8 wrong-data 2632 of 10268 OpenUnderstand entities have no Understand counterpart
15.6 wrong-data 252 references written as 'Java Useby Annotation', which Understand never emits here
15.6 wrong-data 252 references written as 'Java Use Annotation', which Understand never emits here
13.9 wrong-data 138 duplicate entity rows across 108 logical entities
13.9 wrong-data 135 references written as 'Java Extendby Coupleby Implicit External', which Understand never emits here
13.9 wrong-data 135 references written as 'Java Extend Couple Implicit External', which Understand never emits here
12.6 wrong-data 87 references written as 'Java Extend Couple External', which Understand never emits here
12.6 wrong-data 87 references written as 'Java Extendby Coupleby External', which Understand never emits here
12.6 missing-data 6604 references Understand finds are absent, ignoring position (recall 93%)
8.2 missing-data 309 of 7945 entities Understand finds are absent from OpenUnderstand
4.5 wrong-data 4 references written as 'Java Use Constrains Couple', which Understand never emits here
4.5 wrong-data 4 references written as 'Java Useby Constrains Coupleby', which Understand never emits here

testing_legacy_code

Understand OpenUnderstand
Entities 399 235
  • Entities matched: 197 of 399
  • Entities with no Understand counterpart: 38
  • Open findings: 12

Entities by kind family

Family Understand OpenUnderstand Δ
method 170 84 -86
variable 147 63 -84
class 36 20 -16
parameter 35 19 -16
package 6 6 +0
constructor 5 5 +0
file 0 38 +38

Largest reference gaps

Kind Understand OpenUnderstand Missing
Java Typed GenericArgument 24 18 +6
Java Typedby GenericArgument 24 18 +6
Java DotRefby 9 9 +0
Java Useby Deref Partial 158 158 +0
Java Useby GenericArgument 4 4 +0
Java Setby Init 146 146 +0
Java Use Return 4 4 +0
Java End 211 211 +0
Java Useby Castby 15 15 +0
Java Createby 69 69 +0

Open findings

Score Severity Finding
14.4 wrong-data 165 references written as 'Java Useby Annotation', which Understand never emits here
14.4 wrong-data 165 references written as 'Java Use Annotation', which Understand never emits here
11.2 wrong-data 109 of 306 OpenUnderstand entities have no Understand counterpart
8.7 wrong-data 21 references written as 'Java Importby Demand', which Understand never emits here
8.7 wrong-data 21 references written as 'Java Import Demand', which Understand never emits here
7.6 wrong-data 14 references written as 'Java Extend Couple Implicit External', which Understand never emits here
7.6 wrong-data 14 references written as 'Java Extendby Coupleby Implicit External', which Understand never emits here
7.6 missing-data 202 of 399 entities Understand finds are absent from OpenUnderstand
6.5 missing-data 90 references Understand finds are absent, ignoring position (recall 97%)
3.9 wrong-data 3 duplicate entity rows across 1 logical entities
3.1 wrong-data 2 references written as 'Java Extendby Coupleby External', which Understand never emits here
3.1 wrong-data 2 references written as 'Java Extend Couple External', which Understand never emits here

xerces2j

Understand OpenUnderstand
Entities 35218 34259
  • Entities matched: 32700 of 35218
  • Entities with no Understand counterpart: 1559
  • Open findings: 11

Entities by kind family

Family Understand OpenUnderstand Δ
variable 13596 13295 -301
parameter 11203 9753 -1450
method 8554 8554 +0
constructor 862 862 +0
class 801 879 +78
interface 157 157 +0
package 45 48 +3
file 0 711 +711

Largest reference gaps

Kind Understand OpenUnderstand Missing
Java Callby 14304 12030 +2274
Java Useby 57121 55257 +1864
Java Callby Nondynamic 6530 5082 +1448
Java Beginby 10374 9408 +966
Java Endby 10374 9408 +966
Java Begin 10374 9408 +966
Java End 10374 9408 +966
Java Setby Init 11420 10868 +552
Java Set Init 11420 10961 +459
Java Setby 9914 9566 +348

Open findings

Score Severity Finding
20.1 wrong-data 4438 of 37183 OpenUnderstand entities have no Understand counterpart
18.9 wrong-data 821 duplicate entity rows across 569 logical entities
16.3 wrong-data 317 references written as 'Java Extendby Coupleby Implicit External', which Understand never emits here
16.3 wrong-data 317 references written as 'Java Extend Couple Implicit External', which Understand never emits here
15.1 missing-data 36798 references Understand finds are absent, ignoring position (recall 91%)
11.7 wrong-data 61 references written as 'Java Extend Couple External', which Understand never emits here
11.7 wrong-data 61 references written as 'Java Extendby Coupleby External', which Understand never emits here
11.2 missing-data 2473 of 35218 entities Understand finds are absent from OpenUnderstand
8.6 wrong-data 20 references match only once ent and scope are swapped
7.7 wrong-data 24 references have the right line but the wrong column (395224 of 395248 match once the column is included)
2.6 wrong-data 2 entity long names are structurally malformed