Precision100.0% (±0.0%)
MODULE DETAIL
External Corpus Validation (CodeQL) Third-party
Classification claim · 23 test cases · tool: 1.171.0
This corpus is unmodified code from an independent third-party project, not something QShield's team generated — see the corpus notes below for provenance, licensing, and how ground truth was derived.
Every test case in this module: the expected result (ground truth -- for aws_discovery this includes the exact Moto resource configuration each case was scored against) paired with what the real scanner/collector actually reported, for all 23 cases -- not just the false positive/negative samples shown below.
Recall100.0% (±0.0%)
F11.000
Accuracy100.0% (±0.0%)
CONFUSION MATRIX
Outcome counts
True positives12
True negatives11
False positives0
False negatives0
RUN NOTES
Notes
- semgrep available: True
- 23 files vendored unmodified from github/codeql (MIT license, pinned commit d1fed84daf8f3abc59cf9453692b98632932d60f) -- see benchmark/corpus/vendor/codeql_quantum/NOTICE.md
- Ground truth is QShield's own file-level classification of each vendored file by which JCA algorithm-factory calls it contains, not borrowed from any CodeQL query result -- see benchmark/corpus/codeql_quantum.py CLASSIFICATION for the evidence behind every verdict.
CORPUS
Corpus composition
Cases23
Wall time4.2s
- semgrep available: True
- 23 files vendored unmodified from github/codeql (MIT license, pinned commit d1fed84daf8f3abc59cf9453692b98632932d60f) -- see benchmark/corpus/vendor/codeql_quantum/NOTICE.md
- Ground truth is QShield's own file-level classification of each vendored file by which JCA algorithm-factory calls it contains, not borrowed from any CodeQL query result -- see benchmark/corpus/codeql_quantum.py CLASSIFICATION for the evidence behind every verdict.
MISCLASSIFICATIONS
False positives & false negatives
None observed in this run.