Skip to content

Update EXP39-C to use the new dataflow library - #1201

Open
MichaelRFairhurst wants to merge 1 commit into
mainfrom
michaelrfairhurst/use-new-dataflow-exp39-c
Open

MichaelRFairhurst wants to merge 1 commit into
mainfrom
michaelrfairhurst/use-new-dataflow-exp39-c

Conversation

@MichaelRFairhurst

Copy link
Copy Markdown
Collaborator

Description

Update EXP39-C to use the new dataflow library.

This query has other issues I decided not to immediately address -- see #1200. The query actually makes all sources sinks in order to report casts from A* to B* (though this doesn't technically cause UB or violate the rule), only truly requiring dataflow to handle the realloc case.

Updating to the new dataflow began reporting both x and &x in (T*) &x. To continue reporting the issue at the sink, and to exclude T, I check that the node.asExpr() satisfies the expression's sink conditions, that it's either a cast or a dereference.

This query is very slow, partly because its design essentially disables forward/reverse pruning. Roughly timing it on my system indicates that this slowness isn't affected by the dataflow update, so its not something I worried about addressing here.

Change request type

  • Release or process automation (GitHub workflows, internal scripts)
  • Internal documentation
  • External documentation
  • Query files (.ql, .qll, .qls or unit tests)
  • External scripts (analysis report or other code shipped as part of a release)

Rules with added or modified queries

  • No rules added
  • Queries have been added for the following rules:
    • rule number here
  • Queries have been modified for the following rules:
    • EXP39-C

Release change checklist

A change note (development_handbook.md#change-notes) is required for any pull request which modifies:

  • The structure or layout of the release artifacts.
  • The evaluation performance (memory, execution time) of an existing query.
  • The results of an existing query in any circumstance.

If you are only adding new rule queries, a change note is not required.

Author: Is a change note required?

  • Yes
  • No

🚨🚨🚨
Reviewer: Confirm that format of shared queries (not the .qll file, the
.ql file that imports it) is valid by running them within VS Code.

  • Confirmed

Reviewer: Confirm that either a change note is not required or the change note is required and has been added.

  • Confirmed

Query development review checklist

For PRs that add new queries or modify existing queries, the following checklist should be completed by both the author and reviewer:

Author

  • Have all the relevant rule package description files been checked in?
  • Have you verified that the metadata properties of each new query is set appropriately?
  • Do all the unit tests contain both "COMPLIANT" and "NON_COMPLIANT" cases?
  • Are the alert messages properly formatted and consistent with the style guide?
  • Have you run the queries on OpenPilot and verified that the performance and results are acceptable?
    As a rule of thumb, predicates specific to the query should take no more than 1 minute, and for simple queries be under 10 seconds. If this is not the case, this should be highlighted and agreed in the code review process.
  • Does the query have an appropriate level of in-query comments/documentation?
  • Have you considered/identified possible edge cases?
  • Does the query not reinvent features in the standard library?
  • Can the query be simplified further (not golfed!)

Reviewer

  • Have all the relevant rule package description files been checked in?
  • Have you verified that the metadata properties of each new query is set appropriately?
  • Do all the unit tests contain both "COMPLIANT" and "NON_COMPLIANT" cases?
  • Are the alert messages properly formatted and consistent with the style guide?
  • Have you run the queries on OpenPilot and verified that the performance and results are acceptable?
    As a rule of thumb, predicates specific to the query should take no more than 1 minute, and for simple queries be under 10 seconds. If this is not the case, this should be highlighted and agreed in the code review process.
  • Does the query have an appropriate level of in-query comments/documentation?
  • Have you considered/identified possible edge cases?
  • Does the query not reinvent features in the standard library?
  • Can the query be simplified further (not golfed!)

Copilot AI balanced review requested due to automatic review settings September 20, 2026 20:51

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Mandatory QLDoc, line-length, and change-note clarity issues remain.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 3 Low severity

Open (3)
What changed in this PR

Migrates EXP39-C from the deprecated data-flow library to the new CodeQL data-flow API while preserving reported alerts.

Changes:

  • Updates data-flow configuration and sink filtering.
  • Refreshes expected path-graph output.
  • Adds the required change note.
  • Manual VS Code validation remains unconfirmed.
File Description
change_notes/​2026-09-20-update-exp39-c-to-use-new-dataflow.md Documents the migration.
c/​cert/​src/​rules/​EXP39-C/​DoNotAccessVariableViaPointerOfIncompatibleType.ql Migrates the query to new data flow.
c/​cert/​test/​rules/​EXP39-C/​DoNotAccessVariableViaPointerOfIncompatibleType.expected Updates expected paths and locations.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.


class IndirectCastAnalysisUnconvertedCastExpr extends Expr {
IndirectCastAnalysisUnconvertedCastExpr() { this = any(Cast c).getUnconverted() }
class CastedExpr extends Expr {
where
not isExcluded(sink.getNode().asExpr(),
Pointers3Package::doNotAccessVariableViaPointerOfIncompatibleTypeQuery()) and
not isExcluded(sinkExpr, Pointers3Package::doNotAccessVariableViaPointerOfIncompatibleTypeQuery()) and
Comment on lines +2 to +3
- Updated the underlying dataflow library to use the new dataflow from the deprecated old data flow. This may cause behavioral changes in some analyses, usually with improved results and performance.
- Some refactoring required to accommodate functional changes in how dataflow nodes attach to the AST. No functional changes to the analysis from this are expected. No newline at end of file
@MichaelRFairhurst

Copy link
Copy Markdown
Collaborator Author

/test-performance

@github-actions

Copy link
Copy Markdown

🏁 Beep Boop! Performance testing for this PR has been initiated. Please check back later for results. Note that the query package generation step must complete before testing will start so it might be a minute.

💡 If you do not hear back from me please check my status! I will report even if I fail!

@codeql-coding-standards-automation

Copy link
Copy Markdown

🏁 Beep Boop! Performance testing complete! See below for performance of the last 3 runs vs your PR. Times are based on predicate performance. You can find full graphs and stats in the PR that was created for this test in the release engineering repo.


Release                            : v2.61.0
Platform                           : x86-linux
Language                           : cpp
Total_Serialized_Execution_Time_Ms : 4615626
Mean_Predicate_Execution_Time_Ms   : 64.84988900441172
Median_Predicate_Execution_Time_Ms : 0.0
Standard_Deviation_Ms              : 574.4148904892992
Total_Serialized_Execution_Time_s  : 4615.626
Mean_Query_Execution_Time_s        : 0.0648498890044117
Median_Predicate_Execution_Time_s  : 0.0
Percentile95_Ms                    : 165.0
Number_of_Predicates               : 71174

Release                            : v2.61.0
Platform                           : x86-windows
Language                           : c
Total_Serialized_Execution_Time_Ms : 3266370
Mean_Predicate_Execution_Time_Ms   : 63.90362718628947
Median_Predicate_Execution_Time_Ms : 0.0
Standard_Deviation_Ms              : 503.1579843913974
Total_Serialized_Execution_Time_s  : 3266.37
Mean_Query_Execution_Time_s        : 0.0639036271862894
Median_Predicate_Execution_Time_s  : 0.0
Percentile95_Ms                    : 191.0
Number_of_Predicates               : 51114

Release                            : v2.61.0
Platform                           : x86-windows
Language                           : cpp
Total_Serialized_Execution_Time_Ms : 4756186
Mean_Predicate_Execution_Time_Ms   : 66.62911337433282
Median_Predicate_Execution_Time_Ms : 1.0
Standard_Deviation_Ms              : 451.0173047967091
Total_Serialized_Execution_Time_s  : 4756.186
Mean_Query_Execution_Time_s        : 0.0666291133743328
Median_Predicate_Execution_Time_s  : 0.001
Percentile95_Ms                    : 208.0
Number_of_Predicates               : 71383

Release                            : v2.61.0
Platform                           : x86-linux
Language                           : c
Total_Serialized_Execution_Time_Ms : 2784651
Mean_Predicate_Execution_Time_Ms   : 54.388777124553215
Median_Predicate_Execution_Time_Ms : 0.0
Standard_Deviation_Ms              : 610.0917903271135
Total_Serialized_Execution_Time_s  : 2784.651
Mean_Query_Execution_Time_s        : 0.0543887771245532
Median_Predicate_Execution_Time_s  : 0.0
Percentile95_Ms                    : 142.0
Number_of_Predicates               : 51199

Release                            : v2.62.0
Platform                           : x86-linux
Language                           : c
Total_Serialized_Execution_Time_Ms : 2655267
Mean_Predicate_Execution_Time_Ms   : 51.8738546896673
Median_Predicate_Execution_Time_Ms : 0.0
Standard_Deviation_Ms              : 558.1246726024739
Total_Serialized_Execution_Time_s  : 2655.267
Mean_Query_Execution_Time_s        : 0.0518738546896673
Median_Predicate_Execution_Time_s  : 0.0
Percentile95_Ms                    : 138.0
Number_of_Predicates               : 51187

Release                            : v2.62.0
Platform                           : x86-linux
Language                           : cpp
Total_Serialized_Execution_Time_Ms : 4355006
Mean_Predicate_Execution_Time_Ms   : 61.04492507814581
Median_Predicate_Execution_Time_Ms : 0.0
Standard_Deviation_Ms              : 537.0863746976181
Total_Serialized_Execution_Time_s  : 4355.006
Mean_Query_Execution_Time_s        : 0.0610449250781458
Median_Predicate_Execution_Time_s  : 0.0
Percentile95_Ms                    : 165.0
Number_of_Predicates               : 71341

Release                            : v2.62.0
Platform                           : x86-windows
Language                           : cpp
Total_Serialized_Execution_Time_Ms : 4958885
Mean_Predicate_Execution_Time_Ms   : 69.36085545640194
Median_Predicate_Execution_Time_Ms : 1.0
Standard_Deviation_Ms              : 468.2069608070895
Total_Serialized_Execution_Time_s  : 4958.885
Mean_Query_Execution_Time_s        : 0.0693608554564019
Median_Predicate_Execution_Time_s  : 0.001
Percentile95_Ms                    : 218.0
Number_of_Predicates               : 71494

Release                            : v2.62.0
Platform                           : x86-windows
Language                           : c
Total_Serialized_Execution_Time_Ms : 3809287
Mean_Predicate_Execution_Time_Ms   : 74.47432012356057
Median_Predicate_Execution_Time_Ms : 0.0
Standard_Deviation_Ms              : 607.469703694893
Total_Serialized_Execution_Time_s  : 3809.287
Mean_Query_Execution_Time_s        : 0.0744743201235605
Median_Predicate_Execution_Time_s  : 0.0
Percentile95_Ms                    : 196.0
Number_of_Predicates               : 51149

Release                            : 1201
Platform                           : x86-linux
Language                           : c
Total_Serialized_Execution_Time_Ms : 2258413
Mean_Predicate_Execution_Time_Ms   : 44.11996952410721
Median_Predicate_Execution_Time_Ms : 0.0
Standard_Deviation_Ms              : 466.2639140173117
Total_Serialized_Execution_Time_s  : 2258.413
Mean_Query_Execution_Time_s        : 0.0441199695241072
Median_Predicate_Execution_Time_s  : 0.0
Percentile95_Ms                    : 122.0
Number_of_Predicates               : 51188

Release                            : 1201
Platform                           : x86-linux
Language                           : cpp
Total_Serialized_Execution_Time_Ms : 3902106
Mean_Predicate_Execution_Time_Ms   : 54.68197869955157
Median_Predicate_Execution_Time_Ms : 0.0
Standard_Deviation_Ms              : 473.5482442293787
Total_Serialized_Execution_Time_s  : 3902.106
Mean_Query_Execution_Time_s        : 0.0546819786995515
Median_Predicate_Execution_Time_s  : 0.0
Percentile95_Ms                    : 157.0
Number_of_Predicates               : 71360

🏁 Below are the slowest predicates for the last 2 releases vs this PR.


Release           : v2.62.0
Run               : 2026-07-27_15-36-06
Platform          : x86-windows
Language          : cpp
Suite             : misra-default
Predicate         : QualifiedName::getUserTypeNameWithoutArgs/1#8cfc98e9
Execution_Time_Ms : 18257

Release           : v2.62.0
Run               : 2026-07-27_15-36-06
Platform          : x86-windows
Language          : cpp
Suite             : misra-default
Predicate         : IncompatibleFunctionDeclaration::interestedInFunctions/4#95575433
Execution_Time_Ms : 23074

Release           : v2.62.0
Run               : 2026-07-27_15-36-06
Platform          : x86-windows
Language          : cpp
Suite             : autosar-default
Predicate         : FunctionEquivalence::typeSig/1#194ac728
Execution_Time_Ms : 24488

Release           : v2.62.0
Run               : 2026-07-27_15-36-06
Platform          : x86-windows
Language          : cpp
Suite             : autosar-default
Predicate         : Dependency::dependsOnTransitive/2#cbda84a0
Execution_Time_Ms : 26611

Release           : v2.62.0
Run               : 2026-07-27_15-36-06
Platform          : x86-linux
Language          : cpp
Suite             : misra-default
Predicate         : Macro::MacroInvocation.getAnAffectedElement/0#dispred#d1462297_10#join_rhs
Execution_Time_Ms : 31340

Release           : v2.62.0
Run               : 2026-07-27_15-36-06
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : CheckedException::CheckedException#b0aa5ec8
Execution_Time_Ms : 27027

Release           : v2.62.0
Run               : 2026-07-27_15-36-06
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : QualifiedName::getUserTypeNameWithoutArgs/1#8cfc98e9
Execution_Time_Ms : 29311

Release           : v2.62.0
Run               : 2026-07-27_15-36-06
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : CharacterOutsideTheLanguageStandardBasicSourceCharacterSetUsedInTheSourceCode::getUniversalCharacterName/1#36dbaa42
Execution_Time_Ms : 33680

Release           : v2.62.0
Run               : 2026-07-27_15-36-06
Platform          : x86-linux
Language          : c
Suite             : cert-default
Predicate         : IncompatibleFunctionDeclaration::interestedInFunctions/4#95575433
Execution_Time_Ms : 26535

Release           : v2.62.0
Run               : 2026-07-27_15-36-06
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : _Class::Class.getALinkTarget/0#dispred#29b2b38a#bf_Class::Class.getALinkTarget/0#dispred#29b2b38a#bf__#shared
Execution_Time_Ms : 28287

Release           : v2.62.0
Run               : 2026-07-27_15-36-06
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : OutOfBounds::OOB::libraryFunctionNameParamTableSimpleString/5#6de8614f#cpe#1236
Execution_Time_Ms : 35596

Release           : v2.62.0
Run               : 2026-07-27_15-36-06
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : _Macro::Macro.getName/0#dispred#e28b3699_Preprocessor::PreprocessorBranchDirective#bcd2bde4#b_Prepro__#antijoin_rhs
Execution_Time_Ms : 35486

Release           : v2.62.0
Run               : 2026-07-27_15-36-06
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : Macro::MacroInvocation.getAnAffectedElement/0#dispred#d1462297_10#join_rhs
Execution_Time_Ms : 40850

Release           : v2.62.0
Run               : 2026-07-27_15-36-06
Platform          : x86-windows
Language          : cpp
Suite             : misra-default
Predicate         : Macro::MacroInvocation.getAnAffectedElement/0#dispred#d1462297_10#join_rhs
Execution_Time_Ms : 39979

Release           : v2.62.0
Run               : 2026-07-27_15-36-06
Platform          : x86-windows
Language          : c
Suite             : misra-default
Predicate         : _Macro::Macro.getName/0#dispred#e28b3699_Preprocessor::PreprocessorBranchDirective#bcd2bde4#b_Prepro__#antijoin_rhs
Execution_Time_Ms : 31549

Release           : v2.62.0
Run               : 2026-07-27_15-36-06
Platform          : x86-windows
Language          : c
Suite             : misra-default
Predicate         : QualifiedName::getUserTypeNameWithoutArgs/1#8cfc98e9
Execution_Time_Ms : 25356

Release           : v2.62.0
Run               : 2026-07-27_15-36-06
Platform          : x86-windows
Language          : c
Suite             : misra-default
Predicate         : _Class::Class.getALinkTarget/0#dispred#29b2b38a#bf_Class::Class.getALinkTarget/0#dispred#29b2b38a#bf__#shared
Execution_Time_Ms : 25854

Release           : v2.62.0
Run               : 2026-07-27_15-36-06
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : VirtualDispatchPrototype::VirtualDispatch::cannotInheritHelper/4#7c75bd87
Execution_Time_Ms : 27766

Release           : v2.62.0
Run               : 2026-07-27_15-36-06
Platform          : x86-windows
Language          : c
Suite             : misra-default
Predicate         : Macro::MacroInvocation.getAnAffectedElement/0#dispred#d1462297_10#join_rhs
Execution_Time_Ms : 32647

Release           : v2.62.0
Run               : 2026-07-27_15-36-06
Platform          : x86-windows
Language          : c
Suite             : misra-default
Predicate         : OutOfBounds::OOB::libraryFunctionNameParamTableSimpleString/5#6de8614f#cpe#1236
Execution_Time_Ms : 36107

Release           : 1201
Run               : 2026-09-21_14-30-07
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : Macro::MacroInvocation.getAnAffectedElement/0#dispred#d1462297_10#join_rhs
Execution_Time_Ms : 30171

Release           : 1201
Run               : 2026-09-21_14-30-07
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : _Macro::Macro.getName/0#dispred#e28b3699_Preprocessor::PreprocessorBranchDirective#bcd2bde4#b_Prepro__#antijoin_rhs
Execution_Time_Ms : 25232

Release           : 1201
Run               : 2026-09-21_14-30-07
Platform          : x86-linux
Language          : cpp
Suite             : misra-default
Predicate         : OutOfBounds::OOB::libraryFunctionNameParamTableSimpleString/5#0b6a13dd#cpe#1236
Execution_Time_Ms : 26644

Release           : 1201
Run               : 2026-09-21_14-30-07
Platform          : x86-linux
Language          : cpp
Suite             : misra-default
Predicate         : IncompatibleFunctionDeclaration::interestedInFunctions/4#95575433
Execution_Time_Ms : 25016

Release           : 1201
Run               : 2026-09-21_14-30-07
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : QualifiedName::getUserTypeNameWithoutArgs/1#8cfc98e9
Execution_Time_Ms : 24126

Release           : 1201
Run               : 2026-09-21_14-30-07
Platform          : x86-linux
Language          : cpp
Suite             : misra-default
Predicate         : Macro::MacroInvocation.getAnAffectedElement/0#dispred#d1462297_10#join_rhs
Execution_Time_Ms : 23922

Release           : 1201
Run               : 2026-09-21_14-30-07
Platform          : x86-linux
Language          : c
Suite             : cert-default
Predicate         : IncompatibleFunctionDeclaration::interestedInFunctions/4#95575433
Execution_Time_Ms : 26120

Release           : 1201
Run               : 2026-09-21_14-30-07
Platform          : x86-linux
Language          : c
Suite             : cert-default
Predicate         : OutOfBounds::OOB::libraryFunctionNameParamTable/5#79217c12
Execution_Time_Ms : 22188

Release           : 1201
Run               : 2026-09-21_14-30-07
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : OutOfBounds::OOB::libraryFunctionNameParamTableSimpleString/5#6de8614f#cpe#1236
Execution_Time_Ms : 30615

Release           : 1201
Run               : 2026-09-21_14-30-07
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : CharacterOutsideTheLanguageStandardBasicSourceCharacterSetUsedInTheSourceCode::getUniversalCharacterName/1#36dbaa42
Execution_Time_Ms : 27347

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants