Skip to content

Python: Use fastTC explicitly in localFlow - #22249

Open
MathiasVP wants to merge 1 commit into
github:mainfrom
MathiasVP:python-explicit-fasttc-local-flow-step
Open

Python: Use fastTC explicitly in localFlow#22249
MathiasVP wants to merge 1 commit into
github:mainfrom
MathiasVP:python-explicit-fasttc-local-flow-step

Conversation

@MathiasVP

@MathiasVP MathiasVP commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Microsoft hit a performance problem on an internal query. The root-cause was that the compiler suddenly decided not to compile a localFlowStep* to a fastTC, but instead to good old QL recursion (with a suboptimal bound on the sink as a cherry on top). This resulted in:

Pipeline standard for #DataFlowUtil::localFlowStep/2#5174ed45Plus#fb#flipped@38867we6 was evaluated in 15275 iterations totaling 774484ms (delta sizes total: 1576975703).
        1577556584  ~1%    {2} r1 = SCAN `#DataFlowUtil::localFlowStep/2#5174ed45Plus#fb#flipped#prev_delta` OUTPUT In.1, In.0
        1578595232  ~1%    {2}    | JOIN WITH `DataFlowUtil::localFlowStep/2#5174ed45#flipped` ON FIRST 1 OUTPUT Lhs.1, Rhs.1
        1576984587  ~1%    {2}    | AND NOT `#DataFlowUtil::localFlowStep/2#5174ed45Plus#fb#flipped#prev`(FIRST 2)
                           return r1

This PR explicitly uses fastTC to prevent this surprise from biting us all going forward.

We could of course fine-tune the calls to localFlow and ensure that the end-points are optimally bound, but we might as well follow what other languages have done in similar situations (see C#, C++, and Java).

DCA looks fantastic 🎉

@MathiasVP MathiasVP added the no-change-note-required This PR does not need a change note label Jul 29, 2026
@MathiasVP
MathiasVP marked this pull request as ready for review July 29, 2026 22:19
Copilot AI review requested due to automatic review settings July 29, 2026 22:19
@MathiasVP
MathiasVP requested a review from a team as a code owner July 29, 2026 22:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Explicitly uses fastTC for Python local data-flow transitive closure to avoid compiler-dependent performance regressions.

Changes:

  • Adds a private localFlowStepPlus predicate using fastTC.
  • Updates localFlow to include reflexive flow explicitly.
Show a summary per file
File Description
python/ql/lib/semmle/python/dataflow/new/internal/DataFlowUtil.qll Implements explicit fast transitive closure for local flow.

Review details

  • Files reviewed: 1/1 changed files
  • Comments generated: 0
  • Review effort level: Medium

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-change-note-required This PR does not need a change note Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants