Skip to content

Treat discrete JAXOp inputs as disconnected in the gradient - #2418

Open
rishabhsai wants to merge 2 commits into
pymc-devs:mainfrom
rishabhsai:fix-jaxop-grad-integer-inputs
Open

rishabhsai wants to merge 2 commits into
pymc-devs:mainfrom
rishabhsai:fix-jaxop-grad-integer-inputs

Conversation

@rishabhsai

Copy link
Copy Markdown

Description

JAXOp.pullback sent every input through jax.vjp and declared the VJP op's output types as [self.input_types[i] for i in range(num_inputs)], so an integer input came back with an integer gradient and pytensor/gradient.py rejected it with "grad illegally returned an integer-valued variable". Returning DisconnectedType instead was not possible either, because JAXOp had no connection_pattern, so Blockwise assumed every input was connected and refused the disconnected term. That makes grad fail for any wrap_jax function taking an index array, with or without pt.vectorize.

Discrete inputs are now held constant in the VJP, jax.vjp only sees the float inputs, and their gradient is reported as DisconnectedType. JAXOp.connection_pattern marks discrete inputs as disconnected from every output, which Blockwise.connection_pattern picks up for the vectorized case. This is the same arrangement AdvancedSubtensor uses for its index inputs.

Behaviour change: grad with respect to a discrete wrap_jax input now raises DisconnectedInputError rather than silently producing an integer zero.

Related Issue

Checklist

Type of change

  • New feature / enhancement
  • Bug fix
  • Documentation
  • Maintenance
  • Other (please specify):

Comment thread tests/link/jax/test_wrap_jax.py Outdated
with pytest.raises(DisconnectedInputError):
grad(out, [idx])

# Blockwise consults the core Op's connection_pattern, so the same must hold

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not the same, you now used a continuous input? I'd also like to see a test for wrap_jax with mixed input types. The blockwise itself was a symptom downstream, I don't think we need to test it

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dropped the Blockwise block, agreed it was a downstream symptom. On the continuous input: the discrete assertions were the two lines above it (int32 idx, DisconnectedInputError), and the vectorized block did differentiate only the float batched_x, so it was carrying the connection_pattern coverage by accident rather than asserting anything discrete. Replaced it with test_mixed_input_types, which interleaves float x, int32 idx, float y and bool mask, checks the numeric gradients of x and y, asserts both discrete inputs raise DisconnectedInputError, and asserts connection_pattern on the op directly so that half of the fix stays covered without Blockwise.

@rishabhsai
rishabhsai force-pushed the fix-jaxop-grad-integer-inputs branch from 7f52d10 to 7c85996 Compare September 16, 2026 03:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUG: JAXOp.grad fails when inputs include integer-typed tensors

2 participants