Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add clamptype mechanism, to project into cotangent space #16

Closed
wants to merge 10 commits into from

Conversation

mcabbott
Copy link
Member

@mcabbott mcabbott commented May 5, 2021

This adds a mechanism to constrain tangents based on the type of the input. The initial goal is ensuring that real numbers do not accidentally acquire complex gradients, and that some LinearAlgebra structured arrays are preserved.

Edit -- the bulk of the implementation is now at FluxML/Zygote.jl#965, this is just a stub. (Which ought to be non-breaking.) I've moved discussion there.

@mcabbott mcabbott marked this pull request as draft May 5, 2021 20:21
@willtebbutt
Copy link
Member

willtebbutt commented May 6, 2021

I generally really like this, but I wonder whether there's a better name availlable.

This is a bit verbose, but something like project_onto_cotangent_space but might be more informative?

My other question is whether there are any situations in which type information is insufficient to make this work? For example, to know how to clamp a SparseMatrixCSC you need to know the precise sparsity pattern, which is only available at runtime, so this seems like an example where you need the object itself.

On a separate note, what are the considerations regarding doing this here rather than in ChainRules? It's been something I've been thinking about a bit (see here -- I've not gotten very far with it yet though: JuliaDiff/ChainRulesCore.jl#286), so I'd be keen to see it happen in ChainRules. Once we've got Zygote onto ChainRules types, the rule-definition tools here could just utilise this functionality from ChainRules, so it seems plausible that we would just want to implement it there.

@mcabbott
Copy link
Member Author

mcabbott commented May 6, 2021

Can I suggest we take the discussion to the Zygote PR? I put most of the words there now.

Done, thanks! Could delete this.

@mcabbott mcabbott changed the title Add clamptype mechanism, to project onto to tangent space Add clamptype mechanism, to project into cotangent space May 7, 2021
@mcabbott mcabbott closed this Aug 19, 2021
@mcabbott mcabbott deleted the real branch August 19, 2021 03:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants