PythonCall #185
Draft
PythonCall #185
+141
−3
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Comment on lines
+935
to
+987
| function wrap_with_python_boilerplate(code) | ||
| """ | ||
| @isdefined(PythonCall) && PythonCall isa Module && Base.PkgId(PythonCall).uuid == Base.UUID("6099a3de-0909-46bc-b1f4-468b9a2dfc0d") || error("PythonCall must be imported to execute Python code cells with QuartoNotebookRunner") | ||
| let | ||
| code = "$code" | ||
|
|
||
| ast = PythonCall.pyimport("ast") | ||
| tree = ast.parse(code) | ||
|
|
||
| body = tree.body | ||
|
|
||
| result = nothing | ||
| if body !== nothing | ||
| for (i, node) in enumerate(body) | ||
| nodecode = PythonCall.pyconvert(String, ast.unparse(node)) | ||
| if i < length(body) | ||
| PythonCall.pyexec(nodecode, Main.Notebook) | ||
| else | ||
| eval_allowed_nodes = ( | ||
| ast.Expression, # A wrapper for expressions in eval context | ||
| ast.Expr, | ||
| ast.BinOp, # Binary operations like 1 + 1 | ||
| ast.BoolOp, # Boolean operations like "and", "or" | ||
| ast.Call, # Function call like my_func() | ||
| ast.Compare, # Comparisons like a > b | ||
| ast.Constant, # Constants like numbers, strings (Python 3.8+) | ||
| ast.Dict, # Dictionary literals | ||
| ast.List, # List literals | ||
| ast.Name, # Variable names | ||
| ast.Set, # Set literals | ||
| ast.Tuple, # Tuple literals | ||
| ast.UnaryOp, # Unary operations like -1 | ||
| ast.Lambda # Lambda functions | ||
| ) | ||
| if any(t -> PythonCall.pyisinstance(node, t), eval_allowed_nodes) | ||
| result = PythonCall.pyeval(Any, nodecode, Main.Notebook) | ||
| else | ||
| PythonCall.pyexec(nodecode, Main.Notebook) | ||
| if PythonCall.pyisinstance(node, ast.Assign) | ||
| for target in node.targets | ||
| # TODO: how to know whether it's a single value or a one-element tuple? | ||
| # currently throwing away results 2 to n | ||
| result = PythonCall.pyeval(Any, ast.unparse(target), Main.Notebook) | ||
| end | ||
| end | ||
| end | ||
| end | ||
| end | ||
| end | ||
| result | ||
| end | ||
| """ | ||
| end |
There was a problem hiding this comment.
Can this be defined in a reuseable function within the worker package so that it doesn't have to be reparsed/reevaluated on every single cell?
There was a problem hiding this comment.
Yeah right now this is just prototype code I wanted to show, I'm sure most bits can be factored out
|
I played around with this a bit and I had to rebase this to main, to make it work on julia 1.11. This is the error I get: DetailsERROR: Julia server returned error after receiving "run" command:
Failed to run notebook: /home/fcremer/Documents/NFDI4Earth/lhbarticles/Intro_Raster_Data_Analysis_ENG_subset.qmd
ERROR: EvaluationError: Encountered 1 error during evaluation
Error 1 of 1
@ /home/fcremer/Documents/NFDI4Earth/lhbarticles/Intro_Raster_Data_Analysis_ENG_subset.qmd:119
Python: ImportError: /home/fcremer/Documents/NFDI4Earth/lhbarticles/.CondaPkg/env/lib/python3.11/site-packages/rasterio/../../.././libspatialite.so.7: undefined symbol: xmlNanoHTTPCleanup, version LIBXML2_2.4.30
Stacktrace:
[1] pythrow()
@ PythonCall.Core ~/.julia/packages/PythonCall/Nr75f/src/Core/err.jl:92
[2] errcheck
@ ~/.julia/packages/PythonCall/Nr75f/src/Core/err.jl:10 [inlined]
[3] pycallargs(f::Py, args::Py)
@ PythonCall.Core ~/.julia/packages/PythonCall/Nr75f/src/Core/builtins.jl:220
[4] pycall(::Py, ::String, ::Vararg{Any}; kwargs::@Kwargs{})
@ PythonCall.Core ~/.julia/packages/PythonCall/Nr75f/src/Core/builtins.jl:243
[5] pycall(::Py, ::String, ::Vararg{Any})
@ PythonCall.Core ~/.julia/packages/PythonCall/Nr75f/src/Core/builtins.jl:233
[6] (::Py)(::String, ::Vararg{Any}; kwargs::@Kwargs{})
@ PythonCall.Core ~/.julia/packages/PythonCall/Nr75f/src/Core/Py.jl:357
[7] pyexec
@ ~/.julia/packages/PythonCall/Nr75f/src/Core/builtins.jl:1326 [inlined]
[8] pyexec
@ ~/.julia/packages/PythonCall/Nr75f/src/Core/builtins.jl:1331 [inlined]
[9] pyexec(code::String, globals::Module)
@ PythonCall.Core ~/.julia/packages/PythonCall/Nr75f/src/Core/builtins.jl:1331
[10] top-level scope
@ ~/Documents/NFDI4Earth/lhbarticles/Intro_Raster_Data_Analysis_ENG_subset.qmd:159
ERROR: Internal julia server error
Stack trace:
at writeJuliaCommand (file:///opt/quarto/bin/quarto.js:41397:19)
at eventLoopTick (ext:core/01_core.js:175:7)
at async executeJulia (file:///opt/quarto/bin/quarto.js:41291:22)
at async Object.execute (file:///opt/quarto/bin/quarto.js:41028:20)
at async renderExecute (file:///opt/quarto/bin/quarto.js:85764:27)
at async renderFileInternal (file:///opt/quarto/bin/quarto.js:85932:43)
at async renderFiles (file:///opt/quarto/bin/quarto.js:85800:17)
at async render (file:///opt/quarto/bin/quarto.js:90702:21)
at async renderForPreview (file:///opt/quarto/bin/quarto.js:91729:26)
at async render (file:///opt/quarto/bin/quarto.js:91612:29) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Played around with the
astmodule to try and fix the problems of #180, mainly that the "Python" code there actually had to be valid Julia syntax, too, and that it wouldn't work with multiple expressions or any more complex thing.Python only allows
evalto be called on certain expressions and we can useastto analyze whether we got those. Otherwise we callexec. One more special case is assignments, where we firstexecand thenevalthe variable out.I got this working so far: