https://github.com/nnicandro/emacs-jupyter uses a compiled module to communicate directly with a jupyter kernel using zeromq. This requires that you have an emacs with support for compiled modules, and that you are able to compile it. This is the default option in scimax. The older ob-ipython modules might work, but I don’t use or support them anymore.
(require 'scimax-jupyter)Overall, emacs-jupyter is an improvement on ob-ipython. There are a few things I want that don’t come out of the box with emacs-jupyter. Here are a few customizations I have done in scimax.
These are the settings that work well for me.
org-babel-default-header-args:jupyter-pythonSome of these are scimax-specific. For example :results . "both" captures both printed and returned values, which is most consistent with Jupyter notebooks. I set :pandoc . "t" to convert outputs like html to org format where possible.
I find it confusing to have one kernel shared among many files.
- It is easy to mess up the state if you use similar variables in different files
- I often assume the CWD is the file I work from, but the kernel starts in the directory it was started in, which is often different than another org-file
- I want the kernel to shutdown and close when I close the buffer because I don’t need it after that.
scimax makes that happen.
Try it: elisp:scimax-jupyter-org-hydra/body
Easy access to:
- inspect (M-i)
- completion (M-tab)
- editing functions
- kernel management
import numpy as np
?np.linspace
np.linspace
??np.linspace
If you have your cursor on linspace, type M-i or f12-/ to inspect it.
np.linspace
Use M-tab to complete the thing at point. Sometimes you have to type it more than once.
np.geomspace
Figures work like you expect.
import matplotlib.pyplot as plt
import numpy as np
t = np.linspace(0, 20 * np.pi, 350)
x = np.exp(-0.1 * t) * np.sin(t)
y = np.exp(-0.1 * t) * np.cos(t)
plt.plot(x, y)
plt.axis('equal')
plt.figure()
plt.plot(y, x)
plt.axis('equal')
print('Length of t = {}'.format(len(t)))
print('x .dot. y = {}'.format(x @ y))
Emacs still does not natively render html or interactive javascript. Until that happens, I monkey-patched plotly to capture a static image, and save the interactive html so you can still use it in a browser. This isn’t purely scimax, it requires you to pip install pycse, which is a Python package I wrote to do things like this.
from pycse.plotly import *
import plotly.express as px
df = px.data.iris()
fig = px.scatter(df, x="sepal_width", y="sepal_length", color="species",
size='petal_length', hover_data=['petal_width'])
fig.show()
using the pandoc: "t" header generally makes pandas behave well with org-mode. Turn that off if you want with an empty header like :pandoc; note you will get html output then.
import pandas as pd
f = pd.DataFrame([['a', 'b'], [1, 2]])
f
| 0 | 1 | |
|---|---|---|
| 0 | a | b |
| 1 | 1 | 2 |
pycse.orgmode defines several helpful classes to make org figures and tables with attributes.
from pycse.orgmode import *
Table([['x', 'y'],
[1, 2],
[3, 4]],
headers='firstrow',
name='org-data',
caption='The information about the table',
attributes=[('latex', ':environment longtable :align |l||l|')])
| x | y |
|---|---|
| 1 | 2 |
| 3 | 4 |
See Table ref:org-data.
import matplotlib.pyplot as plt
f = './test.png'
plt.plot([1, 4, 17])
plt.savefig(f)
plt.close() # you need this to not see two figures.
Figure(f, name='org-fig', caption='a line plot',
attributes=[('org', ':width 300'),
('latex', ':placement [H]')])
ref:org-fig
import matplotlib.pyplot as plt
import numpy as np
t = np.linspace(0, 20 * np.pi, 350)
x = np.exp(-0.1 * t) * np.sin(t)
y = np.exp(-0.1 * t) * np.cos(t)
plt.plot(x, y)
plt.axis('equal')
plt.savefig('fig-1.png')
plt.close()
plt.figure()
plt.plot(y, x)
plt.axis('equal')
plt.savefig('fig-2.png')
plt.close()
print('Length of t = {}'.format(len(t)))
print('x .dot. y = {}'.format(x @ y))
from pycse.orgmode import Figure, Org
from IPython.display import display
display(Org("\n\n"),
Figure('./fig-1.png', name='clock',
caption='a clockwise line plot'),
Org("\n\n"),
Figure('./fig-2.png', name='counterclock',
caption='a counter-clockwise line plot'))
import pandas as pd
Table(pd.DataFrame([['a', 'b'],
[1, 2],
[5, 6]]),
headers='firstrow',
name='pd-data',
caption='A table from a dataframe')
| 0 | a | b |
|---|---|---|
| 1 | 1 | 2 |
| 2 | 5 | 6 |
There is also a keyword.
Keyword('name', 'fig-1')
and a comment.
Heading('An example of a heading from code', 3)
Comment('A comment for orgmode')
Exceptions go in the results. Type f12 e to jump to the exception in the src block.
print(5)
a = 5
for j in range(5):
1 / 0
print(54)
print(z)
The priority for display is:
- text/org
- image/svg+xml, image/jpeg, image/png
- text/html
- text/markdown
- text/latex
- text/plain
LaTeX is automatically rendered to a png
from sympy import *
init_printing()
x, y, z = symbols('x y z')
Integral(sqrt(1 / x), x)
To get the actual LaTeX, use the :display
from sympy import *
init_printing()
x, y, z = symbols('x y z')
Integral(sqrt(1 / x), x)
and to get it in plain text:
from sympy import *
init_printing()
x, y, z = symbols('x y z')
Integral(sqrt(1 / x), x)
These get converted to org-syntax by pandoc I think. Note that emacs-jupyter and/or pandoc seems to put some \ in the converted results. I use the function scimax-rm-backslashes in a hook to remove these.
from IPython.display import FileLink, Image, display
FileLink('scimax.png')
Image('test.png')
display(FileLink('scimax.png'), Image('test.png'))
Some of these are already orgified, e.g. YouTubeVideo.
from IPython.display import YouTubeVideo
YouTubeVideo('ZXSaLcFSOsU')
The buffer is a great scratch space, but there is also a separate Jupyter scratch buffer. Use it to try out ideas, check values, etc.
Each kernel has a REPL associated with it. Type C-c C-v C-z or f12-z to get to it. It is like an IPython shell! You can explore things there, make plots, etc…
print(3)
3 + 4 # highlight region, C-M-x to run it.
a = 5 # Run C-x C-e here
5 + a # Then, M-i here to inspect a
Put a breakpoint in a function. Define it, then go to the REPL (f12 z) to step through it.
def f(x):
breakpoint()
return 1 / x
learn more about PDB at https://realpython.com/python-debugging-pdb/#getting-started-printing-a-variables-value.
See ox-ipynb. This org-file is not ideal for this export, it has some links that are not supported, and I marked the Known issues section as noexport because it has src-blocks with variables in it.
#+ox-ipynb-language: jupyter-python
(setq org-export-with-broken-links t)
(ox-ipynb-export-to-ipynb-file-and-open)In theory emacs-jupyter supports widgets, if you build it in the emacs-jupyter src directory. I did that, and don’t see any obvious issues, but this does not work. I am not likely to spend time fixing this anytime soon.
(let ((default-directory (file-name-directory (locate-library "jupyter"))))
(shell-command-to-string "make widgets"))This at least outputs something, but I think it should open a browser.
import ipywidgets as widgets
w = widgets.VBox([widgets.Text('#+attr_org: :width 300'),
widgets.Text('#+name: fig-data'),
widgets.Text('#+caption: something here.')])
display(w)
I am not sure why. I don’t think it is related to my changes. See emacs-jupyter/jupyter#333, I am not sure widgets still work.
This just hangs, and does not do anything.
widgets.Image(value=open("test.png", "rb").read(), width=400)
This sequence of cells leads to an error when you run them the first time. The issue is the first cell doesn’t use jupyter, it uses a cached result, so when you get to a * 4 the variable is not defined.
a = 15
print(a)
import time
time.sleep(5)
#+RESULTS[890bb80ec5ce30e9fa7f63b50d7df113702c43c5]: one
15
data
a * 4
Sometimes you get long outputs from things, and especially when it is something that needs fontification, this makes Emacs hard to use. I would like to have a way to truncate long outputs, and maybe write them to a file where you could look at them.
It would be awesome to do this. Probably this could build on ./scimax-literate-programming.el and ./scimax-ob-flycheck.el.
This does not always work when variables are inside a call. I usually see help for the function then.
a = 5
print(a + 5) # inspect a here, I usually see print documentation