Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not able to use magic commands in the Markdown chunks #31

Closed
Hbinfinity opened this issue Mar 31, 2023 · 2 comments
Closed

Not able to use magic commands in the Markdown chunks #31

Hbinfinity opened this issue Mar 31, 2023 · 2 comments

Comments

@Hbinfinity
Copy link

Hbinfinity commented Mar 31, 2023

I'm able to successfully open a connection with Databricks in RStudio and able to execute code in the Markdown chunks.

However I'm wondering now if I can use magic commands (more specifically, %run) in the markdown chunks.
It's an important part of our developments and workflows, I'm trying to do something like:

%run /Users/xxxxx/some_notebook

But it is not working, I get an error:

Error: Error in parse(text = DATABRICKS_CURRENT_TEMP_CMD__) : 
  <text>:2:1: unexpected input
1: {
2: %run /Users/xxxxx/some_notebook
   ^
Error in parse(text = DATABRICKS_CURRENT_TEMP_CMD__): <text>:2:1: unexpected input
Error in parse(text = DATABRICKS_CURRENT_TEMP_CMD__): <text>:2:1: unexpected input
1: {
2: %run /Users/xxxxx/some_notebook
   ^

The other option would be use dbutils.run.notebook command with a python engine, I guess? (What about other magic commands like %sh, %pip commands, %fs, etc though)?
This is ofcourse an "immediate-getting-started" effort with some of our customers;
the more generic question that I'm trying to ask is, how do we work in a modularised manner? Multiple markdown files and calling them in each other? Something like the following in an orchestrating markdown?

rmarkdown::render("test1.Rmd")

How can we use these markdown files to deploy code to higher environments? Can these be used directly somehow or do we need to do something else?

@zacdav-db
Copy link
Contributor

Sorry I totally missed this issue.

Brickster currently doesn't have a way to support %run magic due to how the context API's that Databricks surfaces work. The only supported contexts are those for r, python, sql, scala.

You'd need to work within those confines, so dbutils is completely valid way to go about it.

the more generic question that I'm trying to ask is, how do we work in a modularised manner? Multiple markdown files and calling them in each other? Something like the following in an orchestrating markdown?

I'd probably need a bit more information, if you want to work in a modularised manner then there is the question of why can't that just work as standard R code executed in r or databricks_r chunks? It's just executing R code after all.

I'm hoping developments with Databricks connect V2 and support for R will open up more usage patterns.

@zacdav-db
Copy link
Contributor

Closing as there are no plans to support %run. I have however just added %sh support.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants