-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Isolate databases/queries from one-another #235
Conversation
scripts/build_nsjail.sh
Outdated
mv nsjail .. | ||
cd .. | ||
rm -rf nsjail-checkout | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like how you ended up implementing this, a clean and simple implementation :)
Now that the architecture of ayb is getting a little bit more complex, maybe it's time to create something like an ARCHITECTURE.md file.
src/hosted_db/sandbox.rs
Outdated
.args(["--mount", "none:/tmp:tmpfs:size=100000000"]) // TODO(marcua): Restrict disk size more configurably? | ||
// TODO(marcua): Set resource limits more configurably? | ||
.args(["--max_cpus", "1"]) | ||
.args(["--rlimit_as", "64"]) // in MB |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the builder idea I left in the previous comment is not done, you could add some comments here to explain what is the meaning of each limit :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perfect! Done!
src/hosted_db/sandbox.rs
Outdated
.arg(query) | ||
.stdout(Stdio::piped()) | ||
.stderr(Stdio::piped()) | ||
.stdin(Stdio::piped()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is piping stdin
required?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good eye! :) I was able to remove it with no effect.
src/hosted_db/sqlite.rs
Outdated
Some(isolation) => { | ||
let result = run_in_sandbox(Path::new(&isolation.nsjail_path), path, query).await?; | ||
|
||
if result.stderr.len() > 0 { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if result.stderr.len() > 0 { | |
if !result.stderr.is_empty() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice! Thank you!
(I noticed cargo clippy
also caught this, and realized clippy isn't breaking the build, so I created #250)
src/hosted_db/sqlite.rs
Outdated
result.status.to_string() | ||
), | ||
}) | ||
} else if result.stdout.len() > 0 { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
} else if result.stdout.len() > 0 { | |
} else if !result.stdout.is_empty() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done! Thank you!
src/hosted_db/sqlite.rs
Outdated
}) | ||
} | ||
} | ||
None => Ok(query_sqlite(path, query)?.into()), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You could remove a level of nesting by doing the following at the top
if isolation.is_none() {
return Ok(query_sqlite(path, query)?.into());
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm conflicted on this one. We'd benefit from one lower level of nesting in a relatively short function, but in exchange
- We would stop using a language feature like
match
, which I feel is being used appropriately - We'd have to introduce an
unwrap
, since the compiler won't be able to reason about the knowledge we'd gain with theif
Given we're using match
in the way it was intended and forcing ourselves to enumerate all of the variants, I wonder if we'd gain much in exchange for reducing indentation in a short function.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
New idea! What about using if let Some(isolation) = isolation
and returning the None
case at the end of the function? Seems like a reasonable middleground :)
Note: Deleted a previous comment I made because this suggestion replaces it. The two key-points from that comment:
- Enums like
Option
orResult
are stable and thus won't add any new variants. - When ayb handles a request and panics the server keeps running, thus it's not as bad as it may look like :) (not saying that we should just panic :p)
pub async fn potentially_isolated_sqlite_query(
path: &PathBuf,
query: &str,
isolation: &Option<AybConfigIsolation>,
) -> Result<QueryResult, AybError> {
if let Some(isolation) = isolation {
let result = run_in_sandbox(Path::new(&isolation.nsjail_path), path, query).await?;
if !result.stderr.is_empty() {
let error: AybError = serde_json::from_str(&result.stderr)?;
return Err(error);
}
if result.status != 0 {
return Err(AybError::Other {
message: format!(
"Error status from sandboxed query runner: {}",
result.status
),
});
}
if !result.stdout.is_empty() {
let query_result: QueryResult = serde_json::from_str(&result.stdout)?;
return Ok(query_result);
}
return Err(AybError::Other {
message: "No results from sandboxed query runner".to_string(),
});
}
Ok(query_sqlite(path, query)?)
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Clean! I like it! I appreciated both of your bullet points on the broader point. I'll implement this and we should be good to go! Woohoo! Thank you!!!
Thank you for the very helpful review @sofiaritz! You made the implementation an documentation far more approachable, and I appreciate your feedback. Looking forward to your next round! :) Re: ARCHITECTURE.md, great idea. I added basic documentation on isolation and testing, but also created #252 for a deeper pass. |
Done! Just a few comments about early returns and |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Amazing work!! When this lands, I'll update aybDocker and start hosting a public instance :)
Thank you for the very helpful review, @sofiaritz! I'll merge now, and after #237 lands, I'll release the next version for your public instance! Woohooo! :) |
Resolves #41