Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Isolate databases/queries from one-another #235

Merged
merged 29 commits into from
Dec 28, 2023
Merged

Isolate databases/queries from one-another #235

merged 29 commits into from
Dec 28, 2023

Conversation

marcua
Copy link
Owner

@marcua marcua commented Dec 9, 2023

Resolves #41

Cargo.toml Outdated Show resolved Hide resolved
mv nsjail ..
cd ..
rm -rf nsjail-checkout

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Cargo.toml Show resolved Hide resolved
Cargo.toml Show resolved Hide resolved
Cargo.toml Show resolved Hide resolved
src/hosted_db/sandbox.rs Outdated Show resolved Hide resolved
src/hosted_db/sandbox.rs Outdated Show resolved Hide resolved
src/hosted_db/sqlite.rs Outdated Show resolved Hide resolved
@marcua marcua changed the title [WIP] Isolate dbs Isolate DBs Dec 23, 2023
@marcua marcua requested a review from sofiaritz December 23, 2023 05:12
src/hosted_db/sandbox.rs Outdated Show resolved Hide resolved
Cargo.toml Show resolved Hide resolved
Copy link
Contributor

@sofiaritz sofiaritz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like how you ended up implementing this, a clean and simple implementation :)
Now that the architecture of ayb is getting a little bit more complex, maybe it's time to create something like an ARCHITECTURE.md file.

Cargo.toml Show resolved Hide resolved
src/hosted_db/sandbox.rs Outdated Show resolved Hide resolved
src/hosted_db/sandbox.rs Outdated Show resolved Hide resolved
src/hosted_db/sandbox.rs Show resolved Hide resolved
.args(["--mount", "none:/tmp:tmpfs:size=100000000"]) // TODO(marcua): Restrict disk size more configurably?
// TODO(marcua): Set resource limits more configurably?
.args(["--max_cpus", "1"])
.args(["--rlimit_as", "64"]) // in MB
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the builder idea I left in the previous comment is not done, you could add some comments here to explain what is the meaning of each limit :)

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect! Done!

.arg(query)
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.stdin(Stdio::piped())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is piping stdin required?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good eye! :) I was able to remove it with no effect.

Some(isolation) => {
let result = run_in_sandbox(Path::new(&isolation.nsjail_path), path, query).await?;

if result.stderr.len() > 0 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if result.stderr.len() > 0 {
if !result.stderr.is_empty() {

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Thank you!

(I noticed cargo clippy also caught this, and realized clippy isn't breaking the build, so I created #250)

result.status.to_string()
),
})
} else if result.stdout.len() > 0 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
} else if result.stdout.len() > 0 {
} else if !result.stdout.is_empty() {

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done! Thank you!

src/hosted_db/sqlite.rs Outdated Show resolved Hide resolved
})
}
}
None => Ok(query_sqlite(path, query)?.into()),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could remove a level of nesting by doing the following at the top

if isolation.is_none() {
   return Ok(query_sqlite(path, query)?.into());
}

Copy link
Owner Author

@marcua marcua Dec 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm conflicted on this one. We'd benefit from one lower level of nesting in a relatively short function, but in exchange

  • We would stop using a language feature like match, which I feel is being used appropriately
  • We'd have to introduce an unwrap, since the compiler won't be able to reason about the knowledge we'd gain with the if

Given we're using match in the way it was intended and forcing ourselves to enumerate all of the variants, I wonder if we'd gain much in exchange for reducing indentation in a short function.

Copy link
Contributor

@sofiaritz sofiaritz Dec 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New idea! What about using if let Some(isolation) = isolation and returning the None case at the end of the function? Seems like a reasonable middleground :)

Note: Deleted a previous comment I made because this suggestion replaces it. The two key-points from that comment:

  • Enums like Option or Result are stable and thus won't add any new variants.
  • When ayb handles a request and panics the server keeps running, thus it's not as bad as it may look like :) (not saying that we should just panic :p)
pub async fn potentially_isolated_sqlite_query(
    path: &PathBuf,
    query: &str,
    isolation: &Option<AybConfigIsolation>,
) -> Result<QueryResult, AybError> {
    if let Some(isolation) = isolation {
        let result = run_in_sandbox(Path::new(&isolation.nsjail_path), path, query).await?;
        if !result.stderr.is_empty() {
            let error: AybError = serde_json::from_str(&result.stderr)?;
            return Err(error);
        }
        
        if result.status != 0 {
            return Err(AybError::Other {
                message: format!(
                    "Error status from sandboxed query runner: {}",
                    result.status
                ),
            });
        }
        
        if !result.stdout.is_empty() {
            let query_result: QueryResult = serde_json::from_str(&result.stdout)?;
            return Ok(query_result);
        }
        
        return Err(AybError::Other {
            message: "No results from sandboxed query runner".to_string(),
        });
    }
    
    Ok(query_sqlite(path, query)?)
}

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean! I like it! I appreciated both of your bullet points on the broader point. I'll implement this and we should be good to go! Woohoo! Thank you!!!

@marcua
Copy link
Owner Author

marcua commented Dec 26, 2023

Thank you for the very helpful review @sofiaritz! You made the implementation an documentation far more approachable, and I appreciate your feedback. Looking forward to your next round! :)

Re: ARCHITECTURE.md, great idea. I added basic documentation on isolation and testing, but also created #252 for a deeper pass.

@sofiaritz
Copy link
Contributor

Done! Just a few comments about early returns and match. Really excited for this to land! :)

@marcua marcua requested a review from sofiaritz December 27, 2023 00:46
Copy link
Contributor

@sofiaritz sofiaritz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing work!! When this lands, I'll update aybDocker and start hosting a public instance :)

@marcua
Copy link
Owner Author

marcua commented Dec 28, 2023

Thank you for the very helpful review, @sofiaritz! I'll merge now, and after #237 lands, I'll release the next version for your public instance! Woohooo! :)

@marcua marcua merged commit 785d09f into main Dec 28, 2023
1 check passed
@marcua marcua deleted the isolate-dbs branch December 28, 2023 03:09
@marcua marcua changed the title Isolate DBs Isolate databases/queries from one-another Dec 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Isolate databases from one-another
2 participants