Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Offline Partial Eval #55

Merged
merged 49 commits into from
Mar 18, 2024
Merged

Offline Partial Eval #55

merged 49 commits into from
Mar 18, 2024

Conversation

ncough
Copy link
Collaborator

@ncough ncough commented Mar 14, 2024

Wrapper around existing online partial evaluation work to perform offline partial evaluation. Rather than producing the semantics for a single instruction opcode, this approach produces a lifter for some subset of the architecture.

The process looks roughly like the following:

  1. Convert the decoder & instruction encoding structures into ASL functions
  2. Identify all of the reachable functions from this initial set up to a configurable frontier
  3. Simplify these functions to remove annoying/unsupported features
  4. Transform each function such that loop bounds and bitvector widths are always known
  5. Run the existing online partial evaluation over each instruction encoding function
  6. Do some cleanup and pruning based on the results of the partial evaluation
  7. Run a taint analysis over the evaluated instruction encoding functions to identify
    what is known at lifttime vs runtime
  8. Transform all runtime influenced statements and expressions into IR generating operations
    against an assumed interface
  9. Print the result as an OCaml program with the IR interface instantiated as ASL IR

The above works for a significant portion of AArch64 matching the existing lifters coverage for all but aarch64_memory_vector_single_post_inc, aarch64_memory_vector_single_no_wb, aarch64_memory_vector_multiple_post_inc, aarch64_memory_vector_multiple_no_wb. However, the resulting OCaml program is ~200k lines long and takes ~5min to build. This can mostly be attributed to some aggressive specialisation in stage 4 and the resulting unrolling transforms introduced by stage 5.

There are still a decent number of things to do:

  • Improvements to AArch64 coverage:
    • LExpr BitTuple support
  • Improvements to compile time / size:
    • Break the produced OCaml program into a series of files, rather than one
    • Support symbolic for loops throughout the pipeline, to reduce unrolling and specialisation
    • Simplify based on branch conditions (e.g. if X = '1' then c should replace X with '1' in c)
  • 'Quality' of resulting program:
    • Comparison of online & offline lifter results
    • Implement a form of copy prop over the IR generating operations produced in stage 8
    • Implement some trivial identities within the IR interface

Given the compile time issues, this PR includes a generate lifter that supports no instructions.
A new lifter can be generated using echo ':gen A64 aarch64.+' | dune exec asli from the project directory.
A subsequent dune build will compile the new lifter.
There are two entry points to the lifter:

  • asloff-sem OPCODE will dump the semantics of OPCODE
  • asloff-coverage PAT will run the coverage test for all instruction encodings matching PAT

ncough added 30 commits March 12, 2024 08:14
Correct visit order bug in StatefulIntToBit, support
register type
Some common idioms can be easily rewritten, avoiding
complexity udring dis. Currently unused.
Masks are never symbolic, so they can be trivially
rewritten as a masking and equiv test if needed.
May want to make this a configuration option to dis
Cuts of prefix early, even if prefix does not end with terminal
Assumes no mixed use of array indices
Capable of extracting and simplifying behaviours for
most of the instruction encodings on AArch64.
Offline gen is missing specs for primitives as well
as the prelude required for the ocaml backend.
Some differences in exception printing, unclear
Copy link
Member

@katrinafyi katrinafyi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing!! You can merge this whenever you're ready, #53 can merge cleanly on top of this.

ncough added 2 commits March 18, 2024 09:42
The naive offline transform treats all runtime variables
as temporaries that must be declared in the runtime program.
Often these temporaries simply hold intermediate calculations
and can be trivially removed through a copy prop pass.

As we do not want to introduce a copy prop into the generated
lifter, we can mimic its effects over the produced lifter by
identifying candidate temporaries and changing them to directly
hold the intermediate calculations, rather than representing
runtime variables.
@ncough
Copy link
Collaborator Author

ncough commented Mar 18, 2024

Apologies, made a few improvements. After splitting the OCaml program across multiple files, build times are now down to ~1min. Also managed to get the copy prop pass implemented. Will merge once the tests pass.

@ncough ncough marked this pull request as ready for review March 18, 2024 01:34
@ncough ncough merged commit 4991bcd into partial_eval Mar 18, 2024
1 check passed
@katrinafyi katrinafyi mentioned this pull request Mar 18, 2024
@ncough ncough mentioned this pull request Mar 18, 2024
@ncough ncough deleted the taint branch March 18, 2024 22:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants