Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

poc: remove Rc Cell from parser2 parse scope structs #874

Draft
wants to merge 69 commits into
base: fe-v2
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
d3535a6
Initialize `fe-hir`
Y-Nak Feb 27, 2023
2867a88
Allow leading `mut` before `pat`
Y-Nak Mar 2, 2023
eedb2eb
Remove `assert` statement from the language
Y-Nak Mar 15, 2023
5b0eab9
Add HIR `item` def
Y-Nak Mar 15, 2023
501e541
Add `HIR `attr` def
Y-Nak Mar 15, 2023
61ed75b
Add HIR `body` def
Y-Nak Mar 15, 2023
98baa51
Add HIR `expr` def
Y-Nak Mar 15, 2023
7c416ec
Add HIR `params` def
Y-Nak Mar 15, 2023
97cbd68
Add HIR `pat` def
Y-Nak Mar 15, 2023
55896d3
Add HIR `stmt` def
Y-Nak Mar 15, 2023
080c3bd
Add HIR `type` def
Y-Nak Mar 15, 2023
7ef834a
Add HIR `use_tree` def
Y-Nak Mar 15, 2023
f124569
Add Jar for HIR entities
Y-Nak Mar 15, 2023
932f20e
Add `HirOrigin` type to track the HIR definition origin in the source…
Y-Nak Mar 15, 2023
ec4e35c
Add HIR `path` def
Y-Nak Mar 16, 2023
ce6bc1e
Add HIR lower for `Path`
Y-Nak Mar 16, 2023
eadd043
Define `IngotId` and `FileId`
Y-Nak Mar 16, 2023
488aa2c
Add HIR lower for `TypeId`
Y-Nak Mar 16, 2023
1decf82
Add HIR lower for `params`
Y-Nak Mar 16, 2023
d16b1b3
Add HIR lower for `attr`
Y-Nak Mar 16, 2023
0592bb9
Introduce `MaybeInvalid` type
Y-Nak Mar 17, 2023
5e46510
Add HIR lower for `Item`
Y-Nak Mar 17, 2023
d478b1f
Add syntax to specify generic parameters in `impl` block
Y-Nak Mar 17, 2023
2dd211b
Add HIR lower for `Pat`
Y-Nak Mar 20, 2023
3d07d8c
Add HIR lower for `Stmt`
Y-Nak Mar 20, 2023
9a0a570
Add HIR lower for `Expr`
Y-Nak Mar 21, 2023
3feec5b
Add HIR lower for `Body`
Y-Nak Mar 21, 2023
7a18a38
Define input
Y-Nak Mar 21, 2023
a8ed9e2
Add Id types for more fine-grained reuse of salsa querie results
Y-Nak Mar 21, 2023
7c91c65
Add `mod` item
Y-Nak Mar 22, 2023
2f46b10
Add `ExternFn` item
Y-Nak Mar 22, 2023
5572aa3
Include body in items
Y-Nak Mar 22, 2023
6a7f015
Separate `InputDb` from `HirDb`
Y-Nak Mar 22, 2023
87114fc
Add HIR `Mod` item
Y-Nak Mar 22, 2023
61654c3
Add `IngotModuleTree` and `ModuleItemTree`
Y-Nak Mar 22, 2023
c66aac6
Implement `IngotModuleTreeBuidler`
Y-Nak Mar 22, 2023
b53359b
Add tracked fucntion for parsing
Y-Nak Mar 23, 2023
2535810
Implement `FileLowerCtxt`
Y-Nak Mar 23, 2023
a36c606
Make clippy happy
Y-Nak Mar 23, 2023
3d1c7ca
Rename `MaybeInvalid<T>` to `Partial<T>`
Y-Nak Mar 24, 2023
5a0def9
Add `BodySourceMap`
Y-Nak Mar 27, 2023
ab5dc95
Add `SpannedHirDb`
Y-Nak Mar 27, 2023
3d8c8e1
Define `CompleteDiagnostic`
Y-Nak Mar 27, 2023
5335833
Define `DiagnosticVoucher`
Y-Nak Mar 29, 2023
03cbb4a
Add `ItemSpanState`
Y-Nak Mar 30, 2023
f18cfe1
Define `SpanSeed` and `SpanTransitionChain`
Y-Nak Apr 2, 2023
4d07cf7
Define lazy span for items
Y-Nak Apr 2, 2023
e3cf853
Add lazy span for Path
Y-Nak Apr 4, 2023
fd99142
Add lazy span for use tree
Y-Nak Apr 4, 2023
ac41545
Improve `define_lazy_span_item`
Y-Nak Apr 5, 2023
6475339
Add lazy span for params
Y-Nak Apr 5, 2023
eb3915f
Add lazy span for attribute
Y-Nak Apr 5, 2023
15783cd
Add lazy span for types
Y-Nak Apr 5, 2023
ee0d67a
Add `ChainRoot` trait
Y-Nak Apr 5, 2023
7046250
Add lazy span for pat
Y-Nak Apr 5, 2023
0621fea
Add `EvaluatedSpan` to represent syntactically invalid node's spans
Y-Nak Apr 6, 2023
6dfb646
Add lazy span for expr
Y-Nak Apr 6, 2023
0f755c4
Add lazy span for stmt
Y-Nak Apr 7, 2023
821702e
Add `TestDb`
Y-Nak Apr 7, 2023
ada4c15
Add test for `ItemTree`
Y-Nak Apr 7, 2023
6494f4e
Add test for lazy span
Y-Nak Apr 7, 2023
1b2f5a8
Add an accumulator for `ParseDiagnostic`
Y-Nak Apr 7, 2023
3508787
Make `HirDb` completely span-independent from the perspective of exte…
Y-Nak Apr 8, 2023
bf6e823
Add tests for `ModuleTree`
Y-Nak Apr 9, 2023
05a8107
Make types implementing `LazySpan` comparabale
Y-Nak Apr 19, 2023
44db5e3
Allow more precise span origin tracing
Y-Nak Apr 19, 2023
f679cd9
Add `DynLazySpan`
Y-Nak Apr 19, 2023
c12290e
Add parser2 parsing benchmark
sbillig Apr 19, 2023
ac32a48
poc: remove Rc Cell from parse scope structs
sbillig Apr 19, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
336 changes: 322 additions & 14 deletions Cargo.lock

Large diffs are not rendered by default.

6 changes: 4 additions & 2 deletions crates/analyzer/src/db/queries/ingots.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
use crate::namespace::items::{IngotId, IngotMode, ModuleId, ModuleSource};
use crate::AnalyzerDb;
use crate::{
namespace::items::{IngotId, IngotMode, ModuleId, ModuleSource},
AnalyzerDb,
};
use fe_common::files::{SourceFileId, Utf8Path, Utf8PathBuf};
use indexmap::IndexSet;
use std::rc::Rc;
Expand Down
43 changes: 21 additions & 22 deletions crates/analyzer/src/db/queries/module.rs
Original file line number Diff line number Diff line change
@@ -1,20 +1,25 @@
use crate::context::{Analysis, AnalyzerContext, Constant, NamedThing};
use crate::display::Displayable;
use crate::errors::{self, ConstEvalError, TypeError};
use crate::namespace::items::{
Contract, ContractId, Enum, Function, Impl, ImplId, Item, ModuleConstant, ModuleConstantId,
ModuleId, ModuleSource, Struct, StructId, Trait, TraitId, TypeAlias, TypeDef,
use crate::{
context::{Analysis, AnalyzerContext, Constant, NamedThing},
display::Displayable,
errors::{self, ConstEvalError, TypeError},
namespace::{
items::{
Contract, ContractId, Enum, Function, Impl, ImplId, Item, ModuleConstant,
ModuleConstantId, ModuleId, ModuleSource, Struct, StructId, Trait, TraitId, TypeAlias,
TypeDef,
},
scopes::ItemScope,
types::{self, TypeId},
},
traversal::{const_expr, expressions, types::type_desc},
AnalyzerDb,
};
use crate::namespace::scopes::ItemScope;
use crate::namespace::types::{self, TypeId};
use crate::traversal::{const_expr, expressions, types::type_desc};
use crate::AnalyzerDb;
use fe_common::diagnostics::Label;
use fe_common::files::Utf8Path;
use fe_common::Span;
use fe_common::{diagnostics::Label, files::Utf8Path, Span};
use fe_parser::{ast, node::Node};
use indexmap::indexmap;
use indexmap::map::{Entry, IndexMap};
use indexmap::{
indexmap,
map::{Entry, IndexMap},
};
use smol_str::SmolStr;
use std::rc::Rc;

Expand Down Expand Up @@ -293,13 +298,7 @@ pub fn module_structs(db: &dyn AnalyzerDb, module: ModuleId) -> Rc<[StructId]> {
module
.all_items(db)
.iter()
.chain(
module
.used_items(db)
.values()
.into_iter()
.map(|(_, item)| item),
)
.chain(module.used_items(db).values().map(|(_, item)| item))
.filter_map(|item| match item {
Item::Type(TypeDef::Struct(id)) => Some(*id),
_ => None,
Expand Down
12 changes: 2 additions & 10 deletions crates/codegen/src/yul/runtime/data.rs
Original file line number Diff line number Diff line change
Expand Up @@ -238,11 +238,7 @@ pub(super) fn make_aggregate_init(
let ptr = YulVariable::new("ptr");
let field_num = inner_ty.aggregate_field_num(db.upcast());

let iter_field_args = || {
(0..field_num)
.into_iter()
.map(|i| YulVariable::new(format! {"arg{i}"}))
};
let iter_field_args = || (0..field_num).map(|i| YulVariable::new(format! {"arg{i}"}));

let mut body = vec![];
for (idx, field_arg) in iter_field_args().enumerate() {
Expand Down Expand Up @@ -303,11 +299,7 @@ pub(super) fn make_enum_init(
let ptr = YulVariable::new("ptr");
let disc = YulVariable::new("disc");
let disc_ty = arg_tys[0];
let enum_data = || {
(0..arg_tys.len() - 1)
.into_iter()
.map(|i| YulVariable::new(format! {"arg{i}"}))
};
let enum_data = || (0..arg_tys.len() - 1).map(|i| YulVariable::new(format! {"arg{i}"}));

let tuple_def = TupleDef {
items: arg_tys
Expand Down
17 changes: 17 additions & 0 deletions crates/common2/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
[package]
name = "fe-common2"
version = "0.20.0-alpha"
authors = ["The Fe Developers <[email protected]>"]
edition = "2021"
license = "Apache-2.0"
repository = "https://github.com/ethereum/fe"
description = "Provides HIR definition and lowering for Fe lang."

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
semver = "1.0.17"
camino = "1.1.4"
smol_str = "0.1.24"
salsa = { git = "https://github.com/salsa-rs/salsa", package = "salsa-2022" }
parser = { path = "../parser2", package = "fe-parser2" }
105 changes: 105 additions & 0 deletions crates/common2/src/diagnostics.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
use parser::TextRange;

use crate::InputFile;

#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct CompleteDiagnostic {
pub severity: Severity,
pub message: String,
pub span: Span,
pub sub_diagnostics: Vec<SubDiagnostic>,
pub error_code: GlobalErrorCode,
}

impl CompleteDiagnostic {
pub fn new(
severity: Severity,
message: String,
span: Span,
sub_diagnostics: Vec<SubDiagnostic>,
error_code: GlobalErrorCode,
) -> Self {
Self {
severity,
message,
span,
sub_diagnostics,
error_code,
}
}
}

#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct GlobalErrorCode {
pub pass: AnalysisPass,
pub local_code: u16,
}

impl GlobalErrorCode {
pub fn new(pass: AnalysisPass, local_code: u16) -> Self {
Self { pass, local_code }
}
}

#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct SubDiagnostic {
pub severity: Severity,
pub message: String,
pub span: Span,
}

#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct Span {
pub file: InputFile,
pub range: TextRange,
pub kind: SpanKind,
}

#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub enum SpanKind {
/// A node corresponding is originally written in the source code.
Original,

/// A node corresponding to the span is generated by macro expansion.
Expanded,

/// No span information was found.
/// This happens if analysis code tries to get a span for a node that is
/// generated in lowering phase.
///
/// If span has this kind, it means there is a bug in the analysis code.
/// The reason not to panic is that LSP should continue working even if
/// there are bugs in the span generation(This also makes easier to identify
/// the cause of the bug)
///
/// Range is always the first character of the file in this case.
NotFound,
}

impl Span {
pub fn new(file: InputFile, range: TextRange, kind: SpanKind) -> Self {
Self { file, range, kind }
}
}

#[derive(Debug, Copy, Clone, PartialEq, Eq, Hash)]
pub enum Severity {
Error,
Warning,
Note,
}

#[repr(u16)]
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub enum AnalysisPass {
Parse = 1,
NameResolution,
TyCheck,

ExternalAnalysis(ExternalAnalysisKey) = u16::MAX,
}

#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct ExternalAnalysisKey {
name: String,
}
139 changes: 139 additions & 0 deletions crates/common2/src/input.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
use std::collections::BTreeSet;

use camino::Utf8PathBuf;
use smol_str::SmolStr;

use crate::InputDb;

/// An ingot is a collection of files which are compiled together.
/// Ingot can depend on other ingots.
#[salsa::input(constructor = __new_impl)]
pub struct InputIngot {
/// An absolute path to the ingot root directory.
/// The all files in the ingot should be located under this directory.
#[return_ref]
pub path: Utf8PathBuf,

/// Specifies the kind of the ingot.
pub kind: IngotKind,

/// A version of the ingot.
#[return_ref]
pub version: Version,

/// A list of ingots which the current ingot depends on.
#[return_ref]
pub dependency: BTreeSet<IngotDependency>,

/// A list of files which the current ingot contains.
#[return_ref]
#[set(__set_files_impl)]
pub files: BTreeSet<InputFile>,

#[set(__set_root_file_impl)]
#[get(__get_root_file_impl)]
root_file: Option<InputFile>,
}
impl InputIngot {
pub fn new(
db: &dyn InputDb,
path: &str,
kind: IngotKind,
version: Version,
dependency: BTreeSet<IngotDependency>,
) -> InputIngot {
let path = Utf8PathBuf::from(path);
let root_file = None;
Self::__new_impl(
db,
path,
kind,
version,
dependency,
BTreeSet::default(),
root_file,
)
}

/// Set the root file of the ingot.
/// The root file must be set before the ingot is used.
pub fn set_root_file(self, db: &mut dyn InputDb, file: InputFile) {
self.__set_root_file_impl(db).to(Some(file));
}

/// Set the list of files which the ingot contains.
/// All files must bee set before the ingot is used.
pub fn set_files(self, db: &mut dyn InputDb, files: BTreeSet<InputFile>) {
self.__set_files_impl(db).to(files);
}

/// Returns the root file of the ingot.
/// Panics if the root file is not set.
pub fn root_file(&self, db: &dyn InputDb) -> InputFile {
self.__get_root_file_impl(db).unwrap()
}
}

#[salsa::input]
pub struct InputFile {
/// A ingot id which the file belongs to.
pub ingot: InputIngot,

/// A path to the file from the ingot root directory.
#[return_ref]
pub path: Utf8PathBuf,

#[return_ref]
pub text: String,
}

impl InputFile {
pub fn abs_path(&self, db: &dyn InputDb) -> Utf8PathBuf {
self.ingot(db).path(db).join(self.path(db))
}
}

#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub enum IngotKind {
/// A standalone ingot is a dummy ingot when the compiler is invoked
/// directly on a file.
StandAlone,

/// A local ingot which is the current ingot being compiled.
Local,

/// An external ingot which is depended on by the current ingot.
External,

/// Standard library ingot.
Std,
}

#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct IngotDependency {
/// The ingot may have a alias name from the original ingot name.
pub name: SmolStr,
/// An ingot which the current ingot depends on.
pub ingot: InputIngot,
}
impl IngotDependency {
pub fn new(name: &str, ingot: InputIngot) -> Self {
Self {
name: SmolStr::new(name),
ingot,
}
}
}

impl PartialOrd for IngotDependency {
fn partial_cmp(&self, other: &Self) -> Option<std::cmp::Ordering> {
Some(self.ingot.cmp(&other.ingot))
}
}
impl Ord for IngotDependency {
fn cmp(&self, other: &Self) -> std::cmp::Ordering {
self.ingot.cmp(&other.ingot)
}
}

pub type Version = semver::Version;
14 changes: 14 additions & 0 deletions crates/common2/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
pub mod diagnostics;
pub mod input;

pub use input::{InputFile, InputIngot};

#[salsa::jar(db = InputDb)]
pub struct Jar(InputIngot, InputFile);

pub trait InputDb: salsa::DbWithJar<Jar> {}
impl<DB> InputDb for DB where DB: ?Sized + salsa::DbWithJar<Jar> {}

pub trait Upcast<T: ?Sized> {
fn upcast(&self) -> &T;
}
23 changes: 23 additions & 0 deletions crates/hir/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
[package]
name = "fe-hir"
version = "0.20.0-alpha"
authors = ["The Fe Developers <[email protected]>"]
edition = "2021"
license = "Apache-2.0"
repository = "https://github.com/ethereum/fe"
description = "Provides HIR definition and lowering for Fe lang"

[dependencies]
# We may need to fix this to a specific version,
# but I want to keep up with the latest version until the new Fe implemeentation is merged into the master.
salsa = { git = "https://github.com/salsa-rs/salsa", package = "salsa-2022" }
derive_more = "0.99"
cranelift-entity = "0.91"
num-bigint = "0.4.3"
num-traits = "0.2.15"
camino = "1.1.4"
rustc-hash = "1.1.0"

parser = { path = "../parser2", package = "fe-parser2" }
common = { path = "../common2", package = "fe-common2" }
smallvec = "1.10.0"
Loading