Preparation refactoring towards v5.0 (part 5) #734

aslesarenko · 2021-07-26T14:40:29Z

In this PR:

implemented all eval methods of Value classes (aka ErgoTree operations)
all eval methods contain cost accumulation
costKind descriptors with final cost parameters (based on profiling) declared for all operations

codecov · 2021-07-26T15:07:14Z

Codecov Report

Merging #734 (262a14d) into develop (571571f) will decrease coverage by 2.75%.
The diff coverage is 27.81%.

@@             Coverage Diff             @@
##           develop     #734      +/-   ##
===========================================
- Coverage    69.08%   66.32%   -2.76%     
===========================================
  Files          245      247       +2     
  Lines        17567    18596    +1029     
  Branches       517      548      +31     
===========================================
+ Hits         12136    12334     +198     
- Misses        5431     6262     +831

Impacted Files	Coverage Δ
common/src/main/scala/scalan/AnyVals.scala	`54.54% <0.00%> (-8.62%)`	⬇️
.../src/main/scala/sigmastate/DataValueComparer.scala	`0.00% <0.00%> (ø)`
...ala/sigmastate/interpreter/ErgoTreeEvaluator.scala	`0.00% <0.00%> (ø)`
...te/src/main/scala/sigmastate/lang/SigmaTyper.scala	`74.15% <ø> (ø)`
...mastate/src/main/scala/sigmastate/lang/Terms.scala	`55.07% <7.89%> (-17.93%)`	⬇️
sigmastate/src/main/scala/sigmastate/Values.scala	`69.78% <14.28%> (-11.75%)`	⬇️
...src/main/scala/sigmastate/eval/BigIntegerOps.scala	`47.61% <16.66%> (+15.47%)`	⬆️
.../src/main/scala/sigmastate/utxo/transformers.scala	`56.52% <21.92%> (-42.42%)`	⬇️
sigmastate/src/main/scala/sigmastate/trees.scala	`59.00% <26.08%> (-32.88%)`	⬇️
.../main/scala/org/ergoplatform/ErgoLikeContext.scala	`74.22% <33.33%> (-13.45%)`	⬇️
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 571571f...262a14d. Read the comment docs.

…rpreter into prepare-v5.0-evaluator

kushti

I see no any tests for DataValueComparer.scala, ErgoTreeEvaluator etc

common/src/main/scala/scalan/AnyVals.scala

sigmastate/src/main/scala/org/ergoplatform/ErgoLikeContext.scala

sigmastate/src/main/scala/sigmastate/DataValueComparer.scala

sigmastate/src/main/scala/sigmastate/interpreter/ErgoTreeEvaluator.scala

aslesarenko · 2021-08-25T13:15:53Z

The tests will be in the next (and probably the last incremental PR). A lot of changes in the tests alone (~2K lines). It will be easier to review, because it will be clear that no implementation code is changed and only tests are added.
Also, the new Evaluator will run in parallel with the old version in almost all test cases so that the results are compared. This means the new implementation is fully covered (but in the next PR)
So I suggest to merge this PR without tests.

sigmastate/src/main/scala/sigmastate/eval/BigIntegerOps.scala

kushti · 2021-09-22T18:00:04Z

sigmastate/src/main/scala/sigmastate/serialization/ErgoTreeSerializer.scala

@@ -248,7 +248,7 @@ class ErgoTreeSerializer {

  def substituteConstants(scriptBytes: Array[Byte],


ScalaDoc missed.

description of the second output is missed

greenhat

IR nodes evaluation looks good to me. I'm actually very familiar with this JIT evaluation already since almost all of the sigma-rust interpreter is implemented with this JIT evaluation as a reference (I used direct-tree-eval branch).
I did not check AOT vs. JIT costing equivalency. I suppose there are(will be) tests for that but I did not find them with a quick search. Since we're about to start costing implementation in sigma-rust these tests would be an excellent reference and we would build our costing tests around it.

Please also check notes/questions I made in the comments.

sigmastate/src/main/scala/sigmastate/Values.scala

sigmastate/src/main/scala/sigmastate/types.scala

kushti · 2021-10-18T11:39:11Z

sigmastate/src/main/scala/sigmastate/eval/BigIntegerOps.scala

@@ -72,6 +89,13 @@ object NumericOps {
    override def times(x: BigInt, y: BigInt): BigInt = n.times(x, y)
  }

+  implicit object BigIntIsExactIntegral extends ExactIntegral[BigInt] {


ScalaDoc is missed

sigmastate/src/main/scala/sigmastate/interpreter/ErgoTreeEvaluator.scala

sigmastate/src/main/scala/sigmastate/types.scala

greenhat · 2021-10-20T07:00:51Z

I did not check AOT vs. JIT costing equivalency. I suppose there are(will be) tests for that but I did not find them with a quick search. Since we're about to start costing implementation in sigma-rust these tests would be an excellent reference and we would build our costing tests around it.

I found commented JitCostingSpecification.scala in #738 - https://github.com/ScorexFoundation/sigmastate-interpreter/pull/738/files#diff-dd46271e18b352136f7afb1216c3651fce6cc1691321b544462dab8ff4000e32
I plan to use it as a test vector to ensure sigma-rust costing equivalency

aslesarenko · 2021-10-20T12:07:28Z

Since we're about to start costing implementation in sigma-rust these tests would be an excellent reference and we would build our costing tests around it.

@greenhat The AOT and JIT costs are different, so they are not equivalent.
However JIT costs are lower then AOT costs and as result the node with JIT interpreter can do a full sync without problems. (at least when I did it in May-June)

For sigma-rust you can look at SigmaDslSpecification in #738
It has a lot of test vectors and covers all operations.

…ator # Conflicts: # sigmastate/src/main/scala/sigmastate/eval/BigIntegerOps.scala # sigmastate/src/main/scala/sigmastate/utxo/CostTable.scala

kushti

Approved, but please check comments

kushti · 2021-10-29T08:51:33Z

sigmastate/src/main/scala/sigmastate/serialization/ErgoTreeSerializer.scala

@@ -248,7 +248,7 @@ class ErgoTreeSerializer {

  def substituteConstants(scriptBytes: Array[Byte],


description of the second output is missed

sigmastate/src/main/scala/sigmastate/trees.scala

sigmastate/src/main/scala/sigmastate/types.scala

prepare-v5.0: part 5: implementation of all eval methods

4cfaac4

aslesarenko requested a review from kushti August 11, 2021 12:37

kushti changed the base branch from prepare-v5.0-eval to develop August 19, 2021 21:24

Merge branch 'develop' of github.com:ScorexFoundation/sigmastate-inte…

7514204

…rpreter into prepare-v5.0-evaluator

kushti requested changes Aug 20, 2021

View reviewed changes

kushti reviewed Aug 20, 2021

View reviewed changes

sigmastate/src/main/scala/sigmastate/interpreter/ErgoTreeEvaluator.scala Outdated Show resolved Hide resolved

more ScalaDocs and some refactoring

db181ab

kushti requested changes Sep 22, 2021

View reviewed changes

aslesarenko added 2 commits October 16, 2021 18:11

prepare-v5.0-evaluator: ScalaDocs for substituteConstants

93387a4

prepare-v5.0-evaluator: better ScalaDoc for BigIntIsIntegral.rem method

e64ca79

greenhat requested changes Oct 18, 2021

View reviewed changes

kushti requested changes Oct 18, 2021

View reviewed changes

kushti reviewed Oct 18, 2021

View reviewed changes

sigmastate/src/main/scala/sigmastate/types.scala Outdated Show resolved Hide resolved

kushti reviewed Oct 18, 2021

View reviewed changes

sigmastate/src/main/scala/sigmastate/types.scala Show resolved Hide resolved

kushti reviewed Oct 18, 2021

View reviewed changes

sigmastate/src/main/scala/sigmastate/types.scala Show resolved Hide resolved

aslesarenko added 2 commits October 20, 2021 15:16

prepare-v5.0-evaluator: simplified Tuple.eval

815bc6d

prepare-v5.0-evaluator: more ScalaDocs

3c3f6f1

aslesarenko force-pushed the prepare-v5.0-evaluator branch from c335cbd to 3c3f6f1 Compare October 20, 2021 12:44

aslesarenko added 5 commits October 20, 2021 15:54

prepare-v5.0-evaluator: removed invalid checkWellDefined method

cec2855

prepare-v5.0-evaluator: comment for modQ methods

f248dc1

prepare-v5.0-evaluator: more ScalaDocs and code cleanup

2ad6e44

prepare-v5.0-evaluator: more ScalaDocs and cleanup

c9d5f1a

Merge remote-tracking branch 'origin/develop' into prepare-v5.0-evalu…

455d4a3

…ator # Conflicts: # sigmastate/src/main/scala/sigmastate/eval/BigIntegerOps.scala # sigmastate/src/main/scala/sigmastate/utxo/CostTable.scala

kushti approved these changes Oct 29, 2021

View reviewed changes

prepare-v5.0-evaluator: addressing review comments

262a14d

greenhat approved these changes Nov 1, 2021

View reviewed changes

aslesarenko merged commit 8c7d397 into develop Nov 3, 2021

aslesarenko mentioned this pull request Nov 24, 2021

Release v4.0.5 #751

Merged

aslesarenko deleted the prepare-v5.0-evaluator branch March 12, 2022 09:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Preparation refactoring towards v5.0 (part 5) #734

Preparation refactoring towards v5.0 (part 5) #734

aslesarenko commented Jul 26, 2021

codecov bot commented Jul 26, 2021 •

edited

Loading

kushti left a comment

aslesarenko commented Aug 25, 2021 •

edited

Loading

kushti Sep 22, 2021

aslesarenko Oct 16, 2021

kushti Oct 29, 2021

aslesarenko Nov 1, 2021

greenhat left a comment •

edited

Loading

kushti Oct 18, 2021

aslesarenko Oct 22, 2021

greenhat commented Oct 20, 2021

aslesarenko commented Oct 20, 2021 •

edited

Loading

kushti left a comment

kushti Oct 29, 2021

		@@ -248,7 +248,7 @@ class ErgoTreeSerializer {

		def substituteConstants(scriptBytes: Array[Byte],

Preparation refactoring towards v5.0 (part 5) #734

Preparation refactoring towards v5.0 (part 5) #734

Conversation

aslesarenko commented Jul 26, 2021

codecov bot commented Jul 26, 2021 • edited Loading

Codecov Report

kushti left a comment

Choose a reason for hiding this comment

aslesarenko commented Aug 25, 2021 • edited Loading

kushti Sep 22, 2021

Choose a reason for hiding this comment

aslesarenko Oct 16, 2021

Choose a reason for hiding this comment

kushti Oct 29, 2021

Choose a reason for hiding this comment

aslesarenko Nov 1, 2021

Choose a reason for hiding this comment

greenhat left a comment • edited Loading

Choose a reason for hiding this comment

kushti Oct 18, 2021

Choose a reason for hiding this comment

aslesarenko Oct 22, 2021

Choose a reason for hiding this comment

greenhat commented Oct 20, 2021

aslesarenko commented Oct 20, 2021 • edited Loading

kushti left a comment

Choose a reason for hiding this comment

kushti Oct 29, 2021

Choose a reason for hiding this comment

codecov bot commented Jul 26, 2021 •

edited

Loading

aslesarenko commented Aug 25, 2021 •

edited

Loading

greenhat left a comment •

edited

Loading

aslesarenko commented Oct 20, 2021 •

edited

Loading