Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[6.0] Fix semantics of AvlTree.insert & new AvlTree.insertOrUpdate method #1038

Merged
merged 9 commits into from
Jan 9, 2025
Merged

[6.0] Fix semantics of AvlTree.insert & new AvlTree.insertOrUpdate method #1038

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
c1e5ba3
not throwing exception on duplicate key
kushti Dec 2, 2024
0c74184
insertOrUpdate
kushti Dec 3, 2024
754d624
fixing JS tests and ErgoTree specification
kushti Dec 3, 2024
a506be8
merging w. 6.0.0
kushti Dec 4, 2024
b80b3af
Merge branch 'v6.0.0' of github.com:ScorexFoundation/sigmastate-inter…
kushti Dec 5, 2024
ea31523
polishing, doc and comments update
kushti Dec 16, 2024
2c501a7
Update interpreter/shared/src/main/scala/sigmastate/interpreter/CErgo…
kushti Jan 9, 2025
18a6a77
comments improved
kushti Jan 9, 2025
45a4686
Merge branch 'i908' of github.com:ScorexFoundation/sigmastate-interpr…
kushti Jan 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
polishing, doc and comments update
  • Loading branch information
@kushti
kushti committed Dec 16, 2024
commit ea3152334a2209a964f5e1627d70fba02feef44d
10 changes: 5 additions & 5 deletions data/shared/src/main/scala/sigma/ast/methods.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1765,19 +1765,19 @@ case object SAvlTreeMethods extends MonoTypeMethods {
.withIRInfo(MethodCallIrBuilder)
.withInfo(MethodCall,
"""
| /** Perform insertions of key-value entries into this tree using proof `proof`.
| /** Perform insertions or updates of key-value entries into this tree using proof `proof`.
| * Throws exception if proof is incorrect
| *
| * @note CAUTION! Pairs must be ordered the same way they were in insert ops before proof was generated.
| * Return Some(newTree) if successful
| * Return None if operations were not performed.
| * @param operations collection of key-value pairs to insert in this authenticated dictionary.
| *
| * @note CAUTION! Pairs must be ordered the same way they were in insert ops before proof was generated.
| * @param operations collection of key-value pairs to insert or update in this authenticated dictionary.
| * @param proof
| */
|
""".stripMargin)

/** Implements evaluation of AvlTree.insert method call ErgoTree node.
/** Implements evaluation of AvlTree.insertOrUpdate method call ErgoTree node.
* Called via reflection based on naming convention.
* @see SMethod.evalMethod
*/
Expand Down
2 changes: 1 addition & 1 deletion data/shared/src/main/scala/sigma/eval/AvlTreeVerifier.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ trait AvlTreeVerifier {
* is None.
*
* @param key key to look up
* @param value value to check it was updated
* @param value value to check it was inserted or updated
* @return Success(Some(value)), Success(None), or Failure
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sense to describe each case of returned result.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

*/
def performInsertOrUpdate(key: Array[Byte], value: Array[Byte]): Try[Option[Array[Byte]]]
Expand Down
2 changes: 1 addition & 1 deletion data/shared/src/main/scala/sigma/eval/ErgoTreeEvaluator.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ abstract class ErgoTreeEvaluator {
mc: MethodCall, tree: AvlTree,
operations: KeyValueColl, proof: Coll[Byte]): Option[AvlTree]

/** Implements evaluation of AvlTree.insert method call ErgoTree node. */
/** Implements evaluation of AvlTree.insertOrUpdate method call ErgoTree node. */
def insertOrUpdate_eval(
mc: MethodCall,
tree: AvlTree,
Expand Down
16 changes: 15 additions & 1 deletion docs/LangSpec.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -749,10 +749,24 @@ class AvlTree {
* Return None if operations were not performed.
* @param operations collection of key-value pairs to update in this
* authenticated dictionary.
* @param proof data to reconstruct part of the tree
* @param proof subtree which is enough to check operations
*/
def update(operations: Coll[(Coll[Byte], Coll[Byte])], proof: Coll[Byte]): Option[AvlTree]


/** Perform insertions or updates of key-value entries into this tree using proof `proof`.
* Throws exception if proof is incorrect
*
* @note CAUTION! Pairs must be ordered the same way they were in ops
* before proof was generated.
* Return Some(newTree) if successful
* Return None if operations were not performed.
* @param operations collection of key-value pairs to insert or update in this
* authenticated dictionary.
* @param proof subtree which is enough to check operations
*/
def insertOrUpdate(operations: Coll[(Coll[Byte], Coll[Byte])], proof: Coll[Byte]): Option[AvlTree]

/** Perform removal of entries into this tree using proof `proof`.
* Throws exception if proof is incorrect
* Return Some(newTree) if successful
Expand Down
22 changes: 20 additions & 2 deletions interpreter/shared/src/main/scala/sigmastate/eval/Extensions.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ package sigmastate.eval
import debox.cfor
import org.ergoplatform.ErgoBox
import org.ergoplatform.ErgoBox.TokenId
import scorex.crypto.authds.avltree.batch.{Insert, Lookup, Remove, Update}
import scorex.crypto.authds.avltree.batch.{Insert, InsertOrUpdate, Lookup, Remove, Update}
import scorex.crypto.authds.{ADKey, ADValue}
import scorex.util.encode.Base16
import sigma.ast.SType.AnyOps
Expand Down Expand Up @@ -91,7 +91,7 @@ object Extensions {
val bv = CAvlTreeVerifier(tree, proof)
entries.forall { case (key, value) =>
val insertRes = bv.performOneOperation(Insert(ADKey @@ key.toArray, ADValue @@ value.toArray))
if (insertRes.isFailure) {
if (insertRes.isFailure && !VersionContext.current.isV6SoftForkActivated) {
syntax.error(s"Incorrect insert for $tree (key: $key, value: $value, digest: ${tree.digest}): ${insertRes.failed.get}}")
}
insertRes.isSuccess
Expand Down Expand Up @@ -120,6 +120,24 @@ object Extensions {
}
}

def insertOrUpdate(
entries: Coll[(Coll[Byte], Coll[Byte])],
proof: Coll[Byte]): Option[AvlTree] = {
if (!tree.isInsertAllowed || !tree.isUpdateAllowed) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both flags are required, however for each given tree it will either be insert or update (but not both).
Is it possible to make it more flexible and require the flag only if the corresponding operation is happening for a given tree.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

" for each given tree it will either be insert or update (but not both)" - no, it could be insert for one key and update for another

None
} else {
val bv = CAvlTreeVerifier(tree, proof)
entries.forall { case (key, value) =>
val insertRes = bv.performOneOperation(InsertOrUpdate(ADKey @@ key.toArray, ADValue @@ value.toArray))
insertRes.isSuccess
}
bv.digest match {
case Some(d) => Some(tree.updateDigest(Colls.fromArray(d)))
case _ => None
}
}
}

def remove(operations: Coll[Coll[Byte]], proof: Coll[Byte]): Option[AvlTree] = {
if (!tree.isRemoveAllowed) {
None
Expand Down
143 changes: 0 additions & 143 deletions sc/shared/src/test/scala/sigma/LanguageSpecificationV5.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3174,11 +3174,6 @@ class LanguageSpecificationV5 extends LanguageSpecificationBase { suite =>

type BatchProver = BatchAVLProver[Digest32, Blake2b256.type]

def performInsert(avlProver: BatchProver, key: Coll[Byte], value: Coll[Byte]) = {
avlProver.performOneOperation(Insert(ADKey @@ key.toArray, ADValue @@ value.toArray))
val proof = avlProver.generateProof().toColl
proof
}

def performUpdate(avlProver: BatchProver, key: Coll[Byte], value: Coll[Byte]) = {
avlProver.performOneOperation(Update(ADKey @@ key.toArray, ADValue @@ value.toArray))
Expand All @@ -3202,144 +3197,6 @@ class LanguageSpecificationV5 extends LanguageSpecificationBase { suite =>

type KV = (Coll[Byte], Coll[Byte])

property("AvlTree.insert equivalence") {
val insert = existingFeature((t: (AvlTree, (Coll[KV], Coll[Byte]))) => t._1.insert(t._2._1, t._2._2),
"{ (t: (AvlTree, (Coll[(Coll[Byte], Coll[Byte])], Coll[Byte]))) => t._1.insert(t._2._1, t._2._2) }",
FuncValue(
Vector(
(
1,
STuple(
Vector(
SAvlTree,
STuple(Vector(SCollectionType(STuple(Vector(SByteArray, SByteArray))), SByteArray))
)
)
)
),
BlockValue(
Vector(
ValDef(
3,
List(),
SelectField.typed[Value[STuple]](
ValUse(
1,
STuple(
Vector(
SAvlTree,
STuple(Vector(SCollectionType(STuple(Vector(SByteArray, SByteArray))), SByteArray))
)
)
),
2.toByte
)
)
),
MethodCall.typed[Value[SOption[SAvlTree.type]]](
SelectField.typed[Value[SAvlTree.type]](
ValUse(
1,
STuple(
Vector(
SAvlTree,
STuple(Vector(SCollectionType(STuple(Vector(SByteArray, SByteArray))), SByteArray))
)
)
),
1.toByte
),
SAvlTreeMethods.getMethodByName("insert"),
Vector(
SelectField.typed[Value[SCollection[STuple]]](
ValUse(
3,
STuple(Vector(SCollectionType(STuple(Vector(SByteArray, SByteArray))), SByteArray))
),
1.toByte
),
SelectField.typed[Value[SCollection[SByte.type]]](
ValUse(
3,
STuple(Vector(SCollectionType(STuple(Vector(SByteArray, SByteArray))), SByteArray))
),
2.toByte
)
),
Map()
)
)
))

val testTraceBase = Array(
FixedCostItem(Apply),
FixedCostItem(FuncValue),
FixedCostItem(GetVar),
FixedCostItem(OptionGet),
FixedCostItem(FuncValue.AddToEnvironmentDesc, FixedCost(JitCost(5))),
ast.SeqCostItem(CompanionDesc(BlockValue), PerItemCost(JitCost(1), JitCost(1), 10), 1),
FixedCostItem(ValUse),
FixedCostItem(SelectField),
FixedCostItem(FuncValue.AddToEnvironmentDesc, FixedCost(JitCost(5))),
FixedCostItem(ValUse),
FixedCostItem(SelectField),
FixedCostItem(MethodCall),
FixedCostItem(ValUse),
FixedCostItem(SelectField),
FixedCostItem(ValUse),
FixedCostItem(SelectField),
FixedCostItem(SAvlTreeMethods.isInsertAllowedMethod, FixedCost(JitCost(15)))
)
val costDetails1 = TracedCost(testTraceBase)
val costDetails2 = TracedCost(
testTraceBase ++ Array(
ast.SeqCostItem(NamedDesc("CreateAvlVerifier"), PerItemCost(JitCost(110), JitCost(20), 64), 70),
ast.SeqCostItem(NamedDesc("InsertIntoAvlTree"), PerItemCost(JitCost(40), JitCost(10), 1), 1),
FixedCostItem(SAvlTreeMethods.updateDigestMethod, FixedCost(JitCost(40)))
)
)

forAll(keyCollGen, bytesCollGen) { (key, value) =>
val (tree, avlProver) = createAvlTreeAndProver()
val preInsertDigest = avlProver.digest.toColl
val insertProof = performInsert(avlProver, key, value)
val kvs = Colls.fromItems((key -> value))

{ // positive
val preInsertTree = createTree(preInsertDigest, insertAllowed = true)
val input = (preInsertTree, (kvs, insertProof))
val (res, _) = insert.checkEquality(input).getOrThrow
res.isDefined shouldBe true
insert.checkExpected(input, Expected(Success(res), 1796, costDetails2, 1796, Seq.fill(4)(2102)))
}

{ // negative: readonly tree
val readonlyTree = createTree(preInsertDigest)
val input = (readonlyTree, (kvs, insertProof))
val (res, _) = insert.checkEquality(input).getOrThrow
res.isDefined shouldBe false
insert.checkExpected(input, Expected(Success(res), 1772, costDetails1, 1772, Seq.fill(4)(2078)))
}

{ // negative: invalid key
val tree = createTree(preInsertDigest, insertAllowed = true)
val invalidKey = key.map(x => (-x).toByte) // any other different from key
val invalidKvs = Colls.fromItems((invalidKey -> value)) // NOTE, insertProof is based on `key`
val input = (tree, (invalidKvs, insertProof))
val (res, _) = insert.checkEquality(input).getOrThrow
res.isDefined shouldBe true // TODO v6.0: should it really be true? (looks like a bug) (see https://github.com/ScorexFoundation/sigmastate-interpreter/issues/908)
insert.checkExpected(input, Expected(Success(res), 1796, costDetails2, 1796, Seq.fill(4)(2102)))
}

{ // negative: invalid proof
val tree = createTree(preInsertDigest, insertAllowed = true)
val invalidProof = insertProof.map(x => (-x).toByte) // any other different from proof
val res = insert.checkEquality((tree, (kvs, invalidProof)))
res.isFailure shouldBe true
}
}
}

property("AvlTree.update equivalence") {
val update = existingFeature((t: (AvlTree, (Coll[KV], Coll[Byte]))) => t._1.update(t._2._1, t._2._2),
"{ (t: (AvlTree, (Coll[(Coll[Byte], Coll[Byte])], Coll[Byte]))) => t._1.update(t._2._1, t._2._2) }",
Expand Down
Loading
Loading