Skip to content
This repository has been archived by the owner on Nov 14, 2024. It is now read-only.

Fix or Delete LockWatchValueIntegrationTest#valueStressTest #6699

Open
tpetracca opened this issue Aug 8, 2023 · 1 comment
Open

Fix or Delete LockWatchValueIntegrationTest#valueStressTest #6699

tpetracca opened this issue Aug 8, 2023 · 1 comment

Comments

@tpetracca
Copy link
Contributor

This test is a plague on the repo. It flakes constantly. This is bad for a variety of reasons:

  • hurts developer productivity
  • wastes $$ on CI that has to get re-run
  • breeds a culture of taking failing tests in this test-suite (3) less seriously

I've proposed Ignoring the test outright. But someone should follow-up with either a true fix or just deleting the test.

@tpetracca
Copy link
Contributor Author

tpetracca commented Aug 8, 2023

I can see 6 failures in just the last 5 hours in circleci (across a variety of branches including develop): https://app.circleci.com/pipelines/github/palantir/atlasdb

Examples:

Failures is always:

java.lang.AssertionError: Encountered nonretriable exception
	at com.palantir.atlasdb.timelock.LockWatchValueIntegrationTest.valueStressTest(LockWatchValueIntegrationTest.java:567)
    .... stuff ....
Caused by: java.util.concurrent.ExecutionException: com.palantir.logsafe.exceptions.SafeRuntimeException: Fallback cache threw an exception
	at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:205)
	at com.palantir.atlasdb.timelock.LockWatchValueIntegrationTest.valueStressTest(LockWatchValueIntegrationTest.java:564)
	... 56 more
Caused by: com.palantir.logsafe.exceptions.SafeRuntimeException: Fallback cache threw an exception
	at app//com.palantir.atlasdb.keyvalue.api.ResilientLockWatchProxy.handleException(ResilientLockWatchProxy.java:88)
	at app//com.palantir.atlasdb.keyvalue.api.ResilientLockWatchProxy.handleInvocation(ResilientLockWatchProxy.java:77)
	at app//com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:87)
	at app//com.sun.proxy.$Proxy184.onSuccessfulCommit(Unknown Source)
	at app//com.palantir.lock.watch.LockWatchCacheImpl.onTransactionCommit(LockWatchCacheImpl.java:58)
	at app//com.palantir.atlasdb.keyvalue.api.watch.LockWatchManagerImpl.onTransactionCommit(LockWatchManagerImpl.java:138)
	at app//com.palantir.atlasdb.transaction.impl.SnapshotTransactionManager.lambda$startTransactions$4(SnapshotTransactionManager.java:222)
	at [email protected]/java.util.concurrent.CopyOnWriteArrayList.forEach(CopyOnWriteArrayList.java:807)
	at app//com.palantir.atlasdb.transaction.impl.SnapshotTransaction$SuccessCallbackManager.runCallbacks(SnapshotTransaction.java:2885)
	at app//com.palantir.atlasdb.transaction.impl.SnapshotTransaction.runSuccessCallbacksIfDefinitivelyCommitted(SnapshotTransaction.java:1928)
	at app//com.palantir.atlasdb.transaction.impl.AbstractTransactionManager.runTaskThrowOnConflictWithCallback(AbstractTransactionManager.java:85)
	at app//com.palantir.atlasdb.transaction.impl.SnapshotTransactionManager$OpenTransactionImpl.finishWithCallback(SnapshotTransactionManager.java:270)
	at app//com.palantir.atlasdb.transaction.impl.SnapshotTransactionManager.runTaskWithConditionThrowOnConflict(SnapshotTransactionManager.java:186)
	at app//com.palantir.atlasdb.transaction.impl.SerializableTransactionManager.runTaskWithConditionThrowOnConflict(SerializableTransactionManager.java:58)
	at app//com.palantir.atlasdb.transaction.impl.AbstractConditionAwareTransactionManager.runTaskThrowOnConflict(AbstractConditionAwareTransactionManager.java:57)
	at com.palantir.tritium.proxy.InstrumentedTransactionManager$18.runTaskThrowOnConflict(Unknown Source)
	at app//com.palantir.atlasdb.timelock.LockWatchValueIntegrationTest.randomTransactionTask(LockWatchValueIntegrationTest.java:578)
	at [email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at [email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at [email protected]/java.lang.Thread.run(Thread.java:829)
Caused by: com.palantir.logsafe.exceptions.SafeIllegalStateException: Trying to cache a value which is either locked or is not equal to a currently cached value: {table=default.table, cell=Cell{rowName=617761697473, columnName=636f6c756d6e73}, oldValue=CacheEntry{status=LOCKED, value=com.palantir.atlasdb.keyvalue.api.cache.CacheValue@0}, newValue=CacheEntry{status=UNLOCKED, value=com.palantir.atlasdb.keyvalue.api.cache.CacheValue@0}}
	at app//com.palantir.logsafe.Preconditions.checkState(Preconditions.java:304)
	at app//com.palantir.atlasdb.keyvalue.api.cache.ValueStoreImpl.lambda$putValue$2(ValueStoreImpl.java:98)
	at app//io.vavr.collection.Maps.put(Maps.java:231)
	at app//io.vavr.collection.HashMap.put(HashMap.java:757)
	at app//io.vavr.collection.HashMap.put(HashMap.java:40)
	at app//com.palantir.atlasdb.keyvalue.api.cache.ValueStoreImpl.lambda$putValue$3(ValueStoreImpl.java:97)
	at app//com.palantir.atlasdb.keyvalue.api.cache.StructureHolder.with(StructureHolder.java:36)
	at app//com.palantir.atlasdb.keyvalue.api.cache.ValueStoreImpl.putValue(ValueStoreImpl.java:97)
	at app//com.google.common.collect.RegularImmutableMap.forEach(RegularImmutableMap.java:297)
	at app//com.palantir.atlasdb.keyvalue.api.cache.LockWatchValueScopingCacheImpl$1.invalidateSome(LockWatchValueScopingCacheImpl.java:165)
	at app//com.palantir.atlasdb.keyvalue.api.cache.LockWatchValueScopingCacheImpl$1.invalidateSome(LockWatchValueScopingCacheImpl.java:139)
	at app//com.palantir.lock.watch.CommitUpdate$InvalidateSome.accept(CommitUpdate.java:73)
	at app//com.palantir.atlasdb.keyvalue.api.cache.LockWatchValueScopingCacheImpl.onSuccessfulCommit(LockWatchValueScopingCacheImpl.java:139)
	at jdk.internal.reflect.GeneratedMethodAccessor284.invoke(Unknown Source)
	at [email protected]/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at [email protected]/java.lang.reflect.Method.invoke(Method.java:566)
	at app//com.palantir.atlasdb.keyvalue.api.ResilientLockWatchProxy.handleInvocation(ResilientLockWatchProxy.java:75)
	... 20 more

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant