Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix bitCount is 0 after intersect or union #40

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

SidWeng
Copy link
Contributor

@SidWeng SidWeng commented Aug 29, 2018

use the fact that after i & (i - 1), the right most of set bit of "i"
will be unset
ref: https://www.quora.com/How-do-you-count-the-number-of-1-bits-in-a-number-using-only-bitwise-operations

@SidWeng
Copy link
Contributor Author

SidWeng commented Sep 4, 2018

@alexandrnikitin I cannot figure out why "java.lang.NoClassDefFoundError: sbt/inc/Analysis" happens, any advice?

@alexandrnikitin
Copy link
Owner

@SidWeng I've created a PR #41 with code that uses java.lang.Long.bitCount() instead. But it crashes JVM for some reason I don't quite understand 😞 Also don't know about Scala 2.12 NoClassDefFoundError issue.

@SidWeng
Copy link
Contributor Author

SidWeng commented Sep 14, 2018

@alexandrnikitin Could you post the JVM crash log or reproduce step?

@alexandrnikitin
Copy link
Owner

Sure, I posted logs to this gist. To reproduce I run sbt "project tests" "testOnly *UnsafeBitArraysSpec" on #41. Surprisingly it's green on travis-ci

@SidWeng
Copy link
Contributor Author

SidWeng commented Sep 20, 2018

I try to reproduce it and looks like something wrong with unsafe.getLong() in UnsafeBitArray.set() but still cannot find out the root cause(I guess the index is too big and cause the offset is beyond the allocated memory). Here's my reproduce log

@SidWeng
Copy link
Contributor Author

SidWeng commented Nov 16, 2018

I add some debug log in UnsafeBitArray.scala then run sbt "project tests" "testOnly *UnsafeBitArraysSpec"
following is the debug log just before crash(unsafe.getLong(offset) in UnsafeBitArray.set()):
numberOfBits: 2147483647
indices: 33554432
ptr: 4805672960
end: 5074108416
index: 4294967294
offset: 5342543864

looks like the memory is not big enough or the index is too big
@alexandrnikitin any advice for fixing this error? what should we do if the index is too big, just throw an exception?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants