Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added improvments on string copying, string comparation & calculation of next index in case of collision in custom map #650

Merged
merged 22 commits into from
Jan 29, 2024
Merged
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
cfa1fef
added code
JaimePolidura Jan 25, 2024
55631ee
Fixed pointers bugs
JaimePolidura Jan 27, 2024
16d7d8c
removed my own benchmark
JaimePolidura Jan 27, 2024
09c0862
Merge branch 'gunnarmorling:main' into main
JaimePolidura Jan 27, 2024
6c61123
added comment on how I handle hash collisions
JaimePolidura Jan 27, 2024
78dfea5
Merge branch 'main' of https://github.com/JaimePolidura/1brc
JaimePolidura Jan 27, 2024
e9f6bcc
executed mwvn clean verify
JaimePolidura Jan 27, 2024
81841d8
Merge branch 'gunnarmorling:main' into main
JaimePolidura Jan 27, 2024
486348c
Merge branch 'gunnarmorling:main' into main
JaimePolidura Jan 28, 2024
945edc4
made scripts executable & fixed rounding issues
JaimePolidura Jan 28, 2024
99528ab
Fixed way of dealing with hash collisions
JaimePolidura Jan 28, 2024
5b17296
changed method name sameNameBytes to isSameNameBytes
JaimePolidura Jan 28, 2024
2c4a000
changes script from sh to bash
JaimePolidura Jan 28, 2024
e40332e
Merge branch 'gunnarmorling:main' into main
JaimePolidura Jan 28, 2024
bd1ae2f
fixed chunking bug
JaimePolidura Jan 28, 2024
9ea5bcc
Merge branch 'main' of https://github.com/JaimePolidura/1brc
JaimePolidura Jan 28, 2024
89129f3
Fixed bug in chunking when file size is too small
JaimePolidura Jan 28, 2024
6be7268
added Runtime.getRuntime().availableProcessors
JaimePolidura Jan 28, 2024
174d84a
Merge branch 'gunnarmorling:main' into main
JaimePolidura Jan 29, 2024
9c24822
Merge branch 'gunnarmorling:main' into main
JaimePolidura Jan 29, 2024
7e16eb3
added improvemnts on string copying, calculation of next index of Map…
JaimePolidura Jan 29, 2024
857bbdf
Merge branch 'main' of https://github.com/JaimePolidura/1brc
JaimePolidura Jan 29, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -309,16 +309,14 @@ public SimpleMap(int size) {
}

public void put(long hashToPut, byte[] nameToPut, int nameLength, int valueToPut) {
int index = hashToIndex(hashToPut);
int index = toIndex(hashToPut);

for (;;) {
Result actualEntry = entries[index];

if (actualEntry == null) {
byte[] nameToPutCopy = new byte[nameLength];
for (int i = 0; i < nameLength; i++) {
nameToPutCopy[i] = nameToPut[i];
}
UNSAFE.copyMemory(nameToPut, Unsafe.ARRAY_BYTE_BASE_OFFSET, nameToPutCopy, Unsafe.ARRAY_BYTE_BASE_OFFSET, nameLength);

entries[index] = new Result(hashToPut, nameToPutCopy, nameLength, valueToPut,
valueToPut, valueToPut, 1);
Expand All @@ -331,14 +329,12 @@ public void put(long hashToPut, byte[] nameToPut, int nameLength, int valueToPut
actualEntry.sum = actualEntry.sum + valueToPut;
return;
}
// If the name is not the same, we try to go to the next slot
if (++index >= this.size) {
index = 0;
}

index = toIndex(index + 31);
}
}

private int hashToIndex(long hash) {
private int toIndex(long hash) {
return (int) (((hash >> 32) ^ ((int) hash)) & (this.size - 1));
}
}
Expand Down Expand Up @@ -367,8 +363,15 @@ public boolean isSameName(byte[] otherNameBytes, int otherNameLength) {
}

private boolean isSameNameBytes(byte[] otherNameBytes) {
for (int i = 0; i < this.nameLength; i++) {
if (this.name[i] != otherNameBytes[i]) {
for (int i = 0; i < this.nameLength; i += 8) {
long thisNameBytesAsLong = UNSAFE.getLong(this.name, Unsafe.ARRAY_BYTE_BASE_OFFSET + i);
long otherNameBytesAsLong = UNSAFE.getLong(otherNameBytes, Unsafe.ARRAY_BYTE_BASE_OFFSET + i);

int isPositiveAsInt = (((8 - nameLength + i) >> 31) & 1) ^ 0x01;
int shift = ((8 - nameLength + i) * isPositiveAsInt) * 8;
otherNameBytesAsLong = (otherNameBytesAsLong << shift) >>> shift;

if (thisNameBytesAsLong != otherNameBytesAsLong) {
return false;
}
}
Expand Down
Loading