Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prf eval #795

Closed
wants to merge 118 commits into from
Closed
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
118 commits
Select commit Hold shift + click to select a range
cd46b44
added 25519 prime field
Oct 2, 2023
ce55114
Merge branch 'private-attribution:main' into largeprimefield
danielmasny Oct 2, 2023
e59b214
add files, make ec pub
Oct 2, 2023
1b552f4
last commit before working on shared curve points
Oct 2, 2023
9dc50c4
adding curve points + conversion
Oct 2, 2023
94597b1
from, into scalar for RP25519 + test
Oct 3, 2023
4da8014
test aritmetics for RP25519
Oct 3, 2023
fd6c28e
upgrade share<Fp25519> to share<curve point>
Oct 4, 2023
3c5683e
remove warning
Oct 4, 2023
b216481
add hash curve points
Oct 4, 2023
06f6eb1
implementation of PRF eval protocol
Oct 5, 2023
de5f32c
prf eval test passes
Oct 6, 2023
759f53a
Merge branch 'private-attribution:main' into PRF-Eval
danielmasny Oct 6, 2023
552cc62
fixed clippy complaints, panics when setting invalid curve points or …
Oct 9, 2023
fe3f130
remove comment
Oct 9, 2023
159cd2c
adding some of Martins suggestions
Oct 9, 2023
301a6e5
Alex suggestions, serialize, deserialize
Oct 16, 2023
9a1fa7c
Update src/ff/curve_points.rs
danielmasny Oct 16, 2023
8386d24
Alex suggestions, simplify u32, u64 macros
Oct 16, 2023
3468de5
Merge branch 'PRF-Eval' of https://github.com/danielmasny/ipa into PR…
Oct 16, 2023
5dc07a0
Update src/ff/curve_points.rs
danielmasny Oct 16, 2023
e7e6e63
Merge branch 'PRF-Eval' of https://github.com/danielmasny/ipa into PR…
Oct 16, 2023
e8d9d94
Merge branch 'private-attribution:main' into PRF-Eval
danielmasny Oct 16, 2023
94dda61
Merge branch 'PRF-Eval' of https://github.com/danielmasny/ipa into PR…
Oct 16, 2023
ea8da0d
simplify serde test
Oct 16, 2023
bcea6c6
simplify serde test
Oct 16, 2023
8bca660
error recursion limit
Oct 17, 2023
64dac77
Fix compiler recursion error
akoshelev Oct 17, 2023
884aeec
define WeakSharedValue, adding ipa-prf feature
Oct 17, 2023
a620e34
fix zero
Oct 17, 2023
55cef13
fmt
Oct 17, 2023
9736b71
fix clippy
Oct 17, 2023
2c8a19c
Merge branch 'private-attribution:main' into PRF-Eval
danielmasny Oct 17, 2023
b0ae43f
refactor prf_ipa in separate module
Oct 18, 2023
da13880
fmt
Oct 18, 2023
ecd1885
Add attribution window to OPRF aggregation
taikiy Oct 4, 2023
141a6b1
Optimize by removing redundant multiplications
taikiy Oct 19, 2023
946a9ee
save one more multiplication
taikiy Oct 20, 2023
a5d3587
inline the variables
taikiy Oct 20, 2023
89c86ec
Merge branch 'private-attribution:main' into PRF-Eval
danielmasny Oct 20, 2023
160ba83
adressing Alex's comments
Oct 20, 2023
8407938
fmt
Oct 20, 2023
43696a6
add comments
Oct 20, 2023
4268a05
fix lint
Oct 21, 2023
4208d1b
fmt
Oct 21, 2023
53d71f2
remove did_trigger_get_attributed and refactor attribution flag calcu…
taikiy Oct 23, 2023
1ce4281
1. Increasing max number of breakdowns we can support with OPRF
richajaindce Oct 24, 2023
034f556
Adding a oneshot for oprf
richajaindce Oct 24, 2023
2692af8
Now oprf can run for compact-gate as well
richajaindce Oct 24, 2023
2a798a3
Adding oprf handling in collect_steps.py
richajaindce Oct 24, 2023
8fa6745
refactoring collect_steps
richajaindce Oct 24, 2023
16c8943
Merge pull request #808 from taikiy/attribution_windowing
benjaminsavage Oct 25, 2023
d9d9672
Add test for no contribution + handle 512 breakdowns in test using Gf…
richajaindce Oct 24, 2023
5ea37ca
Supporting the opposite capping logic
benjaminsavage Oct 25, 2023
9fdad64
Merge pull request #815 from private-attribution/support_reverse_order
benjaminsavage Oct 25, 2023
0c5f464
Merge branch 'main' into minor_fixes
richajaindce Oct 25, 2023
c348906
Merge branch 'minor_fixes' into oneshot
richajaindce Oct 25, 2023
090d94f
Merge pull request #813 from richajaindce/minor_fixes
benjaminsavage Oct 25, 2023
dd07816
Peer feedback
richajaindce Oct 25, 2023
51e05fe
Some more fixes
richajaindce Oct 25, 2023
b396557
Merge pull request #814 from richajaindce/oneshot
benjaminsavage Oct 25, 2023
d8c0598
Some code refactor based on Taiki's feedback
richajaindce Oct 25, 2023
31a155c
OPRF test on real world infra
richajaindce Oct 26, 2023
9a59780
Merge pull request #818 from richajaindce/oneshot
benjaminsavage Oct 30, 2023
73468dc
Adding EventType in serialization and deserialization
richajaindce Oct 27, 2023
f77a8b4
Incorporate feedback
richajaindce Oct 30, 2023
2e4b237
Merge branch 'main' into real_world
richajaindce Oct 30, 2023
4d2469e
Make clippy happy
richajaindce Oct 30, 2023
0e9f41b
More fixes
richajaindce Oct 31, 2023
91905cd
Remove output.txt
richajaindce Oct 31, 2023
0747a6f
Rename function
richajaindce Oct 31, 2023
b4760d9
Merge pull request #822 from richajaindce/real_world
benjaminsavage Oct 31, 2023
2b50c45
Fix OPRF attribution and capping circuit
akoshelev Oct 31, 2023
b380eed
Merge pull request #825 from private-attribution/fix-oprf-attribution
benjaminsavage Nov 1, 2023
afeb72c
modulus conversion with retention
martinthomson Nov 1, 2023
6ea4069
Merge pull request #826 from martinthomson/retain-mod-convert
benjaminsavage Nov 1, 2023
da2ee44
Configure logging from env as well
akoshelev Oct 31, 2023
b4d10d6
Add send and receive trace spans
akoshelev Oct 31, 2023
9b7649b
added 25519 prime field
Oct 2, 2023
786d7c8
add files, make ec pub
Oct 2, 2023
c9cb5ee
last commit before working on shared curve points
Oct 2, 2023
31cbde8
adding curve points + conversion
Oct 2, 2023
6219b4c
from, into scalar for RP25519 + test
Oct 3, 2023
c65a93f
test aritmetics for RP25519
Oct 3, 2023
1acb869
upgrade share<Fp25519> to share<curve point>
Oct 4, 2023
644a6fc
remove warning
Oct 4, 2023
851f624
add hash curve points
Oct 4, 2023
45b2a46
implementation of PRF eval protocol
Oct 5, 2023
dd07d7a
prf eval test passes
Oct 6, 2023
e1cb4b7
fixed clippy complaints, panics when setting invalid curve points or …
Oct 9, 2023
b2f5c78
remove comment
Oct 9, 2023
8b0ab77
adding some of Martins suggestions
Oct 9, 2023
4c175bd
Alex suggestions, serialize, deserialize
Oct 16, 2023
537e7af
Alex suggestions, simplify u32, u64 macros
Oct 16, 2023
5325f6d
Update src/ff/curve_points.rs
danielmasny Oct 16, 2023
dd0aac0
Update src/ff/curve_points.rs
danielmasny Oct 16, 2023
5edee6f
simplify serde test
Oct 16, 2023
3fdd719
simplify serde test
Oct 16, 2023
5e65826
error recursion limit
Oct 17, 2023
b89d15a
Fix compiler recursion error
akoshelev Oct 17, 2023
b293fb5
define WeakSharedValue, adding ipa-prf feature
Oct 17, 2023
deeca7d
fix zero
Oct 17, 2023
729c87f
fmt
Oct 17, 2023
c9dc931
fix clippy
Oct 17, 2023
3aaeba4
refactor prf_ipa in separate module
Oct 18, 2023
04fe4d3
fmt
Oct 18, 2023
7db9376
adressing Alex's comments
Oct 20, 2023
7c211b7
fmt
Oct 20, 2023
4d5b105
add comments
Oct 20, 2023
39c7ced
fix lint
Oct 21, 2023
e5db397
fmt
Oct 21, 2023
c6564f0
Merge branch 'PRF-Eval' of https://github.com/danielmasny/ipa into PR…
Nov 2, 2023
0efd323
fix errors after merge
Nov 2, 2023
60d8ab6
fmt
Nov 2, 2023
a17eaa9
add flag
Nov 2, 2023
83d74cb
remove flag
Nov 2, 2023
f8d26c9
add flag
Nov 2, 2023
6c715ff
add flag descriptive gate
Nov 2, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ clap = { version = "4.3.2", optional = true, features = ["derive"] }
comfy-table = { version = "7.0", optional = true }
config = "0.13.2"
criterion = { version = "0.5.1", optional = true, default-features = false, features = ["async_tokio", "plotters", "html_reports"] }
curve25519-dalek = "4.1.1"
danielmasny marked this conversation as resolved.
Show resolved Hide resolved
dashmap = "5.4"
dhat = "0.3.2"
embed-doc-image = "0.1.4"
Expand Down
2 changes: 2 additions & 0 deletions src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,8 @@ pub enum Error {
InvalidReport(#[from] InvalidReportError),
#[error("unsupported: {0}")]
Unsupported(String),
#[error("Decompressing invalid elliptic curve point")]
DecompressingInvalidCurvePoint,
danielmasny marked this conversation as resolved.
Show resolved Hide resolved
}

impl Default for Error {
Expand Down
272 changes: 272 additions & 0 deletions src/ff/curve_points.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,272 @@
use curve25519_dalek::{
constants,
ristretto::{CompressedRistretto, RistrettoPoint},
Scalar,
};
use generic_array::GenericArray;
use hkdf::Hkdf;
use sha2::Sha256;
use typenum::U32;

use crate::{
error::Error,
ff::{ec_prime_field::Fp25519, Field, Serializable},
secret_sharing::{Block, SharedValue},
};

impl Block for CompressedRistretto {
type Size = U32;
}

///ristretto point for curve 25519
#[derive(Clone, Copy, PartialEq, Debug)]
pub struct RP25519(<Self as SharedValue>::Storage);

/// using compressed ristretto point, Zero is generator of the curve, i.e. g^0
impl SharedValue for RP25519 {
type Storage = CompressedRistretto;
const BITS: u32 = 256;
const ZERO: Self = Self(constants::RISTRETTO_BASEPOINT_COMPRESSED);
}

impl Serializable for RP25519 {
type Size = <<RP25519 as SharedValue>::Storage as Block>::Size;

fn serialize(&self, buf: &mut GenericArray<u8, Self::Size>) {
let raw = &self.0.as_bytes()[..buf.len()];
buf.copy_from_slice(raw);
danielmasny marked this conversation as resolved.
Show resolved Hide resolved
}

fn deserialize(buf: &GenericArray<u8, Self::Size>) -> Self {
RP25519(CompressedRistretto::from_slice(buf).unwrap())
danielmasny marked this conversation as resolved.
Show resolved Hide resolved
}
}

impl rand::distributions::Distribution<RP25519> for rand::distributions::Standard {
danielmasny marked this conversation as resolved.
Show resolved Hide resolved
fn sample<R: crate::rand::Rng + ?Sized>(&self, rng: &mut R) -> RP25519 {
//Fp25519(Scalar::random(rng: &mut R))
danielmasny marked this conversation as resolved.
Show resolved Hide resolved
let mut scalar_bytes = [0u8; 64];
rng.fill_bytes(&mut scalar_bytes);
RP25519(RistrettoPoint::from_uniform_bytes(&scalar_bytes).compress())
}
}

impl std::ops::Add for RP25519 {
type Output = Self;

fn add(self, rhs: Self) -> Self::Output {
Self((self.0.decompress().unwrap() + rhs.0.decompress().unwrap()).compress())
}
}

impl std::ops::AddAssign for RP25519 {
#[allow(clippy::assign_op_pattern)]
fn add_assign(&mut self, rhs: Self) {
*self = *self + rhs;
}
}

impl std::ops::Neg for RP25519 {
type Output = Self;

fn neg(self) -> Self::Output {
Self(self.0.decompress().unwrap().neg().compress())
}
}

impl std::ops::Sub for RP25519 {
type Output = Self;

fn sub(self, rhs: Self) -> Self::Output {
Self((self.0.decompress().unwrap() - rhs.0.decompress().unwrap()).compress())
}
}

impl std::ops::SubAssign for RP25519 {
#[allow(clippy::assign_op_pattern)]
fn sub_assign(&mut self, rhs: Self) {
*self = *self - rhs;
}
}

///Scalar Multiplication
///<'a, 'b> `std::ops::Mul<&'b"` Fp25519 for &'a
danielmasny marked this conversation as resolved.
Show resolved Hide resolved
impl RP25519 {

/// # Errors
/// Propagates errors from decompressing invalid curve point
pub fn s_mul(self, rhs: Fp25519) -> Result<RP25519, Error> {
danielmasny marked this conversation as resolved.
Show resolved Hide resolved
self.0
.decompress()
.map_or(Err(Error::DecompressingInvalidCurvePoint), |x| {
Ok((x * Scalar::from(rhs)).compress().into())
})
}
}

///do not use
impl std::ops::Mul for RP25519 {
danielmasny marked this conversation as resolved.
Show resolved Hide resolved
type Output = Self;

fn mul(self, _rhs: RP25519) -> Self::Output {
panic!("Two curve points cannot be multiplied! Do not use *, *= for RP25519 or secret shares of RP25519");
}
}

///do not use
impl std::ops::MulAssign for RP25519 {
fn mul_assign(&mut self, _rhs: RP25519) {
panic!("Two curve points cannot be multiplied! Do not use *, *= for RP25519 or secret shares of RP25519");
}
}

impl From<Scalar> for RP25519 {
danielmasny marked this conversation as resolved.
Show resolved Hide resolved
fn from(s: Scalar) -> Self {
RP25519(RistrettoPoint::mul_base(&s).compress())
}
}

impl From<Fp25519> for RP25519 {
danielmasny marked this conversation as resolved.
Show resolved Hide resolved
fn from(s: Fp25519) -> Self {
RP25519(RistrettoPoint::mul_base(&s.into()).compress())
}
}

impl From<CompressedRistretto> for RP25519 {
fn from(s: CompressedRistretto) -> Self {
RP25519(s)
}
}

impl From<RP25519> for CompressedRistretto {
fn from(s: RP25519) -> Self {
s.0
}
}

macro_rules! cp_hash_impl {
( $u_type:ty, $byte_size:literal) => {
danielmasny marked this conversation as resolved.
Show resolved Hide resolved
impl From<RP25519> for $u_type {
fn from(s: RP25519) -> Self {
let hk = Hkdf::<Sha256>::new(None, s.0.as_bytes());
danielmasny marked this conversation as resolved.
Show resolved Hide resolved
let mut okm = [0u8; $byte_size];
//error invalid length from expand only happens when okm is very large
hk.expand(&[], &mut okm).unwrap();
<$u_type>::from_le_bytes(okm)
}
}

impl From<$u_type> for RP25519 {
danielmasny marked this conversation as resolved.
Show resolved Hide resolved
fn from(s: $u_type) -> Self {
let hk = Hkdf::<Sha256>::new(None, &s.to_le_bytes());
let mut okm = [0u8; 32];
//error invalid length from expand only happens when okm is very large
hk.expand(&[], &mut okm).unwrap();
RP25519::deserialize(&okm.into())
}
}
};
}

cp_hash_impl!(u64, 8);

cp_hash_impl!(u32, 4);
danielmasny marked this conversation as resolved.
Show resolved Hide resolved

/// Daniel had to implement this since Reveal wants it, prefer not to, I dont understand why it is
/// actually needed there, maybe to upgrade it to malicious? but it still shouldn't be needed
impl Field for RP25519 {
danielmasny marked this conversation as resolved.
Show resolved Hide resolved
const ONE: RP25519 = Self(constants::RISTRETTO_BASEPOINT_COMPRESSED);

///both following methods are based on hashing and do not allow to actually convert elements in Fp25519
/// from or into u128. However it is sufficient to generate random elements in Fp25519
fn as_u128(&self) -> u128 {
let hk = Hkdf::<Sha256>::new(None, self.0.as_bytes());
let mut okm = [0u8; 16];
//error invalid length from expand only happens when okm is very large
hk.expand(&[], &mut okm).unwrap();
u128::from_le_bytes(okm)
}

///PRSS uses `truncate_from function`, we need to expand the u128 using a PRG (Sha256) to a [u8;32]
fn truncate_from<T: Into<u128>>(v: T) -> Self {
let hk = Hkdf::<Sha256>::new(None, &v.into().to_le_bytes());
let mut okm = [0u8; 32];
//error invalid length from expand only happens when okm is very large
hk.expand(&[], &mut okm).unwrap();
RP25519::deserialize(&okm.into())
}
}

impl TryFrom<u128> for RP25519 {
danielmasny marked this conversation as resolved.
Show resolved Hide resolved
type Error = crate::error::Error;

fn try_from(v: u128) -> Result<Self, Self::Error> {
let mut bits = [0u8; 32];
bits[..].copy_from_slice(&v.to_le_bytes());
let f: RP25519 = RP25519::ONE;
f.serialize((&mut bits).into());
danielmasny marked this conversation as resolved.
Show resolved Hide resolved
Ok(f)
}
}

#[cfg(all(test, unit_test))]
mod test {
use curve25519_dalek::scalar::Scalar;
use generic_array::GenericArray;
use rand::{thread_rng, Rng};
use typenum::U32;

use crate::{
ff::{curve_points::RP25519, ec_prime_field::Fp25519, Serializable, field::Field},
};

#[test]
fn serde_25519() {
let input: [u8; 32] = [
0x01, 0xff, 0x00, 0xff, 0x00, 0xff, 0x00, 0xff, 0x00, 0xff, 0x00, 0xff, 0x00, 0xff,
0x00, 0xff, 0x00, 0xff, 0x00, 0xff, 0x00, 0xff, 0x00, 0xff, 0x00, 0xff, 0x00, 0xff,
0x00, 0x00, 0x00, 0x00,
];
let mut output: GenericArray<u8, U32> = [0u8; 32].into();
let a = RP25519::deserialize(&input.into());
assert_eq!(a.0.as_bytes()[..32], input);
a.serialize(&mut output);
assert_eq!(a.0.as_bytes()[..32], output.as_slice()[..32]);
assert_eq!(input, output.as_slice()[..32]);
}

#[test]
fn scalar_to_point() {
let a = Scalar::ONE;
let b: RP25519 = a.into();
let d: Fp25519 = a.into();
let c: RP25519 = RP25519::from(d);
assert_eq!(b, RP25519::ONE);
assert_eq!(c, RP25519::ONE);
}

#[test]
fn curve_arithmetics() {
let mut rng = thread_rng();
let fp_a = rng.gen::<Fp25519>();
let fp_b = rng.gen::<Fp25519>();
let fp_c = fp_a + fp_b;
let fp_d = RP25519::from(fp_a) + RP25519::from(fp_b);
assert_eq!(fp_d, RP25519::from(fp_c));
assert_ne!(fp_d, RP25519::ONE);
let fp_e = rng.gen::<Fp25519>();
let fp_f = rng.gen::<Fp25519>();
let fp_g = fp_e * fp_f;
let fp_h = RP25519::from(fp_e).s_mul(fp_f).unwrap();
assert_eq!(fp_h, RP25519::from(fp_g));
assert_ne!(fp_h, RP25519::ONE);
}

#[test]
fn curve_point_to_hash() {
let mut rng = thread_rng();
let fp_a = rng.gen::<RP25519>();
assert_ne!(0u64, u64::from(fp_a));
assert_ne!(0u32, u32::from(fp_a));
}
}
Loading
Loading