Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(act): rebase api changes test merge #55

Merged
merged 43 commits into from
Jun 7, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
8645aca
chore: bump protobuf version to 1.33.0 (#4613) (#4688)
martinconic May 20, 2024
5fe82a6
fix: save stamp issuer after failed uploads (#4684)
istae May 27, 2024
de7eccc
chore: pre-release updates (#4675)
istae May 28, 2024
ca9e74a
chore: bump go ethereum version to 1.14.3 (#4661) (#4689)
martinconic May 28, 2024
68cde4a
fix: allow maxMultiplexForwards to be set to zero (#4682)
ldeffenb May 28, 2024
97e7ee6
chore: bump go version to 1.22 (#4590) (#4687)
martinconic May 28, 2024
79a7991
feat: remove debug api (#4674)
acha-bill May 29, 2024
8c61408
feat: remove auth (#4679)
acha-bill May 31, 2024
fa8315d
feat: add act.go with TODOs
Feb 19, 2024
c4883d0
Diffie-Hellman (#3)
ferencsarai Mar 13, 2024
3b02370
Acces Logic (#8)
aranyia Mar 13, 2024
9b1d4d7
Added context & details to use cases (#6)
aranyia Mar 13, 2024
ade71b0
Add grantee management (#10)
Kexort Mar 14, 2024
e53fc7e
(refactor): from `Get` to `Lookup` to improve clarity and consistency…
ferencsarai Mar 19, 2024
273eea3
Act params rename doc (#14)
ferencsarai Mar 19, 2024
67c4bb4
Move and refactor ACT diffieHellman to Session. Add Key and NewFromKe…
bosi95 Mar 19, 2024
8f35034
Act swarm address (#15)
ferencsarai Mar 19, 2024
2ddeb48
(rename): defaultAct to inMemoryAct (#17)
ferencsarai Mar 19, 2024
64d4b90
(refactor): Update controller_test.go to use NewInMemoryAct, modify S…
ferencsarai Mar 20, 2024
a244177
Act access logic merge (#19)
rolandlor Mar 21, 2024
56430a5
Act kvs merge (#22)
ferencsarai Mar 22, 2024
294f2d4
Session refactor (#24)
bosi95 Mar 26, 2024
64471a0
Access logic refactor (#25)
rolandlor Mar 26, 2024
ce2353c
(refactor:) PR comments (#23)
ferencsarai Mar 27, 2024
4a12e29
Add referenced mock kvs (#26)
Kexort Mar 28, 2024
fee9296
Act kvs test (#27)
ferencsarai Mar 28, 2024
0d45187
Small refactor + al test (#28)
Kexort Apr 2, 2024
f7563b3
Persist grantee list on swarm (#30)
bosi95 Apr 3, 2024
d84fca5
Update package imports to use the v2 version of the modules (#33)
ferencsarai Apr 4, 2024
1056212
chore(mantaray): merge mantaray fix for rebase
LevilkTheReal Apr 9, 2024
f6fc3c4
chore: rebase router after API auth changes
bosi95 Apr 19, 2024
96d4a4e
Act refactor api and ctrl (#36)
bosi95 Apr 19, 2024
2707884
ACT grantee management (#37)
Kexort May 13, 2024
943b219
Start refactoring for new linter rules (#39)
kopi-solarpunk May 16, 2024
272ffcb
refactor(act): typos & docs (#40)
aranyia May 16, 2024
6d805ae
refactor(act): naming and fix remaining PR comments (#42)
bosi95 May 16, 2024
f5d8cb1
Refactor accesslogic.AddGrantee and parallelize tests (#43)
bosi95 May 17, 2024
b80c899
docs(act): following changes to openapi with debug API removal
aranyia May 22, 2024
5e377cc
refactor(act): chunk download and granteelist handling (#46)
bosi95 May 23, 2024
755e038
refactor(act): controller logic (#47)
bosi95 May 24, 2024
4faf2fc
test(act): controller add, revoke and get with history + fix typos (#48)
bosi95 May 29, 2024
370ebf1
Merge branch 'refs/heads/act' into act-rebase-api-changes
aranyia Jun 6, 2024
15a5eb8
Merge branch 'refs/heads/act' into act-rebase-api-changes
aranyia Jun 7, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Start refactoring for new linter rules (#39)
refactor: refactoring to match new linter rules according to #38

Not everything is fixed, just a reference what can and needs to be improved.

* implement grantee management

* Add POST endpoint + fixes

* Save grantees as pubkey list and fix remove error; CHG: act-handler logger names

* Refactor: pass getter, putter to controller functions

* Refactor: error handling in dynamicaccess; Read cache header only for download handlers

* CHG: grantees ref is encrypted and added to history ref + tests

* Fix nil pointer dereference panic

* CHG: put actref in handlegrantees; Add: pin, tag,deferred headers

* CHG: pass loadsave to handlers; check if history address is nil

* FIX: re-init history so that it can be saved; only add publisher if histroy is zero

* make act timestamp optional

* fix revoke grantees

* Fix: Act timestamp header nil check; Uploadhandler UT

* refactor: start refactoring for now linter rules

* refactor: revert non ACT related files

* CHG: accesslogic getkeys refactor

* refactor: fix errcheck and ineffassign linter errors in most cases

* refactor: add headers, and change error handling

* refactor: add headers

---------

Co-authored-by: Kexort <kexort@gmail.com>
Co-authored-by: Bálint Ujvári <balint.ujvari@solarpunk.buzz>
  • Loading branch information
3 people authored and aranyia committed Jun 6, 2024
commit 943b219fe0788d3d1b5ae5f3b1dcee3a15b15bd1
12 changes: 8 additions & 4 deletions pkg/api/dynamicaccess.go
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
// Copyright 2024 The Swarm Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

package api

import (
Expand All @@ -6,6 +10,7 @@ import (
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"time"
Expand Down Expand Up @@ -114,7 +119,6 @@ func (s *Service) actDecryptionHandler() func(h http.Handler) http.Handler {
h.ServeHTTP(w, r.WithContext(setAddressInContext(ctx, reference)))
})
}

}

// actEncryptionHandler is a middleware that encrypts the given address using the publisher's public key
Expand All @@ -133,7 +137,7 @@ func (s *Service) actEncryptionHandler(
if err != nil {
logger.Debug("act failed to encrypt reference", "error", err)
logger.Error(nil, "act failed to encrypt reference")
return swarm.ZeroAddress, err
return swarm.ZeroAddress, fmt.Errorf("act failed to encrypt reference: %w", err)
}
// only need to upload history and kvs if a new history is created,
// meaning that the publsher uploaded to the history for the first time
Expand All @@ -142,13 +146,13 @@ func (s *Service) actEncryptionHandler(
if err != nil {
logger.Debug("done split keyvaluestore failed", "error", err)
logger.Error(nil, "done split keyvaluestore failed")
return swarm.ZeroAddress, err
return swarm.ZeroAddress, fmt.Errorf("done split keyvaluestore failed: %w", err)
}
err = putter.Done(historyReference)
if err != nil {
logger.Debug("done split history failed", "error", err)
logger.Error(nil, "done split history failed")
return swarm.ZeroAddress, err
return swarm.ZeroAddress, fmt.Errorf("done split history failed: %w", err)
}
}

Expand Down
48 changes: 21 additions & 27 deletions pkg/api/dynamicaccess_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -40,23 +40,23 @@ func prepareHistoryFixture(storer api.Storer) (dynamicaccess.History, swarm.Addr

testActRef1 := swarm.NewAddress([]byte("39a5ea87b141fe44aa609c3327ecd891"))
firstTime := time.Date(1994, time.April, 1, 0, 0, 0, 0, time.UTC).Unix()
h.Add(ctx, testActRef1, &firstTime, nil)
_ = h.Add(ctx, testActRef1, &firstTime, nil)

testActRef2 := swarm.NewAddress([]byte("39a5ea87b141fe44aa609c3327ecd892"))
secondTime := time.Date(2000, time.April, 1, 0, 0, 0, 0, time.UTC).Unix()
h.Add(ctx, testActRef2, &secondTime, nil)
_ = h.Add(ctx, testActRef2, &secondTime, nil)

testActRef3 := swarm.NewAddress([]byte("39a5ea87b141fe44aa609c3327ecd893"))
thirdTime := time.Date(2015, time.April, 1, 0, 0, 0, 0, time.UTC).Unix()
h.Add(ctx, testActRef3, &thirdTime, nil)
_ = h.Add(ctx, testActRef3, &thirdTime, nil)

testActRef4 := swarm.NewAddress([]byte("39a5ea87b141fe44aa609c3327ecd894"))
fourthTime := time.Date(2020, time.April, 1, 0, 0, 0, 0, time.UTC).Unix()
h.Add(ctx, testActRef4, &fourthTime, nil)
_ = h.Add(ctx, testActRef4, &fourthTime, nil)

testActRef5 := swarm.NewAddress([]byte("39a5ea87b141fe44aa609c3327ecd895"))
fifthTime := time.Date(2030, time.April, 1, 0, 0, 0, 0, time.UTC).Unix()
h.Add(ctx, testActRef5, &fifthTime, nil)
_ = h.Add(ctx, testActRef5, &fifthTime, nil)

ref, _ := h.Store(ctx)
return h, ref
Expand Down Expand Up @@ -221,10 +221,11 @@ func TestDacEachEndpointWithAct(t *testing.T) {
}
}

// nolint:paralleltest,tparallel
// TestDacWithoutActHeader [negative tests]:
// 1. upload w/ "Swarm-Act" header then try to dowload w/o the header.
// 2. upload w/o "Swarm-Act" header then try to dowload w/ the header.
//
//nolint:paralleltest,tparallel
func TestDacWithoutAct(t *testing.T) {
t.Parallel()
var (
Expand Down Expand Up @@ -321,8 +322,9 @@ func TestDacWithoutAct(t *testing.T) {
})
}

// nolint:paralleltest,tparallel
// TestDacInvalidPath [negative test]: Expect Bad request when the path address is invalid.
//
//nolint:paralleltest,tparallel
func TestDacInvalidPath(t *testing.T) {
t.Parallel()
var (
Expand All @@ -345,9 +347,7 @@ func TestDacInvalidPath(t *testing.T) {
PublicKey: pk.PublicKey,
Dac: mockdac.New(),
})
var (
encryptedRef = "asd"
)
encryptedRef := "asd"

jsonhttptest.Request(t, client, http.MethodGet, fileDownloadResource(encryptedRef), http.StatusBadRequest,
jsonhttptest.WithRequestHeader(api.SwarmActTimestampHeader, strconv.FormatInt(now, 10)),
Expand All @@ -361,7 +361,8 @@ func TestDacInvalidPath(t *testing.T) {
Field: "address",
Error: api.HexInvalidByteError('s').Error(),
},
}}),
},
}),
jsonhttptest.WithRequestHeader(api.ContentTypeHeader, "text/html; charset=utf-8"),
)
})
Expand All @@ -388,7 +389,6 @@ func TestDacHistory(t *testing.T) {
fileName = "sample.html"
now = time.Now().Unix()
)
fmt.Printf("bagoy now: %d\n", now)

t.Run("empty-history-upload-then-download-and-check-data", func(t *testing.T) {
client, _, _, _ := newTestServer(t, testServerOptions{
Expand Down Expand Up @@ -516,9 +516,7 @@ func TestDacHistory(t *testing.T) {
PublicKey: pk.PublicKey,
Dac: mockdac.New(),
})
var (
testfile = "testfile1"
)
testfile := "testfile1"

jsonhttptest.Request(t, client, http.MethodPost, fileUploadResource+"?name="+fileName, http.StatusInternalServerError,
jsonhttptest.WithRequestHeader(api.SwarmActHeader, "true"),
Expand All @@ -541,9 +539,7 @@ func TestDacHistory(t *testing.T) {
PublicKey: pk.PublicKey,
Dac: mockdac.New(mockdac.WithHistory(h, fixtureHref.String())),
})
var (
encryptedRef = "a5df670544eaea29e61b19d8739faa4573b19e4426e58a173e51ed0b5e7e2ade"
)
encryptedRef := "a5df670544eaea29e61b19d8739faa4573b19e4426e58a173e51ed0b5e7e2ade"

jsonhttptest.Request(t, client, http.MethodGet, fileDownloadResource(encryptedRef), http.StatusNotFound,
jsonhttptest.WithRequestHeader(api.SwarmActTimestampHeader, strconv.FormatInt(now, 10)),
Expand Down Expand Up @@ -619,9 +615,7 @@ func TestDacTimestamp(t *testing.T) {
})

t.Run("download-w/o-timestamp", func(t *testing.T) {
var (
encryptedRef = "a5df670544eaea29e61b19d8739faa4573b19e4426e58a173e51ed0b5e7e2ade"
)
encryptedRef := "a5df670544eaea29e61b19d8739faa4573b19e4426e58a173e51ed0b5e7e2ade"
client, _, _, _ := newTestServer(t, testServerOptions{
Storer: storerMock,
Logger: logger,
Expand Down Expand Up @@ -754,7 +748,8 @@ func TestDacPublisher(t *testing.T) {
Field: "Swarm-Act-Publisher",
Error: "malformed public key: invalid length: 32",
},
}}),
},
}),
jsonhttptest.WithRequestHeader(api.ContentTypeHeader, "text/html; charset=utf-8"),
)
})
Expand Down Expand Up @@ -785,9 +780,7 @@ func TestDacPublisher(t *testing.T) {
})

t.Run("download-w/o-publisher", func(t *testing.T) {
var (
encryptedRef = "a5df670544eaea29e61b19d8739faa4573b19e4426e58a173e51ed0b5e7e2ade"
)
encryptedRef := "a5df670544eaea29e61b19d8739faa4573b19e4426e58a173e51ed0b5e7e2ade"
client, _, _, _ := newTestServer(t, testServerOptions{
Storer: storerMock,
Logger: logger,
Expand Down Expand Up @@ -862,7 +855,8 @@ func TestDacGrantees(t *testing.T) {
Field: "address",
Error: api.HexInvalidByteError('s').Error(),
},
}}),
},
}),
)
})
t.Run("add-revoke-grantees", func(t *testing.T) {
Expand Down Expand Up @@ -904,8 +898,8 @@ func TestDacGrantees(t *testing.T) {
jsonhttptest.WithRequestHeader(api.SwarmPostageBatchIdHeader, batchOkStr),
jsonhttptest.WithJSONRequestBody(body),
)

})

t.Run("create-granteelist", func(t *testing.T) {
body := api.GranteesPostRequest{
GranteeList: []string{
Expand Down
65 changes: 36 additions & 29 deletions pkg/dynamicaccess/accesslogic.go
Original file line number Diff line number Diff line change
@@ -1,34 +1,42 @@
// Copyright 2024 The Swarm Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

package dynamicaccess

import (
"context"
"crypto/ecdsa"
"fmt"

encryption "github.com/ethersphere/bee/v2/pkg/encryption"
"github.com/ethersphere/bee/v2/pkg/kvs"
"github.com/ethersphere/bee/v2/pkg/swarm"
"golang.org/x/crypto/sha3"
)

var hashFunc = sha3.NewLegacyKeccak256
var oneByteArray = []byte{1}
var zeroByteArray = []byte{0}
//nolint:gochecknoglobals
var (
hashFunc = sha3.NewLegacyKeccak256
oneByteArray = []byte{1}
zeroByteArray = []byte{0}
)

// Read-only interface for the ACT
// Decryptor is a read-only interface for the ACT.
type Decryptor interface {
// DecryptRef will return a decrypted reference, for given encrypted reference and grantee
DecryptRef(ctx context.Context, storage kvs.KeyValueStore, encryptedRef swarm.Address, publisher *ecdsa.PublicKey) (swarm.Address, error)
// Embedding the Session interface
Session
}

// Control interface for the ACT (does write operations)
// Control interface for the ACT (does write operations).
type Control interface {
// Embedding the Decryptor interface
Decryptor
// Adds a new grantee to the ACT
// AddGrantee adds a new grantee to the ACT
AddGrantee(ctx context.Context, storage kvs.KeyValueStore, publisherPubKey, granteePubKey *ecdsa.PublicKey, accessKey *encryption.Key) error
// Encrypts a Swarm reference for a given grantee
// EncryptRef encrypts a Swarm reference for a given grantee
EncryptRef(ctx context.Context, storage kvs.KeyValueStore, grantee *ecdsa.PublicKey, ref swarm.Address) (swarm.Address, error)
}

Expand All @@ -38,14 +46,14 @@ type ActLogic struct {

var _ Control = (*ActLogic)(nil)

// Adds a new publisher to an empty act
// AddPublisher adds a new publisher to an empty act.
func (al ActLogic) AddPublisher(ctx context.Context, storage kvs.KeyValueStore, publisher *ecdsa.PublicKey) error {
accessKey := encryption.GenerateRandomKey(encryption.KeyLength)

return al.AddGrantee(ctx, storage, publisher, publisher, &accessKey)
}

// Encrypts a SWARM reference for a publisher
// EncryptRef encrypts a SWARM reference for a publisher.
func (al ActLogic) EncryptRef(ctx context.Context, storage kvs.KeyValueStore, publisherPubKey *ecdsa.PublicKey, ref swarm.Address) (swarm.Address, error) {
accessKey, err := al.getAccessKey(ctx, storage, publisherPubKey)
if err != nil {
Expand All @@ -54,13 +62,13 @@ func (al ActLogic) EncryptRef(ctx context.Context, storage kvs.KeyValueStore, pu
refCipher := encryption.New(accessKey, 0, uint32(0), hashFunc)
encryptedRef, err := refCipher.Encrypt(ref.Bytes())
if err != nil {
return swarm.ZeroAddress, err
return swarm.ZeroAddress, fmt.Errorf("failed to encrypt reference: %w", err)
}

return swarm.NewAddress(encryptedRef), nil
}

// Adds a new grantee to the ACT
// AddGrantee adds a new grantee to the ACT.
func (al ActLogic) AddGrantee(ctx context.Context, storage kvs.KeyValueStore, publisherPubKey, granteePubKey *ecdsa.PublicKey, accessKeyPointer *encryption.Key) error {
var (
accessKey encryption.Key
Expand All @@ -79,61 +87,60 @@ func (al ActLogic) AddGrantee(ctx context.Context, storage kvs.KeyValueStore, pu
}

// Encrypt the access key for the new Grantee
keys, err := al.getKeys(granteePubKey)
lookupKey, accessKeyDecryptionKey, err := al.getKeys(granteePubKey)
if err != nil {
return err
}
lookupKey := keys[0]
// accessKeyDecryptionKey is used for encryption of the access key
accessKeyDecryptionKey := keys[1]

// Encrypt the access key for the new Grantee
cipher := encryption.New(encryption.Key(accessKeyDecryptionKey), 0, uint32(0), hashFunc)
granteeEncryptedAccessKey, err := cipher.Encrypt(accessKey)
if err != nil {
return err
return fmt.Errorf("failed to encrypt access key: %w", err)
}

// Add the new encrypted access key for the Act
// Add the new encrypted access key to the Act
return storage.Put(ctx, lookupKey, granteeEncryptedAccessKey)
}

// Will return the access key for a publisher (public key)
// Will return the access key for a publisher (public key).
func (al *ActLogic) getAccessKey(ctx context.Context, storage kvs.KeyValueStore, publisherPubKey *ecdsa.PublicKey) ([]byte, error) {
keys, err := al.getKeys(publisherPubKey)
publisherLookupKey, publisherAKDecryptionKey, err := al.getKeys(publisherPubKey)
if err != nil {
return nil, err
}
publisherLookupKey := keys[0]
publisherAKDecryptionKey := keys[1]
// no need to constructor call if value not found in act
// no need for constructor call if value not found in act
accessKeyDecryptionCipher := encryption.New(encryption.Key(publisherAKDecryptionKey), 0, uint32(0), hashFunc)
encryptedAK, err := storage.Get(ctx, publisherLookupKey)
if err != nil {
return nil, err
return nil, fmt.Errorf("failed go get value from KVS: %w", err)
}

return accessKeyDecryptionCipher.Decrypt(encryptedAK)
}

// Generate lookup key and access key decryption key for a given public key
func (al *ActLogic) getKeys(publicKey *ecdsa.PublicKey) ([][]byte, error) {
return al.Session.Key(publicKey, [][]byte{zeroByteArray, oneByteArray})
func (al *ActLogic) getKeys(publicKey *ecdsa.PublicKey) ([]byte, []byte, error) {
nonces := [][]byte{zeroByteArray, oneByteArray}
// keys := make([][]byte, 0, len(nonces))
keys, err := al.Session.Key(publicKey, nonces)
if keys == nil {
return nil, nil, err
}
return keys[0], keys[1], err
}

// DecryptRef will return a decrypted reference, for given encrypted reference and publisher
func (al ActLogic) DecryptRef(ctx context.Context, storage kvs.KeyValueStore, encryptedRef swarm.Address, publisher *ecdsa.PublicKey) (swarm.Address, error) {
keys, err := al.getKeys(publisher)
lookupKey, accessKeyDecryptionKey, err := al.getKeys(publisher)
if err != nil {
return swarm.ZeroAddress, err
}
lookupKey := keys[0]
accessKeyDecryptionKey := keys[1]

// Lookup encrypted access key from the ACT manifest
encryptedAccessKey, err := storage.Get(ctx, lookupKey)
if err != nil {
return swarm.ZeroAddress, err
return swarm.ZeroAddress, fmt.Errorf("failed to get access key from KVS: %w", err)
}

// Decrypt access key
Expand Down
Loading