diff --git a/.github/dependabot.yml b/.github/dependabot.yml index a7442b6bd044..a9c10069a4f0 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -1,5 +1,6 @@ version: 2 updates: + # Check for updates on our docker image and for celestia-node - package-ecosystem: docker directory: "/ops-bedrock" schedule: diff --git a/.github/workflows/fork-sync.yml b/.github/workflows/fork-sync.yml new file mode 100644 index 000000000000..9d53903e58bf --- /dev/null +++ b/.github/workflows/fork-sync.yml @@ -0,0 +1,20 @@ +name: Sync Fork + +on: + schedule: + - cron: "0 * * * 0" # run once a day + workflow_dispatch: # run manually + +jobs: + sync: + + runs-on: ubuntu-latest + + steps: + - uses: actions/checkout@master + with: + fetch-depth: 500 # greater than the number of commits you made + - uses: tuxcanfly/rebase-upstream-action@master + with: # all args are optional + upstream: ethereum-optimism/optimism + branch: develop diff --git a/README.md b/README.md index bd21be79e6df..d9ab68221bb3 100644 --- a/README.md +++ b/README.md @@ -2,47 +2,100 @@

Optimism +

Optimism is a low-cost and lightning-fast Ethereum L2 blockchain, built with the OP Stack.


-

Optimism is Ethereum, scaled.

+

+

+ Celestia +

Celestia is a modular data availability network that securely scales with the number of users, making it easy for anyone to launch their own blockchain.


-## What is Optimism? +## Celestia + OP Stack tutorial + +If you're looking to run the OP Stack + Celestia setup for this repository, please visit the [Optimism & Celestia guides and tutorials](https://docs.celestia.org/developers/intro-to-op-stack/) to get started. + +## What are Optimism and the OP Stack? [Optimism](https://www.optimism.io/) is a project dedicated to scaling Ethereum's technology and expanding its ability to coordinate people from across the world to build effective decentralized economies and governance systems. The [Optimism Collective](https://app.optimism.io/announcement) builds open-source software for running L2 blockchains and aims to address key governance and economic challenges in the wider cryptocurrency ecosystem. Optimism operates on the principle of **impact=profit**, the idea that individuals who positively impact the Collective should be proportionally rewarded with profit. **Change the incentives and you change the world.** -In this repository, you'll find numerous core components of the OP Stack, the decentralized software stack maintained by the Optimism Collective that powers Optimism and forms the backbone of blockchains like [OP Mainnet](https://explorer.optimism.io/) and [Base](https://base.org). Designed to be "aggressively open source," the OP Stack encourages you to explore, modify, extend, and test the code as needed. Although not all elements of the OP Stack are contained here, many of its essential components can be found within this repository. By collaborating on free, open software and shared standards, the Optimism Collective aims to prevent siloed software development and rapidly accelerate the development of the Ethereum ecosystem. Come contribute, build the future, and redefine power, together. +The OP Stack powers Optimism, an Ethereum L2 blockchain, and forms the technical foundation for the [the Optimism Collective](https://app.optimism.io/announcement)—a group committed to the **impact=profit** principle. This principle rewards individuals for their positive contributions to the collective. + +Optimism addresses critical coordination failures in the crypto ecosystem, such as funding public goods and infrastructure. The OP Stack focuses on creating a shared, open-source system for developing new L2 blockchains within the proposed Superchain ecosystem, promoting collaboration and preventing redundant efforts. + +As Optimism evolves, the OP Stack will adapt, encompassing components ranging from blockchain infrastructure to governance systems. This software suite aims to simplify L2 blockchain creation while supporting the growth and development of the Optimism ecosystem. + +## What is Celestia? + +Celestia is a modular consensus and data network, built to enable anyone to easily deploy their own blockchain with minimal overhead. + +Celestia is a minimal blockchain that only orders and publishes transactions and does not execute them. By decoupling the consensus and application execution layers, Celestia modularizes the blockchain technology stack and unlocks new possibilities for decentralized application builders. Lean more at [Celestia.org](https://celestia.org). + +## Maintenance + +The maintenance guide for this repository can be found in the Wiki tab of the repository or [here](https://github.com/celestiaorg/optimism/wiki). ## Documentation -- If you want to build on top of OP Mainnet, refer to the [Optimism Community Hub](https://community.optimism.io) -- If you want to build your own OP Stack based blockchain, refer to the [OP Stack docs](https://stack.optimism.io) -- If you want to contribute to the OP Stack, check out the [Protocol Specs](./specs) +If you want to build on top of Celestia, take a look at the documentation at [docs.celestia.org](https://docs.celestia.org). + +If you want to learn more about the OP Stack, check out the documentation at [stack.optimism.io](https://stack.optimism.io/). ## Community +### Optimism + General discussion happens most frequently on the [Optimism discord](https://discord.gg/optimism). Governance discussion can also be found on the [Optimism Governance Forum](https://gov.optimism.io/). -## Contributing +### Celestia -Read through [CONTRIBUTING.md](./CONTRIBUTING.md) for a general overview of the contributing process for this repository. +General discussion happens most frequently on the [Celestia discord](https://discord.com/invite/YsnTPcSfWQ). +Other discussions can be found on the [Celestia forum](https://forum.celestia.org). + + + +## e2e testing + +This repository has updated end-to-end tests in the `op-e2e` package to work with +Celestia as the data availability (DA) layer. + +Currently, the tests assume a working [Celestia devnet](https://github.com/rollkit/local-celestia-devnet) running locally: -## Security Policy and Vulnerability Reporting +```bash +docker run --platform linux/amd64 -p 26658:26658 ghcr.io/rollkit/local-celestia-devnet:v0.12.1 +``` -Please refer to the canonical [Security Policy](https://github.com/ethereum-optimism/.github/blob/master/SECURITY.md) document for detailed information about how to report vulnerabilities in this codebase. -Bounty hunters are encouraged to check out [the Optimism Immunefi bug bounty program](https://immunefi.com/bounty/optimism/). -The Optimism Immunefi program offers up to $2,000,042 for in-scope critical vulnerabilities. +The e2e tests can be triggered with: -## The Bedrock Upgrade +```bash +cd $HOME/optimism +cd op-e2e +make test +``` -OP Mainnet is currently preparing for [its next major upgrade, Bedrock](https://dev.optimism.io/introducing-optimism-bedrock/). -You can find detailed specifications for the Bedrock upgrade within the [specs folder](./specs) in this repository. +## Bridging -Please note that a significant number of packages and folders within this repository are part of the Bedrock upgrade and are NOT currently running in production. -Refer to the Directory Structure section below to understand which packages are currently running in production and which are intended for use as part of the Bedrock upgrade. +If you have the OP Stack + Celestia setup running, you can test out bridging from the L1 +to the L2. + +To do this, first navigate to the `packages/contracts-bedrock` directory and create a +`.env` file with the following contents: + +```bash +L1_PROVIDER_URL=http://localhost:8545 +L2_PROVIDER_URL=http://localhost:9545 +PRIVATE_KEY=bf7604d9d3a1c7748642b1b7b05c2bd219c9faa91458b370f85e5a40f3b03af7 +``` + +Then, run the following from the same directory: + +```bash +npx hardhat deposit --network devnetL1 --l1-provider-url http://localhost:8545 --l2-provider-url http://localhost:9545 --amount-eth --to
+``` ## Directory Structure @@ -73,7 +126,7 @@ Refer to the Directory Structure section below to understand which packages are ├── proxyd: Configurable RPC request router and proxy └── specs: Specs of the rollup starting at the Bedrock upgrade - + | <-- commitment --> | +// +// ---------------------------------------- +func (f *FrameRef) MarshalBinary() ([]byte, error) { + ref := make([]byte, 8+len(f.TxCommitment)) + + binary.LittleEndian.PutUint64(ref, f.BlockHeight) + copy(ref[8:], f.TxCommitment) + + return ref, nil +} + +// UnmarshalBinary decodes the binary to FrameRef +// serialization format: height + commitment +// +// ---------------------------------------- +// +// | 8 byte uint64 | 32 byte commitment | +// +// ---------------------------------------- +// +// | <-- height --> | <-- commitment --> | +// +// ---------------------------------------- +func (f *FrameRef) UnmarshalBinary(ref []byte) error { + if len(ref) <= 8 { + return ErrInvalidSize + } + f.BlockHeight = binary.LittleEndian.Uint64(ref[:8]) + f.TxCommitment = ref[8:] + return nil +} diff --git a/op-celestia/celestia/celestia_test.go b/op-celestia/celestia/celestia_test.go new file mode 100644 index 000000000000..955ed446a1aa --- /dev/null +++ b/op-celestia/celestia/celestia_test.go @@ -0,0 +1,51 @@ +package celestia + +import ( + "encoding/hex" + "testing" + + "github.com/stretchr/testify/require" +) + +func TestEncodeDecodeFrameRef(t *testing.T) { + tests := []struct { + name string + frameRefHex string + frameRef FrameRef + isValid bool + err error + }{ + { + "valid frame reference", + "d20400000000000068656c6c6f20776f726c64", // 1234 + "hello world" + FrameRef{BlockHeight: 1234, TxCommitment: []byte{0x68, 0x65, 0x6c, 0x6c, 0x6f, 0x20, 0x77, 0x6f, 0x72, 0x6c, 0x64}}, + true, + nil, + }, + { + "invalid frame reference", + "4269", + FrameRef{}, + false, + ErrInvalidSize, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + frameRef, err := hex.DecodeString(tt.frameRefHex) + require.NoError(t, err) + gotFrameRef := FrameRef{} + err = gotFrameRef.UnmarshalBinary(frameRef) + if !tt.isValid { + require.ErrorIs(t, err, ErrInvalidSize) + return + } + require.NoError(t, err) + require.Equal(t, tt.frameRef, gotFrameRef) + frameRefHex, err := tt.frameRef.MarshalBinary() + require.NoError(t, err) + gotFrameRefHex := hex.EncodeToString(frameRefHex) + require.Equal(t, tt.frameRefHex, gotFrameRefHex) + }) + } +} diff --git a/op-celestia/main.go b/op-celestia/main.go new file mode 100644 index 000000000000..3d23c34066c5 --- /dev/null +++ b/op-celestia/main.go @@ -0,0 +1,44 @@ +package main + +import ( + "context" + "encoding/hex" + "fmt" + "os" + + "github.com/ethereum-optimism/optimism/op-celestia/celestia" + openrpc "github.com/rollkit/celestia-openrpc" + "github.com/rollkit/celestia-openrpc/types/share" +) + +func main() { + if len(os.Args) < 4 { + panic("usage: op-celestia ") + } + + data, _ := hex.DecodeString(os.Args[2]) + + frameRef := celestia.FrameRef{} + frameRef.UnmarshalBinary(data) + + fmt.Printf("celestia block height: %v; tx index: %v\n", frameRef.BlockHeight, frameRef.TxCommitment) + fmt.Println("-----------------------------------------") + client, err := openrpc.NewClient(context.Background(), "http://localhost:26658", os.Args[3]) + if err != nil { + panic(err) + } + nsBytes, err := hex.DecodeString(os.Args[1]) + if err != nil { + panic(err) + } + namespace, err := share.NewBlobNamespaceV0(nsBytes) + if err != nil { + panic(err) + } + + namespacedData, err := client.Blob.GetAll(context.Background(), uint64(frameRef.BlockHeight), []share.Namespace{namespace}) + if err != nil { + panic(err) + } + fmt.Printf("optimism block data on celestia: %x\n", namespacedData[0].Data) +} diff --git a/op-e2e/actions/l2_verifier.go b/op-e2e/actions/l2_verifier.go index 6419a68ce669..ae5635d002ce 100644 --- a/op-e2e/actions/l2_verifier.go +++ b/op-e2e/actions/l2_verifier.go @@ -59,7 +59,9 @@ type L2API interface { func NewL2Verifier(t Testing, log log.Logger, l1 derive.L1Fetcher, eng L2API, cfg *rollup.Config, syncCfg *sync.Config) *L2Verifier { metrics := &testutils.TestDerivationMetrics{} - pipeline := derive.NewDerivationPipeline(log, cfg, l1, eng, metrics, syncCfg) + daCfg, err := rollup.NewDAConfig("http://localhost:26658", "0000e8e5f679bf7116cb", "") + require.NoError(t, err) + pipeline := derive.NewDerivationPipeline(log, cfg, daCfg, l1, eng, metrics, syncCfg) pipeline.Reset() rollupNode := &L2Verifier{ diff --git a/op-e2e/setup.go b/op-e2e/setup.go index 0acd3c67b77f..fc28da8d0e11 100644 --- a/op-e2e/setup.go +++ b/op-e2e/setup.go @@ -77,6 +77,7 @@ func newTxMgrConfig(l1Addr string, privKey *ecdsa.PrivateKey) txmgr.CLIConfig { ReceiptQueryInterval: 50 * time.Millisecond, NetworkTimeout: 2 * time.Second, TxNotInMempoolTimeout: 2 * time.Minute, + NamespaceId: "000008e5f679bf7116cb", } } @@ -617,7 +618,15 @@ func (cfg SystemConfig) Start(t *testing.T, _opts ...SystemConfigOption) (*Syste l.Warn("closed op-node!") }() } - node, err := rollupNode.New(context.Background(), &c, l, snapLog, "", metrics.NewMetrics("")) + + daCfg, err := rollup.NewDAConfig("http://127.0.0.1:26658", "0000e8e5f679bf7116cb", "") + if err != nil { + return nil, err + } + + c.DAConfig = *daCfg + + node, err := rollupNode.New(context.Background(), &c, cfg.Loggers[name], snapLog, "", metrics.NewMetrics("")) if err != nil { didErrAfterStart = true return nil, err diff --git a/op-node/flags/flags.go b/op-node/flags/flags.go index f5aa45d2d06d..874765044084 100644 --- a/op-node/flags/flags.go +++ b/op-node/flags/flags.go @@ -39,6 +39,24 @@ var ( Usage: "Rollup chain parameters", EnvVars: prefixEnvVars("ROLLUP_CONFIG"), } + DaRPC = &cli.StringFlag{ + Name: "da-rpc", + Usage: "Data Availability RPC", + Value: "http://da:26658", + EnvVars: prefixEnvVars("DA_RPC"), + } + NamespaceId = &cli.StringFlag{ + Name: "namespace-id", + Usage: "Namespace ID for DA node", + Value: "000008e5f679bf7116cb", + EnvVars: prefixEnvVars("NAMESPACE_ID"), + } + AuthToken = &cli.StringFlag{ + Name: "auth-token", + Usage: "Authentication Token for DA node", + Value: "", + EnvVars: prefixEnvVars("AUTH_TOKEN"), + } Network = &cli.StringFlag{ Name: "network", Usage: fmt.Sprintf("Predefined network selection. Available networks: %s", strings.Join(chaincfg.AvailableNetworks(), ", ")), @@ -270,6 +288,9 @@ var ( var requiredFlags = []cli.Flag{ L1NodeAddr, L2EngineAddr, + DaRPC, + NamespaceId, + AuthToken, } var optionalFlags = []cli.Flag{ diff --git a/op-node/node/config.go b/op-node/node/config.go index cb970d5ebfd3..d172187dbdc9 100644 --- a/op-node/node/config.go +++ b/op-node/node/config.go @@ -25,6 +25,8 @@ type Config struct { Rollup rollup.Config + DAConfig rollup.DAConfig + // P2PSigner will be used for signing off on published content // if the node is sequencing and if the p2p stack is enabled P2PSigner p2p.SignerSetup diff --git a/op-node/node/node.go b/op-node/node/node.go index c2bcedb0b893..c9cac82b9b76 100644 --- a/op-node/node/node.go +++ b/op-node/node/node.go @@ -21,6 +21,7 @@ import ( "github.com/ethereum-optimism/optimism/op-node/heartbeat" "github.com/ethereum-optimism/optimism/op-node/metrics" "github.com/ethereum-optimism/optimism/op-node/p2p" + "github.com/ethereum-optimism/optimism/op-node/rollup" "github.com/ethereum-optimism/optimism/op-node/rollup/driver" "github.com/ethereum-optimism/optimism/op-node/version" "github.com/ethereum-optimism/optimism/op-service/client" @@ -48,6 +49,7 @@ type OpNode struct { p2pSigner p2p.Signer // p2p gogssip application messages will be signed with this signer tracer Tracer // tracer to get events for testing/debugging runCfg *RuntimeConfig // runtime configurables + daCfg *rollup.DAConfig rollupHalt string // when to halt the rollup, disabled if empty @@ -102,8 +104,16 @@ func New(ctx context.Context, cfg *Config, log log.Logger, snapshotLog log.Logge return n, nil } +func (n *OpNode) initDA(ctx context.Context, cfg *Config) error { + n.daCfg = &cfg.DAConfig + return nil +} + func (n *OpNode) init(ctx context.Context, cfg *Config, snapshotLog log.Logger) error { n.log.Info("Initializing rollup node", "version", n.appVersion) + if err := n.initDA(ctx, cfg); err != nil { + return err + } if err := n.initTracer(ctx, cfg); err != nil { return fmt.Errorf("failed to init the trace: %w", err) } @@ -304,7 +314,7 @@ func (n *OpNode) initL2(ctx context.Context, cfg *Config, snapshotLog log.Logger return err } - n.l2Driver = driver.NewDriver(&cfg.Driver, &cfg.Rollup, n.l2Source, n.l1Source, n, n, n.log, snapshotLog, n.metrics, cfg.ConfigPersistence, &cfg.Sync) + n.l2Driver = driver.NewDriver(&cfg.Driver, &cfg.Rollup, n.daCfg, n.l2Source, n.l1Source, n, n, n.log, snapshotLog, n.metrics, cfg.ConfigPersistence, &cfg.Sync) return nil } diff --git a/op-node/rollup/da_config.go b/op-node/rollup/da_config.go new file mode 100644 index 000000000000..bc1537b8ee84 --- /dev/null +++ b/op-node/rollup/da_config.go @@ -0,0 +1,39 @@ +package rollup + +import ( + "context" + "encoding/hex" + + openrpc "github.com/rollkit/celestia-openrpc" + "github.com/rollkit/celestia-openrpc/types/share" +) + +type DAConfig struct { + Rpc string + Namespace share.Namespace + Client *openrpc.Client + AuthToken string +} + +func NewDAConfig(rpc, token, ns string) (*DAConfig, error) { + nsBytes, err := hex.DecodeString(ns) + if err != nil { + return &DAConfig{}, err + } + + namespace, err := share.NewBlobNamespaceV0(nsBytes) + if err != nil { + return nil, err + } + + client, err := openrpc.NewClient(context.Background(), rpc, token) + if err != nil { + return &DAConfig{}, err + } + + return &DAConfig{ + Namespace: namespace, + Rpc: rpc, + Client: client, + }, nil +} diff --git a/op-node/rollup/derive/calldata_source.go b/op-node/rollup/derive/calldata_source.go index ce914352be75..85f9c2a2fbbf 100644 --- a/op-node/rollup/derive/calldata_source.go +++ b/op-node/rollup/derive/calldata_source.go @@ -2,6 +2,7 @@ package derive import ( "context" + "encoding/hex" "errors" "fmt" "io" @@ -11,6 +12,7 @@ import ( "github.com/ethereum/go-ethereum/core/types" "github.com/ethereum/go-ethereum/log" + "github.com/ethereum-optimism/optimism/op-celestia/celestia" "github.com/ethereum-optimism/optimism/op-node/rollup" "github.com/ethereum-optimism/optimism/op-service/eth" ) @@ -29,16 +31,17 @@ type L1TransactionFetcher interface { type DataSourceFactory struct { log log.Logger cfg *rollup.Config + daCfg *rollup.DAConfig fetcher L1TransactionFetcher } -func NewDataSourceFactory(log log.Logger, cfg *rollup.Config, fetcher L1TransactionFetcher) *DataSourceFactory { - return &DataSourceFactory{log: log, cfg: cfg, fetcher: fetcher} +func NewDataSourceFactory(log log.Logger, cfg *rollup.Config, daCfg *rollup.DAConfig, fetcher L1TransactionFetcher) *DataSourceFactory { + return &DataSourceFactory{log: log, cfg: cfg, daCfg: daCfg, fetcher: fetcher} } // OpenData returns a DataIter. This struct implements the `Next` function. -func (ds *DataSourceFactory) OpenData(ctx context.Context, id eth.BlockID, batcherAddr common.Address) DataIter { - return NewDataSource(ctx, ds.log, ds.cfg, ds.fetcher, id, batcherAddr) +func (ds *DataSourceFactory) OpenData(ctx context.Context, id eth.BlockID, batcherAddr common.Address) (DataIter, error) { + return NewDataSource(ctx, ds.log, ds.cfg, ds.daCfg, ds.fetcher, id, batcherAddr) } // DataSource is a fault tolerant approach to fetching data. @@ -51,6 +54,7 @@ type DataSource struct { // Required to re-attempt fetching id eth.BlockID cfg *rollup.Config // TODO: `DataFromEVMTransactions` should probably not take the full config + daCfg *rollup.DAConfig fetcher L1TransactionFetcher log log.Logger @@ -59,7 +63,7 @@ type DataSource struct { // NewDataSource creates a new calldata source. It suppresses errors in fetching the L1 block if they occur. // If there is an error, it will attempt to fetch the result on the next call to `Next`. -func NewDataSource(ctx context.Context, log log.Logger, cfg *rollup.Config, fetcher L1TransactionFetcher, block eth.BlockID, batcherAddr common.Address) DataIter { +func NewDataSource(ctx context.Context, log log.Logger, cfg *rollup.Config, daCfg *rollup.DAConfig, fetcher L1TransactionFetcher, block eth.BlockID, batcherAddr common.Address) (DataIter, error) { _, txs, err := fetcher.InfoAndTxsByHash(ctx, block.Hash) if err != nil { return &DataSource{ @@ -69,12 +73,23 @@ func NewDataSource(ctx context.Context, log log.Logger, cfg *rollup.Config, fetc fetcher: fetcher, log: log, batcherAddr: batcherAddr, - } + }, nil } else { + data, err := DataFromEVMTransactions(cfg, daCfg, batcherAddr, txs, log.New("origin", block)) + if err != nil { + return &DataSource{ + open: false, + id: block, + cfg: cfg, + fetcher: fetcher, + log: log, + batcherAddr: batcherAddr, + }, err + } return &DataSource{ open: true, - data: DataFromEVMTransactions(cfg, batcherAddr, txs, log.New("origin", block)), - } + data: data, + }, nil } } @@ -85,7 +100,11 @@ func (ds *DataSource) Next(ctx context.Context) (eth.Data, error) { if !ds.open { if _, txs, err := ds.fetcher.InfoAndTxsByHash(ctx, ds.id.Hash); err == nil { ds.open = true - ds.data = DataFromEVMTransactions(ds.cfg, ds.batcherAddr, txs, log.New("origin", ds.id)) + ds.data, err = DataFromEVMTransactions(ds.cfg, ds.daCfg, ds.batcherAddr, txs, log.New("origin", ds.id)) + if err != nil { + // already wrapped + return nil, err + } } else if errors.Is(err, ethereum.NotFound) { return nil, NewResetError(fmt.Errorf("failed to open calldata source: %w", err)) } else { @@ -104,7 +123,7 @@ func (ds *DataSource) Next(ctx context.Context) (eth.Data, error) { // DataFromEVMTransactions filters all of the transactions and returns the calldata from transactions // that are sent to the batch inbox address from the batch sender address. // This will return an empty array if no valid transactions are found. -func DataFromEVMTransactions(config *rollup.Config, batcherAddr common.Address, txs types.Transactions, log log.Logger) []eth.Data { +func DataFromEVMTransactions(config *rollup.Config, daCfg *rollup.DAConfig, batcherAddr common.Address, txs types.Transactions, log log.Logger) ([]eth.Data, error) { var out []eth.Data l1Signer := config.L1Signer() for j, tx := range txs { @@ -119,8 +138,24 @@ func DataFromEVMTransactions(config *rollup.Config, batcherAddr common.Address, log.Warn("tx in inbox with unauthorized submitter", "index", j, "err", err) continue // not an authorized batch submitter, ignore } - out = append(out, tx.Data()) + + if daCfg != nil { + frameRef := celestia.FrameRef{} + frameRef.UnmarshalBinary(tx.Data()) + if err != nil { + log.Warn("unable to decode frame reference", "index", j, "err", err) + return nil, err + } + log.Info("requesting data from celestia", "namespace", hex.EncodeToString(daCfg.Namespace), "height", frameRef.BlockHeight) + blob, err := daCfg.Client.Blob.Get(context.Background(), frameRef.BlockHeight, daCfg.Namespace, frameRef.TxCommitment) + if err != nil { + return nil, NewResetError(fmt.Errorf("failed to resolve frame from celestia: %w", err)) + } + out = append(out, blob.Data) + } else { + out = append(out, tx.Data()) + } } } - return out + return out, nil } diff --git a/op-node/rollup/derive/calldata_source_test.go b/op-node/rollup/derive/calldata_source_test.go index 9ec43aafef00..4d1a5f4f448f 100644 --- a/op-node/rollup/derive/calldata_source_test.go +++ b/op-node/rollup/derive/calldata_source_test.go @@ -121,8 +121,9 @@ func TestDataFromEVMTransactions(t *testing.T) { } } - out := DataFromEVMTransactions(cfg, batcherAddr, txs, testlog.Logger(t, log.LvlCrit)) + out, err := DataFromEVMTransactions(cfg, nil, batcherAddr, txs, testlog.Logger(t, log.LvlCrit)) require.ElementsMatch(t, expectedData, out) + require.NoError(t, err) } } diff --git a/op-node/rollup/derive/l1_retrieval.go b/op-node/rollup/derive/l1_retrieval.go index 586e125acc32..db53ab2fff23 100644 --- a/op-node/rollup/derive/l1_retrieval.go +++ b/op-node/rollup/derive/l1_retrieval.go @@ -11,7 +11,7 @@ import ( ) type DataAvailabilitySource interface { - OpenData(ctx context.Context, id eth.BlockID, batcherAddr common.Address) DataIter + OpenData(ctx context.Context, id eth.BlockID, batcherAddr common.Address) (DataIter, error) } type NextBlockProvider interface { @@ -53,7 +53,10 @@ func (l1r *L1Retrieval) NextData(ctx context.Context) ([]byte, error) { } else if err != nil { return nil, err } - l1r.datas = l1r.dataSrc.OpenData(ctx, next.ID(), l1r.prev.SystemConfig().BatcherAddr) + l1r.datas, err = l1r.dataSrc.OpenData(ctx, next.ID(), l1r.prev.SystemConfig().BatcherAddr) + if err != nil { + return nil, err + } } l1r.log.Debug("fetching next piece of data") @@ -73,7 +76,7 @@ func (l1r *L1Retrieval) NextData(ctx context.Context) ([]byte, error) { // Note that we open up the `l1r.datas` here because it is requires to maintain the // internal invariants that later propagate up the derivation pipeline. func (l1r *L1Retrieval) Reset(ctx context.Context, base eth.L1BlockRef, sysCfg eth.SystemConfig) error { - l1r.datas = l1r.dataSrc.OpenData(ctx, base.ID(), sysCfg.BatcherAddr) + l1r.datas, _ = l1r.dataSrc.OpenData(ctx, base.ID(), sysCfg.BatcherAddr) l1r.log.Info("Reset of L1Retrieval done", "origin", base) return io.EOF } diff --git a/op-node/rollup/derive/pipeline.go b/op-node/rollup/derive/pipeline.go index 643dc312cce9..e37d1f07459f 100644 --- a/op-node/rollup/derive/pipeline.go +++ b/op-node/rollup/derive/pipeline.go @@ -82,11 +82,11 @@ type DerivationPipeline struct { } // NewDerivationPipeline creates a derivation pipeline, which should be reset before use. -func NewDerivationPipeline(log log.Logger, cfg *rollup.Config, l1Fetcher L1Fetcher, engine Engine, metrics Metrics, syncCfg *sync.Config) *DerivationPipeline { +func NewDerivationPipeline(log log.Logger, cfg *rollup.Config, daCfg *rollup.DAConfig, l1Fetcher L1Fetcher, engine Engine, metrics Metrics, syncCfg *sync.Config) *DerivationPipeline { // Pull stages l1Traversal := NewL1Traversal(log, cfg, l1Fetcher) - dataSrc := NewDataSourceFactory(log, cfg, l1Fetcher) // auxiliary stage for L1Retrieval + dataSrc := NewDataSourceFactory(log, cfg, daCfg, l1Fetcher) // auxiliary stage for L1Retrieval l1Src := NewL1Retrieval(log, dataSrc, l1Traversal) frameQueue := NewFrameQueue(log, l1Src) bank := NewChannelBank(log, cfg, frameQueue, l1Fetcher, metrics) diff --git a/op-node/rollup/driver/driver.go b/op-node/rollup/driver/driver.go index 1db8d5dff87e..6e2fd5403d99 100644 --- a/op-node/rollup/driver/driver.go +++ b/op-node/rollup/driver/driver.go @@ -115,13 +115,13 @@ type SequencerStateListener interface { } // NewDriver composes an events handler that tracks L1 state, triggers L2 derivation, and optionally sequences new L2 blocks. -func NewDriver(driverCfg *Config, cfg *rollup.Config, l2 L2Chain, l1 L1Chain, altSync AltSync, network Network, log log.Logger, snapshotLog log.Logger, metrics Metrics, sequencerStateListener SequencerStateListener, syncCfg *sync.Config) *Driver { +func NewDriver(driverCfg *Config, cfg *rollup.Config, daCfg *rollup.DAConfig, l2 L2Chain, l1 L1Chain, altSync AltSync, network Network, log log.Logger, snapshotLog log.Logger, metrics Metrics, sequencerStateListener SequencerStateListener, syncCfg *sync.Config) *Driver { l1 = NewMeteredL1Fetcher(l1, metrics) l1State := NewL1State(log, metrics) sequencerConfDepth := NewConfDepth(driverCfg.SequencerConfDepth, l1State.L1Head, l1) findL1Origin := NewL1OriginSelector(log, cfg, sequencerConfDepth) verifConfDepth := NewConfDepth(driverCfg.VerifierConfDepth, l1State.L1Head, l1) - derivationPipeline := derive.NewDerivationPipeline(log, cfg, verifConfDepth, l2, metrics, syncCfg) + derivationPipeline := derive.NewDerivationPipeline(log, cfg, daCfg, verifConfDepth, l2, metrics, syncCfg) attrBuilder := derive.NewFetchingAttributesBuilder(cfg, l1, l2) engine := derivationPipeline meteredEngine := NewMeteredEngine(cfg, engine, metrics, log) diff --git a/op-node/service.go b/op-node/service.go index 64c564cc517b..148ba52912df 100644 --- a/op-node/service.go +++ b/op-node/service.go @@ -70,13 +70,18 @@ func NewConfig(ctx *cli.Context, log log.Logger) (*node.Config, error) { if haltOption == "none" { haltOption = "" } + daCfg, err := rollup.NewDAConfig(flags.DaRPC.Value, flags.AuthToken.Value, flags.NamespaceId.Value) + if err != nil { + return nil, fmt.Errorf("failed to load da config: %w", err) + } cfg := &node.Config{ - L1: l1Endpoint, - L2: l2Endpoint, - L2Sync: l2SyncEndpoint, - Rollup: *rollupConfig, - Driver: *driverConfig, + L1: l1Endpoint, + L2: l2Endpoint, + L2Sync: l2SyncEndpoint, + Rollup: *rollupConfig, + DAConfig: *daCfg, + Driver: *driverConfig, RPC: node.RPCConfig{ ListenAddr: ctx.String(flags.RPCListenAddr.Name), ListenPort: ctx.Int(flags.RPCListenPort.Name), diff --git a/op-program/client/boot.go b/op-program/client/boot.go index e0b2de0d71e3..673c59841237 100644 --- a/op-program/client/boot.go +++ b/op-program/client/boot.go @@ -36,6 +36,7 @@ type BootInfo struct { L2ChainConfig *params.ChainConfig RollupConfig *rollup.Config + DAConfig *rollup.DAConfig } type oracleClient interface { diff --git a/op-program/client/driver/driver.go b/op-program/client/driver/driver.go index 5487d8f19fd5..697c37afe8cc 100644 --- a/op-program/client/driver/driver.go +++ b/op-program/client/driver/driver.go @@ -35,8 +35,8 @@ type Driver struct { targetBlockNum uint64 } -func NewDriver(logger log.Logger, cfg *rollup.Config, l1Source derive.L1Fetcher, l2Source L2Source, targetBlockNum uint64) *Driver { - pipeline := derive.NewDerivationPipeline(logger, cfg, l1Source, l2Source, metrics.NoopMetrics, &sync.Config{}) +func NewDriver(logger log.Logger, cfg *rollup.Config, daCfg *rollup.DAConfig, l1Source derive.L1Fetcher, l2Source L2Source, targetBlockNum uint64) *Driver { + pipeline := derive.NewDerivationPipeline(logger, cfg, daCfg, l1Source, l2Source, metrics.NoopMetrics, &sync.Config{}) pipeline.Reset() return &Driver{ logger: logger, diff --git a/op-program/client/program.go b/op-program/client/program.go index b43950bfb871..3216a0b5834f 100644 --- a/op-program/client/program.go +++ b/op-program/client/program.go @@ -51,6 +51,7 @@ func RunProgram(logger log.Logger, preimageOracle io.ReadWriter, preimageHinter return runDerivation( logger, bootInfo.RollupConfig, + bootInfo.DAConfig, bootInfo.L2ChainConfig, bootInfo.L1Head, bootInfo.L2OutputRoot, @@ -62,7 +63,7 @@ func RunProgram(logger log.Logger, preimageOracle io.ReadWriter, preimageHinter } // runDerivation executes the L2 state transition, given a minimal interface to retrieve data. -func runDerivation(logger log.Logger, cfg *rollup.Config, l2Cfg *params.ChainConfig, l1Head common.Hash, l2OutputRoot common.Hash, l2Claim common.Hash, l2ClaimBlockNum uint64, l1Oracle l1.Oracle, l2Oracle l2.Oracle) error { +func runDerivation(logger log.Logger, cfg *rollup.Config, daCfg *rollup.DAConfig, l2Cfg *params.ChainConfig, l1Head common.Hash, l2OutputRoot common.Hash, l2Claim common.Hash, l2ClaimBlockNum uint64, l1Oracle l1.Oracle, l2Oracle l2.Oracle) error { l1Source := l1.NewOracleL1Client(logger, l1Oracle, l1Head) engineBackend, err := l2.NewOracleBackedL2Chain(logger, l2Oracle, l2Cfg, l2OutputRoot) if err != nil { @@ -71,7 +72,7 @@ func runDerivation(logger log.Logger, cfg *rollup.Config, l2Cfg *params.ChainCon l2Source := l2.NewOracleEngine(cfg, logger, engineBackend) logger.Info("Starting derivation") - d := cldr.NewDriver(logger, cfg, l1Source, l2Source, l2ClaimBlockNum) + d := cldr.NewDriver(logger, cfg, daCfg, l1Source, l2Source, l2ClaimBlockNum) for { if err = d.Step(context.Background()); errors.Is(err, io.EOF) { break diff --git a/op-service/txmgr/cli.go b/op-service/txmgr/cli.go index 30f4225ecc45..26a2c82d3ef9 100644 --- a/op-service/txmgr/cli.go +++ b/op-service/txmgr/cli.go @@ -32,6 +32,9 @@ const ( TxSendTimeoutFlagName = "txmgr.send-timeout" TxNotInMempoolTimeoutFlagName = "txmgr.not-in-mempool-timeout" ReceiptQueryIntervalFlagName = "txmgr.receipt-query-interval" + DaRpcFlagName = "da-rpc" + NamespaceIdFlagName = "namespace-id" + AuthTokenFlagName = "auth-token" ) var ( @@ -155,6 +158,24 @@ func CLIFlagsWithDefaults(envPrefix string, defaults DefaultFlagValues) []cli.Fl Value: defaults.ReceiptQueryInterval, EnvVars: prefixEnvVars("TXMGR_RECEIPT_QUERY_INTERVAL"), }, + &cli.StringFlag{ + Name: DaRpcFlagName, + Usage: "RPC URL of the DA layer", + Value: "http://da:26658", + EnvVars: opservice.PrefixEnvVar(envPrefix, "DA_RPC"), + }, + &cli.StringFlag{ + Name: NamespaceIdFlagName, + Usage: "Namespace ID of the DA layer", + Value: "e8e5f679bf7116cb", + EnvVars: opservice.PrefixEnvVar(envPrefix, "NAMESPACE_ID"), + }, + &cli.StringFlag{ + Name: AuthTokenFlagName, + Usage: "Authentication Token of the DA layer", + Value: "", + EnvVars: opservice.PrefixEnvVar(envPrefix, "AUTH_TOKEN"), + }, }, opsigner.CLIFlags(envPrefix)...) } @@ -174,6 +195,9 @@ type CLIConfig struct { NetworkTimeout time.Duration TxSendTimeout time.Duration TxNotInMempoolTimeout time.Duration + DaRpc string + NamespaceId string + AuthToken string } func NewCLIConfig(l1RPCURL string, defaults DefaultFlagValues) CLIConfig { @@ -216,6 +240,15 @@ func (m CLIConfig) Check() error { if m.SafeAbortNonceTooLowCount == 0 { return errors.New("SafeAbortNonceTooLowCount must not be 0") } + if m.DaRpc == "" { + return errors.New("must provide a DA RPC url") + } + if m.NamespaceId == "" { + return errors.New("must provide a DA namespace ID") + } + if m.AuthToken == "" { + return errors.New("must provide a DA auth token") + } if err := m.SignerCLIConfig.Check(); err != nil { return err } @@ -239,6 +272,9 @@ func ReadCLIConfig(ctx *cli.Context) CLIConfig { NetworkTimeout: ctx.Duration(NetworkTimeoutFlagName), TxSendTimeout: ctx.Duration(TxSendTimeoutFlagName), TxNotInMempoolTimeout: ctx.Duration(TxNotInMempoolTimeoutFlagName), + DaRpc: ctx.String(DaRpcFlagName), + NamespaceId: ctx.String(NamespaceIdFlagName), + AuthToken: ctx.String(AuthTokenFlagName), } } @@ -285,6 +321,9 @@ func NewConfig(cfg CLIConfig, l log.Logger) (Config, error) { ReceiptQueryInterval: cfg.ReceiptQueryInterval, NumConfirmations: cfg.NumConfirmations, SafeAbortNonceTooLowCount: cfg.SafeAbortNonceTooLowCount, + DaRpc: cfg.DaRpc, + NamespaceId: cfg.NamespaceId, + AuthToken: cfg.AuthToken, Signer: signerFactory(chainID), From: from, }, nil @@ -331,6 +370,15 @@ type Config struct { // confirmation. SafeAbortNonceTooLowCount uint64 + // DaRpc is the HTTP provider URL for the Data Availability node. + DaRpc string + + // NamespaceId is the id of the namespace of the Data Availability node. + NamespaceId string + + // AuthToken is the authentication token for the Data Availability node. + AuthToken string + // Signer is used to sign transactions when the gas price is increased. Signer opcrypto.SignerFn From common.Address diff --git a/op-service/txmgr/txmgr.go b/op-service/txmgr/txmgr.go index 0a2109af8647..71e0622fd22a 100644 --- a/op-service/txmgr/txmgr.go +++ b/op-service/txmgr/txmgr.go @@ -2,6 +2,7 @@ package txmgr import ( "context" + "encoding/hex" "errors" "fmt" "math/big" @@ -19,6 +20,12 @@ import ( "github.com/ethereum-optimism/optimism/op-service/retry" "github.com/ethereum-optimism/optimism/op-service/txmgr/metrics" + + "github.com/ethereum-optimism/optimism/op-celestia/celestia" + openrpc "github.com/rollkit/celestia-openrpc" + "github.com/rollkit/celestia-openrpc/types/blob" + openrpcns "github.com/rollkit/celestia-openrpc/types/namespace" + "github.com/rollkit/celestia-openrpc/types/share" ) const ( @@ -89,6 +96,9 @@ type SimpleTxManager struct { name string chainID *big.Int + daClient *openrpc.Client + namespace openrpcns.Namespace + backend ETHBackend l log.Logger metr metrics.TxMetricer @@ -113,13 +123,34 @@ func NewSimpleTxManagerFromConfig(name string, l log.Logger, m metrics.TxMetrice if err := conf.Check(); err != nil { return nil, fmt.Errorf("invalid config: %w", err) } + + daClient, err := openrpc.NewClient(context.Background(), conf.DaRpc, conf.AuthToken) + if err != nil { + return nil, err + } + + if conf.NamespaceId == "" { + return nil, errors.New("namespace id cannot be blank") + } + nsBytes, err := hex.DecodeString(conf.NamespaceId) + if err != nil { + return nil, err + } + + namespace, err := share.NewBlobNamespaceV0(nsBytes) + if err != nil { + return nil, err + } + return &SimpleTxManager{ - chainID: conf.ChainID, - name: name, - cfg: conf, - backend: conf.Backend, - l: l.New("service", name), - metr: m, + chainID: conf.ChainID, + name: name, + cfg: conf, + daClient: daClient, + namespace: namespace.ToAppNamespace(), + backend: conf.Backend, + l: l.New("service", name), + metr: m, }, nil } @@ -172,6 +203,40 @@ func (m *SimpleTxManager) send(ctx context.Context, candidate TxCandidate) (*typ ctx, cancel = context.WithTimeout(ctx, m.cfg.TxSendTimeout) defer cancel() } + // TODO: this is a hack to route only batcher transactions through celestia + // SimpleTxManager is used by both batcher and proposer but since proposer + // writes to a smart contract, we overwrite _only_ batcher candidate as the + // frame pointer to celestia, while retaining the proposer pathway that + // writes the state commitment data to ethereum. + if candidate.To.Hex() == "0xFf00000000000000000000000000000000000901" { + dataBlob, err := blob.NewBlobV0(m.namespace.Bytes(), candidate.TxData) + com, err := blob.CreateCommitment(dataBlob) + if err != nil { + m.l.Warn("unable to create blob commitment to celestia", "err", err) + return nil, err + } + err = m.daClient.Header.SyncWait(ctx) + if err != nil { + m.l.Warn("unable to wait for celestia header sync", "err", err) + return nil, err + } + height, err := m.daClient.Blob.Submit(ctx, []*blob.Blob{dataBlob}, openrpc.DefaultSubmitOptions()) + if err != nil { + m.l.Warn("unable to publish tx to celestia", "err", err) + return nil, err + } + if height == 0 { + m.l.Warn("unexpected response from celestia got", "height", height) + return nil, errors.New("unexpected response code") + } + frameRef := celestia.FrameRef{ + BlockHeight: height, + TxCommitment: com, + } + frameRefData, _ := frameRef.MarshalBinary() + candidate.TxData = frameRefData + m.l.Info("submitting txdata", "celestia height", height, "txdata", hex.EncodeToString(frameRefData)) + } tx, err := retry.Do(ctx, 30, retry.Fixed(2*time.Second), func() (*types.Transaction, error) { tx, err := m.craftTx(ctx, candidate) if err != nil { @@ -222,6 +287,7 @@ func (m *SimpleTxManager) craftTx(ctx context.Context, candidate TxCandidate) (* Data: rawTx.Data, Value: rawTx.Value, }) + m.l.Warn("estimating gas", "candidate", candidate, "gasFeeCap", gasFeeCap, "gasTipCap", gasTipCap, "err", err) if err != nil { return nil, fmt.Errorf("failed to estimate gas: %w", err) } diff --git a/ops-bedrock/docker-compose-testnet.yml b/ops-bedrock/docker-compose-testnet.yml new file mode 100644 index 000000000000..ad42f62062ff --- /dev/null +++ b/ops-bedrock/docker-compose-testnet.yml @@ -0,0 +1,233 @@ +version: '3.4' + +# This Compose file is expected to be used with the testnet-up.sh script. +# The volumes below mount the configs generated by the script into each +# service. + +volumes: + l1_data: + l2_data: + op_log: + + +services: +version: '3.4' + +# This Compose file is expected to be used with the devnet-up.sh script. +# The volumes below mount the configs generated by the script into each +# service. + +volumes: + l1_data: + l2_data: + op_log: + + +services: + op_stack_go_builder: # Not an actual service, but builds the prerequisite go images + build: + context: ../ + dockerfile: ops/docker/op-stack-go/Dockerfile + args: + GIT_COMMIT: "dev" + GIT_DATE: "0" + image: us-docker.pkg.dev/oplabs-tools-artifacts/images/op-stack-go:devnet + entrypoint: ["echo", "build complete"] + da: + #container_name: celestia-light-node + user: root + platform: "${PLATFORM}" + image: "ghcr.io/celestiaorg/celestia-node:v0.12.0" + command: celestia light start --core.ip rpc-mocha.pops.one --p2p.network mocha --log.level debug --gateway + environment: + - NODE_TYPE=light + - P2P_NETWORK=mocha + volumes: + - $HOME/.celestia-light-mocha-4/:/home/celestia/.celestia-light-mocha-4/ + ports: + - "26657:26657" + - "26658:26658" + - "26659:26659" + + l1: + build: + context: . + dockerfile: Dockerfile.l1 + ports: + - "8545:8545" + - "8546:8546" + - "7060:6060" + volumes: + - "l1_data:/db" + - "${PWD}/../.devnet/genesis-l1.json:/genesis.json" + - "${PWD}/test-jwt-secret.txt:/config/test-jwt-secret.txt" + environment: + GETH_MINER_RECOMMIT: 100ms + + l2: + build: + context: . + dockerfile: Dockerfile.l2 + ports: + - "9545:8545" + - "8060:6060" + volumes: + - "l2_data:/db" + - "${PWD}/../.devnet/genesis-l2.json:/genesis.json" + - "${PWD}/test-jwt-secret.txt:/config/test-jwt-secret.txt" + entrypoint: # pass the L2 specific flags by overriding the entry-point and adding extra arguments + - "/bin/sh" + - "/entrypoint.sh" + - "--authrpc.jwtsecret=/config/test-jwt-secret.txt" + environment: + GETH_MINER_RECOMMIT: 100ms + + op-node: + depends_on: + - op_stack_go_builder + - l1 + - l2 + - da + build: + context: ../ + dockerfile: ./op-node/Dockerfile + args: + OP_STACK_GO_BUILDER: us-docker.pkg.dev/oplabs-tools-artifacts/images/op-stack-go:devnet + image: us-docker.pkg.dev/oplabs-tools-artifacts/images/op-node:devnet + command: > + op-node + --l1=ws://l1:8546 + --l2=http://l2:8551 + --l2.jwt-secret=/config/test-jwt-secret.txt + --sequencer.enabled + --sequencer.l1-confs=0 + --verifier.l1-confs=0 + --p2p.sequencer.key=8b3a350cf5c34c9194ca85829a2df0ec3153be0318b5e2d3348e872092edffba + --rollup.config=/rollup.json + --rpc.addr=0.0.0.0 + --rpc.port=8545 + --p2p.listen.ip=0.0.0.0 + --p2p.listen.tcp=9003 + --p2p.listen.udp=9003 + --p2p.scoring.peers=light + --p2p.ban.peers=true + --snapshotlog.file=/op_log/snapshot.log + --p2p.priv.path=/config/p2p-node-key.txt + --metrics.enabled + --metrics.addr=0.0.0.0 + --metrics.port=7300 + --pprof.enabled + --rpc.enable-admin + --da-rpc=http://da:26658 + --namespace-id=000008e5f679bf7116cb + --auth-token=$CELESTIA_NODE_AUTH_TOKEN + environment: + OP_NODE_AUTH_TOKEN: $CELESTIA_NODE_AUTH_TOKEN + ports: + - "7545:8545" + - "9003:9003" + - "7300:7300" + - "6060:6060" + volumes: + - "${PWD}/p2p-sequencer-key.txt:/config/p2p-sequencer-key.txt" + - "${PWD}/p2p-node-key.txt:/config/p2p-node-key.txt" + - "${PWD}/test-jwt-secret.txt:/config/test-jwt-secret.txt" + - "${PWD}/../.devnet/rollup.json:/rollup.json" + - op_log:/op_log + + op-proposer: + depends_on: + - op_stack_go_builder + - l1 + - l2 + - op-node + build: + context: ../ + dockerfile: ./op-proposer/Dockerfile + args: + OP_STACK_GO_BUILDER: us-docker.pkg.dev/oplabs-tools-artifacts/images/op-stack-go:devnet + image: us-docker.pkg.dev/oplabs-tools-artifacts/images/op-proposer:devnet + ports: + - "6062:6060" + - "7302:7300" + - "6546:8545" + environment: + OP_PROPOSER_L1_ETH_RPC: http://l1:8545 + OP_PROPOSER_ROLLUP_RPC: http://op-node:8545 + OP_PROPOSER_POLL_INTERVAL: 1s + OP_PROPOSER_NUM_CONFIRMATIONS: 1 + OP_PROPOSER_MNEMONIC: test test test test test test test test test test test junk + OP_PROPOSER_L2_OUTPUT_HD_PATH: "m/44'/60'/0'/0/1" + OP_PROPOSER_L2OO_ADDRESS: "${L2OO_ADDRESS}" + OP_PROPOSER_PPROF_ENABLED: "true" + OP_PROPOSER_METRICS_ENABLED: "true" + OP_PROPOSER_ALLOW_NON_FINALIZED: "true" + OP_PROPOSER_RPC_ENABLE_ADMIN: "true" + OP_PROPOSER_DA_RPC: "http://da:26658" + OP_PROPOSER_AUTH_TOKEN: $CELESTIA_NODE_AUTH_TOKEN + + op-batcher: + depends_on: + op_stack_go_builder: + condition: service_started + l1: + condition: service_started + l2: + condition: service_started + op-node: + condition: service_started + da: + condition: service_healthy + build: + context: ../ + dockerfile: ./op-batcher/Dockerfile + args: + OP_STACK_GO_BUILDER: us-docker.pkg.dev/oplabs-tools-artifacts/images/op-stack-go:devnet + image: us-docker.pkg.dev/oplabs-tools-artifacts/images/op-batcher:devnet + ports: + - "6061:6060" + - "7301:7300" + - "6545:8545" + environment: + OP_BATCHER_L1_ETH_RPC: http://l1:8545 + OP_BATCHER_L2_ETH_RPC: http://l2:8545 + OP_BATCHER_ROLLUP_RPC: http://op-node:8545 + OFFLINE_GAS_ESTIMATION: null + OP_BATCHER_MAX_CHANNEL_DURATION: 1 + OP_BATCHER_SUB_SAFETY_MARGIN: 4 # SWS is 15, ChannelTimeout is 40 + OP_BATCHER_POLL_INTERVAL: 1s + OP_BATCHER_NUM_CONFIRMATIONS: 1 + OP_BATCHER_MNEMONIC: test test test test test test test test test test test junk + OP_BATCHER_SEQUENCER_HD_PATH: "m/44'/60'/0'/0/2" + OP_BATCHER_PPROF_ENABLED: "true" + OP_BATCHER_METRICS_ENABLED: "true" + OP_BATCHER_RPC_ENABLE_ADMIN: "true" + OP_BATCHER_BATCH_TYPE: 0 + OP_BATCHER_NAMESPACE_ID: "000008e5f679bf7116cb" + OP_BATCHER_DA_RPC: http://da:26658 + OP_BATCHER_AUTH_TOKEN: $CELESTIA_NODE_AUTH_TOKEN + + artifact-server: + depends_on: + - l1 + image: nginx:1.25-alpine + ports: + - "8080:80" + volumes: + - "${PWD}/../.devnet/:/usr/share/nginx/html/:ro" + security_opt: + - "no-new-privileges:true" + +# stateviz: +# build: +# context: ../ +# dockerfile: ./ops-bedrock/Dockerfile.stateviz +# command: +# - stateviz +# - -addr=0.0.0.0:8080 +# - -snapshot=/op_log/snapshot.log +# - -refresh=10s +# ports: +# - "9090:8080" +# volumes: +# - op_log:/op_log:ro diff --git a/ops-bedrock/docker-compose.yml b/ops-bedrock/docker-compose.yml index 12216ab510ff..2cf26faaf9df 100644 --- a/ops-bedrock/docker-compose.yml +++ b/ops-bedrock/docker-compose.yml @@ -20,6 +20,19 @@ services: GIT_DATE: "0" image: us-docker.pkg.dev/oplabs-tools-artifacts/images/op-stack-go:devnet entrypoint: ["echo", "build complete"] + da: + platform: linux/x86_64 + image: "ghcr.io/rollkit/local-celestia-devnet:v0.12.1" + ports: + - "26657:26657" + - "26658:26658" + - "26659:26659" + healthcheck: + test: ["CMD", "curl", "-f", "http://localhost:26659/header/1"] + interval: 10s + timeout: 5s + retries: 5 + start_period: 30s l1: build: @@ -59,6 +72,7 @@ services: - op_stack_go_builder - l1 - l2 + - da build: context: ../ dockerfile: ./op-node/Dockerfile @@ -89,6 +103,11 @@ services: --metrics.port=7300 --pprof.enabled --rpc.enable-admin + --da-rpc=http://da:26658 + --namespace-id=000008e5f679bf7116cb + --auth-token=$CELESTIA_NODE_AUTH_TOKEN + environment: + OP_NODE_AUTH_TOKEN: $CELESTIA_NODE_AUTH_TOKEN ports: - "7545:8545" - "9003:9003" @@ -129,13 +148,21 @@ services: OP_PROPOSER_METRICS_ENABLED: "true" OP_PROPOSER_ALLOW_NON_FINALIZED: "true" OP_PROPOSER_RPC_ENABLE_ADMIN: "true" + OP_PROPOSER_DA_RPC: "http://da:26658" + OP_PROPOSER_AUTH_TOKEN: $CELESTIA_NODE_AUTH_TOKEN op-batcher: depends_on: - - op_stack_go_builder - - l1 - - l2 - - op-node + op_stack_go_builder: + condition: service_started + l1: + condition: service_started + l2: + condition: service_started + op-node: + condition: service_started + da: + condition: service_healthy build: context: ../ dockerfile: ./op-batcher/Dockerfile @@ -150,6 +177,7 @@ services: OP_BATCHER_L1_ETH_RPC: http://l1:8545 OP_BATCHER_L2_ETH_RPC: http://l2:8545 OP_BATCHER_ROLLUP_RPC: http://op-node:8545 + OFFLINE_GAS_ESTIMATION: null OP_BATCHER_MAX_CHANNEL_DURATION: 1 OP_BATCHER_SUB_SAFETY_MARGIN: 4 # SWS is 15, ChannelTimeout is 40 OP_BATCHER_POLL_INTERVAL: 1s @@ -160,6 +188,9 @@ services: OP_BATCHER_METRICS_ENABLED: "true" OP_BATCHER_RPC_ENABLE_ADMIN: "true" OP_BATCHER_BATCH_TYPE: 0 + OP_BATCHER_NAMESPACE_ID: "000008e5f679bf7116cb" + OP_BATCHER_DA_RPC: http://da:26658 + OP_BATCHER_AUTH_TOKEN: $CELESTIA_NODE_AUTH_TOKEN artifact-server: depends_on: diff --git a/ops/docker/op-stack-go/Dockerfile.dockerignore b/ops/docker/op-stack-go/Dockerfile.dockerignore index ccd041a996f6..962bd607dcb5 100644 --- a/ops/docker/op-stack-go/Dockerfile.dockerignore +++ b/ops/docker/op-stack-go/Dockerfile.dockerignore @@ -8,6 +8,7 @@ !/op-bootnode !/op-chain-ops !/op-challenger +!/op-celestia !/op-heartbeat !/op-node !/op-preimage