Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automated CipherNode deployment #195

Merged
merged 26 commits into from
Dec 10, 2024
Merged

Automated CipherNode deployment #195

merged 26 commits into from
Dec 10, 2024

Conversation

hmzakhalid
Copy link
Member

@hmzakhalid hmzakhalid commented Dec 3, 2024

Summary by CodeRabbit

Release Notes

  • New Features

    • Introduced a new GitHub Actions workflow for building and deploying the Ciphernode application.
    • Added new services (cn1, cn2, cn3, aggregator) and a network (cn-network) in the development Docker Compose configuration.
    • Enhanced configuration management with new parameters (quic_port, enable_mdns) across various configuration files.
  • Bug Fixes

    • Improved error handling and logging in event extraction and processing logic.
  • Documentation

    • Updated configuration files to reflect new parameters for better flexibility.
  • Chores

    • Removed deprecated environment variables and adjusted Docker Compose service configurations.

@hmzakhalid hmzakhalid linked an issue Dec 3, 2024 that may be closed by this pull request
@hmzakhalid hmzakhalid marked this pull request as draft December 3, 2024 17:17
Copy link
Contributor

coderabbitai bot commented Dec 3, 2024

Walkthrough

This pull request introduces significant changes to the Ciphernode application, including the addition of a new GitHub Actions workflow for building and deploying the application to EC2. It also implements new Docker Compose configurations for development and production environments, enhancing service management. Additionally, several configuration files are updated to support new parameters for QUIC and mDNS functionalities. Multiple Rust files are modified to incorporate these new configurations and improve event handling, error management, and logging, thereby enhancing the overall functionality of the Ciphernode application.

Changes

File Path Change Summary
.github/workflows/ec2-deployment.yml New GitHub Actions workflow for building and deploying Ciphernode application to EC2.
docker-compose.dev.yml New Docker Compose file defining services cn1, cn2, cn3, and aggregator with a shared network.
docker-compose.yml New Docker Compose configuration for deploying cipher nodes and aggregator with persistent volumes.
packages/ciphernode/Cargo.toml Added dependencies: shellexpand = "3.1.0" and tempfile = "3.14.0".
packages/ciphernode/Dockerfile Multi-stage Dockerfile for building and running the Ciphernode application.
packages/ciphernode/ciphernode-entrypoint.sh New entrypoint script for initializing the Ciphernode environment with error handling.
packages/ciphernode/config/Cargo.toml Added shellexpand as a dependency and tempfile as a development dependency.
packages/ciphernode/config/src/app_config.rs Enhanced AppConfig structure with new fields quic_port and enable_mdns, and updated methods.
packages/ciphernode/config/src/lib.rs New module yaml added for YAML configuration handling.
packages/ciphernode/config/src/yaml.rs New function load_yaml_with_env for reading YAML with environment variable substitution.
packages/ciphernode/enclave_node/src/aggregator.rs Updated setup_aggregator function to include quic_port and enable_mdns parameters.
packages/ciphernode/enclave_node/src/ciphernode.rs Updated setup_ciphernode function to include quic_port and enable_mdns parameters.
packages/ciphernode/evm/src/ciphernode_registry_sol.rs Enhanced event handling and logging for CiphernodeRegistrySolReader and CiphernodeRegistrySol.
packages/ciphernode/net/src/bin/p2p_test.rs Added environment variable handling for enable_mdns in p2p_test.
packages/ciphernode/net/src/network_manager.rs Updated setup_with_peer method to accept quic_port and enable_mdns parameters.
packages/ciphernode/net/src/network_peer.rs Modified NodeBehaviour and create_mdns_kad_behaviour to include enable_mdns parameter.
packages/ciphernode/net/tests/docker-compose.yaml Updated environment variable from BLOCK_MDNS to ENABLE_MDNS with default value changed to true.
packages/ciphernode/net/tests/entrypoint.sh Removed mDNS blocking logic from the entrypoint script.
tests/basic_integration/lib/ag/config.yaml Added quic_port and enable_mdns parameters to configuration.
tests/basic_integration/lib/cn1/config.yaml Added quic_port and enable_mdns parameters to configuration.
tests/basic_integration/lib/cn2/config.yaml Added quic_port and enable_mdns parameters to configuration.
tests/basic_integration/lib/cn3/config.yaml Added quic_port and enable_mdns parameters to configuration.
tests/basic_integration/lib/cn4/config.yaml Added quic_port and enable_mdns parameters to configuration.

Possibly related PRs

Suggested reviewers

  • nginnever
  • ryardley
  • auryn-macmillan

Poem

🐇 In the meadow where code does play,
New workflows bloom, bright as day.
With Docker's dance and secrets tight,
Ciphernodes leap into the night.
QUIC and mDNS, a hop, skip, and cheer,
Deployments are here, let’s give a big cheer! 🎉


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between e99feb4 and bc9e6db.

📒 Files selected for processing (1)
  • packages/ciphernode/Dockerfile (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/ciphernode/Dockerfile

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Experiment)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@hmzakhalid hmzakhalid added Ciphernode Related to the ciphernode package chore labels Dec 3, 2024
@hmzakhalid hmzakhalid self-assigned this Dec 3, 2024
@hmzakhalid hmzakhalid changed the title Automated CipherNode deployment to ECR Automated CipherNode deployment Dec 9, 2024
hmzakhalid and others added 5 commits December 9, 2024 12:45
* Env substitution works

* Move to dev-dependencies

* Formatting

* Ensure that there are no quotes or anything around substitution

* use non normal env vars to avoid future issues
@hmzakhalid hmzakhalid marked this pull request as ready for review December 10, 2024 11:12
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 13

🧹 Outside diff range and nitpick comments (12)
packages/ciphernode/net/src/network_peer.rs (1)

119-165: Consider Adding Unit Tests for enable_mdns Functionality

To ensure that the conditional mDNS initialization works as intended, consider adding unit tests for scenarios where enable_mdns is both true and false.

tests/basic_integration/lib/cn4/config.yaml (1)

4-5: Consider distributed system architecture implications

The configuration changes across multiple nodes introduce important architectural considerations:

  1. Service Discovery:

    • mDNS enables automatic service discovery but may not be suitable for all deployment scenarios
    • Consider alternative service discovery mechanisms for production (e.g., DNS-SD, consul)
  2. Network Architecture:

    • Sequential QUIC ports (9091-9094) suggest a tightly coupled configuration
    • Consider making ports configurable through environment variables
    • Document the network topology and port allocation strategy
  3. Security:

    • Evaluate the security implications of mDNS in different deployment environments
    • Consider implementing network segmentation
    • Add configuration validation for production deployments

Would you like assistance in implementing any of these architectural improvements?

packages/ciphernode/config/src/yaml.rs (1)

23-50: Enhance test coverage

While the current test is good, consider adding tests for:

  1. Missing environment variables
  2. Invalid YAML syntax
  3. File read errors
  4. Nested environment variables

Would you like me to provide additional test cases?

packages/ciphernode/Dockerfile (1)

1-5: Consider using a more specific Node.js base image version

While using node:22 works, it's better to specify a more precise version (e.g., node:22.0.0-slim) to ensure consistent builds. Also, consider using the -slim variant to reduce the image size since this is just a build stage.

-FROM node:22 AS evm-builder
+FROM node:22.0.0-slim AS evm-builder
packages/ciphernode/Cargo.toml (1)

58-58: Consider moving tempfile to dev-dependencies

The tempfile crate is typically used for testing. Unless it's needed in production code, consider moving it to the [dev-dependencies] section.

docker-compose.yml (2)

12-16: Consider securing UDP ports

The services expose UDP ports in host mode, which bypasses Docker's network isolation. Consider:

  1. Using bridge mode instead of host mode
  2. Implementing proper firewall rules

90-93: Fix trailing whitespace and improve secrets management

  1. Remove trailing whitespace on line 93
  2. Consider using Docker secrets management for swarm mode
secrets:
  secrets.json:
    file: ./configs/secrets.json
-    
🧰 Tools
🪛 yamllint (1.35.1)

[error] 93-93: trailing spaces

(trailing-spaces)

packages/ciphernode/net/src/bin/p2p_test.rs (2)

41-41: Consider security implications of enabling mDNS

mDNS can pose security risks in production environments. Ensure this feature is properly controlled through configuration and documented.

Consider:

  1. Adding a warning log when mDNS is enabled
  2. Documenting security implications in deployment guides
  3. Adding configuration validation in production environments

41-41: Make network test timeouts configurable

The hardcoded 10-second timeout for message reception might not be suitable for all environments, especially in CI/CD pipelines or slower networks.

+const DEFAULT_MESSAGE_TIMEOUT_SECS: u64 = 10;
+
 #[tokio::main]
 async fn main() -> Result<()> {
+    let message_timeout = env::var("MESSAGE_TIMEOUT_SECS")
+        .ok()
+        .and_then(|s| s.parse::<u64>().ok())
+        .unwrap_or(DEFAULT_MESSAGE_TIMEOUT_SECS);
packages/ciphernode/evm/src/ciphernode_registry_sol.rs (1)

Line range hint 89-95: Consider enhancing error logging with additional context.

When parsing events fails, it would be helpful to include the raw event data in the error log to aid debugging.

-                error!("Error parsing event CiphernodeAdded after topic was matched!");
+                error!(
+                    data=?data,
+                    "Error parsing event CiphernodeAdded after topic was matched!"
+                );
packages/ciphernode/net/src/network_manager.rs (1)

74-75: Document the new parameters.

The new parameters quic_port and enable_mdns should be documented with comments explaining their purpose and valid ranges.

+    /// Port number for QUIC transport (valid range: 1024-65535)
     quic_port: u16,
+    /// When true, enables mDNS for peer discovery in local networks
     enable_mdns: bool,
packages/ciphernode/config/src/app_config.rs (1)

523-579: Enhance environment variable test coverage.

The test case verifies basic env var substitution but should also test:

  • Invalid port numbers
  • Boolean parsing for enable_mdns

Add these test cases:

#[test]
fn test_config_invalid_port() {
    // Test port < 1024
    // Test port > 65535
}

#[test]
fn test_config_mdns_parsing() {
    // Test true/false values
    // Test invalid values
}
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 3a89686 and e99feb4.

⛔ Files ignored due to path filters (1)
  • packages/ciphernode/Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (25)
  • .github/workflows/ec2-deployment.yml (1 hunks)
  • docker-compose.dev.yml (1 hunks)
  • docker-compose.yml (1 hunks)
  • packages/ciphernode/Cargo.toml (2 hunks)
  • packages/ciphernode/Dockerfile (1 hunks)
  • packages/ciphernode/ciphernode-entrypoint.sh (1 hunks)
  • packages/ciphernode/config/Cargo.toml (1 hunks)
  • packages/ciphernode/config/src/app_config.rs (6 hunks)
  • packages/ciphernode/config/src/lib.rs (1 hunks)
  • packages/ciphernode/config/src/yaml.rs (1 hunks)
  • packages/ciphernode/enclave_node/src/aggregator.rs (1 hunks)
  • packages/ciphernode/enclave_node/src/ciphernode.rs (1 hunks)
  • packages/ciphernode/evm/src/ciphernode_registry_sol.rs (1 hunks)
  • packages/ciphernode/net/src/bin/p2p_test.rs (1 hunks)
  • packages/ciphernode/net/src/network_manager.rs (3 hunks)
  • packages/ciphernode/net/src/network_peer.rs (4 hunks)
  • packages/ciphernode/net/tests/docker-compose.yaml (3 hunks)
  • packages/ciphernode/net/tests/entrypoint.sh (0 hunks)
  • tests/basic_integration/base.sh (1 hunks)
  • tests/basic_integration/lib/ag/config.yaml (1 hunks)
  • tests/basic_integration/lib/cn1/config.yaml (1 hunks)
  • tests/basic_integration/lib/cn2/config.yaml (1 hunks)
  • tests/basic_integration/lib/cn3/config.yaml (1 hunks)
  • tests/basic_integration/lib/cn4/config.yaml (1 hunks)
  • tests/basic_integration/persist.sh (1 hunks)
💤 Files with no reviewable changes (1)
  • packages/ciphernode/net/tests/entrypoint.sh
✅ Files skipped from review due to trivial changes (2)
  • packages/ciphernode/config/src/lib.rs
  • docker-compose.dev.yml
🧰 Additional context used
🪛 actionlint (1.7.4)
.github/workflows/ec2-deployment.yml

34-34: the runner of "actions/checkout@v3" action is too old to run on GitHub Actions. update the action's version to fix this issue

(action)


38-38: shellcheck reported issue in this script: SC2086:info:1:54: Double quote to prevent globbing and word splitting

(shellcheck)


50-50: shellcheck reported issue in this script: SC2086:info:1:17: Double quote to prevent globbing and word splitting

(shellcheck)


50-50: shellcheck reported issue in this script: SC2086:info:1:69: Double quote to prevent globbing and word splitting

(shellcheck)


50-50: shellcheck reported issue in this script: SC2086:info:2:13: Double quote to prevent globbing and word splitting

(shellcheck)


50-50: shellcheck reported issue in this script: SC2086:info:2:25: Double quote to prevent globbing and word splitting

(shellcheck)


58-58: shellcheck reported issue in this script: SC2086:info:1:12: Double quote to prevent globbing and word splitting

(shellcheck)


58-58: shellcheck reported issue in this script: SC2086:info:1:24: Double quote to prevent globbing and word splitting

(shellcheck)


58-58: shellcheck reported issue in this script: SC2086:info:1:35: Double quote to prevent globbing and word splitting

(shellcheck)


58-58: shellcheck reported issue in this script: SC2086:info:2:13: Double quote to prevent globbing and word splitting

(shellcheck)

🪛 yamllint (1.35.1)
.github/workflows/ec2-deployment.yml

[error] 35-35: trailing spaces

(trailing-spaces)


[error] 45-45: trailing spaces

(trailing-spaces)


[error] 66-66: trailing spaces

(trailing-spaces)


[error] 69-69: trailing spaces

(trailing-spaces)


[error] 81-81: trailing spaces

(trailing-spaces)


[error] 84-84: trailing spaces

(trailing-spaces)


[warning] 86-86: too many blank lines

(1 > 0) (empty-lines)

docker-compose.yml

[error] 93-93: trailing spaces

(trailing-spaces)

🔇 Additional comments (25)
packages/ciphernode/net/src/network_peer.rs (6)

10-10: Import of Toggle Added

The import statement correctly includes Toggle, which is necessary for conditionally enabling mDNS behavior.


17-17: Updated mpsc Imports

The import of channel, Receiver, and Sender from tokio::sync::mpsc is appropriate and necessary for message passing.


26-26: Added mdns Field to NodeBehaviour

The mdns field has been added to the NodeBehaviour struct wrapped in a Toggle to conditionally enable mDNS.


55-55: Updated Behavior Initialization with enable_mdns

The behavior is correctly initialized using the enable_mdns parameter within the .with_behaviour closure.


119-165: Implementation of create_mdns_kad_behaviour with Conditional mDNS

The create_mdns_kad_behaviour function has been updated to initialize mDNS conditionally based on the enable_mdns flag. This change allows for greater flexibility in network behavior configuration.


47-47: Verify All Instantiations of NetworkPeer::new Include enable_mdns Parameter

The enable_mdns: bool parameter has been added to NetworkPeer::new. Ensure that all calls to this method across the codebase have been updated to include this new parameter to prevent compile-time errors.

Run the following script to locate and verify calls without the enable_mdns parameter:

✅ Verification successful

All NetworkPeer::new Calls Include enable_mdns Parameter - No Updates Required

The verification shows that all existing calls to NetworkPeer::new already include the enable_mdns parameter:

  • In network_manager.rs (lines 97-103): enable_mdns is passed as the last parameter
  • In p2p_test.rs (line 41): enable_mdns is passed as the last parameter
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Find all calls to `NetworkPeer::new` missing the `enable_mdns` parameter.

# Search for calls with four arguments (indicating missing `enable_mdns`)
ast-grep --pattern 'NetworkPeer::new($_, $_, $_, $_)' --ignore-case

# Expected result: No matches found. If matches are found, those instances need to be updated.

Length of output: 270


Script:

#!/bin/bash
# Let's first find all instances of NetworkPeer::new to analyze its usage
ast-grep --pattern 'NetworkPeer::new($$$)'

# Also search for the struct definition to understand the constructor
ast-grep --pattern 'impl NetworkPeer {
  $$$
  pub fn new($$$) -> $$ {
    $$$
  }
  $$$
}'

Length of output: 814

packages/ciphernode/config/Cargo.toml (2)

12-12: Added shellexpand Dependency

The addition of shellexpand to [dependencies] is appropriate for expanding environment variables within configuration files.


15-17: Added tempfile to [dev-dependencies]

Including tempfile in [dev-dependencies] is suitable for handling temporary files during testing and development.

tests/basic_integration/lib/ag/config.yaml (1)

4-5: Ensure quic_port Values Are Unique Across Configurations

The quic_port is set to 9095. Verify that all quic_port settings in different configuration files are unique to prevent port conflicts during integration tests.

Run the following script to check for duplicate quic_port values:

✅ Verification successful

QUIC Ports Are Properly Configured

The configuration files show that each instance uses a unique QUIC port (9091, 9092, 9093, 9094, 9095), ensuring there will be no port conflicts during integration tests.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check for duplicate `quic_port` values in configuration files.

# Find all `config.yaml` files and extract `quic_port` values.
fd config.yaml tests/ | xargs grep 'quic_port' | sort | uniq -c | grep -v '^\s*1\s'

# Expected result: No output means there are no duplicate `quic_port` values.

Length of output: 83


Script:

#!/bin/bash
# Let's first check if there are any config.yaml files and their content
fd config.yaml tests/ -x cat {}

Length of output: 1995

tests/basic_integration/lib/cn3/config.yaml (1)

4-5: Ensure quic_port Values Are Unique Across Configurations

The quic_port is set to 9093. Confirm that this port number is unique among all configuration files to avoid any network conflicts.

Use the same script provided earlier to check for duplicates.

tests/basic_integration/lib/cn1/config.yaml (1)

4-5: Review security implications of mDNS in production

While mDNS is useful for service discovery in development environments, enabling it in production should be carefully considered as it could:

  • Expose service information to the local network
  • Create potential attack vectors through service discovery
  • Impact network performance in large deployments

Consider making enable_mdns environment-specific or adding security controls.

tests/basic_integration/lib/cn2/config.yaml (1)

4-5: Verify port allocation strategy across nodes

The sequential port allocation (9092) follows a logical pattern, but consider:

  • Documenting the port allocation strategy
  • Adding port configuration validation
  • Implementing port conflict detection during startup
✅ Verification successful

Port allocation follows consistent sequential pattern

The verification confirms a clear and consistent port allocation strategy:

  • CN1: 9091
  • CN2: 9092 (current change)
  • CN3: 9093
  • CN4: 9094
  • AG: 9095

The sequential port allocation (9091-9095) demonstrates a well-organized approach, and the new port 9092 for CN2 fits perfectly within this pattern.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Check port allocation across all config files
echo "Analyzing port allocation pattern..."
fd -e yaml -e yml . tests/basic_integration/lib/ -x grep -l "quic_port"

# Look for any port-related configuration patterns
echo "Checking for other port configurations..."
rg "port:.*909[0-9]" 

Length of output: 862

tests/basic_integration/lib/cn4/config.yaml (1)

4-5: Verify configuration completeness and consistency

The configuration follows the pattern but raises some concerns:

  • Port 9094 implies existence of cn3 (port 9093)
  • Need to ensure all node configurations are updated consistently
✅ Verification successful

Configuration is complete and consistent across all nodes

The verification shows that:

  • All node configurations (cn1-cn4 and ag) are present
  • Port sequence is complete (9091-9095)
  • Configuration structure is consistent with quic_port and enable_mdns settings across all nodes
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Check for missing node configurations
echo "Verifying all node configurations..."
fd -e yaml -e yml . tests/basic_integration/lib/ -x grep -l "quic_port"

# Verify configuration consistency across all nodes
echo "Checking configuration structure consistency..."
for f in $(fd -e yaml -e yml . tests/basic_integration/lib/); do
  echo "=== $f ==="
  grep -A 1 "quic_port\|enable_mdns" "$f"
done

Length of output: 2022

packages/ciphernode/net/tests/docker-compose.yaml (1)

14-14: Security improvement: Verify mDNS requirement in test environment

The change from BLOCK_MDNS to ENABLE_MDNS with a default of true could potentially affect network isolation in the test environment. While this might be necessary for service discovery, consider:

  1. Whether mDNS is required for all test scenarios
  2. If it should be disabled by default in test environments
  3. The security implications of enabling multicast DNS in containerized tests

Also applies to: 26-26, 37-37

✅ Verification successful

Let me check the content of the run.sh file to understand how mDNS is being used in the tests.


Let me check the test implementation to understand if mDNS is actually being tested for functionality.


Let me check the test image's entrypoint script to understand how the ENABLE_MDNS flag is used.


Let me check the Dockerfile to understand the test environment setup.


mDNS configuration is correctly implemented for test environment

The change from BLOCK_MDNS to ENABLE_MDNS with default true is appropriate because:

  1. The test harness explicitly tests both mDNS-enabled and mDNS-blocked scenarios (as seen in run.sh)
  2. The implementation in network_peer.rs shows proper handling of mDNS discovery with appropriate security controls:
    • mDNS is properly toggled through the enable_mdns flag
    • Discovered peers are explicitly added/removed from the gossipsub network
    • The feature is part of the test suite to verify peer discovery functionality
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Check if mDNS is used in test files
echo "Checking mDNS usage in tests..."
rg -l "mdns" packages/ciphernode/net/tests/

Length of output: 153


Script:

#!/bin/bash
# Check how mDNS is used in run.sh
echo "Content of run.sh:"
cat packages/ciphernode/net/tests/run.sh

Length of output: 537


Script:

#!/bin/bash
# Search for test files that might contain mDNS-related tests
echo "Searching for test files:"
fd -e rs -e ts -e js test packages/ciphernode/net/tests/

echo -e "\nChecking test content for mDNS functionality:"
rg -i "mdns" packages/ciphernode/net/tests/ -A 5

Length of output: 2282


Script:

#!/bin/bash
# Look for the entrypoint script and any related configuration
echo "Searching for entrypoint.sh:"
fd entrypoint.sh packages/ciphernode/net/tests/

echo -e "\nChecking entrypoint content:"
cat packages/ciphernode/net/tests/entrypoint.sh 2>/dev/null || echo "Entrypoint not found in tests directory"

echo -e "\nSearching for other relevant files:"
fd -t f . packages/ciphernode/net/tests/

Length of output: 615


Script:

#!/bin/bash
# Check Dockerfile content
echo "Content of Dockerfile:"
cat packages/ciphernode/net/tests/Dockerfile

echo -e "\nChecking for mDNS usage in the codebase:"
rg -i "mdns" packages/ciphernode/net/ --type rust -A 5

Length of output: 8297

packages/ciphernode/Dockerfile (3)

24-30: LGTM: Proper security practices implemented

Good security practices observed:

  • Creating a non-root user
  • Setting up proper directory permissions
  • Using specific UIDs

37-38: Verify entrypoint script existence and permissions

The Dockerfile references ciphernode-entrypoint.sh, but it's not visible in the current PR. Ensure this file exists and contains proper error handling.

✅ Verification successful

Let me check the content of the entrypoint script to verify its error handling.


Entrypoint script verified with proper error handling

The ciphernode-entrypoint.sh script exists and includes comprehensive error handling:

  • Checks for required config and secrets files
  • Validates presence of required secrets (private_key and password)
  • Uses set -e for failing on any error
  • Proper exit codes and error messages
  • Clear execution paths for both aggregator and non-aggregator modes
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Check if entrypoint script exists
fd -t f "ciphernode-entrypoint.sh$"

Length of output: 82


Script:

#!/bin/bash
cat packages/ciphernode/ciphernode-entrypoint.sh

Length of output: 1162


19-21: Consider security implications of installed packages

The iptables package requires elevated privileges. Ensure this is absolutely necessary for the application's functionality, as it increases the security surface area.

packages/ciphernode/Cargo.toml (1)

42-42: Verify shellexpand usage for security implications

The shellexpand crate can potentially execute shell commands if used incorrectly. Ensure it's only used for safe path expansion.

✅ Verification successful

Let me check the exact usage pattern by looking at the full context of the code.


Let me try a different approach to get the full context.


Safe usage of shellexpand confirmed

The shellexpand crate is being used safely in this codebase. It's only used for environment variable substitution in YAML configuration files via shellexpand::env(), which specifically handles environment variable expansion and doesn't execute shell commands. The implementation in load_yaml_with_env function is secure as it only performs environment variable substitution on the file content.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for shellexpand usage patterns
rg "shellexpand::" -A 3

Length of output: 243


Script:

#!/bin/bash
# Get the full context of shellexpand usage
ast-grep --pattern 'fn $_($_) {
  $$$
  shellexpand::$$$
  $$$
}'

Length of output: 67


Script:

#!/bin/bash
# Get the content of the file where shellexpand is used
cat packages/ciphernode/config/src/yaml.rs

Length of output: 1462

docker-compose.yml (1)

100-102: Verify overlay network requirements

The use of overlay network implies Docker Swarm mode. Ensure this is intentional and that the deployment environment supports it.

✅ Verification successful

Overlay network and Swarm mode configuration confirmed as intentional

The use of overlay network driver along with deploy configurations in the docker-compose.yml file indicates this is indeed a Swarm-mode setup. The configuration shows:

  • All services (cn1, cn2, cn3, aggregator) are configured with deployment specifications
  • Each service has replicas defined
  • All services are connected to the overlay network
  • Port publishing is configured in host mode

This is a valid Swarm-mode configuration for a distributed system of cipher nodes.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for other network configurations
rg "networks:" -A 5 "docker-compose*.yml"

Length of output: 101


Script:

#!/bin/bash
# Let's first find the docker-compose files in the repository
fd "docker-compose" --type f

# Then check their content for network configurations
fd "docker-compose" --type f --exec cat {}

Length of output: 3584


Script:

#!/bin/bash
# Let's check for Docker Swarm or orchestration related configurations
fd "stack|swarm|deploy" --type f

# And specifically look for deploy configurations
rg "deploy:" -A 3

Length of output: 947

.github/workflows/ec2-deployment.yml (1)

3-26: Verify branch protection rules for the release branch

Since this workflow deploys to production when pushing to the release branch, ensure that branch protection rules are in place to prevent unauthorized deployments.

packages/ciphernode/enclave_node/src/ciphernode.rs (1)

73-74: LGTM! Network configuration changes look good

The addition of QUIC port and mDNS configuration parameters is well-integrated and consistent with the network manager's interface.

packages/ciphernode/enclave_node/src/aggregator.rs (1)

84-85: LGTM! Network configuration changes are consistent

The network configuration changes mirror those in ciphernode.rs, maintaining consistency across the codebase.

tests/basic_integration/persist.sh (1)

54-55: ⚠️ Potential issue

Critical: Replace sleep with robust synchronization in persistence test

Using a sleep command in a persistence test is particularly risky as service restart timings can vary significantly in different environments, especially in CI/CD pipelines.

The same synchronization mechanism suggested for base.sh should be implemented here. Additionally, consider adding health checks for the aggregator service restart:

-sleep 2
+# Function to check if committee is ready
+check_committee_ready() {
+    local max_attempts=30
+    local attempt=1
+    while [ $attempt -le $max_attempts ]; do
+        if yarn committee:status --network localhost | grep -q "ready"; then
+            return 0
+        fi
+        sleep 1
+        ((attempt++))
+    done
+    return 1
+}
+
+# Wait for committee to be ready with timeout
+if ! check_committee_ready; then
+    echo "Timeout waiting for committee to be ready"
+    exit 1
+fi
packages/ciphernode/evm/src/ciphernode_registry_sol.rs (1)

6-6: LGTM! Import cleanup looks good.

The simplified import improves code clarity by removing unused imports.

packages/ciphernode/config/src/app_config.rs (1)

181-182: Review default values for security and usability.

  • Default port 9091 is good (non-privileged)
  • Default enable_mdns as false is secure

packages/ciphernode/ciphernode-entrypoint.sh Show resolved Hide resolved
packages/ciphernode/ciphernode-entrypoint.sh Show resolved Hide resolved
packages/ciphernode/config/src/yaml.rs Show resolved Hide resolved
packages/ciphernode/config/src/yaml.rs Show resolved Hide resolved
packages/ciphernode/Dockerfile Outdated Show resolved Hide resolved
.github/workflows/ec2-deployment.yml Show resolved Hide resolved
tests/basic_integration/base.sh Outdated Show resolved Hide resolved
packages/ciphernode/net/src/bin/p2p_test.rs Show resolved Hide resolved
packages/ciphernode/config/src/app_config.rs Show resolved Hide resolved
Copy link
Contributor

@ryardley ryardley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖

@ryardley ryardley merged commit 44aac5b into main Dec 10, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
chore Ciphernode Related to the ciphernode package
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Automated deployment
2 participants