Moving Cache and Staging responsibility into HolonSpace

[evomimic]

This enhancement shifts responsibility for both caching and staging holons to its owning HolonSpace. This allows the HolonCacheManager to be replaced and the scope of the CommitManager to be reduced.

Dependencies

None

Current State

• The HolonCacheManager manages a set of caches keyed by HolonSpaceId
• CommitManager maintains a vector of staged holon references. The contents of this vector are converted to/from StagingArea as part of ping-ponging session state between client and guest.
• The StagingArea struct is defined in the Dances layer

Structural Changes

Cache Responsibility

• Eliminate CacheManager . HolonSpace (not Context) should own its staged holons and its cache. Proxies (when supported) can own their own cache.
• Move the following HolonCacheManager functions to HolonSpace:
  • Merge HolonCacheManager's new function into HolonSpace's new function -- create local_cache
  • get_cache(holon_id: HolonId)->Result<Rc<RefCell<HolonCache>>, HolonError> should return a reference to either its own cache or the cache of the appropriate inbound proxy
  • fetch_local_holon(local_id: &LocalId) -> Result<Holon, HolonError>
  • get_rc_holon( &self, holon_id: &HolonId, ) -> Result<Rc<RefCell<Holon>>, HolonError>

Staging and Commit Responsibilities

Split the current functionality of the CommitManager as follows:

• Move responsibility for staging new holons (including those introduced by Issue Add stage_ new_from_clone and stage_new_version Dances #118) from CommitManager to a new Nursery object.
  • Move the CommitManager struct definition to a new nursery.rs module offering the following struct:

      pub struct Nursery {
         staged_holons: Vec<Rc<RefCell<Holon>>>, // Contains all holons staged for commit
         keyed_index: BTreeMap<MapString, usize>, // Allows lookup by key to staged holons for which keys are defined
     };  

NOTE: The above fields are private and, thus, only accessible via Nursery methods.

• Define a HolonsNursery trait in the nursery.rs that defines the signatures for the set of methods used to access and update staged holons

  • get_holon_by_key -- finds and returns a shared reference (Rc<RefCell>) to the staged holon matching the specified key. NOTE: Only staged holons are searched and some holon types do not define unique keys. This means that:

    • even if this function returns None a holon with the specified key may exist in the DHT
    • There might be some holons staged for update that you cannot find by key

  • get_holon(&self, reference: &StagedReference) -> Result<Ref<Holon>, HolonError>

  • get_mut_holon_internal

  • get_mut_holon

  • get_mut_holon_by_index

  • to_staged_reference (by index)

• Implement HolonsNursery trait for Nursery

  • Move the implementation of the following methods to Nursery from CommitManager

  • get_holon_by_key -- finds and returns a shared reference (Rc<RefCell>) to the staged holon matching the specified key. NOTE: Only staged holons are searched and some holon types do not define unique keys. This means that:

    • even if this function returns None a holon with the specified key may exist in the DHT
    • There might be some holons staged for update that you cannot find by key

  • get_holon(&self, reference: &StagedReference) -> Result<Ref<Holon>, HolonError>

  • get_mut_holon_internal

  • get_mut_holon

  • get_mut_holon_by_index

  • to_staged_reference (by index)

• Implement HolonsNursery trait in HolonSpace -- these implementations should just borrow the Nursery (possibly mutably) and then delegate the call to the nursery

• Change definition of HolonSpace to:

pub struct HolonSpace {
    space: Holon,
    cache: HolonCache,
    nursery: Nursery, 
}

• [ ]stage_new_holon as a method on HolonSpace should include a described_by: HolonReference parameter.

• Move all CommitManager methods to HolonSpaceManager except commit()

• CommitManager is just a service. It's only struct field is context:

  pub struct CommitManager<'a> {
      context: &'a HolonsContext, 
  }
  

• Elevate the commit() method to HolonSpace. This is the public interface for commit. HolonSpace should borrow its commit service and then delegate the commit call to its commit service.

Testing

This enhancement does not introduce any new functionality, so no new test cases are required.

Definition of Done

• All existing regression tests pass

[evomimic]

@nphias : I wanted to capture my understanding of where we landed after yesterday's meeting and also share some thoughts that occurred to me since our meeting.

• Rename SpaceManager to HolonSpaceManager. Variables that refer to the HolonSpaceManager can be named space_manager (i.e. a full descriptive name in snake case, but the variable name can omit the 'holon' prefix),

• Retain HolonSpace as a DTO -- we didn't discuss this, but in reviewing the code for your PR, I noticed you had moved the accessor methods from HolonSpace into HolonSpaceManager and then commented them out with a question about whether they were needed or not. I propose we retain a HolonSpace object as a newtype wrapper on Holon with accessor methods. You can move those methods back to HolonSpace.

• Dancer -- shift responsibility for creating the HolonSpace and initializing the Nursery from the context to the HolonSpaceManager itself (via its new method as described below) -- although perhaps name it other than 'new' given how much work it is doing.

• CommitService -- store the (lifetimed) context reference in the CommitService struct:

pub struct HolonSpaceManager<'a> {
    context: &'a HolonsContext, // Reference to the context where the HolonSpace will be persisted
}

impl<'a> CommitService<'a> {
    // Constructor for the service
    pub fn new(context: &'a HolonsContext) -> Self {
        CommitService{ context }
    }

This eliminates the need to pass context to the commit() method.

Define the following traits to segregate different behaviors:

HolonStagingBehavior trait

This trait should be implemented by Nursery, but we can elevate this API to HolonSpaceManager by also having it implement this trait and just delegate the calls defined by this trait to its nursery. This encapsulates knowledge of the Nursery within the HolonSpace Manager.

pub trait HolonStagingBehavior {
    /// Retrieve immutable access to a staged holon using a staged reference.
    fn get_holon(
        &self,
        reference: &StagedReference,
    ) -> Result<Ref<Holon>, HolonError>;

    /// Retrieve mutable access to a staged holon using a staged reference.
    fn get_mut_holon(
        &self,
        reference: &StagedReference,
    ) -> Result<RefMut<Holon>, HolonError>;

    /// Retrieve mutable access to a staged holon using a direct index.
    fn get_mut_holon_by_index(
        &self,
        index: StagedIndex,
    ) -> Result<RefMut<Holon>, HolonError>;

    /// Retrieve a holon by its key, if it exists.
    fn get_holon_by_key(&self, key: MapString) -> Option<Ref<Holon>>;

    /// Stage a new holon in the nursery, returning a staged reference.
    fn stage_new_holon(&mut self, holon: Holon) -> Result<StagedReference, HolonError>;

    /// Convert a staged index into a staged reference.
    fn to_staged_reference(&self, staged_index: StagedIndex) -> Result<StagedReference, HolonError>;
}

Trait HolonCacheBehavior

This trait offers cache methods based on LocalId. It is implemented by CacheManager and (in the future) HolonSpaceProxy. In the latter case, the proxy just needs to delegate this call to its own cache. The functions provided by this trait are ONLY intended to be called by HolonSpaceManager.

/// get_rc_holon returns a shared reference to a cached holon given its LocalId. If a holon with that id is not already resident in the cache, then first retrieve the holon from the persistent store and then add it to the cache.
pub fn get_rc_holon(&self, local_id: LocalId)-> Result<Rc<RefCell>, HolonError>

HolonSpaceBehavior trait.

    - [ ] `pub fn new(context: &HolonsContext) -> Self` -- This method creates a new HolonSpaceManager from a context and initializes its nursery from the context.  This method can encapsulate the process of: (1) checking if the context already contains a HolonSpace, (2) if not, attempting to load it from the DHT, and (3) if still not found, creating the HolonSpace in the dht.
    - [ ] `pub fn get_rc_holon(&self, holon_id: HolonId)-> Result<Rc<RefCell>, HolonError>` -- returns a shared reference to a cached holon given its _HolonId_. If _holon_id_ is a `LocalId`, just delegate the call to the HolonSpaceManager's cache, passing the `LocalId`. _**In a future enhancement:**_ If holon_id as an ExternalId, find the HolonSpaceProxy associated with the  HolonSpaceId and delegate the call to that proxy, passing the `LocalId` portion of the `ExternalId`.
    - [ ] 

Implementations

HolonSpaceManager

• Do NOT store the context as a field within the HolonSpaceManager struct
• Implement HolonSpaceBehavior trait
• Implement HolonStagingBehaivor trait -- and simply delegate calls on this trait's functions to its nursery
• Implement HolonCacheBehavior trait
• Implement the following private functions:

fn ensure_local_holon_space_in_context(&self) -> Result<HolonReference, HolonError>
fn create_local_holon_space(&self) -> Result<HolonReference, HolonError>
fn fetch_and_set_local_holon_space(&self) -> Result<HolonReference, HolonError>

Nursery

• Implement the HolonStagingBehavior trait
• Implement the following as public functions for use by the HolonSpaceManager (but not as part of the trait definition):

pub fn new() -> Self
pub fn clear_staged_objects(&mut self)