storage-library-api

A custom Python API which is using for storing a dictionary in a custom destination
The API supports JSON and XML storage formats and File System and Amazon S3 destinations. Any storage format can be used with any destination.

For adding new storage format or destination, you can follow the guide in For Developers section.

This repo also contains a sub-repo which is a simple Stack interface and its implementation (and tests) under stack_interface folder in this repo.

1. Stack Interface

This repo contains two Python files:

• stack_impl.py
  • Simple Stack Interface and its implementation.
• test_stack_impl.py
  • Testing of the interface implementation with unittest.TestCase module.

stack_impl.py

It contains following methods:

def size(self):
    """Returns an integer representing the total number of items in the stack."""

def push(self, element):
    """Pushes the element onto the top of the stack."""

def pop(self):
    """Removes the top element from the stack and returns its value."""

def peek(self):
    """Retrieves the top element from the stack without removing it, and returns its value."""

def empty(self):
    """Tests whether the stack is empty. Returns True if the stack is empty, False otherwise."""

The stack implementation holds the stack in a private Python list called elements.
As defaults in a Python list, the stack can contain any type of element in it.

Also, in addition to stack and its implementation, this code contains two custom Exception class called NullElementException and EmptyStackException.

Technical Details

The interface IStackInterface is a subclass of Python's ABC (Abstract Base Classes) module.

You can extend or implement this interface by inheriting this interface.
Note that, you should implement all methods in this interface to inherit.

2. Storage Library API

Usage

The main.py in this repository contains a demonstration of this library with JSON storage format and File System destination.

Simple usage:

# Create a sample dict
some_dict = {'a': 5, 'b': '4'}
    
# Create a sample record
record = Record('sample', some_dict)

# Call Custom destination and format
destination = FileDestination()
json_format = JsonFormat()

# Custom record operations
record.insert({"a": 55, "c": 14, "d": 'value'})

# Store the record
stored_file = destination.store(record, json_format)

print('{} stored in file system.'.format(stored_file))

# Get the record from stored file 
record_from_system = destination.parse(stored_file, json_format)
print('\nContent:')
print(record_from_system)

Detailed Explanation

All process can be separated into a few steps:

• Preparing key/value data
  • Create a dict which represents all the key/value pairs.
• Preparing Record
  • Create a Record object with the dict and a name.
    • Note that the name will be used as storage file name.
  • Insert/Update/Delete on the Record object.
• Preparing storage format & destination
  • Preparing Storage Format (Supported ones (for now): JSON, XML)
    • Just import the format from formats.
  • Preparing Destination Store (Supported ones (for now): File System, Amazon S3)
    • Just import the store from destinations.
• Store/parse operation
  • From the destination object, call the store/parse methods with appropriate format.

For Developers

If you want to register/extend new storage format or destination for this library, you can use information under this section.

Extending Storage Format

All the default storage formats are stored under formats.py file.

In this file, we have a base abstract class called Format.
For adding a new storage format, you should just create a subclass which inherits Format.

Format class has two functions that requires to implement: to_format and from_format.

def to_format(self, attrs):
This function accepts a dictionary as input and converts it to string representation of desired format.
It returns a formatted string.

def from_format(self, formatted: str):
This function accepts a formatted string of desired format as input and converts it to a dictionary object.
It returns a dict object.

Extending Destination

All the default destinations are stored under destinations.py file.

In this file, we have a base abstract class called Destination.
For adding a new destination, you should just create a subclass which inherits Destination.

Destination class has two functions that requires to implement: store and parse.

def store(self, record: Record, storage_format: Format):
This function accepts a Record and a Format object and stores the Record's content (i.e. attributes, dict) into a file in the destination. The file's storage format is the given Format.
It returns the record's name on successful addition. However, this is not required.

def parse(self, record_name: str, storage_format: Format):
This function accepts a record's name and a Format object and gets the Record with given name and format from the destination and returns a Record object.
In other words, it parses the formatted object into Record object.