Skip to content

Commit

Permalink
Moves the KG builder code to the experimental folder
Browse files Browse the repository at this point in the history
  • Loading branch information
alexthomas93 committed Aug 13, 2024
1 parent cc48eef commit 48503c0
Show file tree
Hide file tree
Showing 42 changed files with 133 additions and 110 deletions.
30 changes: 15 additions & 15 deletions docs/source/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,55 +12,55 @@ Components
KGWriter
========

.. autoclass:: neo4j_genai.components.kg_writer.KGWriter
.. autoclass:: neo4j_genai.experimental.components.kg_writer.KGWriter
:members: run

Neo4jWriter
===========

.. autoclass:: neo4j_genai.components.kg_writer.Neo4jWriter
.. autoclass:: neo4j_genai.experimental.components.kg_writer.Neo4jWriter
:members: run

TextSplitter
============

.. autoclass:: neo4j_genai.components.text_splitters.base.TextSplitter
.. autoclass:: neo4j_genai.experimental.components.text_splitters.base.TextSplitter
:members: run

LangChainTextSplitterAdapter
============================

.. autoclass:: neo4j_genai.components.text_splitters.langchain.LangChainTextSplitterAdapter
.. autoclass:: neo4j_genai.experimental.components.text_splitters.langchain.LangChainTextSplitterAdapter
:members: run

LlamaIndexTextSplitterAdapter
=============================

.. autoclass:: neo4j_genai.components.text_splitters.llamaindex.LlamaIndexTextSplitterAdapter
.. autoclass:: neo4j_genai.experimental.components.text_splitters.llamaindex.LlamaIndexTextSplitterAdapter
:members: run

TextChunkEmbedder
=================

.. autoclass:: neo4j_genai.components.embedder.TextChunkEmbedder
.. autoclass:: neo4j_genai.experimental.components.embedder.TextChunkEmbedder
:members: run

SchemaBuilder
=============

.. autoclass:: neo4j_genai.components.schema.SchemaBuilder
.. autoclass:: neo4j_genai.experimental.components.schema.SchemaBuilder
:members: run

EntityRelationExtractor
=======================

.. autoclass:: neo4j_genai.components.entity_relation_extractor.EntityRelationExtractor
.. autoclass:: neo4j_genai.experimental.components.entity_relation_extractor.EntityRelationExtractor
:members: run

LLMEntityRelationExtractor
==========================

.. autoclass:: neo4j_genai.components.entity_relation_extractor.LLMEntityRelationExtractor
.. autoclass:: neo4j_genai.experimental.components.entity_relation_extractor.LLMEntityRelationExtractor
:members: run

.. _retrievers-section:
Expand Down Expand Up @@ -219,11 +219,11 @@ Errors

* :class:`neo4j_genai.exceptions.LLMGenerationError`

* :class:`neo4j_genai.pipeline.exceptions.PipelineDefinitionError`
* :class:`neo4j_genai.experimental.pipeline.exceptions.PipelineDefinitionError`

* :class:`neo4j_genai.pipeline.exceptions.PipelineMissingDependencyError`
* :class:`neo4j_genai.experimental.pipeline.exceptions.PipelineMissingDependencyError`

* :class:`neo4j_genai.pipeline.exceptions.PipelineStatusUpdateError`
* :class:`neo4j_genai.experimental.pipeline.exceptions.PipelineStatusUpdateError`


Neo4jGenAiError
Expand Down Expand Up @@ -327,19 +327,19 @@ LLMGenerationError
PipelineDefinitionError
=======================

.. autoclass:: neo4j_genai.pipeline.exceptions.PipelineDefinitionError
.. autoclass:: neo4j_genai.experimental.pipeline.exceptions.PipelineDefinitionError
:show-inheritance:


PipelineMissingDependencyError
==============================

.. autoclass:: neo4j_genai.pipeline.exceptions.PipelineMissingDependencyError
.. autoclass:: neo4j_genai.experimental.pipeline.exceptions.PipelineMissingDependencyError
:show-inheritance:


PipelineStatusUpdateError
=========================

.. autoclass:: neo4j_genai.pipeline.exceptions.PipelineStatusUpdateError
.. autoclass:: neo4j_genai.experimental.pipeline.exceptions.PipelineStatusUpdateError
:show-inheritance:
20 changes: 10 additions & 10 deletions docs/source/types.rst
Original file line number Diff line number Diff line change
Expand Up @@ -36,49 +36,49 @@ RagResultModel
TextChunk
=========

.. autoclass:: neo4j_genai.components.types.TextChunk
.. autoclass:: neo4j_genai.experimental.components.types.TextChunk

TextChunks
==========

.. autoclass:: neo4j_genai.components.types.TextChunks
.. autoclass:: neo4j_genai.experimental.components.types.TextChunks

Neo4jNode
=========

.. autoclass:: neo4j_genai.components.types.Neo4jNode
.. autoclass:: neo4j_genai.experimental.components.types.Neo4jNode

Neo4jRelationship
=================

.. autoclass:: neo4j_genai.components.types.Neo4jRelationship
.. autoclass:: neo4j_genai.experimental.components.types.Neo4jRelationship

Neo4jGraph
==========

.. autoclass:: neo4j_genai.components.types.Neo4jGraph
.. autoclass:: neo4j_genai.experimental.components.types.Neo4jGraph

KGWriterModel
=============

.. autoclass:: neo4j_genai.components.kg_writer.KGWriterModel
.. autoclass:: neo4j_genai.experimental.components.kg_writer.KGWriterModel

SchemaProperty
==============

.. autoclass:: neo4j_genai.components.schema.SchemaProperty
.. autoclass:: neo4j_genai.experimental.components.schema.SchemaProperty

SchemaEntity
============

.. autoclass:: neo4j_genai.components.schema.SchemaEntity
.. autoclass:: neo4j_genai.experimental.components.schema.SchemaEntity

SchemaRelation
==============

.. autoclass:: neo4j_genai.components.schema.SchemaEntity
.. autoclass:: neo4j_genai.experimental.components.schema.SchemaEntity

SchemaConfig
============

.. autoclass:: neo4j_genai.components.schema.SchemaConfig
.. autoclass:: neo4j_genai.experimental.components.schema.SchemaConfig
8 changes: 4 additions & 4 deletions docs/source/user_guide_pipeline.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@ such as chunking documents or saving results to Neo4j.
This package includes a few default components, but developers can create
their own by following these steps:

1. Create a subclass of the Pydantic `neo4j_genai.pipeline.DataModel` to represent the data being returned by the component
2. Create a subclass of `neo4j_genai.pipeline.Component`
1. Create a subclass of the Pydantic `neo4j_genai.experimental.pipeline.DataModel` to represent the data being returned by the component
2. Create a subclass of `neo4j_genai.experimental.pipeline.Component`
3. Create a run method in this new class and specify the required inputs and output model using the just created `DataModel`
4. Implement the run method: it's an `async` method, allowing tasks to be parallelized and awaited within this method.

Expand All @@ -29,7 +29,7 @@ the resulting sum:

.. code:: python
from neo4j_genai.pipeline import Component, DataModel
from neo4j_genai.experimental.pipeline import Component, DataModel
class IntResultModel(DataModel):
result: int
Expand All @@ -53,7 +53,7 @@ Here's how to create a simple pipeline and propagate results from one component
.. code:: python
import asyncio
from neo4j_genai.pipeline import Pipeline
from neo4j_genai.experimental.pipeline import Pipeline
pipe = Pipeline()
pipe.add_component("a", ComponentAdd())
Expand Down
16 changes: 9 additions & 7 deletions examples/pipeline/kg_builder.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,22 +20,24 @@

import neo4j
from langchain_text_splitters import CharacterTextSplitter
from neo4j_genai.components.embedder import TextChunkEmbedder
from neo4j_genai.components.entity_relation_extractor import (
from neo4j_genai.embeddings.openai import OpenAIEmbeddings
from neo4j_genai.experimental.components.embedder import TextChunkEmbedder
from neo4j_genai.experimental.components.entity_relation_extractor import (
LLMEntityRelationExtractor,
OnError,
)
from neo4j_genai.components.kg_writer import Neo4jWriter
from neo4j_genai.components.schema import (
from neo4j_genai.experimental.components.kg_writer import Neo4jWriter
from neo4j_genai.experimental.components.schema import (
SchemaBuilder,
SchemaEntity,
SchemaProperty,
SchemaRelation,
)
from neo4j_genai.components.text_splitters.langchain import LangChainTextSplitterAdapter
from neo4j_genai.embeddings.openai import OpenAIEmbeddings
from neo4j_genai.experimental.components.text_splitters.langchain import (
LangChainTextSplitterAdapter,
)
from neo4j_genai.experimental.pipeline import Pipeline
from neo4j_genai.llm import OpenAILLM
from neo4j_genai.pipeline import Pipeline

# set log level to DEBUG for all neo4j_genai.* loggers
logging.config.dictConfig(
Expand Down
6 changes: 3 additions & 3 deletions examples/pipeline/kg_builder_with_schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,12 @@
import logging
from typing import Any

from neo4j_genai.components.schema import (
from neo4j_genai.experimental.components.schema import (
SchemaBuilder,
SchemaEntity,
SchemaRelation,
)
from neo4j_genai.pipeline import Component, DataModel
from neo4j_genai.experimental.pipeline import Component, DataModel
from pydantic import BaseModel, validate_call

logging.basicConfig(level=logging.DEBUG)
Expand Down Expand Up @@ -87,7 +87,7 @@ async def run(self, graph: Neo4jGraph) -> WriterModel:


if __name__ == "__main__":
from neo4j_genai.pipeline import Pipeline
from neo4j_genai.experimental.pipeline import Pipeline

# Instantiate Entity and Relation objects
entities = [
Expand Down
10 changes: 7 additions & 3 deletions examples/pipeline/rag.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,15 @@

import neo4j
from neo4j_genai.embeddings.openai import OpenAIEmbeddings
from neo4j_genai.experimental.pipeline import Component, Pipeline
from neo4j_genai.experimental.pipeline.component import DataModel
from neo4j_genai.experimental.pipeline.types import (
ComponentConfig,
ConnectionConfig,
PipelineConfig,
)
from neo4j_genai.generation import PromptTemplate, RagTemplate
from neo4j_genai.llm import LLMInterface, OpenAILLM
from neo4j_genai.pipeline import Component, Pipeline
from neo4j_genai.pipeline.component import DataModel
from neo4j_genai.pipeline.types import ComponentConfig, ConnectionConfig, PipelineConfig
from neo4j_genai.retrievers import VectorRetriever
from neo4j_genai.retrievers.base import Retriever

Expand Down
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,9 @@
# limitations under the License.
from pydantic import validate_call

from neo4j_genai.components.types import TextChunk, TextChunks
from neo4j_genai.embedder import Embedder
from neo4j_genai.pipeline.component import Component
from neo4j_genai.experimental.components.types import TextChunk, TextChunks
from neo4j_genai.experimental.pipeline.component import Component


class TextChunkEmbedder(Component):
Expand All @@ -29,9 +29,9 @@ class TextChunkEmbedder(Component):
.. code-block:: python
from neo4j_genai.components.embedder import TextChunkEmbedder
from neo4j_genai.experimental.components.embedder import TextChunkEmbedder
from neo4j_genai.embeddings.openai import OpenAIEmbeddings
from neo4j_genai.pipeline import Pipeline
from neo4j_genai.experimental.pipeline import Pipeline
embedder = OpenAIEmbeddings(model="text-embedding-3-large")
chunk_embedder = TextChunkEmbedder(embedder)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,18 +24,18 @@

from pydantic import ValidationError, validate_call

from neo4j_genai.components.schema import SchemaConfig
from neo4j_genai.components.types import (
from neo4j_genai.exceptions import LLMGenerationError
from neo4j_genai.experimental.components.schema import SchemaConfig
from neo4j_genai.experimental.components.types import (
Neo4jGraph,
Neo4jNode,
Neo4jRelationship,
TextChunk,
TextChunks,
)
from neo4j_genai.exceptions import LLMGenerationError
from neo4j_genai.experimental.pipeline.component import Component
from neo4j_genai.generation.prompts import ERExtractionTemplate, PromptTemplate
from neo4j_genai.llm import LLMInterface
from neo4j_genai.pipeline.component import Component

logger = logging.getLogger(__name__)

Expand Down Expand Up @@ -163,9 +163,9 @@ class LLMEntityRelationExtractor(EntityRelationExtractor):
.. code-block:: python
from neo4j_genai.components.entity_relation_extractor import LLMEntityRelationExtractor
from neo4j_genai.experimental.components.entity_relation_extractor import LLMEntityRelationExtractor
from neo4j_genai.llm import OpenAILLM
from neo4j_genai.pipeline import Pipeline
from neo4j_genai.experimental.pipeline import Pipeline
llm = OpenAILLM(model_name="gpt-4o", model_params={"temperature": 0, "response_format": {"type": "object"}})
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,14 @@
import neo4j
from pydantic import validate_call

from neo4j_genai.components.types import Neo4jGraph, Neo4jNode, Neo4jRelationship
from neo4j_genai.experimental.components.types import (
Neo4jGraph,
Neo4jNode,
Neo4jRelationship,
)
from neo4j_genai.experimental.pipeline.component import Component, DataModel
from neo4j_genai.indexes import upsert_vector, upsert_vector_on_relationship
from neo4j_genai.neo4j_queries import UPSERT_NODE_QUERY, UPSERT_RELATIONSHIP_QUERY
from neo4j_genai.pipeline.component import Component, DataModel

logger = logging.getLogger(__name__)

Expand Down Expand Up @@ -66,8 +70,8 @@ class Neo4jWriter(KGWriter):
.. code-block:: python
from neo4j import GraphDatabase
from neo4j_genai.components.kg_writer import Neo4jWriter
from neo4j_genai.pipeline import Pipeline
from neo4j_genai.experimental.components.kg_writer import Neo4jWriter
from neo4j_genai.experimental.pipeline import Pipeline
URI = "neo4j://localhost:7687"
AUTH = ("neo4j", "password")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
from pydantic import BaseModel, ValidationError, model_validator, validate_call

from neo4j_genai.exceptions import SchemaValidationError
from neo4j_genai.pipeline import Component, DataModel
from neo4j_genai.experimental.pipeline import Component, DataModel


class SchemaProperty(BaseModel):
Expand Down Expand Up @@ -107,13 +107,13 @@ class SchemaBuilder(Component):
.. code-block:: python
from neo4j_genai.components.schema import (
from neo4j_genai.experimental.components.schema import (
SchemaBuilder,
SchemaEntity,
SchemaProperty,
SchemaRelation,
)
from neo4j_genai.pipeline import Pipeline
from neo4j_genai.experimental.pipeline import Pipeline
entities = [
SchemaEntity(
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@

from abc import abstractmethod

from neo4j_genai.components.types import TextChunks
from neo4j_genai.pipeline.component import Component
from neo4j_genai.experimental.components.types import TextChunks
from neo4j_genai.experimental.pipeline.component import Component


class TextSplitter(Component):
Expand Down
Loading

0 comments on commit 48503c0

Please sign in to comment.