Feature/hybrid node #188
Feature/hybrid node #188
Conversation
MahdiPresario001
commented
Dec 11, 2024
•
MahdiPresario001
commented
- renamed query_node to synthetic_node + removed organic queries bit from it
- renamed entry_node to organic_node + replace redis bit with the usage of the same functions used by synthetic_node
- moved those steaming/nonstreaming functions onto a new "common" folder, used by synth_node & organic_node
- added keypair loading via secret seed via env var
- exposed redis/psql ports in order to be able to reach from outside
| if chat_request.stream: | ||
| return StreamingResponse(generator, media_type="text/event-stream") | ||
| else: | ||
| return await _handle_nonstream_response(generator) |
There was a problem hiding this comment.
separate minimal functions like _handle_no_stream and _handle_no_stream_comp as in latest prod would be more appropriate for /v1/completions and /v1/chat/completions routes
There was a problem hiding this comment.
_handle_no_stream and _handle_no_stream_comp have a lot of the same code that's why i put it in one function directly
| docker compose rm -f -v grafana |
| @@ -22,6 +23,8 @@ services: | |||
| container_name: nineteen-otel-collector | |||
| image: otel/opentelemetry-collector-contrib:0.106.1 | |||
| restart: always | |||
| ports: | |||
| - "4317:4317" | |||
There was a problem hiding this comment.
why exactly is this required just to confirm? to connect organic node running outside docker network?
There was a problem hiding this comment.
yeah for organic nodes on render (redis too)
MahdiPresario001
force-pushed
the
feature/hybrid-node
branch
from
b7f8ace to
b231ee8
Compare
|
|
||
| logger = get_logger(__name__) | ||
|
|
||
| def load_hotkey_keypair_from_seed(secret_seed: str) -> Keypair: |
|
|
||
| T = TypeVar("T", bound=BaseModel) | ||
|
|
||
| load_dotenv() |
| except Exception as e: | ||
| raise ValueError(f"Failed to load keypair: {str(e)}") | ||
|
|
||
| def create_redis_pool( |
|
|
||
| _config = None | ||
|
|
||
| async def factory_config(): |
There was a problem hiding this comment.
dont need separate functions for this, one can do both jobs
| choice = chunk_data["choices"][0] | ||
|
|
||
| if first_chunk: | ||
| if "delta" in choice: |
There was a problem hiding this comment.
loads of magic strings here which are very dangerous. Why?
Was this unchanged from before or all new? If new, how come?
…nx issues over network
MahdiPresario001
force-pushed
the
feature/hybrid-node
branch
from
cf2bdfb to
dad8181
Compare
