Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Demo mode v2 & moving background tasks outside of the server #2561

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
e5001e9
Demo mode
Matvey-Kuk Nov 18, 2024
31a3e88
Docs
Matvey-Kuk Nov 18, 2024
be886ff
Merge branch 'main' into Matvey-Kuk/demo_mode
Matvey-Kuk Nov 18, 2024
c682491
Fix
Matvey-Kuk Nov 18, 2024
c63e469
up
Matvey-Kuk Nov 18, 2024
7daf2ce
minor
Matvey-Kuk Nov 18, 2024
9b8c52b
fix
Matvey-Kuk Nov 18, 2024
5bc2ed9
Read or Create topology
Matvey-Kuk Nov 18, 2024
294a792
Fingerprint randomization
Matvey-Kuk Nov 18, 2024
d045a28
Merge branch 'main' into Matvey-Kuk/demo_mode
Matvey-Kuk Nov 18, 2024
42cbb09
conditional import
Matvey-Kuk Nov 18, 2024
b34c1df
Update keep/api/config.py
Matvey-Kuk Nov 18, 2024
ac3b392
Update docs/deployment/configuration.mdx
Matvey-Kuk Nov 18, 2024
fe22a75
Merge branch 'main' into Matvey-Kuk/demo_mode
Matvey-Kuk Nov 18, 2024
b8edba1
Fix
Matvey-Kuk Nov 18, 2024
a93fa0c
Merge remote-tracking branch 'refs/remotes/origin/Matvey-Kuk/demo_mod…
Matvey-Kuk Nov 18, 2024
ce121da
Up
Matvey-Kuk Nov 18, 2024
89faba0
More logs
Matvey-Kuk Nov 18, 2024
3fe4071
Waiting for API to get live
Matvey-Kuk Nov 18, 2024
0e329b2
when_ready
Matvey-Kuk Nov 19, 2024
d96465b
Fix
Matvey-Kuk Nov 19, 2024
0fe790d
fix: fix
talboren Nov 19, 2024
353e42c
fix: fix
talboren Nov 19, 2024
e333e9e
fix: fix
talboren Nov 19, 2024
2256347
wait for the connection
Matvey-Kuk Nov 19, 2024
baacc13
fix: fix
talboren Nov 19, 2024
c1dd51a
fix: fix
talboren Nov 19, 2024
1e83a5c
fix: timeout
talboren Nov 19, 2024
d11b44c
Providers
Matvey-Kuk Nov 19, 2024
bfdd576
customer_id:acme
Matvey-Kuk Nov 19, 2024
d3ece2a
Merge remote-tracking branch 'refs/remotes/origin/Matvey-Kuk/demo_mod…
Matvey-Kuk Nov 19, 2024
5ce3629
Fix provider reading
Matvey-Kuk Nov 20, 2024
ddcfcdf
Debug logs
Matvey-Kuk Nov 20, 2024
85b1d89
Moving bg tasks out of the server
Matvey-Kuk Nov 20, 2024
d90654b
Merge branch 'main' into Matvey-Kuk/moving-background-tasks-outside-o…
Matvey-Kuk Nov 20, 2024
bb6452f
Wrapping background tasks
Matvey-Kuk Nov 20, 2024
e63bd84
Formatting
Matvey-Kuk Nov 20, 2024
634780d
Excessive LIVE_DEMO_MODE
Matvey-Kuk Nov 20, 2024
84b5eef
Cleaning
Matvey-Kuk Nov 20, 2024
4e20033
Merge branch 'main' into Matvey-Kuk/moving-background-tasks-outside-o…
Matvey-Kuk Nov 20, 2024
900cdf2
cleaning
Matvey-Kuk Nov 20, 2024
c05007e
entrypoint.sh
Matvey-Kuk Nov 20, 2024
242cf40
POSTHOG_DISABLED for dev
Matvey-Kuk Nov 20, 2024
2a41bc7
Clean
Matvey-Kuk Nov 20, 2024
98810dd
Fix imports
Matvey-Kuk Nov 20, 2024
7f293ca
Analytics from the backend
Matvey-Kuk Nov 20, 2024
cb65b37
TenantS
Matvey-Kuk Nov 20, 2024
0e502d5
Fixes
Matvey-Kuk Nov 21, 2024
9377db9
Merge branch 'main' into Matvey-Kuk/moving-background-tasks-outside-o…
Matvey-Kuk Nov 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion docker/Dockerfile.api
Original file line number Diff line number Diff line change
Expand Up @@ -35,4 +35,7 @@ RUN chgrp -R 0 /app && chmod -R g=u /app
RUN chown -R keep:keep /app
RUN chown -R keep:keep /venv
USER keep
ENTRYPOINT ["gunicorn", "keep.api.api:get_app", "--bind" , "0.0.0.0:8080" , "--workers", "4" , "-k" , "uvicorn.workers.UvicornWorker", "-c", "/venv/lib/python3.11/site-packages/keep/api/config.py"]

ENTRYPOINT ["/venv/lib/python3.11/site-packages/keep/entrypoint.sh"]

CMD ["gunicorn", "keep.api.api:get_app", "--bind" , "0.0.0.0:8080" , "--workers", "4" , "-k" , "uvicorn.workers.UvicornWorker", "-c", "/venv/lib/python3.11/site-packages/keep/api/config.py"]
2 changes: 2 additions & 0 deletions docker/Dockerfile.dev.api
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ RUN . /venv/bin/activate && poetry install --no-root
ENV PYTHONPATH="/app:${PYTHONPATH}"
ENV PATH="/venv/bin:${PATH}"
ENV VIRTUAL_ENV="/venv"
ENV POSTHOG_DISABLED="true"

ENTRYPOINT ["/app/keep/entrypoint.sh"]

CMD ["gunicorn", "keep.api.api:get_app", "--bind" , "0.0.0.0:8080" , "--workers", "1" , "-k" , "uvicorn.workers.UvicornWorker", "-c", "./keep/api/config.py", "--reload"]
1 change: 1 addition & 0 deletions docs/deployment/configuration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ General configuration variables control the core behavior of the Keep server. Th
| **KEEP_API_URL** | Specifies the Keep API URL | No | Constructed from HOST and PORT | Valid URL |
| **KEEP_STORE_RAW_ALERTS** | Enables storing of raw alerts | No | "false" | "true" or "false" |
| **TENANT_CONFIGURATION_RELOAD_TIME** | Time in minutes to reload tenant configurations | No | 5 | Positive integer |
| **KEEP_LIVE_DEMO_MODE** | Keep will simulate incoming alerts and other activity | No | "false" | "true" or "false" |

### Logging and Environment
<Info>
Expand Down
3 changes: 1 addition & 2 deletions keep/api/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
import keep.api.logging
from keep.api.api import AUTH_TYPE
from keep.api.core.db_on_start import migrate_db, try_create_single_tenant
from keep.api.core.report_uptime import launch_uptime_reporting
from keep.api.core.dependencies import SINGLE_TENANT_UUID
from keep.identitymanager.identitymanagerfactory import IdentityManagerTypes

Expand All @@ -19,7 +18,6 @@ def on_starting(server=None):
logger.info("Keep server starting")

migrate_db()
launch_uptime_reporting()

# Create single tenant if it doesn't exist
if AUTH_TYPE in [
Expand Down Expand Up @@ -54,4 +52,5 @@ def on_starting(server=None):
public_url = ngrok_connection.public_url
logger.info(f"ngrok tunnel: {public_url}")
os.environ["KEEP_API_URL"] = public_url

logger.info("Keep server started")
44 changes: 44 additions & 0 deletions keep/api/core/db.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@
from keep.api.models.db.preset import * # pylint: disable=unused-wildcard-import
from keep.api.models.db.provider import * # pylint: disable=unused-wildcard-import
from keep.api.models.db.rule import * # pylint: disable=unused-wildcard-import
from keep.api.models.db.system import * # pylint: disable=unused-wildcard-import
from keep.api.models.db.tenant import * # pylint: disable=unused-wildcard-import
from keep.api.models.db.topology import * # pylint: disable=unused-wildcard-import
from keep.api.models.db.workflow import * # pylint: disable=unused-wildcard-import
Expand Down Expand Up @@ -4482,3 +4483,46 @@ def get_resource_ids_by_resource_type(
# Execute the query and return results
result = session.exec(query)
return result.all()

def get_or_creat_posthog_instance_id(
session: Optional[Session] = None
):
POSTHOG_INSTANCE_ID_KEY = "posthog_instance_id"
with Session(engine) as session:
system = session.exec(select(System).where(System.name == POSTHOG_INSTANCE_ID_KEY)).first()
if system:
return system.value

system = System(
id=str(uuid4()),
name=POSTHOG_INSTANCE_ID_KEY,
value=str(uuid4()),
)
session.add(system)
session.commit()
session.refresh(system)
return system.value

def get_activity_report(
session: Optional[Session] = None
):
from keep.api.models.db.user import User

last_24_hours = datetime.utcnow() - timedelta(hours=24)
activity_report = {}
with Session(engine) as session:
activity_report['tenants_count'] = session.query(Tenant).count()
activity_report['providers_count'] = session.query(Provider).count()
activity_report['users_count'] = session.query(User).count()
activity_report['last_24_hours_incidents_count'] = session.query(Incident).filter(
Incident.creation_time >= last_24_hours).count()
activity_report['last_24_hours_alerts_count'] = session.query(Alert).filter(
Alert.timestamp >= last_24_hours).count()
activity_report['last_24_hours_rules_created'] = session.query(Rule).filter(
Rule.creation_time >= last_24_hours).count()
activity_report['last_24_hours_workflows_created'] = session.query(Workflow).filter(
Workflow.creation_time >= last_24_hours).count()
activity_report['last_24_hours_workflows_executed'] = session.query(WorkflowExecution).filter(
WorkflowExecution.started >= last_24_hours).count()

return activity_report
Loading
Loading