From 0799ad70218644a595c9b9a1566eb645e9a273f8 Mon Sep 17 00:00:00 2001 From: Michelle Fu Date: Tue, 9 Jul 2024 13:27:25 -0700 Subject: [PATCH 1/8] restructure docs and clarify ingest setup --- src/docs/environment/index.mdx | 119 ++++++++++++++++----------------- 1 file changed, 57 insertions(+), 62 deletions(-) diff --git a/src/docs/environment/index.mdx b/src/docs/environment/index.mdx index 5e2b05de67..9e7f0f297c 100644 --- a/src/docs/environment/index.mdx +++ b/src/docs/environment/index.mdx @@ -49,6 +49,55 @@ You can create other users with `sentry createuser`. displayed correctly. +### Frontend Only & Backend Only + +Please refer to [Frontend Development Server](/frontend/development-server/) and [Backend Development Server](/backend/development-server/) for alternative ways to bring up the Sentry UI. + +### Enabling HTTPS + +You may wish to run the development server in HTTPS mode. This can be done by generating and installing local certificates. + +We will be using [mkcert](https://github.com/FiloSottile/mkcert) to create and install a locally-trusted, development certificate. The following will install `mkcert` and then create and install the local certificates. + +```shell +brew install mkcert +brew install nss # if you use Firefox +yarn mkcert-localhost +``` + +Running `sentry devserver` will automatically use HTTPS when the certificates have been installed. + + +### Ingestion Pipeline (Relay) + +Some services are not run in all situations. Among those are Relay and the ingest workers. + +If you need to ingest errors, but don't require metrics ingestion: Set `SENTRY_USE_RELAY=True` in `~/.sentry/sentry.conf.py`. If `sentry devservices` is currently up, make sure to restart it after you make the change. This will launch Relay as part of the `devserver` workflow. + +Additionally, you can explicitly control this during `devserver` usage with the `--ingest` and `--no-ingest` flags. The `sentry devservices` command will not update Relay automatically in that case, to do this manually run: + +```shell +sentry devservices up --skip-only-if relay +sentry devserver --workers --ingest +``` + +If you want to enable the entire metrics ingestion pipeline: You need to add the following to your config at `~/.sentry/sentry.conf.py`: + +```python +SENTRY_USE_RELAY = True +SENTRY_USE_METRICS_DEV = True +SENTRY_EVENTSTREAM = "sentry.eventstream.kafka.KafkaEventStream" +SENTRY_FEATURES['organizations:metrics-extraction'] = True # Enables session metrics +SENTRY_FEATURES['organizations:transaction-metrics-extraction'] = True # Enables transaction metrics +``` + +After enabling `KafkaEventStream` the `snuba` service has to be reset to pick up the new configuration: + +```shell +sentry devservices rm snuba +sentry devservices up snuba +``` + ## Running siloed instances By default `sentry devserver` will run a monolith mode application server. You can also run ``devserver`` with siloed application instances. Before you do, you need to create split silo databases. @@ -72,24 +121,20 @@ In the above setup your local environment will use org slug domains, and send re ### Ngrok and siloed servers -To combine ngrok and local development servers you’ll need to reserve multiple domains in ngrok, and create a configuration file for ngrok: +To combine ngrok and local development servers you’ll need to reserve two domains in ngrok, and create a configuration file for ngrok: ```yaml version: '2' authtoken: tunnels: - acme-org: - proto: http - hostname: acme..ngrok.dev - addr: 8000 control-silo: proto: http - hostname: .ngrok.dev + hostname: $yourname.ngrok.dev host_header: 'rewrite' addr: 8000 region-silo: proto: http - hostname: us..ngrok.dev + hostname: us.$yourname.ngrok.dev addr: 8010 host_header: 'rewrite' ``` @@ -98,16 +143,16 @@ Then run all the required servers ```shell # Run a control silo with ngrok -sentry devserver --silo=control --ngrok .ngrok.dev +sentry devserver --silo=control --ngrok $yourname.ngrok.dev # Run a region silo without ngrok -sentry devserver --silo=region --ngrok .ngrok.dev +sentry devserver --silo=region --ngrok $yourname.ngrok.dev # Run ngrok ngrok start --all --config regions.yml ``` -This setup will result in both the region and control servers responding to different domains. The multi-region setup with ngrok also enables customer-domains and you'll need ngrok domains for each organization you plan on using. In this configuration, CORS will work similar to production. For ngrok setup with non-siloed development server see developement server. +This setup will result in both the region and control servers responding to different domains, and CORS will work similar to production. ### Siloed Django Shell @@ -152,60 +197,10 @@ environment, you can create `getsentry/conf/settings/devlocal.py` and put the configuration option overrides there. This module will be automatically imported by `dev.py` if it exists. - -### Frontend Only & Backend Only - -Please refer to [Frontend Development Server](/frontend/development-server/) and [Backend Development Server](/backend/development-server/) for alternative ways to bring up the Sentry UI. - -### Enabling HTTPS - -You may wish to run the development server in HTTPS mode. This can be done by generating and installing local certificates. - -We will be using [mkcert](https://github.com/FiloSottile/mkcert) to create and install a locally-trusted, development certificate. The following will install `mkcert` and then create and install the local certificates. - -```shell -brew install mkcert -brew install nss # if you use Firefox -yarn mkcert-localhost -``` - -Running `sentry devserver` will automatically use HTTPS when the certificates have been installed. - - -### Ingestion Pipeline (Relay) - -Some services are not run in all situations. Among those are Relay and the ingest workers. If you need -a more production-like environment in development, you can set `SENTRY_USE_RELAY=True` in `~/.sentry/sentry.conf.py`. If `sentry devservices` is currently up ,make sure to restart it after you make the change. This will launch Relay -as part of the `devserver` workflow. - -Additionally, you can explicitly control this during `devserver` usage with the `--ingest` and `--no-ingest` flags. The `sentry devservices` -command will not update Relay automatically in that case, to do this manually run: - -```shell -sentry devservices up --skip-only-if relay -sentry devserver --workers --ingest +To enable the ingest workers, follow the steps described here and run ```shell +getsentry devserver --workers --ingest ``` -If you want to enable the entire metrics ingestion pipeline, you need to add the following to your config at `~/.sentry/sentry.conf.py`: - -```python -SENTRY_USE_RELAY = True -SENTRY_USE_METRICS_DEV = True -SENTRY_EVENTSTREAM = "sentry.eventstream.kafka.KafkaEventStream" -SENTRY_FEATURES['organizations:metrics-extraction'] = True # Enables session metrics -SENTRY_FEATURES['organizations:transaction-metrics-extraction'] = True # Enables transaction metrics -``` - -After enabling `KafkaEventStream` the `snuba` service has to be reset to pick up the new configuration: - -```shell -sentry devservices rm snuba -sentry devservices up snuba -``` - - -## Troubleshooting - You might also be interested in troubleshooting CI. --- From 1dc2953e061a3010e967473f97d2616c5096cf54 Mon Sep 17 00:00:00 2001 From: Michelle Fu Date: Tue, 9 Jul 2024 14:00:12 -0700 Subject: [PATCH 2/8] fix copy paste errors --- src/docs/environment/index.mdx | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/src/docs/environment/index.mdx b/src/docs/environment/index.mdx index 9e7f0f297c..ecec7627be 100644 --- a/src/docs/environment/index.mdx +++ b/src/docs/environment/index.mdx @@ -67,14 +67,15 @@ yarn mkcert-localhost Running `sentry devserver` will automatically use HTTPS when the certificates have been installed. - ### Ingestion Pipeline (Relay) -Some services are not run in all situations. Among those are Relay and the ingest workers. +Some services are not run in all situations. Among those are Relay and the ingest workers. -If you need to ingest errors, but don't require metrics ingestion: Set `SENTRY_USE_RELAY=True` in `~/.sentry/sentry.conf.py`. If `sentry devservices` is currently up, make sure to restart it after you make the change. This will launch Relay as part of the `devserver` workflow. +If you need to ingest errors, but don't require metrics ingestion: Set `SENTRY_USE_RELAY=True` in `~/.sentry/sentry.conf.py`. If `sentry devservices` +is currently up, make sure to restart it after you make the change. This will launch Relay as part of the `devserver` workflow. -Additionally, you can explicitly control this during `devserver` usage with the `--ingest` and `--no-ingest` flags. The `sentry devservices` command will not update Relay automatically in that case, to do this manually run: +Additionally, you can explicitly control this during `devserver` usage with the `--ingest` and `--no-ingest` flags. The `sentry devservices` +command will not update Relay automatically in that case, to do this manually run: ```shell sentry devservices up --skip-only-if relay @@ -121,20 +122,24 @@ In the above setup your local environment will use org slug domains, and send re ### Ngrok and siloed servers -To combine ngrok and local development servers you’ll need to reserve two domains in ngrok, and create a configuration file for ngrok: +To combine ngrok and local development servers you’ll need to reserve multiple domains in ngrok, and create a configuration file for ngrok: ```yaml version: '2' authtoken: tunnels: + acme-org: + proto: http + hostname: acme..ngrok.dev + addr: 8000 control-silo: proto: http - hostname: $yourname.ngrok.dev + hostname: .ngrok.dev host_header: 'rewrite' addr: 8000 region-silo: proto: http - hostname: us.$yourname.ngrok.dev + hostname: us..ngrok.dev addr: 8010 host_header: 'rewrite' ``` @@ -143,16 +148,16 @@ Then run all the required servers ```shell # Run a control silo with ngrok -sentry devserver --silo=control --ngrok $yourname.ngrok.dev +sentry devserver --silo=control --ngrok .ngrok.dev # Run a region silo without ngrok -sentry devserver --silo=region --ngrok $yourname.ngrok.dev +sentry devserver --silo=region --ngrok .ngrok.dev # Run ngrok ngrok start --all --config regions.yml ``` -This setup will result in both the region and control servers responding to different domains, and CORS will work similar to production. +This setup will result in both the region and control servers responding to different domains. The multi-region setup with ngrok also enables customer-domains and you'll need ngrok domains for each organization you plan on using. In this configuration, CORS will work similar to production. For ngrok setup with non-siloed development server see developement server. ### Siloed Django Shell @@ -199,7 +204,8 @@ by `dev.py` if it exists. To enable the ingest workers, follow the steps described here and run ```shell getsentry devserver --workers --ingest -``` + +## Troubleshooting You might also be interested in troubleshooting CI. From 91898ec64ddc1aeaf10a4f67cfda0c644bf2e3f5 Mon Sep 17 00:00:00 2001 From: Michelle Fu Date: Tue, 9 Jul 2024 14:01:26 -0700 Subject: [PATCH 3/8] fix link location --- src/docs/environment/index.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/docs/environment/index.mdx b/src/docs/environment/index.mdx index ecec7627be..801ae38da8 100644 --- a/src/docs/environment/index.mdx +++ b/src/docs/environment/index.mdx @@ -202,7 +202,7 @@ environment, you can create `getsentry/conf/settings/devlocal.py` and put the configuration option overrides there. This module will be automatically imported by `dev.py` if it exists. -To enable the ingest workers, follow the steps described here and run ```shell +To enable the ingest workers, follow the steps described here and run ```shell getsentry devserver --workers --ingest ## Troubleshooting From 0b3456d1ad040e906746fec00eb52fac7646088b Mon Sep 17 00:00:00 2001 From: Michelle Fu Date: Tue, 9 Jul 2024 14:08:03 -0700 Subject: [PATCH 4/8] fix link location again --- src/docs/environment/index.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/docs/environment/index.mdx b/src/docs/environment/index.mdx index 801ae38da8..c2dd53be02 100644 --- a/src/docs/environment/index.mdx +++ b/src/docs/environment/index.mdx @@ -202,7 +202,7 @@ environment, you can create `getsentry/conf/settings/devlocal.py` and put the configuration option overrides there. This module will be automatically imported by `dev.py` if it exists. -To enable the ingest workers, follow the steps described here and run ```shell +To enable the ingest workers, follow the steps described here and run ```shell getsentry devserver --workers --ingest ## Troubleshooting From 7fbf95e3a460ed1b2c7fbcf28430bbbd74a7852d Mon Sep 17 00:00:00 2001 From: Michelle Fu Date: Tue, 9 Jul 2024 14:10:47 -0700 Subject: [PATCH 5/8] fix missing backticks --- src/docs/environment/index.mdx | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/docs/environment/index.mdx b/src/docs/environment/index.mdx index c2dd53be02..7f7d9c1578 100644 --- a/src/docs/environment/index.mdx +++ b/src/docs/environment/index.mdx @@ -202,8 +202,9 @@ environment, you can create `getsentry/conf/settings/devlocal.py` and put the configuration option overrides there. This module will be automatically imported by `dev.py` if it exists. -To enable the ingest workers, follow the steps described here and run ```shell -getsentry devserver --workers --ingest +To enable the ingest workers, follow the steps described here and run +```shell +getsentry devserver --workers --ingest``` ## Troubleshooting From 5f7abec2e76b685ab08076e3de7b1af1b7ec72e4 Mon Sep 17 00:00:00 2001 From: Michelle Fu Date: Tue, 9 Jul 2024 14:25:04 -0700 Subject: [PATCH 6/8] move getsentry instructions above siloed instance instructions --- src/docs/environment/index.mdx | 71 +++++++++++++++++----------------- 1 file changed, 36 insertions(+), 35 deletions(-) diff --git a/src/docs/environment/index.mdx b/src/docs/environment/index.mdx index 7f7d9c1578..43c7a916bf 100644 --- a/src/docs/environment/index.mdx +++ b/src/docs/environment/index.mdx @@ -99,6 +99,42 @@ sentry devservices rm snuba sentry devservices up snuba ``` +## Running the Getsentry Development Server + + + Only Sentry employees have access to getsentry. + + +See also: Sentry vs Getsentry + +Just like running `sentry` (see above), you can start the `devservices` using the following command in the `getsentry` folder: + +```shell +sentry devservices up +``` + +After that, you can start the development server inside the `getsentry` folder: + +```shell +getsentry devserver --workers +``` + +**Note**: You **cannot** have both sentry and getsentry devserver running at the same time. + +After the server warms up for a little while, you must access it +at [http://dev.getsentry.net:8000](http://dev.getsentry.net:8000/). +Using localhost doesn't work. + +If you need to overwrite configuration options for your local +environment, you can create `getsentry/conf/settings/devlocal.py` and put the +configuration option overrides there. This module will be automatically imported +by `dev.py` if it exists. + +To enable the ingest workers, follow the steps described here and run +```shell +getsentry devserver --workers --ingest``` + + ## Running siloed instances By default `sentry devserver` will run a monolith mode application server. You can also run ``devserver`` with siloed application instances. Before you do, you need to create split silo databases. @@ -171,41 +207,6 @@ SENTRY_SILO_DEVSERVER=1 SENTRY_SILO_MODE=CONTROL getsentry django shell SENTRY_SILO_DEVSERVER=1 SENTRY_SILO_MODE=REGION SENTRY_REGION=us getsentry django shell ``` -## Running the Getsentry Development Server - - - Only Sentry employees have access to getsentry. - - -See also: Sentry vs Getsentry - -Just like running `sentry` (see above), you can start the `devservices` using the following command in the `getsentry` folder: - -```shell -sentry devservices up -``` - -After that, you can start the development server inside the `getsentry` folder: - -```shell -getsentry devserver --workers -``` - -**Note**: You **cannot** have both sentry and getsentry devserver running at the same time. - -After the server warms up for a little while, you must access it -at [http://dev.getsentry.net:8000](http://dev.getsentry.net:8000/). -Using localhost doesn't work. - -If you need to overwrite configuration options for your local -environment, you can create `getsentry/conf/settings/devlocal.py` and put the -configuration option overrides there. This module will be automatically imported -by `dev.py` if it exists. - -To enable the ingest workers, follow the steps described here and run -```shell -getsentry devserver --workers --ingest``` - ## Troubleshooting You might also be interested in troubleshooting CI. From 13c9f7201716b6bce4b4abe1a20e7ecd18a34158 Mon Sep 17 00:00:00 2001 From: Michelle Fu Date: Tue, 9 Jul 2024 14:26:22 -0700 Subject: [PATCH 7/8] bug --- src/docs/environment/index.mdx | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/docs/environment/index.mdx b/src/docs/environment/index.mdx index 43c7a916bf..935a582f87 100644 --- a/src/docs/environment/index.mdx +++ b/src/docs/environment/index.mdx @@ -132,7 +132,8 @@ by `dev.py` if it exists. To enable the ingest workers, follow the steps described here and run ```shell -getsentry devserver --workers --ingest``` +getsentry devserver --workers --ingest +``` ## Running siloed instances From c9e534856fd7b6880848bbf4ba0ec47e6653e802 Mon Sep 17 00:00:00 2001 From: Michelle Fu Date: Tue, 9 Jul 2024 14:58:17 -0700 Subject: [PATCH 8/8] properly bold text --- src/docs/environment/index.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/docs/environment/index.mdx b/src/docs/environment/index.mdx index 935a582f87..f12dd79681 100644 --- a/src/docs/environment/index.mdx +++ b/src/docs/environment/index.mdx @@ -71,7 +71,7 @@ Running `sentry devserver` will automatically use HTTPS when the certificates ha Some services are not run in all situations. Among those are Relay and the ingest workers. -If you need to ingest errors, but don't require metrics ingestion: Set `SENTRY_USE_RELAY=True` in `~/.sentry/sentry.conf.py`. If `sentry devservices` +**If you need to ingest errors, but don't require metrics ingestion:** Set `SENTRY_USE_RELAY=True` in `~/.sentry/sentry.conf.py`. If `sentry devservices` is currently up, make sure to restart it after you make the change. This will launch Relay as part of the `devserver` workflow. Additionally, you can explicitly control this during `devserver` usage with the `--ingest` and `--no-ingest` flags. The `sentry devservices` @@ -82,7 +82,7 @@ sentry devservices up --skip-only-if relay sentry devserver --workers --ingest ``` -If you want to enable the entire metrics ingestion pipeline: You need to add the following to your config at `~/.sentry/sentry.conf.py`: +**If you want to enable the entire metrics ingestion pipeline:** You need to add the following to your config at `~/.sentry/sentry.conf.py`: ```python SENTRY_USE_RELAY = True