Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Trace Analytics support for custom sources #2112

Merged
merged 11 commits into from
Sep 4, 2024
Merged

Trace Analytics support for custom sources #2112

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
9799886
improve traces table
joshuali925 Aug 8, 2024
38488d6
improve queries
joshuali925 Aug 8, 2024
dd421e0
add support for custom trace indices
ps48 Sep 3, 2024
e276980
update mode picker
ps48 Sep 3, 2024
361213d
added services view for custom trace source
ps48 Sep 4, 2024
9341e38
updated snapshots
ps48 Sep 4, 2024
6a5a7c8
update DSL filter tests
ps48 Sep 4, 2024
fb6cf82
update app analytics snapshots
ps48 Sep 4, 2024
51317ad
remove console.log
ps48 Sep 4, 2024
55e7e62
update service map inputs, remove commented code
ps48 Sep 4, 2024
850a8c8
remove console.log, update copy and tests
ps48 Sep 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 11 additions & 5 deletions common/constants/trace_analytics.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,20 @@ export const JAEGER_INDEX_NAME = '*jaeger-span-*';
export const JAEGER_SERVICE_INDEX_NAME = '*jaeger-service*';
export const DATA_PREPPER_INDEX_NAME = 'otel-v1-apm-span-*';
export const DATA_PREPPER_SERVICE_INDEX_NAME = 'otel-v1-apm-service-map*';
export const TRACE_ANALYTICS_DATE_FORMAT = 'MM/DD/YYYY HH:mm:ss';
export const TRACE_ANALYTICS_PLOTS_DATE_FORMAT = 'MMM D, YYYY HH:mm:ss';
export const TRACE_ANALYTICS_DATE_FORMAT = 'MM/DD/YYYY HH:mm:ss.SSS';
export const TRACE_ANALYTICS_PLOTS_DATE_FORMAT = 'MMM D, YYYY HH:mm:ss.SSS';
export const SERVICE_MAP_MAX_NODES = 500;
// size limit when requesting edge related queries, not necessarily the number of edges
export const SERVICE_MAP_MAX_EDGES = 1000;
export const TRACES_MAX_NUM = 3000;
export const TRACE_ANALYTICS_DOCUMENTATION_LINK = 'https://opensearch.org/docs/latest/observability-plugin/trace/index/';
export const TRACE_ANALYTICS_DOCUMENTATION_LINK =
'https://opensearch.org/docs/latest/observability-plugin/trace/index/';

export const TRACE_ANALYTICS_JAEGER_INDICES_ROUTE = '/api/observability/trace_analytics/jaeger_indices';
export const TRACE_ANALYTICS_DATA_PREPPER_INDICES_ROUTE = '/api/observability/trace_analytics/data_prepper_indices';
export const TRACE_ANALYTICS_JAEGER_INDICES_ROUTE =
'/api/observability/trace_analytics/jaeger_indices';
export const TRACE_ANALYTICS_DATA_PREPPER_INDICES_ROUTE =
'/api/observability/trace_analytics/data_prepper_indices';
export const TRACE_ANALYTICS_DSL_ROUTE = '/api/observability/trace_analytics/query';

export const TRACE_CUSTOM_SPAN_INDEX_SETTING = 'observability:traceAnalyticsSpanIndices';
export const TRACE_CUSTOM_SERVICE_INDEX_SETTING = 'observability:traceAnalyticsServiceIndices';
2 changes: 2 additions & 0 deletions common/types/trace_analytics.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,3 +52,5 @@ export interface GraphVisEdge {
to: number;
color: string;
}

export type TraceAnalyticsMode = 'jaeger' | 'data_prepper' | 'custom_data_prepper';
Binary file removed public/components/.DS_Store
View file
Open in desktop
Binary file not shown.
384 changes: 160 additions & 224 deletions public/components/application_analytics/__tests__/__snapshots__/create.test.tsx.snap
View file
Open in desktop

Large diffs are not rendered by default.

82 changes: 38 additions & 44 deletions public/components/application_analytics/__tests__/__snapshots__/service_config.test.tsx.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -986,30 +986,16 @@ exports[`Service Config component renders empty service config 1`] = `
/>
</EuiHorizontalRule>
<EuiFlexGroup
alignItems="center"
gutterSize="s"
justifyContent="spaceBetween"
>
<div
className="euiFlexGroup euiFlexGroup--gutterSmall euiFlexGroup--alignItemsCenter euiFlexGroup--directionRow euiFlexGroup--responsive"
className="euiFlexGroup euiFlexGroup--gutterLarge euiFlexGroup--justifyContentSpaceBetween euiFlexGroup--directionRow euiFlexGroup--responsive"
>
<EuiFlexItem
grow={false}
grow={7}
>
<div
className="euiFlexItem euiFlexItem--flexGrowZero"
>
<EuiText>
<div
className="euiText euiText--medium"
>
Focus on
</div>
</EuiText>
</div>
</EuiFlexItem>
<EuiFlexItem>
<div
className="euiFlexItem"
className="euiFlexItem euiFlexItem--flexGrow7"
>
<EuiCompressedFieldSearch
compressed={true}
Expand All @@ -1021,25 +1007,36 @@ exports[`Service Config component renders empty service config 1`] = `
onChange={[Function]}
onSearch={[Function]}
placeholder="Service name"
prepend="Focus on"
value=""
>
<EuiFormControlLayout
compressed={true}
fullWidth={false}
icon="search"
isLoading={false}
prepend="Focus on"
>
<div
className="euiFormControlLayout euiFormControlLayout--compressed"
className="euiFormControlLayout euiFormControlLayout--compressed euiFormControlLayout--group"
>
<EuiFormLabel
className="euiFormControlLayout__prepend"
>
<label
className="euiFormLabel euiFormControlLayout__prepend"
>
Focus on
</label>
</EuiFormLabel>
<div
className="euiFormControlLayout__childrenWrapper"
>
<EuiValidatableControl
isInvalid={false}
>
<input
className="euiFieldSearch euiFieldSearch--compressed"
className="euiFieldSearch euiFieldSearch--compressed euiFieldSearch--inGroup"
onChange={[Function]}
onKeyUp={[Function]}
placeholder="Service name"
Expand Down Expand Up @@ -1196,10 +1193,10 @@ exports[`Service Config component renders empty service config 1`] = `
</div>
</EuiPanel>
<EuiSpacer
size="xl"
size="m"
>
<div
className="euiSpacer euiSpacer--xl"
className="euiSpacer euiSpacer--m"
/>
</EuiSpacer>
</ServiceMap>
Expand Down Expand Up @@ -2215,30 +2212,16 @@ exports[`Service Config component renders with one service selected 1`] = `
/>
</EuiHorizontalRule>
<EuiFlexGroup
alignItems="center"
gutterSize="s"
justifyContent="spaceBetween"
>
<div
className="euiFlexGroup euiFlexGroup--gutterSmall euiFlexGroup--alignItemsCenter euiFlexGroup--directionRow euiFlexGroup--responsive"
className="euiFlexGroup euiFlexGroup--gutterLarge euiFlexGroup--justifyContentSpaceBetween euiFlexGroup--directionRow euiFlexGroup--responsive"
>
<EuiFlexItem
grow={false}
grow={7}
>
<div
className="euiFlexItem euiFlexItem--flexGrowZero"
>
<EuiText>
<div
className="euiText euiText--medium"
>
Focus on
</div>
</EuiText>
</div>
</EuiFlexItem>
<EuiFlexItem>
<div
className="euiFlexItem"
className="euiFlexItem euiFlexItem--flexGrow7"
>
<EuiCompressedFieldSearch
compressed={true}
Expand All @@ -2250,25 +2233,36 @@ exports[`Service Config component renders with one service selected 1`] = `
onChange={[Function]}
onSearch={[Function]}
placeholder="Service name"
prepend="Focus on"
value=""
>
<EuiFormControlLayout
compressed={true}
fullWidth={false}
icon="search"
isLoading={false}
prepend="Focus on"
>
<div
className="euiFormControlLayout euiFormControlLayout--compressed"
className="euiFormControlLayout euiFormControlLayout--compressed euiFormControlLayout--group"
>
<EuiFormLabel
className="euiFormControlLayout__prepend"
>
<label
className="euiFormLabel euiFormControlLayout__prepend"
>
Focus on
</label>
</EuiFormLabel>
<div
className="euiFormControlLayout__childrenWrapper"
>
<EuiValidatableControl
isInvalid={false}
>
<input
className="euiFieldSearch euiFieldSearch--compressed"
className="euiFieldSearch euiFieldSearch--compressed euiFieldSearch--inGroup"
onChange={[Function]}
onKeyUp={[Function]}
placeholder="Service name"
Expand Down Expand Up @@ -2425,10 +2419,10 @@ exports[`Service Config component renders with one service selected 1`] = `
</div>
</EuiPanel>
<EuiSpacer
size="xl"
size="m"
>
<div
className="euiSpacer euiSpacer--xl"
className="euiSpacer euiSpacer--m"
/>
</EuiSpacer>
</ServiceMap>
Expand Down
4 changes: 1 addition & 3 deletions public/components/application_analytics/components/application.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@
const [serviceFlyoutName, setServiceFlyoutName] = useState<string>('');
const [traceFlyoutId, setTraceFlyoutId] = useState<string>('');
const [spanFlyoutId, setSpanFlyoutId] = useState<string>('');
const [spanDSL, setSpanDSL] = useState<any>({});

Check warning on line 137 in public/components/application_analytics/components/application.tsx

View workflow job for this annotation

GitHub Actions / Lint 

Unexpected any. Specify a different type
const [totalSpans, setTotalSpans] = useState<number>(0);
const [editVizId, setEditVizId] = useState<string>('');
const [visWithAvailability, setVisWithAvailability] = useState<EuiSelectOption[]>([]);
Expand All @@ -157,7 +157,7 @@
sessionStorage.setItem(`${application.name}EndTime`, newEndTime);
};

const addSpanFilter = (field: string, value: any) => {

Check warning on line 160 in public/components/application_analytics/components/application.tsx

View workflow job for this annotation

GitHub Actions / Lint 

Unexpected any. Specify a different type
const newFilters = [...filters];
const index = newFilters.findIndex(({ field: filterField }) => field === filterField);
if (index === -1) {
Expand Down Expand Up @@ -286,7 +286,7 @@
},
];

const nameColumnAction = (item: any) => openServiceFlyout(item);

Check warning on line 289 in public/components/application_analytics/components/application.tsx

View workflow job for this annotation

GitHub Actions / Lint 

Unexpected any. Specify a different type
const traceColumnAction = () => switchToTrace();

const getService = () => {
Expand Down Expand Up @@ -314,8 +314,6 @@
setSelectedTab(TAB_TRACE_ID);
};

const traceIdColumnAction = (item: any) => openTraceFlyout(item);

const getTrace = () => {
return (
<>
Expand All @@ -325,7 +323,7 @@
page="app"
parentBreadcrumb={parentBreadcrumbs[0]}
childBreadcrumbs={childBreadcrumbs}
traceIdColumnAction={traceIdColumnAction}
openTraceFlyout={openTraceFlyout}
startTime={appStartTime}
endTime={appEndTime}
setStartTime={setStartTimeForApp}
Expand Down
2 changes: 1 addition & 1 deletion ...c/components/application_analytics/components/flyout_components/service_detail_flyout.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,10 +49,10 @@
openSpanFlyout,
mode,
} = props;
const [fields, setFields] = useState<any>({});

Check warning on line 52 in public/components/application_analytics/components/flyout_components/service_detail_flyout.tsx

View workflow job for this annotation

GitHub Actions / Lint 

Unexpected any. Specify a different type
const [serviceMap, setServiceMap] = useState<ServiceObject>({});
const [total, setTotal] = useState(0);
const [DSL, setDSL] = useState<any>({});

Check warning on line 55 in public/components/application_analytics/components/flyout_components/service_detail_flyout.tsx

View workflow job for this annotation

GitHub Actions / Lint 

Unexpected any. Specify a different type
const [serviceMapIdSelected, setServiceMapIdSelected] = useState<
'latency' | 'error_rate' | 'throughput'
>('latency');
Expand Down Expand Up @@ -133,7 +133,7 @@
handleServiceViewRequest(serviceName, http, serviceDSL, setFields, mode);
handleServiceMapRequest(http, serviceDSL, mode, '', setServiceMap, serviceName);
const spanDSL = filtersToDsl(mode, filters, query, startTime, endTime, 'app', appConfigs);
spanDSL.query.bool.must.push({
spanDSL.query.bool.filter.push({
term: {
serviceName,
},
Expand Down
19 changes: 12 additions & 7 deletions public/components/application_analytics/components/flyout_components/trace_detail_render.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,28 +3,33 @@
* SPDX-License-Identifier: Apache-2.0
*/

import { EuiText, EuiSpacer, EuiHorizontalRule, EuiCodeBlock } from '@elastic/eui';
import { EuiCodeBlock, EuiHorizontalRule, EuiSpacer, EuiText } from '@elastic/eui';
import React, { useEffect, useMemo, useState } from 'react';
import { HttpStart } from '../../../../../../../src/core/public';
import { TraceAnalyticsMode } from '../../../../../common/types/trace_analytics';
import { ServiceBreakdownPanel } from '../../../trace_analytics/components/traces/service_breakdown_panel';
import { SpanDetailPanel } from '../../../trace_analytics/components/traces/span_detail_panel';
import {
handleTraceViewRequest,
handleServicesPieChartRequest,
handlePayloadRequest,
handleServicesPieChartRequest,
handleTraceViewRequest,
} from '../../../trace_analytics/requests/traces_request_handler';
import { HttpStart } from '../../../../../../../src/core/public';
import { getListItem } from '../../helpers/utils';
import { TraceAnalyticsMode } from '../../../../../public/components/trace_analytics/home';

interface TraceDetailRenderProps {
traceId: string;
http: HttpStart;
openSpanFlyout: (spanId: string) => void;
mode : TraceAnalyticsMode
mode: TraceAnalyticsMode;
}

export const TraceDetailRender = ({ traceId, http, openSpanFlyout, mode }: TraceDetailRenderProps) => {
export const TraceDetailRender = ({
traceId,
http,
openSpanFlyout,
mode,
}: TraceDetailRenderProps) => {
const [fields, setFields] = useState<any>({});

Check warning on line 32 in public/components/application_analytics/components/flyout_components/trace_detail_render.tsx

View workflow job for this annotation

GitHub Actions / Lint 

Unexpected any. Specify a different type
const [serviceBreakdownData, setServiceBreakdownData] = useState([]);
const [payloadData, setPayloadData] = useState('');
const [colorMap, setColorMap] = useState({});
Expand Down Expand Up @@ -83,13 +88,13 @@
) : null}
</>
);
}, [traceId, fields, serviceBreakdownData, colorMap, payloadData]);

Check warning on line 91 in public/components/application_analytics/components/flyout_components/trace_detail_render.tsx

View workflow job for this annotation

GitHub Actions / Lint 

React Hook useMemo has missing dependencies: 'http', 'mode', and 'openSpanFlyout'. Either include them or remove the dependency array

useEffect(() => {
handleTraceViewRequest(traceId, http, fields, setFields, mode);
handleServicesPieChartRequest(traceId, http, setServiceBreakdownData, setColorMap, mode);
handlePayloadRequest(traceId, http, payloadData, setPayloadData, mode);
}, [traceId]);

Check warning on line 97 in public/components/application_analytics/components/flyout_components/trace_detail_render.tsx

View workflow job for this annotation

GitHub Actions / Lint 

React Hook useEffect has missing dependencies: 'fields', 'http', 'mode', and 'payloadData'. Either include them or remove the dependency array

return renderContent;
};
13 changes: 7 additions & 6 deletions public/components/trace_analytics/components/common/__tests__/helper_functions.test.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@

import { configure, mount, shallow } from 'enzyme';
import Adapter from 'enzyme-adapter-react-16';
import { TraceAnalyticsMode } from 'public/components/trace_analytics/home';
import React from 'react';
import { TEST_SERVICE_MAP, TEST_SERVICE_MAP_GRAPH } from '../../../../../../test/constants';
import {
Expand Down Expand Up @@ -38,7 +37,9 @@ describe('Helper functions', () => {

it('renders no match and missing configuration messages', () => {
const noMatchMessage = shallow(<NoMatchMessage size="s" />);
const missingConfigurationMessage = shallow(<MissingConfigurationMessage mode='data_prepper'/>)
const missingConfigurationMessage = shallow(
<MissingConfigurationMessage mode="data_prepper" />
);
expect(noMatchMessage).toMatchSnapshot();
expect(missingConfigurationMessage).toMatchSnapshot();
});
Expand Down Expand Up @@ -137,16 +138,16 @@ describe('Helper functions', () => {
);
const existsDSL = getTestDslFromFilters();
expect(JSON.stringify(existsDSL)).toEqual(
'{"query":{"bool":{"must":[{"range":{"startTime":{"gte":"now-5m","lte":"now"}}},{"query_string":{"query":"order"}},{"exists":{"field":"traceGroup"}}],"filter":[],"should":[],"must_not":[]}},"custom":{"timeFilter":[{"range":{"startTime":{"gte":"now-5m","lte":"now"}}}],"serviceNames":[],"serviceNamesExclude":[],"traceGroup":[],"traceGroupExclude":[],"percentiles":{"query":{"bool":{"should":[]}}}}}'
'{"query":{"bool":{"must":[{"exists":{"field":"traceGroup"}}],"filter":[{"range":{"startTime":{"gte":"now-5m","lte":"now"}}},{"query_string":{"query":"order"}}],"should":[],"must_not":[]}},"custom":{"timeFilter":[{"range":{"startTime":{"gte":"now-5m","lte":"now"}}}],"serviceNames":[],"serviceNamesExclude":[],"traceGroup":[],"traceGroupExclude":[],"percentiles":{"query":{"bool":{"should":[]}}}}}'
);

const isDSL = getTestDslFromFilters('traceGroup', 'is');
expect(JSON.stringify(isDSL)).toEqual(
'{"query":{"bool":{"must":[{"range":{"startTime":{"gte":"now-5m","lte":"now"}}},{"query_string":{"query":"order"}},{"term":{"traceGroup":{"from":"100","to":"∞"}}}],"filter":[],"should":[],"must_not":[]}},"custom":{"timeFilter":[{"range":{"startTime":{"gte":"now-5m","lte":"now"}}}],"serviceNames":[],"serviceNamesExclude":[],"traceGroup":[],"traceGroupExclude":[],"percentiles":{"query":{"bool":{"should":[]}}}}}'
'{"query":{"bool":{"must":[{"term":{"traceGroup":{"from":"100","to":"∞"}}}],"filter":[{"range":{"startTime":{"gte":"now-5m","lte":"now"}}},{"query_string":{"query":"order"}}],"should":[],"must_not":[]}},"custom":{"timeFilter":[{"range":{"startTime":{"gte":"now-5m","lte":"now"}}}],"serviceNames":[],"serviceNamesExclude":[],"traceGroup":[],"traceGroupExclude":[],"percentiles":{"query":{"bool":{"should":[]}}}}}'
);
const isBetweenDSL = getTestDslFromFilters('durationInNanos', 'is between');
expect(JSON.stringify(isBetweenDSL)).toEqual(
`{"query":{"bool":{"must":[{"range":{"startTime":{"gte":"now-5m","lte":"now"}}},{"query_string":{"query":"order"}},{"range":{"durationInNanos":{"gte":"100"}}}],"filter":[],"should":[],"must_not":[]}},"custom":{"timeFilter":[{"range":{"startTime":{"gte":"now-5m","lte":"now"}}}],"serviceNames":[],"serviceNamesExclude":[],"traceGroup":[],"traceGroupExclude":[],"percentiles":{"query":{"bool":{"should":[]}}}}}`
'{"query":{"bool":{"must":[{"range":{"durationInNanos":{"gte":"100"}}}],"filter":[{"range":{"startTime":{"gte":"now-5m","lte":"now"}}},{"query_string":{"query":"order"}}],"should":[],"must_not":[]}},"custom":{"timeFilter":[{"range":{"startTime":{"gte":"now-5m","lte":"now"}}}],"serviceNames":[],"serviceNamesExclude":[],"traceGroup":[],"traceGroupExclude":[],"percentiles":{"query":{"bool":{"should":[]}}}}}'
);

const customDSL = filtersToDsl(
Expand All @@ -166,7 +167,7 @@ describe('Helper functions', () => {
'now'
);
expect(JSON.stringify(customDSL)).toEqual(
`{"query":{"bool":{"must":[{"range":{"startTime":{"gte":"now-5m","lte":"now"}}},{"query_string":{"query":"order"}}],"filter":[],"should":["test"],"must_not":[],"minimum_should_match":1}},"custom":{"timeFilter":[{"range":{"startTime":{"gte":"now-5m","lte":"now"}}}],"serviceNames":[],"serviceNamesExclude":[],"traceGroup":[],"traceGroupExclude":[],"percentiles":{"query":{"bool":{"should":["test"],"minimum_should_match":1}}}}}`
'{"query":{"bool":{"must":[],"filter":[{"range":{"startTime":{"gte":"now-5m","lte":"now"}}},{"query_string":{"query":"order"}}],"should":["test"],"must_not":[],"minimum_should_match":1}},"custom":{"timeFilter":[{"range":{"startTime":{"gte":"now-5m","lte":"now"}}}],"serviceNames":[],"serviceNamesExclude":[],"traceGroup":[],"traceGroupExclude":[],"percentiles":{"query":{"bool":{"should":["test"],"minimum_should_match":1}}}}}'
);
});
});
Loading
Loading