Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fabric memory stats #295

Merged
merged 22 commits into from
Oct 23, 2024
Merged

Fabric memory stats #295

Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
dcd4aed
Dependency update (#286)
TApplencourt Sep 4, 2024
481dd72
Add archive (#287)
TApplencourt Sep 4, 2024
41248b2
Single rank profiling (#288)
TApplencourt Sep 17, 2024
767ae9c
fabricPort sampling
solo2abera May 23, 2024
b71a880
Fabric Timeline
solo2abera May 31, 2024
19fe26a
Sampling rewrite
solo2abera Jun 6, 2024
1484f94
Timeline cleanup
solo2abera Jun 13, 2024
1492018
include telemetry handles
solo2abera Jun 27, 2024
2de225b
uuid based timeline
solo2abera Jul 10, 2024
61d663e
Memory sampling
solo2abera Jul 23, 2024
e57f503
zes_support
Sep 18, 2024
44e66f7
rebased
Sep 25, 2024
578874a
Code cleanup and fix
Oct 3, 2024
fcd21fc
Update ze/btx_zeinterval_callbacks.cpp
sbekele81 Oct 4, 2024
ad21873
Update xprof/xprof.rb.in
sbekele81 Oct 4, 2024
6fb44fb
Update xprof/xprof.rb.in
sbekele81 Oct 4, 2024
50a28e6
PR corrections
Oct 16, 2024
d549063
separate sampling stream
Oct 18, 2024
720ccd0
Merge branch 'devel' into fabric_memory_stats
sbekele81 Oct 18, 2024
320a81e
Deltas and Hash-return handled
Oct 22, 2024
39b4cab
Remove Ze calls for subDevice
Oct 23, 2024
f7d207f
minor change on delta
Oct 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
211 changes: 121 additions & 90 deletions .github/workflows/presubmit.yml
View file
Open in desktop

Large diffs are not rendered by default.

4 changes: 4 additions & 0 deletions integration_tests/general.bats
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,10 @@ teardown_file() {
rm out.pftrace
}

@test "archive_summary" {
$IPROF --archive $THAPI_TEST_BIN
}

@test "replay_summary" {
$IPROF $THAPI_TEST_BIN
$IPROF -r
Expand Down
9 changes: 9 additions & 0 deletions utils/babeltrace_thapi.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,7 @@ def get_components(names)
components_classes = {
'source.ctf.fs' => BT2::BTPlugin.find('ctf').get_source_component_class_by_name('fs'),
'source.ctf.lttng_live' => BT2::BTPlugin.find('ctf').get_source_component_class_by_name('lttng-live'),
'source.ctf.lttng_archive' => BT2::BTPlugin.find('ctf').get_source_component_class_by_name('lttng-archive'),
'filter.utils.muxer' => BT2::BTPlugin.find('utils').get_filter_component_class_by_name('muxer'),
'sink.text.pretty' => BT2::BTPlugin.find('text').get_sink_component_class_by_name('pretty'),
'sink.ctf.fs' => BT2::BTPlugin.find('ctf').get_sink_component_class_by_name('fs'),
Expand Down Expand Up @@ -200,6 +201,10 @@ def get_and_add_components(graph, names, l_inputs)
graph.add(comp, 'source_live',
params: { 'inputs' => $options[:inputs],
'session-not-found-action' => 'end' })
when 'source.ctf.lttng_archive'
graph.add(comp, 'source_archive',
params: { 'session-name' => $options[:archive],
'session-found-file-path' => $options[:'archive-session-found-file-path'] })
when 'source.ctf.fs'
s = Find.find(*l_inputs)
.reject { |path| FileTest.directory?(path) }
Expand Down Expand Up @@ -281,6 +286,8 @@ def bt_graphs(inputs)
@bt_graphs[inputs] ||= begin
g_comps = [if $options[:live]
'source.ctf.lttng_live'
elsif $options[:archive]
'source.ctf.lttng_archive'
else
'source.ctf.fs'
end]
Expand Down Expand Up @@ -354,6 +361,8 @@ class BabeltraceParserThapi < OptionParserWithDefaultAndValidation
'Format: backend_name[:backend_level],...',
default: ['mpi:3', 'omp:2', 'cl:1', 'ze:1', 'cuda:1', 'hip:1'])
on('--debug', default: false)
on('--archive SESSION-NAME')
on('--archive-session-found-file-path PATH')
on('--[no-]muxer')
on('-v', '--version', 'Print the version string') do
puts File.read(File.join(DATADIR, 'version'))
Expand Down
9 changes: 6 additions & 3 deletions utils/xprof_utils.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,9 @@ typedef intptr_t process_id_t;
typedef uintptr_t thread_id_t;
typedef std::string hostname_t;
typedef std::string thapi_function_name;
typedef uintptr_t thapi_device_id;
typedef uint64_t thapi_device_id;
typedef uint64_t thapi_telemetry_handle;
typedef uintptr_t thapi_fabricPort_id;
typedef uint32_t thapi_domain_idx;
typedef uint32_t thapi_sdevice_idx;

Expand All @@ -69,9 +71,10 @@ typedef std::tuple<hostname_t, process_id_t, thread_id_t, thapi_device_id, thapi
thapi_function_name>
hpt_device_function_name_t;
typedef std::tuple<hostname_t, process_id_t, thapi_device_id> hp_device_t;
typedef std::tuple<hostname_t, thapi_device_id> h_device_t;
typedef std::tuple<hostname_t, process_id_t, thapi_device_id, thapi_device_id> hp_dsd_t;
typedef std::tuple<hostname_t, process_id_t, thapi_device_id, thapi_domain_idx> hp_ddomain_t;
typedef std::tuple<hostname_t, process_id_t, thapi_device_id, thapi_sdevice_idx> hp_dsdev_t;
typedef std::tuple<hostname_t, process_id_t, thapi_device_id, thapi_telemetry_handle, thapi_domain_idx> hp_ddomain_t;
typedef std::tuple<hostname_t, process_id_t, thapi_device_id, thapi_telemetry_handle, thapi_sdevice_idx, bool> hp_dfsdev_t;
typedef std::tuple<long, long> sd_t;
typedef std::tuple<thread_id_t, thapi_function_name, long> tfn_ts_t;
typedef std::tuple<thapi_function_name, long> fn_ts_t;
Expand Down
130 changes: 130 additions & 0 deletions xprof/btx_interval_model.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,16 @@
:type: integer_unsigned
:field_value_range: 64
:cast_type: uint64_t
- :name: deviceIdx
:field_class:
:type: integer_unsigned
:field_value_range: 32
:cast_type: uint32_t
- :name: hFrequency
:field_class:
:type: integer_unsigned
:field_value_range: 64
:cast_type: uint64_t
- :name: domain
:field_class:
:type: integer_unsigned
Expand All @@ -121,6 +131,16 @@
:type: integer_unsigned
:field_value_range: 64
:cast_type: uint64_t
- :name: deviceIdx
:field_class:
:type: integer_unsigned
:field_value_range: 32
:cast_type: uint32_t
- :name: hPower
:field_class:
:type: integer_unsigned
:field_value_range: 64
:cast_type: uint64_t
- :name: domain
:field_class:
:type: integer_unsigned
Expand All @@ -140,6 +160,16 @@
:type: integer_unsigned
:field_value_range: 64
:cast_type: uint64_t
- :name: deviceIdx
:field_class:
:type: integer_unsigned
:field_value_range: 32
:cast_type: uint32_t
- :name: hEngine
:field_class:
:type: integer_unsigned
:field_value_range: 64
:cast_type: uint64_t
- :name: subDevice
:field_class:
:type: integer_unsigned
Expand All @@ -158,6 +188,16 @@
:type: integer_unsigned
:field_value_range: 64
:cast_type: uint64_t
- :name: deviceIdx
:field_class:
:type: integer_unsigned
:field_value_range: 32
:cast_type: uint32_t
- :name: hEngine
:field_class:
:type: integer_unsigned
:field_value_range: 64
:cast_type: uint64_t
- :name: subDevice
:field_class:
:type: integer_unsigned
Expand All @@ -167,3 +207,93 @@
:field_class:
:type: single
:cast_type: float
- :name: lttng:fabricPort
:payload_field_class:
:type: structure
:members:
- :name: did
:field_class:
:type: integer_unsigned
:field_value_range: 64
:cast_type: uint64_t
- :name: deviceIdx
:field_class:
:type: integer_unsigned
:field_value_range: 32
:cast_type: uint32_t
- :name: hFabricPort
:field_class:
:type: integer_unsigned
:field_value_range: 64
:cast_type: uint64_t
- :name: subDevice
:field_class:
:type: integer_unsigned
:field_value_range: 32
:cast_type: uint32_t
- :name: portId
:field_class:
:type: integer_unsigned
:field_value_range: 32
:cast_type: uint32_t
- :name: remotePortId
:field_class:
:type: integer_unsigned
:field_value_range: 32
:cast_type: uint32_t
- :name: rxThroughput
:field_class:
:type: double
:cast_type: float
- :name: txThroughput
:field_class:
:type: double
:cast_type: float
- :name: rxSpeed
:field_class:
:type: double
:cast_type: float
- :name: txSpeed
:field_class:
:type: double
:cast_type: float
- :name: lttng:memModule
:payload_field_class:
:type: structure
:members:
- :name: did
:field_class:
:type: integer_unsigned
:field_value_range: 64
:cast_type: uint64_t
- :name: deviceIdx
:field_class:
:type: integer_unsigned
:field_value_range: 32
:cast_type: uint32_t
- :name: hMemModule
:field_class:
:type: integer_unsigned
:field_value_range: 64
:cast_type: uint64_t
- :name: subDevice
:field_class:
:type: integer_unsigned
:field_value_range: 32
:cast_type: uint32_t
- :name: pBandwidth
:field_class:
:type: double
:cast_type: float
- :name: rdBandwidth
:field_class:
:type: double
:cast_type: float
- :name: wtBandwidth
:field_class:
:type: double
:cast_type: float
- :name: occupancy
:field_class:
:type: double
:cast_type: float
Loading