Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AP_HAL_ESP32: more scheduler stats to help trying to find why copter… #25911

Closed

Conversation

davidbuzz
Copy link
Collaborator

… loops are slow

emits equivalent of tasks.txt to console after approx 2 minutes after boot.

eg:
TasksV2
AP_InertialSensor::update* MIN= 63 MAX= 979 AVG= 157 OVR= 0 SLP= 0, TOT= 3.4%
Copter::run_rate_controller* MIN= 148 MAX=1328 AVG= 247 OVR= 0 SLP= 0, TOT= 5.4%
Copter::motors_output* MIN= 294 MAX=1369 AVG= 327 OVR= 0 SLP= 0, TOT= 7.1%
Copter::read_AHRS* MIN=1237 MAX=6024 AVG=1991 OVR=389 SLP= 0, TOT=43.0%
Copter::read_inertia* MIN= 64 MAX= 980 AVG= 130 OVR= 0 SLP= 0, TOT= 2.8%
Copter::check_ekf_reset* MIN= 20 MAX= 925 AVG= 48 OVR= 0 SLP= 0, TOT= 1.1%
Copter::update_flight_mode* MIN= 397 MAX=1884 AVG= 665 OVR= 0 SLP= 0, TOT=14.4%
Copter::update_home_from_EKF* MIN= 10 MAX=1007 AVG= 21 OVR= 0 SLP= 0, TOT= 0.4%
Copter::update_land_and_crash_de MIN= 45 MAX=1045 AVG= 63 OVR= 0 SLP= 0, TOT= 1.4%
Copter::update_rangefinder_terra MIN= 12 MAX=1015 AVG= 22 OVR= 0 SLP= 0, TOT= 0.5%
Copter::Log_Video_Stabilisation* MIN= 17 MAX=1038 AVG= 33 OVR= 0 SLP= 0, TOT= 0.7%
Copter::rc_loop MIN= 57 MAX=1203 AVG= 90 OVR= 0 SLP= 0, TOT= 2.0%
Copter::throttle_loop MIN= 47 MAX=1105 AVG= 92 OVR= 23 SLP= 0, TOT= 0.2%
AP_GPS::update MIN= 87 MAX=1505 AVG= 195 OVR= 56 SLP= 0, TOT= 0.5%
Copter::update_batt_compass MIN= 68 MAX=1073 AVG= 160 OVR= 11 SLP= 0, TOT= 0.1%
RC_Channels::read_aux_all MIN= 15 MAX= 178 AVG= 21 OVR= 1 SLP= 0, TOT= 0.0%
Copter::arm_motors_check MIN= 14 MAX=1001 AVG= 67 OVR= 3 SLP= 0, TOT= 0.0%
Copter::auto_disarm_check MIN= 7 MAX= 19 AVG= 8 OVR= 0 SLP= 0, TOT= 0.0%
Copter::auto_trim MIN= 7 MAX= 154 AVG= 14 OVR= 2 SLP= 0, TOT= 0.0%
Copter::read_rangefinder MIN= 56 MAX=1052 AVG= 126 OVR= 12 SLP= 0, TOT= 0.1%
AP_Proximity::update MIN= 10 MAX=1075 AVG= 54 OVR= 50 SLP= 0, TOT= 0.5%
AP_Beacon::update MIN= 10 MAX=1084 AVG= 41 OVR=162 SLP= 0, TOT= 0.8%
Copter::update_altitude MIN= 165 MAX=1479 AVG= 310 OVR= 45 SLP= 0, TOT= 0.1%
Copter::run_nav_updates MIN= 9 MAX=1065 AVG= 19 OVR= 4 SLP= 0, TOT= 0.1%
Copter::update_throttle_hover MIN= 6 MAX= 992 AVG= 18 OVR= 8 SLP= 0, TOT= 0.1%
ModeSmartRTL::save_position MIN= 13 MAX= 43 AVG= 32 OVR= 0 SLP= 0, TOT= 0.0%
AC_Sprayer::update MIN= 10 MAX= 11 AVG= 10 OVR= 0 SLP= 0, TOT= 0.0%
Copter::three_hz_loop MIN= 31 MAX= 35 AVG= 33 OVR= 0 SLP= 0, TOT= 0.0%
AP_ServoRelayEvents::update_even MIN= 7 MAX=1022 AVG= 23 OVR= 7 SLP= 0, TOT= 0.1%
AP_Baro::accumulate MIN= 8 MAX= 149 AVG= 11 OVR= 2 SLP= 0, TOT= 0.0%
Copter::update_precland MIN= 11 MAX=1083 AVG= 38 OVR=162 SLP= 0, TOT= 0.7%
Copter::loop_rate_logging MIN= 9 MAX=1041 AVG= 17 OVR=119 SLP= 0, TOT= 0.3%
Compass::cal_update MIN= 16 MAX= 995 AVG= 31 OVR= 7 SLP= 0, TOT= 0.1%
AP_Notify::update MIN= 6 MAX= 627 AVG= 13 OVR= 1 SLP= 0, TOT= 0.0%
Copter::one_hz_loop MIN= 187 MAX= 663 AVG= 284 OVR= 5 SLP= 0, TOT= 0.0%
Copter::ekf_check MIN= 11 MAX=1056 AVG= 60 OVR= 3 SLP= 0, TOT= 0.0%
Copter::check_vibration MIN= 48 MAX=1031 AVG= 104 OVR= 26 SLP= 0, TOT= 0.1%
Copter::gpsglitch_check MIN= 13 MAX= 179 AVG= 23 OVR= 1 SLP= 0, TOT= 0.0%
Copter::takeoff_check MIN= 3 MAX= 164 AVG= 11 OVR= 2 SLP= 0, TOT= 0.0%
Copter::standby_update MIN= 6 MAX= 318 AVG= 16 OVR= 9 SLP= 0, TOT= 0.1%
Copter::lost_vehicle_check MIN= 17 MAX= 960 AVG= 40 OVR= 1 SLP= 0, TOT= 0.0%
GCS::update_receive MIN= 90 MAX=2048 AVG= 201 OVR=442 SLP= 0, TOT= 4.1%
GCS::update_send MIN= 157 MAX=1688 AVG= 379 OVR=705 SLP= 0, TOT= 5.9%
Copter::ten_hz_logging_loop MIN= 456 MAX=1654 AVG= 631 OVR= 45 SLP= 0, TOT= 0.3%
Copter::twentyfive_hz_logging MIN= 14 MAX= 370 AVG= 32 OVR= 5 SLP= 0, TOT= 0.0%
AP_Logger::periodic_tasks MIN= 74 MAX=1142 AVG= 125 OVR=447 SLP= 0, TOT= 2.2%
AP_InertialSensor::periodic MIN= 5 MAX= 357 AVG= 12 OVR=133 SLP= 0, TOT= 0.2%
AP_Scheduler::update_logging MIN= 23 MAX= 23 AVG= 23 OVR= 0 SLP= 0, TOT= 0.0%
AP_TempCalibration::update MIN= 6 MAX= 8 AVG= 6 OVR= 0 SLP= 0, TOT= 0.0%
AP_Winch::update MIN= 9 MAX= 29 AVG= 13 OVR= 0 SLP= 0, TOT= 0.0%
AP_Stats::update MIN= 26 MAX= 31 AVG= 28 OVR= 0 SLP= 0, TOT= 0.0%
AP_NMEA_Output::update MIN= 5 MAX= 338 AVG= 10 OVR= 2 SLP= 0, TOT= 0.0%
AP_RunCam::update MIN= 5 MAX= 128 AVG= 6 OVR= 1 SLP= 0, TOT= 0.0%
AP_Vehicle::update_dynamic_notch MIN= 10 MAX=1020 AVG= 22 OVR=299 SLP= 0, TOT= 0.4%
AP_VideoTX::update MIN= 5 MAX= 7 AVG= 6 OVR= 0 SLP= 0, TOT= 0.0%
AP_Tramp::update MIN= 7 MAX= 944 AVG= 17 OVR= 5 SLP= 0, TOT= 0.0%
AP_Vehicle::send_watchdog_reset_ MIN= 9 MAX= 9 AVG= 9 OVR= 0 SLP= 0, TOT= 0.0%
AP_Vehicle::publish_osd_info MIN= 37 MAX= 41 AVG= 39 OVR= 5 SLP= 0, TOT= 0.0%
AP_TemperatureSensor::update MIN= 6 MAX= 16 AVG= 8 OVR= 0 SLP= 0, TOT= 0.0%
AP_Vehicle::accel_cal_update MIN= 16 MAX= 27 AVG= 19 OVR= 0 SLP= 0, TOT= 0.0%
AP_AIS::update MIN= 7 MAX= 16 AVG= 9 OVR= 0 SLP= 0, TOT= 0.0%
AP_EFI::update MIN= 4 MAX= 152 AVG= 9 OVR= 0 SLP= 0, TOT= 0.0%
AP_Vehicle::one_Hz_update MIN= 13 MAX= 130 AVG= 37 OVR= 1 SLP= 0, TOT= 0.0%
AP_Filters::update MIN= 9 MAX= 10 AVG= 9 OVR= 0 SLP= 0, TOT= 0.0%
AP_Vehicle::update_arming MIN= 79 MAX= 83 AVG= 81 OVR= 5 SLP= 0, TOT= 0.0%

loop_rate: actual: 161Hz, expected: 400Hz

@davidbuzz davidbuzz requested a review from tridge January 6, 2024 05:37
@davidbuzz davidbuzz added the esp32 label Jan 6, 2024
@davidbuzz davidbuzz force-pushed the esp32-more-scheduler-stats branch 2 times, most recently from 48b0b23 to 818aa84 Compare January 8, 2024 09:42
@davidbuzz davidbuzz force-pushed the esp32-more-scheduler-stats branch from 818aa84 to 08efdd3 Compare March 1, 2024 22:20
…loops are slow

emits equivalent of tasks.txt to console after approx 2 minutes after boot.

eg:
TasksV2
AP_InertialSensor::update*       MIN=  63 MAX= 979 AVG= 157 OVR=  0 SLP=  0, TOT= 3.4%
Copter::run_rate_controller*     MIN= 148 MAX=1328 AVG= 247 OVR=  0 SLP=  0, TOT= 5.4%
Copter::motors_output*           MIN= 294 MAX=1369 AVG= 327 OVR=  0 SLP=  0, TOT= 7.1%
Copter::read_AHRS*               MIN=1237 MAX=6024 AVG=1991 OVR=389 SLP=  0, TOT=43.0%
Copter::read_inertia*            MIN=  64 MAX= 980 AVG= 130 OVR=  0 SLP=  0, TOT= 2.8%
Copter::check_ekf_reset*         MIN=  20 MAX= 925 AVG=  48 OVR=  0 SLP=  0, TOT= 1.1%
Copter::update_flight_mode*      MIN= 397 MAX=1884 AVG= 665 OVR=  0 SLP=  0, TOT=14.4%
Copter::update_home_from_EKF*    MIN=  10 MAX=1007 AVG=  21 OVR=  0 SLP=  0, TOT= 0.4%
Copter::update_land_and_crash_de MIN=  45 MAX=1045 AVG=  63 OVR=  0 SLP=  0, TOT= 1.4%
Copter::update_rangefinder_terra MIN=  12 MAX=1015 AVG=  22 OVR=  0 SLP=  0, TOT= 0.5%
Copter::Log_Video_Stabilisation* MIN=  17 MAX=1038 AVG=  33 OVR=  0 SLP=  0, TOT= 0.7%
Copter::rc_loop                  MIN=  57 MAX=1203 AVG=  90 OVR=  0 SLP=  0, TOT= 2.0%
Copter::throttle_loop            MIN=  47 MAX=1105 AVG=  92 OVR= 23 SLP=  0, TOT= 0.2%
AP_GPS::update                   MIN=  87 MAX=1505 AVG= 195 OVR= 56 SLP=  0, TOT= 0.5%
Copter::update_batt_compass      MIN=  68 MAX=1073 AVG= 160 OVR= 11 SLP=  0, TOT= 0.1%
RC_Channels::read_aux_all        MIN=  15 MAX= 178 AVG=  21 OVR=  1 SLP=  0, TOT= 0.0%
Copter::arm_motors_check         MIN=  14 MAX=1001 AVG=  67 OVR=  3 SLP=  0, TOT= 0.0%
Copter::auto_disarm_check        MIN=   7 MAX=  19 AVG=   8 OVR=  0 SLP=  0, TOT= 0.0%
Copter::auto_trim                MIN=   7 MAX= 154 AVG=  14 OVR=  2 SLP=  0, TOT= 0.0%
Copter::read_rangefinder         MIN=  56 MAX=1052 AVG= 126 OVR= 12 SLP=  0, TOT= 0.1%
AP_Proximity::update             MIN=  10 MAX=1075 AVG=  54 OVR= 50 SLP=  0, TOT= 0.5%
AP_Beacon::update                MIN=  10 MAX=1084 AVG=  41 OVR=162 SLP=  0, TOT= 0.8%
Copter::update_altitude          MIN= 165 MAX=1479 AVG= 310 OVR= 45 SLP=  0, TOT= 0.1%
Copter::run_nav_updates          MIN=   9 MAX=1065 AVG=  19 OVR=  4 SLP=  0, TOT= 0.1%
Copter::update_throttle_hover    MIN=   6 MAX= 992 AVG=  18 OVR=  8 SLP=  0, TOT= 0.1%
ModeSmartRTL::save_position      MIN=  13 MAX=  43 AVG=  32 OVR=  0 SLP=  0, TOT= 0.0%
AC_Sprayer::update               MIN=  10 MAX=  11 AVG=  10 OVR=  0 SLP=  0, TOT= 0.0%
Copter::three_hz_loop            MIN=  31 MAX=  35 AVG=  33 OVR=  0 SLP=  0, TOT= 0.0%
AP_ServoRelayEvents::update_even MIN=   7 MAX=1022 AVG=  23 OVR=  7 SLP=  0, TOT= 0.1%
AP_Baro::accumulate              MIN=   8 MAX= 149 AVG=  11 OVR=  2 SLP=  0, TOT= 0.0%
Copter::update_precland          MIN=  11 MAX=1083 AVG=  38 OVR=162 SLP=  0, TOT= 0.7%
Copter::loop_rate_logging        MIN=   9 MAX=1041 AVG=  17 OVR=119 SLP=  0, TOT= 0.3%
Compass::cal_update              MIN=  16 MAX= 995 AVG=  31 OVR=  7 SLP=  0, TOT= 0.1%
AP_Notify::update                MIN=   6 MAX= 627 AVG=  13 OVR=  1 SLP=  0, TOT= 0.0%
Copter::one_hz_loop              MIN= 187 MAX= 663 AVG= 284 OVR=  5 SLP=  0, TOT= 0.0%
Copter::ekf_check                MIN=  11 MAX=1056 AVG=  60 OVR=  3 SLP=  0, TOT= 0.0%
Copter::check_vibration          MIN=  48 MAX=1031 AVG= 104 OVR= 26 SLP=  0, TOT= 0.1%
Copter::gpsglitch_check          MIN=  13 MAX= 179 AVG=  23 OVR=  1 SLP=  0, TOT= 0.0%
Copter::takeoff_check            MIN=   3 MAX= 164 AVG=  11 OVR=  2 SLP=  0, TOT= 0.0%
Copter::standby_update           MIN=   6 MAX= 318 AVG=  16 OVR=  9 SLP=  0, TOT= 0.1%
Copter::lost_vehicle_check       MIN=  17 MAX= 960 AVG=  40 OVR=  1 SLP=  0, TOT= 0.0%
GCS::update_receive              MIN=  90 MAX=2048 AVG= 201 OVR=442 SLP=  0, TOT= 4.1%
GCS::update_send                 MIN= 157 MAX=1688 AVG= 379 OVR=705 SLP=  0, TOT= 5.9%
Copter::ten_hz_logging_loop      MIN= 456 MAX=1654 AVG= 631 OVR= 45 SLP=  0, TOT= 0.3%
Copter::twentyfive_hz_logging    MIN=  14 MAX= 370 AVG=  32 OVR=  5 SLP=  0, TOT= 0.0%
AP_Logger::periodic_tasks        MIN=  74 MAX=1142 AVG= 125 OVR=447 SLP=  0, TOT= 2.2%
AP_InertialSensor::periodic      MIN=   5 MAX= 357 AVG=  12 OVR=133 SLP=  0, TOT= 0.2%
AP_Scheduler::update_logging     MIN=  23 MAX=  23 AVG=  23 OVR=  0 SLP=  0, TOT= 0.0%
AP_TempCalibration::update       MIN=   6 MAX=   8 AVG=   6 OVR=  0 SLP=  0, TOT= 0.0%
AP_Winch::update                 MIN=   9 MAX=  29 AVG=  13 OVR=  0 SLP=  0, TOT= 0.0%
AP_Stats::update                 MIN=  26 MAX=  31 AVG=  28 OVR=  0 SLP=  0, TOT= 0.0%
AP_NMEA_Output::update           MIN=   5 MAX= 338 AVG=  10 OVR=  2 SLP=  0, TOT= 0.0%
AP_RunCam::update                MIN=   5 MAX= 128 AVG=   6 OVR=  1 SLP=  0, TOT= 0.0%
AP_Vehicle::update_dynamic_notch MIN=  10 MAX=1020 AVG=  22 OVR=299 SLP=  0, TOT= 0.4%
AP_VideoTX::update               MIN=   5 MAX=   7 AVG=   6 OVR=  0 SLP=  0, TOT= 0.0%
AP_Tramp::update                 MIN=   7 MAX= 944 AVG=  17 OVR=  5 SLP=  0, TOT= 0.0%
AP_Vehicle::send_watchdog_reset_ MIN=   9 MAX=   9 AVG=   9 OVR=  0 SLP=  0, TOT= 0.0%
AP_Vehicle::publish_osd_info     MIN=  37 MAX=  41 AVG=  39 OVR=  5 SLP=  0, TOT= 0.0%
AP_TemperatureSensor::update     MIN=   6 MAX=  16 AVG=   8 OVR=  0 SLP=  0, TOT= 0.0%
AP_Vehicle::accel_cal_update     MIN=  16 MAX=  27 AVG=  19 OVR=  0 SLP=  0, TOT= 0.0%
AP_AIS::update                   MIN=   7 MAX=  16 AVG=   9 OVR=  0 SLP=  0, TOT= 0.0%
AP_EFI::update                   MIN=   4 MAX= 152 AVG=   9 OVR=  0 SLP=  0, TOT= 0.0%
AP_Vehicle::one_Hz_update        MIN=  13 MAX= 130 AVG=  37 OVR=  1 SLP=  0, TOT= 0.0%
AP_Filters::update               MIN=   9 MAX=  10 AVG=   9 OVR=  0 SLP=  0, TOT= 0.0%
AP_Vehicle::update_arming        MIN=  79 MAX=  83 AVG=  81 OVR=  5 SLP=  0, TOT= 0.0%

loop_rate: actual: 161Hz, expected: 400Hz
@davidbuzz davidbuzz force-pushed the esp32-more-scheduler-stats branch from 08efdd3 to 0d665ea Compare September 2, 2024 00:16
@davidbuzz davidbuzz closed this Sep 30, 2024
@davidbuzz
Copy link
Collaborator Author

Old

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant