Berbs (#2357)

## About The Pull Request
I'm not calling them biddle verbs.

Basically, adds verbs to a queue so that they won't be run if the MC is
overtiming since by default BYOND will just run them even if it means
the MC won't get much/any time. tl;dr possible lag/overtime reduction
from verbs like input

Ports:
tgstation/tgstation#65589
tgstation/tgstation#68990
tgstation/tgstation#70647
tgstation/tgstation#71520

## Why It's Good For The Game
Overtime bad

## Changelog

:cl:
/:cl:

<!-- Both :cl:'s are required for the changelog to work! You can put
your name to the right of the first :cl: if you want to overwrite your
GitHub username as author ingame. -->
<!-- You can use multiple of the same prefix (they're only used for the
icon ingame) and delete the unneeded ones. Despite some of the tags,
changelogs should generally represent how a player might be affected by
the changes rather than a summary of the PR's contents. -->

---------

Co-authored-by: KylerAce <[email protected]>
Co-authored-by: Kyle Spier-Swenson <[email protected]>
Co-authored-by: LemonInTheDark <[email protected]>
Co-authored-by: Mothblocks <[email protected]>

.github/TICK_ORDER.md

@@ -0,0 +1,21 @@
+The byond tick proceeds as follows:
+1. procs sleeping via walk() are resumed (i dont know why these are first)
+
+2. normal sleeping procs are resumed, in the order they went to sleep in the first place, this is where the MC wakes up and processes subsystems. a consequence of this is that the MC almost never resumes before other sleeping procs, because it only goes to sleep for 1 tick 99% of the time, and 99% of procs either go to sleep for less time than the MC (which guarantees that they entered the sleep queue earlier when its time to wake up) and/or were called synchronously from the MC's execution, almost all of the time the MC is the last sleeping proc to resume in any given tick. This is good because it means the MC can account for the cost of previous resuming procs in the tick, and minimizes overtime.
+
+3. control is passed to byond after all of our code's procs stop execution for this tick
+
+4. a few small things happen in byond internals
+
+5. SendMaps is called for this tick, which processes the game state for all clients connected to the game and handles sending them changes
+in appearances within their view range. This is expensive and takes up a significant portion of our tick, about 0.45% per connected player
+as of 3/20/2022. meaning that with 50 players, 22.5% of our tick is being used up by just SendMaps, after all of our code has stopped executing. Thats only the average across all rounds, for most highpop rounds it can look like 0.6% of the tick per player, which is 30% for 50 players.
+
+6. After SendMaps ends, client verbs sent to the server are executed, and its the last major step before the next tick begins.
+During the course of the tick, a client can send a command to the server saying that they have executed any verb. The actual code defined
+for that /verb/name() proc isnt executed until this point, and the way the MC is designed makes this especially likely to make verbs
+"overrun" the bounds of the tick they executed in, stopping the other tick from starting and thus delaying the MC firing in that tick.
+
+The master controller can derive how much of the tick was used in: procs executing before it woke up (because of world.tick_usage), and SendMaps (because of world.map_cpu, since this is a running average you cant derive the tick spent on maptick on any particular tick). It cannot derive how much of the tick was used for sleeping procs resuming after the MC ran, or for verbs executing after SendMaps. 
+
+It is for these reasons why you should heavily limit processing done in verbs, while procs resuming after the MC are rare, verbs are not, and are much more likely to cause overtime since theyre literally at the end of the tick. If you make a verb, try to offload any expensive work to the beginning of the next tick via a verb management subsystem.

code/__DEFINES/MC.dm

@@ -19,6 +19,15 @@
 #define MC_AVG_FAST_UP_SLOW_DOWN(average, current) (average > current ? MC_AVERAGE_SLOW(average, current) : MC_AVERAGE_FAST(average, current))
 #define MC_AVG_SLOW_UP_FAST_DOWN(average, current) (average < current ? MC_AVERAGE_SLOW(average, current) : MC_AVERAGE_FAST(average, current))
 
+///creates a running average of "things elapsed" per time period when you need to count via a smaller time period.
+///eg you want an average number of things happening per second but you measure the event every tick (50 milliseconds).
+///make sure both time intervals are in the same units. doesnt work if current_duration > total_duration or if total_duration == 0
+#define MC_AVG_OVER_TIME(average, current, total_duration, current_duration) ((((total_duration) - (current_duration)) / (total_duration)) * (average) + (current))
+
+#define MC_AVG_MINUTES(average, current, current_duration) (MC_AVG_OVER_TIME(average, current, 1 MINUTES, current_duration))
+
+#define MC_AVG_SECONDS(average, current, current_duration) (MC_AVG_OVER_TIME(average, current, 1 SECONDS, current_duration))
+
 #define NEW_SS_GLOBAL(varname) if(varname != src){if(istype(varname)){Recover();qdel(varname);}varname = src;}
 
 #define START_PROCESSING(Processor, Datum) if (!(Datum.datum_flags & DF_ISPROCESSING)) {Datum.datum_flags |= DF_ISPROCESSING;Processor.processing += Datum}
@@ -99,3 +108,12 @@
 	ss_id="processing_[#X]";\
 }\
 /datum/controller/subsystem/processing/##X
+
+#define VERB_MANAGER_SUBSYSTEM_DEF(X) GLOBAL_REAL(SS##X, /datum/controller/subsystem/verb_manager/##X);\
+/datum/controller/subsystem/verb_manager/##X/New(){\
+	NEW_SS_GLOBAL(SS##X);\
+	PreInit();\
+	ss_id="verb_manager_[#X]";\
+}\
+/datum/controller/subsystem/verb_manager/##X/fire() {..() /*just so it shows up on the profiler*/} \
+/datum/controller/subsystem/verb_manager/##X

code/__DEFINES/callbacks.dm

@@ -2,4 +2,6 @@
 /// A shorthand for the callback datum, [documented here](datum/callback.html)
 #define CALLBACK new /datum/callback
 #define INVOKE_ASYNC world.ImmediateInvokeAsync
+/// like CALLBACK but specifically for verb callbacks
+#define VERB_CALLBACK new /datum/callback/verb_callback
 #define CALLBACK_NEW(typepath, args) CALLBACK(GLOBAL_PROC, /proc/___callbacknew, typepath, args)

code/__DEFINES/input.dm

@@ -0,0 +1,2 @@
+///if the running average click latency is above this amount then clicks will never queue and will execute immediately
+#define MAXIMUM_CLICK_LATENCY (0.5 DECISECONDS)

code/__DEFINES/subsystems.dm

@@ -187,6 +187,7 @@
 #define FIRE_PRIORITY_SOUND_LOOPS 800
 #define FIRE_PRIORITY_OVERMAP_MOVEMENT 850
 #define FIRE_PRIORITY_SPEECH_CONTROLLER 900
+#define FIRE_PRIORITY_DELAYED_VERBS 950
 #define FIRE_PRIORITY_INPUT 1000 // This must always always be the max highest priority. Player input must never be lost.
 
 //Pipeline rebuild helper defines, these suck but it'll do for now

code/__DEFINES/time.dm

@@ -46,6 +46,10 @@ When using time2text(), please use "DDD" to find the weekday. Refrain from using
 #define SATURDAY "Sat"
 #define SUNDAY "Sun"
 
+#define MILLISECONDS *0.01
+
+#define DECISECONDS *1 //the base unit all of these defines are scaled by, because byond uses that as a unit of measurement for some fucking reason
+
 #define SECONDS *10
 
 #define MINUTES SECONDS*60
@@ -54,8 +58,6 @@ When using time2text(), please use "DDD" to find the weekday. Refrain from using
 
 #define TICKS *world.tick_lag
 
-#define MILLISECONDS * 0.01
-
 #define DS2TICKS(DS) ((DS)/world.tick_lag)
 
 #define TICKS2DS(T) ((T) TICKS)

code/__DEFINES/verb_manager.dm

@@ -0,0 +1,36 @@
+/**
+ * verb queuing thresholds. remember that since verbs execute after SendMaps the player wont see the effects of the verbs on the game world
+ * until SendMaps executes next tick, and then when that later update reaches them. thus most player input has a minimum latency of world.tick_lag + player ping.
+ * however thats only for the visual effect of player input, when a verb processes the actual latency of game state changes or semantic latency is effectively 1/2 player ping,
+ * unless that verb is queued for the next tick in which case its some number probably smaller than world.tick_lag.
+ * so some verbs that represent player input are important enough that we only introduce semantic latency if we absolutely need to.
+ * its for this reason why player clicks are handled in SSinput before even movement - semantic latency could cause someone to move out of range
+ * when the verb finally processes but it was in range if the verb had processed immediately and overtimed.
+ */
+
+///queuing tick_usage threshold for verbs that are high enough priority that they only queue if the server is overtiming.
+///ONLY use for critical verbs
+#define VERB_OVERTIME_QUEUE_THRESHOLD 100
+///queuing tick_usage threshold for verbs that need lower latency more than most verbs.
+#define VERB_HIGH_PRIORITY_QUEUE_THRESHOLD 95
+///default queuing tick_usage threshold for most verbs which can allow a small amount of latency to be processed in the next tick
+#define VERB_DEFAULT_QUEUE_THRESHOLD 85
+
+///attempt to queue this verb process if the server is overloaded. evaluates to FALSE if queuing isnt necessary or if it failed.
+///_verification_args... are only necessary if the verb_manager subsystem youre using checks them in can_queue_verb()
+///if you put anything in _verification_args that ISNT explicitely put in the can_queue_verb() override of the subsystem youre using,
+///it will runtime.
+#define TRY_QUEUE_VERB(_verb_callback, _tick_check, _subsystem_to_use, _verification_args...) (_queue_verb(_verb_callback, _tick_check, _subsystem_to_use, _verification_args))
+///queue wrapper for TRY_QUEUE_VERB() when you want to call the proc if the server isnt overloaded enough to queue
+#define QUEUE_OR_CALL_VERB(_verb_callback, _tick_check, _subsystem_to_use, _verification_args...) \
+	if(!TRY_QUEUE_VERB(_verb_callback, _tick_check, _subsystem_to_use, _verification_args)) {\
+		_verb_callback:InvokeAsync() \
+		};
+
+//goes straight to SSverb_manager with default tick threshold
+#define DEFAULT_TRY_QUEUE_VERB(_verb_callback, _verification_args...) (TRY_QUEUE_VERB(_verb_callback, VERB_DEFAULT_QUEUE_THRESHOLD, null, _verification_args))
+#define DEFAULT_QUEUE_OR_CALL_VERB(_verb_callback, _verification_args...) QUEUE_OR_CALL_VERB(_verb_callback, VERB_DEFAULT_QUEUE_THRESHOLD, null, _verification_args)
+
+//default tick threshold but nondefault subsystem
+#define TRY_QUEUE_VERB_FOR(_verb_callback, _subsystem_to_use, _verification_args...) (TRY_QUEUE_VERB(_verb_callback, VERB_DEFAULT_QUEUE_THRESHOLD, _subsystem_to_use, _verification_args))
+#define QUEUE_OR_CALL_VERB_FOR(_verb_callback, _subsystem_to_use, _verification_args...) QUEUE_OR_CALL_VERB(_verb_callback, VERB_DEFAULT_QUEUE_THRESHOLD, _subsystem_to_use, _verification_args)

code/__HELPERS/unsorted.dm

@@ -660,6 +660,11 @@ will handle it, but:
 	if(!istype(AM))
 		return
 
+	//Find coordinates
+	var/turf/T = get_turf(AM) //use checked_atom's turfs, as it's coords are the same as checked_atom's AND checked_atom's coords are lost if it is inside another atom
+	if(!T)
+		return null
+
 	//Find AM's matrix so we can use it's X/Y pixel shifts
 	var/matrix/M = matrix(AM.transform)
 
@@ -678,10 +683,6 @@ will handle it, but:
 	var/rough_x = round(round(pixel_x_offset,world.icon_size)/world.icon_size)
 	var/rough_y = round(round(pixel_y_offset,world.icon_size)/world.icon_size)
 
-	//Find coordinates
-	var/turf/T = get_turf(AM) //use AM's turfs, as it's coords are the same as AM's AND AM's coords are lost if it is inside another atom
-	if(!T)
-		return null
 	var/final_x = T.x + rough_x
 	var/final_y = T.y + rough_y
 
@@ -1379,7 +1380,7 @@ GLOBAL_DATUM_INIT(dview_mob, /mob/dview, new)
 
 // Makes a call in the context of a different usr
 // Use sparingly
-/world/proc/PushUsr(mob/M, datum/callback/CB, ...)
+/world/proc/push_usr(mob/M, datum/callback/CB, ...)
 	var/temp = usr
 	usr = M
 	if (length(args) > 2)

code/_onclick/click.dm

@@ -38,9 +38,10 @@
  *
  * Note that this proc can be overridden, and is in the case of screen objects.
  */
-/atom/Click(location,control,params)
+/atom/Click(location, control, params)
 	if(flags_1 & INITIALIZED_1)
 		SEND_SIGNAL(src, COMSIG_CLICK, location, control, params, usr)
+
 		usr.ClickOn(src, params)
 
 /atom/DblClick(location,control,params)

code/controllers/master.dm

@@ -367,6 +367,7 @@ GLOBAL_REAL(Master, /datum/controller/master) = new
 	while (1)
 		tickdrift = max(0, MC_AVERAGE_FAST(tickdrift, (((REALTIMEOFDAY - init_timeofday) - (world.time - init_time)) / world.tick_lag)))
 		var/starting_tick_usage = TICK_USAGE
+
 		if (init_stage != init_stage_completed)
 			return MC_LOOP_RTN_NEWSTAGES
 		if (processing <= 0)

code/controllers/subsystem/input.dm

@@ -1,15 +1,28 @@
-SUBSYSTEM_DEF(input)
+VERB_MANAGER_SUBSYSTEM_DEF(input)
 	name = "Input"
-	wait = 1 //SS_TICKER means this runs every tick
 	init_order = INIT_ORDER_INPUT
 	init_stage = INITSTAGE_EARLY
 	flags = SS_TICKER
 	priority = FIRE_PRIORITY_INPUT
 	runlevels = RUNLEVELS_DEFAULT | RUNLEVEL_LOBBY
 
+	use_default_stats = FALSE
+
 	var/list/macro_sets
 
-/datum/controller/subsystem/input/Initialize()
+	///running average of how many clicks inputted by a player the server processes every second. used for the subsystem stat entry
+	var/clicks_per_second = 0
+	///count of how many clicks onto atoms have elapsed before being cleared by fire(). used to average with clicks_per_second.
+	var/current_clicks = 0
+	///acts like clicks_per_second but only counts the clicks actually processed by SSinput itself while clicks_per_second counts all clicks
+	var/delayed_clicks_per_second = 0
+	///running average of how many movement iterations from player input the server processes every second. used for the subsystem stat entry
+	var/movements_per_second = 0
+	///running average of the amount of real time clicks take to truly execute after the command is originally sent to the server.
+	///if a click isnt delayed at all then it counts as 0 deciseconds.
+	var/average_click_delay = 0
+
+/datum/controller/subsystem/verb_manager/input/Initialize()
 	setup_default_macro_sets()
 
 	initialized = TRUE
@@ -19,7 +32,7 @@ SUBSYSTEM_DEF(input)
 	return ..()
 
 // This is for when macro sets are eventualy datumized
-/datum/controller/subsystem/input/proc/setup_default_macro_sets()
+/datum/controller/subsystem/verb_manager/input/proc/setup_default_macro_sets()
 	var/list/static/default_macro_sets
 
 	if(default_macro_sets)
@@ -86,12 +99,59 @@ SUBSYSTEM_DEF(input)
 	macro_sets = default_macro_sets
 
 // Badmins just wanna have fun ♪
-/datum/controller/subsystem/input/proc/refresh_client_macro_sets()
+/datum/controller/subsystem/verb_manager/input/proc/refresh_client_macro_sets()
 	var/list/clients = GLOB.clients
 	for(var/i in 1 to clients.len)
 		var/client/user = clients[i]
 		user.set_macros()
 
-/datum/controller/subsystem/input/fire()
+/datum/controller/subsystem/verb_manager/input/can_queue_verb(datum/callback/verb_callback/incoming_callback, control)
+	//make sure the incoming verb is actually something we specifically want to handle
+	if(control != "mapwindow.map")
+		return FALSE
+
+	if(average_click_delay > MAXIMUM_CLICK_LATENCY || !..())
+		current_clicks++
+		average_click_delay = MC_AVG_FAST_UP_SLOW_DOWN(average_click_delay, 0)
+		return FALSE
+
+	return TRUE
+
+///stupid workaround for byond not recognizing the /atom/Click typepath for the queued click callbacks
+/atom/proc/_Click(location, control, params)
+	if(usr)
+		Click(location, control, params)
+
+/datum/controller/subsystem/verb_manager/input/fire()
+	..()
+
+	var/moves_this_run = 0
 	for(var/mob/user as anything in GLOB.keyloop_list)
-		user.focus?.keyLoop(user.client)
+		moves_this_run += user.focus?.keyLoop(user.client)//only increments if a player moves due to their own input
+
+	movements_per_second = MC_AVG_SECONDS(movements_per_second, moves_this_run, wait TICKS)
+
+/datum/controller/subsystem/verb_manager/input/run_verb_queue()
+	var/deferred_clicks_this_run = 0 //acts like current_clicks but doesnt count clicks that dont get processed by SSinput
+
+	for(var/datum/callback/verb_callback/queued_click as anything in verb_queue)
+		if(!istype(queued_click))
+			stack_trace("non /datum/callback/verb_callback instance inside SSinput's verb_queue!")
+			continue
+
+		average_click_delay = MC_AVG_FAST_UP_SLOW_DOWN(average_click_delay, TICKS2DS((DS2TICKS(world.time) - queued_click.creation_time)) SECONDS)
+		queued_click.InvokeAsync()
+
+		current_clicks++
+		deferred_clicks_this_run++
+
+	verb_queue.Cut() //is ran all the way through every run, no exceptions
+
+	clicks_per_second = MC_AVG_SECONDS(clicks_per_second, current_clicks, wait SECONDS)
+	delayed_clicks_per_second = MC_AVG_SECONDS(delayed_clicks_per_second, deferred_clicks_this_run, wait SECONDS)
+	current_clicks = 0
+
+/datum/controller/subsystem/verb_manager/input/stat_entry(msg)
+	. = ..()
+	. += "M/S:[round(movements_per_second,0.01)] | C/S:[round(clicks_per_second,0.01)] ([round(delayed_clicks_per_second,0.01)] | CD: [round(average_click_delay / (1 SECONDS),0.01)])"
+

code/controllers/subsystem/speech_controller.dm

@@ -1,54 +1,5 @@
-SUBSYSTEM_DEF(speech_controller)
+/// verb_manager subsystem just for handling say's
+VERB_MANAGER_SUBSYSTEM_DEF(speech_controller)
 	name = "Speech Controller"
 	wait = 1
-	flags = SS_TICKER
 	priority = FIRE_PRIORITY_SPEECH_CONTROLLER//has to be high priority, second in priority ONLY to SSinput
-	init_order = INIT_ORDER_SPEECH_CONTROLLER
-	runlevels = RUNLEVELS_DEFAULT | RUNLEVEL_LOBBY
-
-	///used so that an admin can force all speech verbs to execute immediately instead of queueing
-	var/FOR_ADMINS_IF_BROKE_immediately_execute_all_speech = FALSE
-
-	///list of the form: list(client mob, message that mob is queued to say, other say arguments (if any)).
-	///this is our process queue, processed every tick.
-	var/list/queued_says_to_execute = list()
-
-///queues mob_to_queue into our process list so they say(message) near the start of the next tick
-/datum/controller/subsystem/speech_controller/proc/queue_say_for_mob(mob/mob_to_queue, message, message_type)
-
-	if(!TICK_CHECK || FOR_ADMINS_IF_BROKE_immediately_execute_all_speech)
-		process_single_say(mob_to_queue, message, message_type)
-		return TRUE
-
-	queued_says_to_execute += list(list(mob_to_queue, message, message_type))
-
-	return TRUE
-
-/datum/controller/subsystem/speech_controller/fire(resumed)
-
-	///	cache for sanic speed (lists are references anyways)
-	var/list/says_to_process = queued_says_to_execute.Copy()
-	queued_says_to_execute.Cut()//we should be going through the entire list every single iteration
-
-	for(var/list/say_to_process as anything in says_to_process)
-
-		var/mob/mob_to_speak = say_to_process[MOB_INDEX]//index 1 is the mob, 2 is the message, 3 is the message category
-		var/message = say_to_process[MESSAGE_INDEX]
-		var/message_category = say_to_process[CATEGORY_INDEX]
-
-		process_single_say(mob_to_speak, message, message_category)
-
-///used in fire() to process a single mobs message through the relevant proc.
-///only exists so that sleeps in the message pipeline dont cause the whole queue to wait
-/datum/controller/subsystem/speech_controller/proc/process_single_say(mob/mob_to_speak, message, message_category)
-	set waitfor = FALSE
-
-	switch(message_category)
-		if(SPEECH_CONTROLLER_QUEUE_SAY_VERB)
-			mob_to_speak.say(message)
-
-		if(SPEECH_CONTROLLER_QUEUE_WHISPER_VERB)
-			mob_to_speak.whisper(message)
-
-		if(SPEECH_CONTROLLER_QUEUE_EMOTE_VERB)
-			mob_to_speak.emote("me",1,message,TRUE)