How to use SendRawInputGraphCallback ? #5

dyanni3 · 2020-04-14T16:10:25Z

Good afternoon, thanks again for your work here and I just bought the socketio client from the marketplace for a little support.

I've managed to get this almost fully working for what I'm trying to do, just a little unclear on how to use the function SendRawInputGraphCallback in MachineLearningRemoteComponent.h.

For context, what I'm trying to do is send an array of floats (state and reward) to the server (for a q learning agent to process) and then receive an array of floats (actions) in order to try some reinforcement learning.

I'm able to send floats to the server

void ATPSPlayer::Tick(float DeltaTime)
{
ATPSCharacter::Tick(DeltaTime);
//PlayerMLR2 is a MachineLearningRemoteComponent connected to player
UPlayerMLR2 * thisMLR = FindComponentByClass<UPlayerMLR2>();
TArray<float> InputData = { 10.0, 100.0, 1000. };
thisMLR->SendRawInput(InputData, TEXT("on_float_array_input"));
}

My server receives these floats every tick, and can e.g. write them to log.

def on_float_array_input(self, float_array_input):
	ue.log("got a float array")
	for val in float_array_input:
		ue.log(val)
	#return an array output
	return ([9.0, 8.0, 7.0])

This all works successfully. Receiving a response from the server though has me confused. I can try

void ATPSPlayer::Tick(float DeltaTime)
{
ATPSCharacter::Tick(DeltaTime);
UPlayerMLR2 * thisMLR = FindComponentByClass<UPlayerMLR2>();
TArray<float> InputData = { 10.0, 100.0, 1000. };
TArray<float> ResultData = { 0.0 };
//thisMLR->SendRawInput(InputData, TEXT("on_float_array_input"));
struct FLatentActionInfo LatentInfo; //what do here?
thisMLR->SendRawInputGraphCallback(InputData, ResultData, LatentInfo, TEXT("on_float_array_input"));
GEngine->AddOnScreenDebugMessage(-1, 10.f, FColor::Purple, FString::SanitizeFloat(ResultData[0]));
}

I would like to get to the point where the on screen debug message prints e.g. 9.0

I literally can't find any documentation on what sort of struct FLatentActionInfo should be -- google just sends me back to your repos.

Again, thanks so much in advance!

The text was updated successfully, but these errors were encountered:

getnamo · 2020-04-15T06:33:04Z

Thanks for the support and for helping ironing out the bugs in this early ml-remote prototype :)

FLatentAction is really about receiving data in blueprint graphs, it resumes a blueprint latent function.

e.g. this graph callback function, notice how the result returns as if it was a normal function, but this will be called in a latent (async) manner and can only exist in event graphs. The little clock symbol next to the function indicates it's a latent function.

I quickly looked at the current plugin API and it seems to be blueprint biased in this early prototype. I went ahead and added native lambda callback variants via overloading the SendRawInput SendStringInput. This landed as v0.3.1, pull latest master to get the changes and check out https://github.com/getnamo/machine-learning-remote-ue4/blob/master/README.md#c-api for instructions on using lambda callback variants.

Basically just add an appropriate lambda as the second parameter and it will use the lamba variant e.g.

//Let's say you want to send some raw data
TArray<float> InputData;
//... fill

MLComponent->SendRawInput(InputData, [this](TArray<float>& ResultData)
{
	//Now we got our results back, do something with them here
}, TEXT("on_float_array_input"));

That said you can also use the SendRawInput without a lambda and bind to the OnRawInputResult multicast delegate, but you shouldn't be using the graph callback variants in C++.

dyanni3 · 2020-04-15T12:17:34Z

My man this is incredible! I am currently on baby duty but will pull latest and try out in a few hours. Thanks so much! I wish I could send you a Xmas present or something!

…

On Wed, Apr 15, 2020 at 2:33 AM Jan Kaniewski ***@***.***> wrote: FLatentAction is really about receiving data in blueprint graphs, it resumes a blueprint latent function. <https://camo.githubusercontent.com/72754dd079dd531338d85c9e36e0e83d9369be3d/68747470733a2f2f692e696d6775722e636f6d2f576a6d464c41752e706e67> *e.g. this graph callback function, notice you draw off the result as if it was a normal function, but this will be called in a latent manner and can only exist in event graphs.* I quickly looked at the current API and it seems to be blueprint biased in this early prototype. I went ahead and added native lambda callback variants via overloading the SendRawInput/SendStringInput. Pull latest master to get the changes and check out https://github.com/getnamo/machine-learning-remote-ue4/blob/master/README.md#c-api for instructions on using lambda callback variants. That said you can also use the SendRawInput and bind to the OnRawInputResult multicast delegate. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#5 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEVVN5YGZHMQETM4MMCK6ILRMVIK3ANCNFSM4MH3VUPQ> .

dyanni3 · 2020-04-15T17:56:20Z

Okay, this is great. I hate to pester but I am still having one issue which is that I can't seem to get the lambda callbacks to get called? Taking an example from the updated readme I am trying to test out just sending a string and printing returned string. In UE4 project

FString InputString = TEXT("The Sent Data");
UE_LOG(LogTemp, Warning, TEXT("UE_LOG Works"));

RemoteMLComponent->SendStringInput(InputString, [this](const FString& ResultData)
{
	//e.g. just print the result
	UE_LOG(LogTemp, Warning, TEXT("SendStringInput Callback Was Called"));
	UE_LOG(LogTemp, Warning, TEXT("Got some results: %s"), *ResultData);
}, TEXT("on_test_string"));

And then in hello.py (the script that RemoteMLComponent successfully runs) I have

def on_test_string(self, string_input):
	print("got some data")
	ue.log(string_input)
	return("sending some string-like stuff back")

When I play this the server's console prints out "got some data". The log at localhost:8080 prints "The Sent Data". However, the UE_LOG only shows the first warning "UE_LOG WORKS", it does not log anything like "SendStringInput Callback Was Called" or "sending some string-like stuff back"

Do I need to explicitly call the callback function from the python side?

Thanks again

getnamo · 2020-04-15T19:47:12Z

Pull latest on https://github.com/getnamo/ml-remote-server as well, I believe I updated custom functions to work the same way as native on_json_input and on_float_array_input (commit: getnamo/ml-remote-server@36d6267). Let me know if that fixes it

dyanni3 · 2020-04-15T19:59:36Z

Hey thanks,

Unfortunately I was already on latest commit: getnamo/ml-remote-server@36d6267, but the problem persists.

Also, if I don't use a custom function, and instead use on_float_array_input I get the same behavior (server receives the input floats and prints them out but the callback doesn't get called)

So I changed line 68 in server.py to elif data[functionFieldName] == 'on_float_array_input': and this gets some sort of callback to get called I believe. I now see the server printing statements like

run_on_gt callback: [9.0, 8.0, 7.0]

and

sendInput return: [9.0, 8.0, 7.0]

but now running into some other complaints that looks related to this

MLBaseLog: Warning: SendRawInput: Expected float array wrapped object, got [[9,8,7]]

Edited to add some more context

For context, here's the code in ue4

#include "newMLR.h"

void UnewMLR::BeginPlay() {
	Super::BeginPlay();
	GEngine->AddOnScreenDebugMessage(-1, 5.f, FColor::Red, TEXT("Began Play!"));
	TArray<float> InputData = { 0.0, 1.0, 2.3 };
	TArray<float> ResultData = { 0.0 };
	UE_LOG(LogTemp, Warning, TEXT("testing ue log"));
	//SendRawInput(InputData, TEXT("on_float_array_input"));


	SendRawInput(InputData, [this](TArray<float>& ResultData)
	{
	//Now we got our results back, do something with them here
		//UE_LOG(LogTemp, Log, TEXT("Got some results: %s"), FString::SanitizeFloat(*ResultData[0]));
		GEngine->AddOnScreenDebugMessage(-1, 5.f, FColor::Green, FString::SanitizeFloat(ResultData[0]));
		UE_LOG(LogTemp, Warning, TEXT("callback called"));
		float a = ResultData[0];
	}, TEXT("on_float_array_input"));
	GEngine->AddOnScreenDebugMessage(-1, 5.f, FColor::Purple, FString::SanitizeFloat(ResultData[0]));

	FString InputString = TEXT("Some Data");

	SendStringInput(InputString, [this](const FString& ResultData)
	{
		//e.g. just print the result
		UE_LOG(LogTemp, Warning, TEXT("SendStringInput Callback Was Called"));
		UE_LOG(LogTemp, Warning, TEXT("Got some results: %s"), *ResultData);
	}, TEXT("on_test_string"));


}

Here's the functions in hello.py

def on_float_array_input(self, float_array_input):
	print("got a float array")
	ue.log(float_array_input)
	ret_list = [9, 8, 7]
	return([float(i) for i in ret_list])

def on_test_string(self, string_input):
	print("got some data")
	ue.log(string_input)
	return("sending back some stringy stuff")

Here's what the server prints

connect  a82cd287a5f94733b8410499ddfad54d
sendInput: {'inputData': [0.0, 1.0, 2.299999952316284], 'targetFunction': 'on_float_array_input'}
got a float array
[0.0, 1.0, 2.299999952316284]
sendInput: {'inputData': 'Some Data', 'targetFunction': 'on_test_string'}
run_on_gt callback: [9.0, 8.0, 7.0]
got some data
sendInput return: [9.0, 8.0, 7.0]
Some Data
[9.0, 8.0, 7.0]
loading <hello>
hello import
loaded.
hello on_setup
hi
started.
hello on_begin_training
disconnect a82cd287a5f94733b8410499ddfad54d

getnamo · 2020-04-15T20:23:52Z

Ok I'll have a look to see if I can't replicate this bug in C++ later today, thanks for the detailed info.

getnamo · 2020-04-16T23:39:23Z

Haven't had the time to replicate yet, but can you confirm the blueprint version of callback is working for you first (in e.g. a blueprint not a c++ file)? It would help narrow down if it's the current state of both parts or just the new c++ lambdas giving problems

dyanni3 · 2020-04-17T14:55:16Z

Yeah will definitely try the blueprint version. Sorry it might take me a minute I'm still new to ue4.

…

On Thu, Apr 16, 2020 at 7:39 PM Jan Kaniewski ***@***.***> wrote: Haven't had the time to replicate yet, but can you confirm the blueprint variant is working for you first? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#5 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEVVN5Z6DKCLWR5WJGQ7DKLRM6JLRANCNFSM4MH3VUPQ> .

dyanni3 · 2020-04-17T20:06:56Z

Okay so I tried the blueprint version with the latest release. Here's how my blueprint script looks

The machine learning remote component is setup to run hello.py, which I modified to print "the blueprint version is running" on setup. Here's the server output

      connect 45a76c9b1c264f0cb1391c37d51ccc90
     connect 8c1fabff512146a9aabe130f90853b02
     loading
     hello import
     loaded.
     hello on_setup
     the blueprint version is running!
     hi
     started.
     loading
     hello import
     loaded.
     hello on_setup
     the blueprint version is running!
     hi
     started.
     disconnect 8c1fabff512146a9aabe130f90853b02
     disconnect 45a76c9b1c264f0cb1391c37d51ccc90

here's a snippet of hello.py

def on_setup(self):
	print('hello on_setup')
	print('the blueprint version is running!')
	ue.log('hi')
	pass
	
#optional api: parse input object and return a result object, which will be converted to json for UE4
def on_json_input(self, input):
	print('hello on_json_input')
	return {}

#optional api: start training your network
def on_begin_training(self):
	#print('hello on_begin_training')
	pass

def on_float_array_input(self, float_array_input):
	print("got a float array")
	ue.log(float_array_input)
	return(9.0)

def on_test_string(self, string_input):
	print("got some data")
	ue.log(string_input)
	return("sending back some stringy stuff")

So it appears either I'm doing something incorrectly or in the current blueprint version it's not calling the on_float_array_input function.

Thanks!

Edited: whoops

When I try to call a different custom function like on_test_string I get the following. First here's the blueprint

connect 4395c96839d74a7cba3250ad1bb03e10
sendInput: {'inputData': 'the sent string', 'targetFunction': 'on_test_string'}
got some data
the sent string
got some data
the sent string
loading
hello import
loaded.
hello on_setup
the blueprint version is running!
hi
started.
loading
hello import
loaded.
hello on_setup
the blueprint version is running!
hi
started.
disconnect 4395c96839d74a7cba3250ad1bb03e10

Okay so it looks like the same behavior as c++ to me. The function on_test_string is getting called but the returned value isn't getting printed in the log.

Nothing is getting printed to the unreal engine log, the localhost:8080 or the game screen.

getnamo · 2020-04-20T05:28:32Z

That's strange behavior, I'll try to find some time to run an example both in BP and C++ to confirm. I did have the BP variant working in my tests a few days ago, so I wonder what could be the difference in setup.

dyanni3 · 2020-04-20T12:30:33Z

Okay thanks. Like I said I’m new to ue4 so there could be something obvious I’m missing. Also in case it’s relevant: I’m on Windows 10, VS2017, ue4.24.3 I opened up a blank third person shooter template, adjusted the build.cs file, added the plugins folder, attached a machine learning remote component to the character and that’s it.

…

On Mon, Apr 20, 2020 at 1:28 AM Jan Kaniewski ***@***.***> wrote: That's strange behavior, I'll try to find some time to run an example both in BP and C++ to confirm. I did have the BP variant working in my tests a few days ago, so I wonder what could be the difference in setup. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#5 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEVVN52ITZBRXUDZZ6STSMTRNPMQ3ANCNFSM4MH3VUPQ> .

dyanni3 · 2020-04-20T15:38:00Z

More updates. Sorry to spam.

I got SendSIOJsonInput to work in blueprints with the result data successfully printed to game screen

I had to change line 57 of server.py to if data[functionFieldName] == 'on_json_input': to match the function name in the mlpluginapi. Then I have to make sure to send properly formatted json string.

It looks like it's only the custom functions, send float array, and send string that are broken.

Since the JSON is working I will probably use this to communicate state and actions back and forth. Thanks!

getnamo added the question Further information is requested label Apr 15, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How to use SendRawInputGraphCallback ? #5

How to use SendRawInputGraphCallback ? #5

dyanni3 commented Apr 14, 2020 •

edited

Loading

getnamo commented Apr 15, 2020 •

edited

Loading

dyanni3 commented Apr 15, 2020 via email

dyanni3 commented Apr 15, 2020 •

edited

Loading

getnamo commented Apr 15, 2020 •

edited

Loading

dyanni3 commented Apr 15, 2020 •

edited

Loading

getnamo commented Apr 15, 2020

getnamo commented Apr 16, 2020 •

edited

Loading

dyanni3 commented Apr 17, 2020 via email

dyanni3 commented Apr 17, 2020 •

edited

Loading

getnamo commented Apr 20, 2020

dyanni3 commented Apr 20, 2020 via email

dyanni3 commented Apr 20, 2020

How to use SendRawInputGraphCallback ? #5

How to use SendRawInputGraphCallback ? #5

Comments

dyanni3 commented Apr 14, 2020 • edited Loading

getnamo commented Apr 15, 2020 • edited Loading

dyanni3 commented Apr 15, 2020 via email

dyanni3 commented Apr 15, 2020 • edited Loading

getnamo commented Apr 15, 2020 • edited Loading

dyanni3 commented Apr 15, 2020 • edited Loading

getnamo commented Apr 15, 2020

getnamo commented Apr 16, 2020 • edited Loading

dyanni3 commented Apr 17, 2020 via email

dyanni3 commented Apr 17, 2020 • edited Loading

getnamo commented Apr 20, 2020

dyanni3 commented Apr 20, 2020 via email

dyanni3 commented Apr 20, 2020

dyanni3 commented Apr 14, 2020 •

edited

Loading

getnamo commented Apr 15, 2020 •

edited

Loading

dyanni3 commented Apr 15, 2020 •

edited

Loading

getnamo commented Apr 15, 2020 •

edited

Loading

dyanni3 commented Apr 15, 2020 •

edited

Loading

getnamo commented Apr 16, 2020 •

edited

Loading

dyanni3 commented Apr 17, 2020 •

edited

Loading