Issue formatting log data ("['response_context_system'] not found in axis")

[renatodossantosleal]

When trying to format my log data in jupyter notebook (local) i get this error:

KeyError: "['response_context_system'] not found in axis"

It is just two lines of logs extracted today (22/09) and I'm using assistant-improve-toolkit==1.3.6 and python 3.7.10

[MoonMess]

I have exactly the same error with my logs. I just check the format of the logs and it seems they are different from the example ("https://raw.githubusercontent.com/watson-developer-cloud/assistant-improve-recommendations-notebook/master/notebook/data/workspace.json"). I don't know if it's because of a format update's of the logs in Watson Assistant or just that the logs in the exemple have a special format... I think we need to modify the function format_data()

[Abonia1]

Hello @renatodossantosleal @MoonMess ,
I had the same issue and do some work around to mitigate it as follow:

#There is a missing field inside the df_logs, column response, field ['context']['system']
#Let's add it 
field_to_add = {'system':{}} 
for index, row in df_logs.iterrows():
    row['response']['context'].update(field_to_add)

# Format the logs data from the workspace
df_formatted = format_data(df_logs)

Hope it helps with the issue.

Happy coding!