Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scrub args alternate implementation #95

Closed
wants to merge 5 commits into from

Conversation

sibson
Copy link
Contributor

@sibson sibson commented Feb 25, 2016

Based on discussion on https://github.com/rollbar/pyrollbar/pull/91/files#r54032504 I'm submitting this as a discussion piece. I expect it will change how args are displayed in rollbar.


# Add any varargs
if arginfo.varargs is not None:
args.extend(local_vars[arginfo.varargs])
if SETTINGS['locals']['scrub_varargs']:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

*args could contain anything, including passwords so the safest thing is to scrub by default

@coryvirok
Copy link
Contributor

Thanks for starting up this convo.

I'll summarize to see if I understand the intent:

Since varargs can contain any type of data, including passwords, we want to scrub them by default.

I think this is a great idea. There are a few implementation details here that we need to fix before we can merge this:

  • Python 2.6 doesn't support dict comprehensions
  • safe_repr needs to be adhered to, (need to use _transform() for named args)

I like that you're using arginfo.varargs for the name of the varargs. I hadn't thought to do that. If you can fix the build errors for the different Python versions and the safe_repr errors I can review and test again. It'd be great to get this merged in.


# Add any varargs
if arginfo.varargs is not None:
args.extend(local_vars[arginfo.varargs])
if SETTINGS['locals']['scrub_varargs']:
args[arginfo.varargs] = ['*' * randint(3, 20) for i in range(len(local_vars[arginfo.varargs]))]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'*' * randint() here feels janky but it's not obvious to me what the better solution is

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should see if it's possible to use ScrubTransform here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, this may be a bit more complicated than I anticipated. I'm going to make a PR to this branch containing the Scrub code.

@sibson
Copy link
Contributor Author

sibson commented Mar 10, 2016

I've fixed the py2.6 issue and merged master but I'm not sure I understand the safe_repr comment. I think that this _serialize_frame_data_() call is doing the necessary scrubbing. I do wonder if the comment was related to my other PR that has been backed out. If you can explain the problem in more detail I'll try to address it.

@ezarowny
Copy link
Contributor

ezarowny commented Apr 1, 2016

Hey @sibson, I'll be working on this with you in the coming week. Looking forwards to merging it!

@ezarowny
Copy link
Contributor

ezarowny commented Apr 4, 2016

@sibson can you pull in changes from master again?

@ezarowny
Copy link
Contributor

ezarowny commented Apr 4, 2016

@sibson, we had a discussion after reviewing your PR and I think we have a way to both properly scrub args and simplify the notifier code.

Basically, args would become a list of argument names sourced from ArgSpec. We would substitute argument names for argument values for in Rollbar's UI by sourcing the values from locals (which come from ArgInfo). The Rollbar UI won't actually change but the data format will (since we are going to deprecate “args” and add “argspec”).

The plan as it stands would take a bump of pyrollbar's minor version and a completely new branch without any of your commits. Will this address your original PR?

@sibson
Copy link
Contributor Author

sibson commented Apr 5, 2016

@ezarowny let me rephrase to see if I understand.

  1. You're going to transmit argnames rather than argvalues to rollbar.com
  2. The UI will "dynamically" lookup locals[argname] for the value to display
  3. We're assuming locals is properly scrubbed, thus avoiding the issue of varargs containing sensitive data

If that is correct it seems reasonable. A possible gotcha is what happens if I've set locals=False in my config? I suppose just displaying the argname is acceptable.

@ezarowny
Copy link
Contributor

ezarowny commented Apr 5, 2016

Yep, that's correct!

I think what will happen in this case is that we will not include locals which means the UI will not have any args or locals in it. I'd have to double check that though.

@ezarowny
Copy link
Contributor

ezarowny commented Apr 5, 2016

I think what would happen is that we wouldn't inspect the frame at all if locals.enabled was False. Only the trace would be reported.

@ezarowny
Copy link
Contributor

ezarowny commented Apr 6, 2016

After clarification, would the proposed changes still address your original PR?

@sibson
Copy link
Contributor Author

sibson commented Apr 6, 2016

yes I think it addresses this PR.

The grandfather of this was #91, where the issue around str(exc) is still outstanding, https://github.com/rollbar/pyrollbar/pull/91/files#diff-1fcbac3a897889dbaf8984fdfe14c99cR633.

A proposed solution involved #67 but I don't think it would be secure by default as it stands.

@ezarowny
Copy link
Contributor

Closing in favor of #113

@ezarowny ezarowny closed this May 26, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants