-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
is the test suite something that could be factored out/generalized? #40
Comments
cc @mstange and @Jake-Shadle |
I think something like this would be great. The MacOS dump_syms stuff I added was loosely based on the end-to-end testing I did in https://github.com/EmbarkStudios/crash-handling/tree/main/minidumper-test though there I at the moment just doing the basic "does this minidump contain the expected crash reason" without checking the symbolicated stack traces for the thread(s) also match, but that was something I wanted to add eventually. I guess the one big difference to me is that the dream scenario you specified above feels a bit too manual for me, but maybe this is just a different workflow for doing changes before making automated tests? |
yeah i made the dream slightly manual just so we could do absolutely whatever we want and extract only the parts we care about for individual runs. i.e. "this old dump_syms version had this problem, I want a sym file showing that for my tests, so I will making automated tests I think would look closer to |
basically it would be useful as a
|
Ok here's a rough diagram of what a full deployment architecture looks like. I've broken the architecture into 3 major sections:
The client and server are further broken up into separate processes. Rough atlas:
Note that Mozilla (and Sentry?) don't use any of the "client" stuff right now except for minidump-writer in some specific situations. Migrating to minidump-writer completely is desirable for Mozilla (kill breakpad), but the other stuff is less clearly important to us. graph TD;
subgraph build
cargo
dump_syms
http-symbol-sender(http-symbol-sender)
binary{.exe/.pdb}
sym{.sym}
end
subgraph client
subgraph app
YOUR_ACTUAL_APP(YOUR ACTUAL APP)
minidumper::Client
crash-handler
end
subgraph crash-monitor
minidumper::Service
minidump-writer
http-dump-sender(http-dump-sender)
context{CrashContext}
dump{.dump}
end
end
subgraph server
subgraph crash-server
http-dump-receiver(http-dump-receiver)
minidump-processor
minidump
structMinidump{Minidump}
report{report.json}
end
subgraph symbol-server
tecken(tecken)
http-symbol-receiver(http-symbol-receiver)
end
subgraph crash-stats
socorro(socorro)
end
end
cargo -.-> binary
binary -.-> dump_syms
binary -.-> deploy(deploy)
dump_syms -.-> sym
sym -.-> http-symbol-sender
crash-handler ---> minidumper::Client
minidumper::Client ---> minidumper::Service
minidumper::Service ---> minidump-writer
minidump-writer -.-> dump
dump -.-> http-dump-sender
minidumper::Service ---> http-dump-sender
minidumper::Service -.-> context
context -.-> minidump-writer
http-dump-receiver ---> minidump-processor
http-dump-receiver ---> minidump
minidump -.-> structMinidump
structMinidump -.-> minidump-processor
tecken -. .sym .-> minidump-processor
minidump-processor -.-> report
report -.-> http-dump-receiver
http-dump-receiver -. report.json .-> socorro
http-symbol-sender -. .sym .-> http-symbol-receiver
http-dump-sender -. .dump .-> http-dump-receiver
graph source
|
You've done some incredible stuff in there making it possible to generate dumps, use dump_syms, and then load dumps again. I've been wanting a tool that is two binaries,
minidump-harness
andfutility
.futility
(it seems like
src/bin/test.rs
is basically this)The futile utility application, providing a "crash suite"
minidump-harness
(it seems like
tests/
is kinda this)Orchestrates building/analyzing/torturing futility and storing/diffing the results. it would serve three roles:
The first is straight-forward and you mostly have that setup (although #30 still needs to be done). The second is more subtle. You can see my initial attempts at this problem with my socc-pair utility which largely automates diffing/benching a local version of minidump-stackwalk against what mozilla has in production. This is great for testing if I fixed a bug in production, but it's still more manual than I'd like.
The Dream
The Design
minidump-harness run $DIR
would:minidump-stackwalk --cyborg --verbose=trace --symbols-path=$DIR/symbols
on each minidump, putting:minidump-harness diff $DIR1 $DIR2
would:Right now you have some of these things, but a lot of the test implementations are in the harness and kind of manually adhoced. I don't fully understand your design, and all the constraints there are. I'm happy to do a lot of legwork here, but you've clearly done a lot stuff that's worth using/extending instead of me kicking off a green-field project.
The text was updated successfully, but these errors were encountered: