Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Debugger breakpoints: wrong bank #512

Closed
andrew-davie opened this issue Aug 11, 2019 · 84 comments
Closed

Debugger breakpoints: wrong bank #512

andrew-davie opened this issue Aug 11, 2019 · 84 comments
Assignees
Milestone

Comments

@andrew-davie
Copy link

It seems that when you have a breakpoint, only the PC address is used/recorded for the breakpoint. If I am stepping through code and I click in the breakpoint column to add a breakpoint, and then continue - the breakpoint can be (is!) triggered by access to the PC address from a different bank. This is not good behaviour, because I don't want to break on code in a different bank - I want to break on code in the bank I set the breakpoint in!

In other words, breakpoints should keep track of the bank they belong to. Yes, this isn't possible to know if you set a breakpoint via label ("break label"), but in that case you would want to break when PC = label in ANY bank. But when you do it manually, you're stepping through code, you most definitely do NOT want it to break whenever PC=addr. You want it to break ONLY when PC==addr AND bank=bank that was switched in when breakpoint was set.

In other words, I think breakpoints also need a bank field. Normally this should mean "any bank" but in the case where set via step-through and set-breakpoint-here type usage, then the bank should be constrained to the one you are currently in.

@thrust26 thrust26 added this to the Prio 2 milestone Aug 11, 2019
@sa666666
Copy link
Member

Also related to #217.

@thrust26
Copy link
Member

thrust26 commented Aug 18, 2019

We could use the already existing conditional breakpoints (e.g. generally or via config or in case of a multi bank ROM).

Currently all normal breakpoints (and traps) only check for the PC. This is done via a 64k bitset for speed reasons. But maybe this is not that relevant anymore?

@thrust26 thrust26 self-assigned this Aug 18, 2019
@andrew-davie
Copy link
Author

Keep using the 64k bitset for initial check, but once a breakpoint is discovered, then check if it is the correct bank and if not, ignore...?

@sa666666
Copy link
Member

I've though about this in the past, but never had time to revisit it. It also needs work in the UI, since clicking an address in the disassembly right now creates a naive breakpoint (ie, one that only looks at address, not bank). There needs to be a way to graphically set the kind you want (address AND bank). And perhaps have them be different colours/shapes too, to indicate which is active.

I agree with @thrust26 that the easiest way with our current infrastructure is to just use a CBP. So the former looks like this:
break $f050
while the CBP would be like this:
breakif {_pc==$f050 && _bank==x} (where 'x' is the bank we want)

So the infrastructure is there, just with a little more typing. Presenting a choice to the user in UI mode would then allow the code to 'write' that breakpoint for us. In the end, I think that's the best approach.

@thrust26
Copy link
Member

thrust26 commented Aug 19, 2019

Keep using the 64k bitset for initial check, but once a breakpoint is discovered, then check if it is the correct bank and if not, ignore...?

I thought about that too. A problem are multiple breakpoints at the same address but different banks. We would have to add a counter for each address. Else deleting one breakpoint would affect the other ones too.

Maybe something for later, if the new solution turns out to be too slow.

@sa666666
Copy link
Member

I still think a CBP is what we need. The bitset is for a simple use-case: unconditional breakpoint. As soon as we add an extra condition (ie, a corresponding bank), then it becomes a conditional breakpoint, hence CBP. The infrastructure is already there; there's no need to complicate the bitset approach to do something it wasn't designed to do. Just use a CBP and be done with it. Also, they're not that slow.

@sa666666
Copy link
Member

In any event, the UI work still needs to be done no matter what approach we take. It should be easy to just point and click, and add the different types of BP without having to type anything. But until that happens, the typing approach works right now.

@thrust26
Copy link
Member

Agreed. I will have a look.

@sa666666
Copy link
Member

Well, I didn't mean to imply that it had to be you, or even that we need to do this before 6.1. I just wanted to point out that this is more of a UI issue than anything else. The bankend can already do it; we just need to extend the functionality to the frontend.

@thrust26
Copy link
Member

If we completely remove the unconditional breakpoints, we have no UI issue anymore.

@sa666666
Copy link
Member

There are two issues with going that route:

  • the unconditional BP are faster; but maybe that doesn't matter so much anymore?
  • more importantly, what if someone want to set a BP and not consider banks? Again, maybe that isn't a use case we need to worry about??

@sa666666
Copy link
Member

I defer to the experience of people actually using Stella for real 2600 development; you and @andrew-davie.

@andrew-davie
Copy link
Author

andrew-davie commented Aug 19, 2019 via email

@thrust26
Copy link
Member

And one could still define a conditional breakpoint manually, without providing the bank as argument.

@andrew-davie
Copy link
Author

andrew-davie commented Aug 19, 2019 via email

@sa666666
Copy link
Member

As to using the command-line, if I type “break label” then perhaps it could add a CPB breakpoint for EVERY bank at that label address. Then I get the original behaviour. If that’s not what I wanted I can disable each breakpoint that I didn’t want via UI as I hit it.

This is probably the most elegant solution. If we make all BP conditional, then the whole problem goes away (UI issues included). Then the only potential issue is that CBP are slower. But we can test that right now, before any work is started. I don't think it will be an issue on any modern computer, and even if it slows down slightly, that's to be expected when enabling debugging features.

@thrust26
Copy link
Member

thrust26 commented Aug 19, 2019

@andrew-davie Yup.

And you can type break label or break label bank. At least that's the plan.

Though removing break points added without bank by clicking in the breakpoint column needs some thought.:

  • We can either add N (one for each bank) breakpoints initially and then you remove one.
  • Or we add one breakpoint without a bank.
    • Then, if the first one is removed via breakpoints column, we can either remove that global breakpoint.
    • Or we remove that one and replace it with N-1 breakpoints in the other banks.

What's better?

@sa666666
Copy link
Member

This address mirroring in the 2600 really complicates things, doesn't it? 😈

@thrust26
Copy link
Member

thrust26 commented Aug 19, 2019

@sa666666 You won't be surprised if I tell you that I already started? I love challenges! 😄

One more thought: "break label" could alternatively automatically add the current bank. Then the problem described above is gone.

@sa666666
Copy link
Member

@sa666666 You won't be surprised if I tell you that I already started? I love challenges!

Sigh. I used to feel that way too, and I miss it. I'm just so overloaded with stuff now that I'm getting a little burnt out. Hopefully I can get back to the 'challenging' stuff soon too. Multi-bank disassembly is one that I really want to work on ...

@thrust26
Copy link
Member

thrust26 commented Aug 19, 2019

When I am back at work next Tuesday, I will feel more like you do now.

Multi-bank disassembly will become very challenging, DiStella was never meant for that and the code shows. I happily leave that task for up. 😈

BTW: Is there an issue for this?

@sa666666
Copy link
Member

Yes, #284. In fact you created it!

@thrust26
Copy link
Member

I just checked the maximum frame rates with multiple conditional breakpoints. The slowdown is quite noticeable on my notebook (i7-7820HQ):

  • 0 breakpoints >600 frames/s
  • 5 breakpoints: ~585 frames/s
  • 10 breakpoints: ~530 frames/s
  • 15 breakpoints: ~465 frames/s
  • 20 breakpoints: ~425 frames/s

So for slower machines we should consider some optimizations.

@sa666666
Copy link
Member

I don't think that's actually that bad. How many times would one have 20 BP at a time? And even if the machine is 10x slower and gets 60fps max, now maybe it only gets 40fps max (assuming this scales linearly). Of course I'm all for efficiency, but at the same time, if you're doing heavy debugging then you have to accept more CPU usage and potential slowdown.

@thrust26
Copy link
Member

Yea, the single thread performance of my Z8350 based mini notebook is only about 5x slower than the i7. So that should be sufficient.

And I did some minor optimizations (e.g. no bank check for non-bankswitched ROMs).

@sa666666
Copy link
Member

You can probably also extend that optimization to ROMs that support only one bank but still have bankswitching. Some of the more complicated schemes only report one bank, since it's much too complicated otherwise. In those cases the bank check is also redundant.

@thrust26
Copy link
Member

But couldn't their banks share the same addresses?

@thrust26 thrust26 pinned this issue Aug 23, 2019
@thrust26
Copy link
Member

thrust26 commented Aug 23, 2019

@andrew-davie
I am currently working on this. Since the parser cannot use ALL (only numbers), we have to use -1. Weather this indicates toggling in all banks (what I would prefer) or limits to the current one (like you prefer), or there will be separate command is undecided yet. But there will be an option to set breaks in the current bank only, and another one to set in all banks.

But here is another thing:
The real 6507 addresses are only 13 bits (addr & 0x1fff). So we have at least the following options for e.g. break f000:

  • set break only for f000
  • set 8 breaks for 1000, 3000, ... f000
  • set one break for 1000 (= f000 & 1fff, first mirror)
  • any more???

Problems:

  • In the first case, the debugger will only break if the PC has the correct 16 bit address. But the full PC address is only updated in case of JSR, JMP, RTS and BRK. Else the high bits may e.g. result from a previous bank with a different ORG and the break may never trigger.
  • The seconds option will get the breakpoint, but it will clutter the command result and 'listbreaks'. And e.g. break 5000 will clear all breaks created by break f000.
  • The last option may be irritating (or maybe not), because the command will be reported and listed as break 1000. We could store the original address for reporting and listing, but internally it will always be be 1000. So e.g. break 5000 would clear it.

The root problem are the upper three bits of the address. In reality they do not exist. But we developers got used to them and also used and still use them to workaround our tools' (e.g. DiStella) flaws.

BTW: If we decide for one thing or the other we should make this consistent for the debugger.

@DirtyHairy
Copy link
Member

I'd mask bits 15-13. If I understand correctly, this is option 3 😏

@thrust26
Copy link
Member

Correct. I am leaning towards it too. But I am not exactly happy with it.

@thrust26
Copy link
Member

I went for option 3 and also reimplemented the breakpoints which now use a hash map. So there are no slowdowns, even with a LOT of breakpoints.

As of now -1 for the bank enables a bp in all banks. This may change.

@andrew-davie
Copy link
Author

Thank you, I will try and test soon.
"We" could always have a different named command to distinguish the two.

@thrust26
Copy link
Member

Agreed.

IMO breakshould be reserved for the normal breakpoints, because it is logical to autofill the current bank like it does with the current address too. And it stays in sync with setting a breakpoint via mouse.

So I need a good name for putting a breakpoint in all banks. Then you will also be able to type just goodnamehere and get a breakpoint in all banks at the current address.

@andrew-davie
Copy link
Author

IMO breakshould be reserved for the normal breakpoints, because it is logical to autofill the current bank like it does with the current address too. And it stays in sync with setting a breakpoint via mouse.

This is NOT my preferred course of action, as I truly believe if I say "break address" then I should get a breakpoint no matter what bank happens to be switched in when the CPU hits that address. As stated earlier, if I want to break in a specific bank and only that bank, then I should specify it.

Having said that, and hopefully making it totally clear I do not think this is a good solution... how about using "breakall" as the command to implement it the way you want?

@thrust26
Copy link
Member

This is NOT my preferred course of action, as I truly believe if I say "break address" then I should get a breakpoint no matter what bank happens to be switched in when the CPU hits that address. As stated earlier, if I want to break in a specific bank and only that bank, then I should specify it.

Following that argument, break would have to set a breakpoint at every address. 😄 I prefer things to be consistent.

To me,breakall sounds more like the solution you prefer.

@andrew-davie
Copy link
Author

Following that argument, break would have to set a breakpoint at every address. 😄 I prefer things to be consistent.

To me,breakall sounds more like the solution you prefer.

I believe "break" should maintain its existing behaviour and break at every bank with the given address. Yes. And if you don't want that behaviour, then specify the bank you want. Or, disable the breaks as they trigger in the banks you don't want them, via UI click.

So no, "breakall" is not the solution I prefer. It is an ugly hack/compromise to get around the issues you are having with parsing numbers/text.

I prefer "break" to do what it has always done - break at every bank with the given address. And if that's not what the user wants, with the extra 'bank' option, you get the choice to limit it to one bank. THAT's what I prefer.

@thrust26
Copy link
Member

thrust26 commented Aug 26, 2019

To me we fixed a long standing bug with break command and toggling in the ROM listing (thanks for remembering us). For break both parameters are optional and automatically filled with the current values if not provided. Just like when clicking into the ROM listing. So that is fully consistent behavior which is more relevant to me than emulating old behavior.

I suppose this is one of those cases where we both cannot agree, except that we agree on that. 😄

@andrew-davie
Copy link
Author

I'm not really wanting to argue this much more. But I will emphasise what I think are the important points regarding using 'break' in the debugger with a bankswitched game.

Of prime importance is that it is possible, even probable that the person doing the debugging has absolutely no idea of the actual bank in which code is living. All they have is a label (say, the entry to a subroutine). So, let's say they want to put a breakpoint at the entry to the subroutine 'DrawTheScreen'.

How do they do that in the new system via your method? They have to use 'breakall' - in other words, put a breakpoint in every single bank so that the 'DrawTheScreen' PC address will be caught.

In other words, what I expect to be the DEFAULT behaviour for most breakpoints usage when debugging (because you almost certainly do not know the bank), is now to use 'breakall'. You have therefore changed usage from 'break' (which used to do this - that is, effectively put breaks in all banks) to a new one 'breakall'. What has that actually achieved? Nothing but confusing existing users.

I think it's extremely unlikely and very rare that anyone will actually type the bank number when entering a breakpoint. After all, there is no bank number available in the symbol table, unless you specifically add a label for it to tie label/bank together somehow.

So to me, what you are coding for is the least used case of all. In fact the most used case is the previous behaviour (break on all banks), and the whole discussion was to allow those extra bank breakpoints to be removed if required. And that lead to having the CBPs added, and thus easy removal of the breaks that were not wanted.

In summary, then, I think your use-case is not well considered. The previous behaviour was the most likely scenario (and not, IMHO, a bug). It only needed the change to CPB to allow individual ones to be removed as required.

Perhaps we just use the debugger totally differently. In any case, hopefully I'll fall silent now and you can implement it however you think is best. I appreciate all your hard work on this!

@andrew-davie
Copy link
Author

Here is another use-case, which I have just encountered. Your game crashes. Stella halts, and you can see addresses on the stack. F809. You want to put a breakpoint on that location - but you don't, of course, know which bank it was from. The most common requirement here would be to want a break in EVERY bank so you could find out where it came from.

@thrust26
Copy link
Member

thrust26 commented Aug 28, 2019

@andrew-davie
First, I get your points even though I seem to develop and debug differently (knowing my banks). So they are relevant to me. I hope for the same vice versa for my arguments.

The problems we are facing are a direct result of multiple complexities:

  1. the 6507 address space is 13 bit only
  2. for extended space, we need bankswitching
  3. neither DASM nor DiStella can provide labels with the correct bank
  4. the PC may or may not have the upper 3 bits set as expected, so breakpoints may be missed

Previously the break command and click did not care for any of them. And now we are trying to fix that. Providing a bank is only part of the fix (2.), reducing the break addresses to 13 bit fixes 1. + 4. But that makes 3. even worse, because the upper 3 bits have a meaning during development because they are often used to differentiate banks. I will come back to this.

When designing a UI, the complexity should be mitigated by the design. Making the UI consistent is a major point here. Keeping it backward compatible is another one.

Setting and clearing a breakpoint via break command or via clicking into the ROM list behaved identical in previous versions. This is fine. Therefore we would break backward compatibility and consistency and increase complexity if we would make the behavior different now. And the break command itself would become inconsistent too: One argument would be defaulted to the current state and the other one not. So that's why I decided to provide the current address and bank if an argument is not given.

I then added the option to break in all banks to suit your needs (even though you are not happy with it). But by now I realized that this is not sufficient and your suggestion too. When you are searching for a label and its bank, the upper 3 bits usually (depending on coding style) become important. When you use the upper 3 bits to differentiate banks and there are no more than 8 banks, each label becomes totally unique. And even for more banks, they are still quite unique. So there is no need to set breakpoints in all banks (if you assume that the PC is set correctly, see 4.). Instead the break command should use all 16 bits in this case.

Therefore, depending on the situation, it makes sense to search for 13 or 16 bit addresses. I do not like to handle this via extra parameter as it adds extra complexity to the UI. Therefore we need a new command for the latter (e.g. break16 or breaklabel). This new command will not even need a bank, it will (usually) find the correct bank from the upper 3 bits and therefore will default to any bank. And when this breakpoint has been found, I plan that the user can convert it into a normal break address bank one easily (disable and re-enable breakpoint). And last not least, this will also avoid cluttering the breakpoint list with unnecessary or even invalid (middle of instruction or data) breakpoints.

So that's my plan, the best solution I found by now. But I am open for more arguments and suggestions as long as my points (especially UI consistency) given above are considered too.

@thrust26
Copy link
Member

thrust26 commented Sep 4, 2019

Any more feedback on this one? Else I will close it in some days.

@andrew-davie
Copy link
Author

Initial test has me confused. I have 3E bankswitching, and put a breakpoint on a label I KNOW is in bank $F. "Reset". That's the start vector entry point to the game. So, 'breaklabel Restart' and then 'run'. It breaks, but instead of in bank $F it's showing me a correct disassembly in bank $B. I switch the bank to anything via the manual bank-select droplist in the 3E tab, and nothing changes.

I think the issue here is that the fixed bank ($F) is NOT one of the selectable banks - that is, the disassembly is showing the non-bankswitched code, but the title of it is as if it's a bank. Perhaps it would be useful to change the wording to include bank # ($F000-$F800) + fixed ($f800-$FFFF) or at least something. I know, this was my misunderstanding, but still...

Back to 'breaklabel'... it worked as advertised for a label in the fixed bank.
It did not work for me in a label in code that is copied to RAM. Specifically, the ORIGIN of the code was set to 0 (hence the label symbol is located at $000B). This code is copied into RAM which lives at $f800, so the PC address would be $f809, but the label is $0009, so there's that.
I would be happy to relocate the code/assembly which shouldn't make much difference, I think, but at least here is an example where using breaklabel does not work for the intended use-case, that is copying code to multiple banks and breaking on the label address.
I know this is a marginal case, and perhaps it could be argued that my code should not have an origin at 0. Let's discuss?

@thrust26
Copy link
Member

thrust26 commented Sep 4, 2019

I have to check the first case. Actually I am not quite clear what happens. Does it break at the wrong place or is the disassembly shown just the wrong one? Or is just the bank number display "wrong"?

"breaklabel" will of course only work correctly if the origin on assembly matches the one on execution. There will be cases were this problem circumvented and maybe there are other cases where this turns out difficult to impossible. The complexity strikes again here and probably we have to find a good compromise here. E.g. with 1K slice sizes, code with was assembled in slice 3, could go to slice 0, 1, 2 or 3 (even changing and/or multiple ones). There is no way for Stella to know this.

In your case, a correct ORG should fix the problem.

@andrew-davie
Copy link
Author

The first case:

It broke in the correct place, say $FC2F. That's in bank $F (fixed).
But the bank being displayed was (say) $B. Even when I went to the 3E tab and manually switched banks, it still displayed the correct disassembled code, because no matter what bank is switched via $3F, the fixed bank is static. So, code in $Fc2F is always visible.
It was just a misunderstanding on my part on what I was seeing. I thought that since it said "bank $b" I was actually looking at code in bank $B. But bank $B was switched into address range $f000-$F7FF, and of course the fixed bank is switched to $F800-$FFFF so of course I always see code at $FC2F, no matter what bank is switched in. In that case, I was suggesting that the title to the window could perhaps state that I am actually seeing the switched bank AND the fixed bank.

@thrust26
Copy link
Member

thrust26 commented Sep 4, 2019

Yes, that's irritating. For a general solution, we would have to either list all slices (only two in your case). IMO that's probably irritating as well.

Or we detect the bank from the address and react accordingly. So when it breaks in the first slice(s), display "bank #N" from that slice, else we display e.g. "bank #15" or "fixed bank".

@andrew-davie
Copy link
Author

How about put a little colour box next to the bank # display in the title, and use that colour as the background for the displayed bank ONLY. Thus, if you see a background of a different colour, you know it's NOT the bank displayed. Also, when you click anywhere in the disassembly window, the bank # should update to show correctly what bank is currently being seen at that point/line?

@thrust26
Copy link
Member

thrust26 commented Sep 4, 2019

That would probably require a lot of work. The disassembler and the breakpoint handling are in very different places of the code. Making them interact would need quite some refactoring.

@andrew-davie
Copy link
Author

That would probably require a lot of work. The disassembler and the breakpoint handling are in very different places of the code. Making them interact would need quite some refactoring.

But the UI already has the ability to know what address a click is. You can set/clear breakpoints on any line, and the current PC line is highlighted... so clearly the address of each line is available. Given that, it would be pretty straightforward in the draw of the disassembly to change the background colour based on address on that line...?

@thrust26
Copy link
Member

thrust26 commented Sep 5, 2019

Yes, that would be possible. But that wouldn't be sufficient.

The root problem is, that there is no interface in the Cartridge class which provides information about the memory layout of the 4K. So we do not know the detailed bank slice sizes in the debugger. This is handled internally in the Cart... classes and in the according widget classes. That would be ~100 classes affected.

@thrust26
Copy link
Member

thrust26 commented Sep 7, 2019

@andrew-davie
I just enhanced the bank returned for some cart classes (incl. 3E). This now considers the address. This should help with the breakpoints (breaking and displaying better).

@andrew-davie
Copy link
Author

@andrew-davie
I just enhanced the bank returned for some cart classes (incl. 3E). This now considers the address. This should help with the breakpoints (breaking and displaying better).

TY!

@thrust26
Copy link
Member

I suppose this is as good as it gets for now.

@thrust26 thrust26 unpinned this issue Sep 18, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants