Friday, May 23, 2008

It's a real pleasure to be able to be able to return to proper disassembly work after the interval I had to spend writing the delinker tool. Subjectively, I know that writing the tool was a necessary step that would save me issues in the long run, but it was still annoying to have to write and then spend all the time fiddling with it to get it work, particularly since I'd never really bothered to look at the structure of executables, and was learning from scratch.

Getting stuck into Rex Nebular, I've started debugging it using the DOSBOX debugger. I was a bit unsure about using it at first, but it's turned out to be an excellent tool. Back when I was disassembling Lure of the Temptress, I made do with DEBUG, which wasn't bad, but obviously had issues with trying to debug graphics programs.. I'd actually had to hack the game executable to stop it setting graphics mode so that I could use DEBUG to trace through all the startup code. Now with DOSBOX, I can trace anywhere in the program, since the debugger itself is in a separate window, which is wonderful.

About the only real annoyance remaining is the RTLink library itself. Although I do have a full disassembly based on a processed executable, I still need to do my debugging on the original executable (if the game worked without the extra space it provides, they wouldn't have needed it in the first place). The issue is that due to the way RTLink works, a call through a proxy stub can actually unload the calling code, which then gets reloaded when the called function exits.. but not necessarily in the same place.

Which means that you might try stepping over a function call, and suddenly the entire game sequence will play out, because the location returned to after the function call may now be at another segment address. Confused me at first, until I happened to trace out of a method and realised what was happening. :)

No comments: