Sherlock Testing Resounding Success

Hi everyone,

It seems like the testing of the Sherlock games has been a success. There were lots of bug reported, and all the ones reported so far have been fixed. The foreign language versions haven't all been fully tested yet, but hopefully now the immediate crashes with conversations and inventory in both Serrated Scalpel and Rose Tattoo foreign language versions have been fixed, and the rest of the games can be tested. I'd like to thank everyone who's tested so far for your efforts, and feel free to post any more bugs you come across. Though hopefully there won't be too many more to find :). And if one else has copies of either game, particularly different foreign versions, any other testers would be appreciated.

Right at the moment I'm at the sweet point where all the outstanding bugs for Sherlock have been resolved, so unless new ones come in, I can turn my attention to other things.

So, whats coming next for me? Several things:

Serrated Scalpel 3DO

Serrated Scalpel 3DO still isn't completable. There are some areas, such as the darts game to fix up, and there are still some differences in sprite positioning that would need to be accounted for. It's somewhat constrained by the fact that we don't have any original source for it, and I don't have any experience with reverse engineering 3DO games. It may simply be a case of do as best we can with hardcoded fixes as necessary.

The 3DO version also has a few missing things compared to the PC version; notably it lacks the journal the PC version has. A "nice to have" for the future would be to reintroduce it, so that the 3DO version could be the definitive version of the game, containing all the PC version functionality as well as the video for all conversations. We'd likely create a tool that extracts necessary graphics for UI buttons and the journal background from the PC version, and produce a Dat file that ScummVM can use automatically when playing the 3DO version.

RTLink Overlay manager

Next, there's the RTLink/Plus overly handling in Companions of Xanth. I'd previously had some luck writing a tool to process Rex Nebular and create a flat executable with all the segments suitable for disassembling, but doing the same for Companions of Xanth proved elusive. Over the years I made several attempts, but none bore fruit. Until now. As of yesterday, I was finally able to write a new version of the tool that successfully generated a flat executable that could be disassembled. So one day, after a great deal of disassembly work, ScummVM may support the game.

My only disappointment is that RTLink/Plus seems to have had quite a number of variations. Rex and Xanth's RTLink mechanism were fundamentally similar, with all the RTLink code, segment list, and method thunks/stubs in the executable and/or overlay file. For several other games with RTLink that I tried, however, they seem to use a bizarre alternate method where a file called 'rtlinkst.com' is loaded, then an '.RTL' file for the game, and finally only then the game executable. Which means that my tool doesn't work with them, and I'd need to figure out this new mechanism from scratch if I want the tool to be general-purpose enough to handle any RTLink game anyone might want to use it on in the future.

Guess that can be another long-term project to muck around with. Hopefully it won't take as long as it did to finally get Xanth properly disassembled. :). I'll probably make another posting in a day or so about RTLink in more detail, for those that are interested.

Might & Magic, World of Xeen

Next there's my work on re-implementing Might & Magic - World of Xeen (and Clouds of Xeen, Dark Side of Xeen, and Swords of Xeen). With some free time last weekend, I finally returned to working on them, and likewise was finally able to properly disassemble the  algorithm the original used for scaling. So as of now, sprites are now correctly scaled, as before I was only using a rough guess scaling code I'd nicked from another game engine. I've also fixed some other drawing bugs for drawing outdoor areas (outside town). So as of now the game scenes now display properly! :)

That's right, you can now walk around, fight monsters, go visit the various buildings in town, and even leave the town! In fact, most of the functionality for the games are already implemented. Apart from lots of testing and minor bugfixes that will be needed, only the following major areas remain to be implemented:

• Introduction/ending sequences for the games
• character management, and title screens.
• Sound. I'm hoping I can simply slot in one of ScummVM's audio decoders without much further work.
• Savegames

At the moment I'm concentrating on getting World of Xeen (which combines Might & Magic 4 and 5 together) working. But then afterwards I'll implement separate support for 4 and 5, as well as for Swords of Xeen. It might even be feasible to handle Might and Magic 3 - Isles of Terra as well, since I'm given to understand the engines are nearly the same.

Those interested in following the progress can see it at the brand-spanking new RogueVM Github account. That's right; after all the years of idle talk, I've finally set up a place to properly store RPG related game engines. No website yet, but at least it's a start. :) I'll likely spend the near-term focusing mostly on finishing support for the game before I move onto any other adventure games, considering how far along the engine is already.

Return of the Phantom

Strangerke has put a lot of work recently into implementing scene logic for Return of Phantom, the next MADS game that was published after Rex Nebular. There are quite a few stubs for missing engine functionality that was added in though. So when I do return to working on adventures, it will likely be to assist him in completing the game.

DreamMaster.

Rose Tattoo is in progress

Hi all,

Work is progressing well on Rose Tattoo, and as of tonight I hit a major milestone.. the entire game intro sequence is now completable. See the screenshots below:

There are still some minor graphic glitches at various points, and one of the scenes which is a double-side scene isn't properly scrolled horizontally yet, but even so, it's a great step forward in supporting the game. We're actually lucky in that Rose Tattoo implemented all of the introduction using standard game scenes. So it saved a lot of effort implementing manual introduction code like had to be done for Scalpel.

Now that the introduction is working, more or less, I'll be devoting more time to implementing the game-play. I'd already been spending some of my working on it, so some interaction is possible.. you can look at objects, open up the inventory, and conversations with characters are partially working. The game map is also already implemented, so you can get to other game locations as well.

On another subject, I had good luck implementing the original EA logo at the start of Serrated Scalpel. I was able to complete support for it in the TsAGE engine, and then used that as a basis for copying necessary code into the Sherlock engine. With some most welcome assistance from others, the EA logo at the start of the game now displays when you start the game, just like the original does.

Finally, on yet another tack, there have been some promising first steps towards supporting the 3DO version of Serrated Scalpel by m-kiewitz, with some assistance from clone2727.  The 3DO was a superior version of game, and included 16-bit color, video portraits, and full speech for every conversation in the game. Supporting this would be great. It would be nice if, one day, the 3DO version could be re-released with ScummVM, so a wider audience could properly enjoy the game.

Happy Birthday, Sir Arthur Conan Doyle

Hey everyone.

Looking back it seems, to my chagrin, that it's now been over a year since my last news posting was over a year ago. Whoops :P. Not that I haven't been keeping busy over the last year or so, with the release of Voyeur, Amazon - Guardians of Eden, Rex Nebular (finally), and of course, the newest game.. The Lost Files of Sherlock Holmes: Case of the Serrated Scalpel.

Many thanks go to EA for providing us access to the original source for this game. Also to forum user sirlemonhead, and to James Ferguson, who patiently over the last few years tried to make this happen. I've always been a big Sherlock Holmes fan, so it was fun to work on this project.  It feels fitting to merge the game into master on the 22nd May, which is the birthday of the character's creator, Sir Arthur Conan Doyle. The engine isn't quite ready for serious testing yet.. it's still missing music playback, and there's also a starting logo animation that's not present. It should be finished soon, though, so expect to see an official testing announcement in the near future.

So.. as things stand, what I am up to right now?

The Logo
Apart from the game proper, one of the more interesting things about the game is at the very beginning, where the publisher EA logo is shown. This logo display was actually implemented in a separate executable using the TsAGE engine, of all things. Original source for this couldn't be located, so it means that I'm having to reverse engineer it. Luckily, since we've already had experience with several other TsAGE titles, I've been able to make excellent progress in figuring out all the various TsAGE classes and their methods within the executable.

At the current point in time, I've identified the bulk of the core TsAGE classes, and the custom logic for the "game", which is contained a single scene class. This scene class consists of several scene objects, a few palette containers, and an "Action" class for coordinating what happens in the logo display. There's only a minor variation in how object sprites are loaded compared to the games that I still need to figure out.

I've already started implementing a new sub-module within TsAGE for the game logo. Once I finish that, it will be easier to analyse all the movement and frame changes of the images with the scene.  Hopefully, based on that, I'll be able to simulate a similar sequence in our new Sherlock engine using the bare necessities from TsAGE - likely just the RLB archive manager and sprite loader. Particularly given the thoughtfulness of EA in providing us access to the original source, it would be nice to give them (the company) proper attribution by showing their logo just like the original game does.

The Sequel
Apart from that, we have also been given access to source for the sequel, The Case of the Rose Tattoo. Implementing this is likely to be much more challenging, as the sequel changed over to a 640x480 display, and significantly altered the user interface. As such, it's likely it will need a lot of re-factoring of the code base to add support for it to the existing engine. If you thought a lot of re-factoring was done during the pull request, you 'aint seen nothing yet. :)

I'm also somewhat constrained by the fact that the original uses DOS4GW and a 32-bit code segment. Whilst we do have the original source, I need to be able to run the game in DosBox so I can actually see the code running, and check registers and memory contents at given points in the program. I've had some significant trouble with the DosBox debugger, trying to set breakpoints in the code so I can inspect the game's state.  Doing so crashes either crashes DosBox, or the game executable, or the breakpoints simply aren't hit.

So far, I've only done some preliminary loading of scene resources in the second game, and the lack of a way to display the program state meant that I had to take a more laborious route of poring over the various resource structures and scene loading code in both games, to try and figure out what the differences were between the two, so my code can support it. Likely, as I proceed with implementing more of the game, this will cause real issues that will make finding bugs a lot harder.

World of Xeen
It's been somewhat on a back-burner since I started work on the Sherlock Holmes games, but I had previously made real progress on re-implementing World of Xeen using the ScummVM framework. See below:

As you can see, I have much of the game interface implemented. You can move around, fight monsters (with a few minor glitches), and even leave the town. There are really only a few main areas left to implement, which includes sound support, logic for all the various spells, savegames, and the intro/ending cut-scenes. I probably won't return to working on it until after Rose Tattoo is finished, though. But when I do, I don't anticipate it will take long to finish the remaining areas, and then it would simply be a matter of playing the game through in earnest, identifying and fixing minor bugs as they're identified.

Of course, as an RPG, World of Xeen is a bit outside the scope of ScummVM proper. At that point, it may be time to finally launch the RPG sister project Strangerke and I have been wanting to do. :)

Rex-tacular

What a difference a mere week can make in terms of progress. At the beginning of last week only the basic background from one scene was showing, although there was more code implemented in the engine that wasn't yet working. Now, a week later, behold the progress of Rex Nebular on ScummVM:

That's right, in a rush of effort this week, the Rex Nebular engine has already passed several milestones in development:

• The user interface is now partly implemented. As you can see from the above picture, the user interface background is showing, with the sections for the actions, items, and the animated inventory item showing.
• The scene background is now displaying, along with basic sprite animations within the scene. The fires are burning, and the hamster "auxiliary power" :) is running.
• Basic player display. Rex is in his default position for the scene, facing the door.
• Behind the scenes, we've already completed the logic for a couple of the game scenes, although we're obviously not able to test them properly yet.

The focus for the upcoming week will now be on the beginnings of user interaction and action handling. This means getting walking working, selections in the user interface, and actually being able to execute actions. A lot of the needed code is already in place, but will require some heavy duty debugging and comparison against the original to figure out where things are going wrong. We're somewhat hampered by the complexity of the core engine when doing debugging, but we'll get there eventually. :)

Rexing it Up

Work has been progressing furiously with the MADS engine, and Rex Nebular support. Whilst I was able to us some code from the previously engine version, a lot of it turned out to be more M4 specific than I'd anticipated, and had to be freshly disassembled and/or rewritten. Additionally, with my greater experience writing engines, I've been able to lay out a cleaner separation of methods into classes than previously, particularly keeping in mind extensibility for having separate game logic for the other MADS games. As a result, the new MADS engine is already significantly different from the old M4 engine.

After a great deal of implementation, not to mention debugging of my code and comparison against the original running in DOSBox, I finally have the background of my test scene showing! See below for the first 'new engine' view of Rex Nebular in all it's glory:

Particularly auspicious considering I see that Rex Nebular has just been released for sale on GOG. So presuming that the engine gets finished this time, people will easily be able to obtain a copy. :)

What's next? In order to reach my original milestone of having an animation sequence playing correctly, it meant that I needed to properly implement the entire frame step and rendering logic from the game, which had a great deal of code, and many different secondary methods that I've encapsulated into a multitude of classes for sprite sets, pending sprites, active sequences, text display, and lots of other things. Not to mention a pretty complicated precursor to M4's RGBList, where resources loaded in for both the scene and for sprites are allocated chunks of the palette space.

I'm currently focusing on debugging the standard sprite drawing, which is used by the animation class. I'm hoping with a bit more work, I can it to properly show the animation sequence. Doing so will also help ensure that all the sprite display logic for the scene also works correctly. This will make it easier to start work later on for full blown game scenes, since all the necessary sprite display will already be done, and I can concentrate on things like player movement, action handling, etc.

Explosive decompression

Looking back, despite my earlier promises of posting more regularly, it seems I've been a bit tardy in posting updates, and quite a bit has happened. So lets get up to date with what's been happening with me. Firstly, we finally got some proper bug reports for Return to Ringworld, and have been working to fix them. Well, moreso Strangerke than me, so thanks go to him for his work in fixing bugs when he has the time.  As of right now, the card mini-game still isn't fully functional, but at least the main game is that much more polished now than it was.

Secondly, as of Monday, I finally merged the next game I was working on, Voyeur, into the project's master branch. This is a weird little game that Strangerke put me onto, because the original DOS executable had debug information embedded in it, which made it somewhat easier to implement. Still a lot of work to understand the contents of methods, reimplement them, and get the game in a stable state. But it made a nice change of pace from all the TsAGE work, where we had to disassemble everything from scratch. As of right now, the Voyeur engine only supports the DOS version. Apart from DOS, there were also CDi and Macintosh versions released. The Mac version at least may be supported at some point in the future, although there is some extra complexities we'd have to worry about due to the Mac-specific data in it, such as rasterized fonts that the PC version doesn't have. Adding support for it might make a nice future mini-project for a Mac enthusiast.

So what's next? Apart from further bugfixes for both Return to Ringworld and Voyeur, I'm already onto my next project.. once again tackling the white elephant of Nex Nebular, for the 3rd time. This time, though, things will hopefully be different. Using my experience with disassembling the TsAGE sound system, I tackled the sound system first, which was the major previous remaining stumbling block. And as of last Sunday, I finally got it working! That's right, my new embryonic MADS engine was able to play back the explosive decompression sound you get when you fail the copy protection check. It was a wonderful feeling, after spending a week reverse engineering and then implementing the code, then having spent an afternoon trying to debug my code and compare bytes being written to the ScummVM FM_OPL driver against the port writes in the original running under DosBox, to finally get everything right, and hear the sound coming out of my speakers.

 It turns out that despite some funky assembly tricks in the player that I had to work around, the player itself is actually a fairly compact, simple implementation. I'm no sound expert (as my frequent prior complaints make clear :) ), but I understood enough to give the various methods in my code halfway decent names, and name at least some of the fields. Hopefully a further analysis, now that I have cleaner C++ code, will help me better understand what's going on, and give the rest of the fields more proper names.

Now that the issue of sound support is taken care of, how am I going to proceed from here? I have a lot of code back from the earlier combined m4 engine which had code from both Rex Nebular and Orion Burger. Part of the trouble with the earlier attempts, I think, was that we were overcomplicating matters. As such, I'll only gradually put in code as I need it, and not worry so much about M4. That'll help me keep the focus more on getting Rex Nebular to work. Not that I'm going to just chuck out all M4 code from the engine. I've already started work on making the core classes, such as the previously named 'M4Surface' class, having a cleaner structure more suited for multiple games. For example, it'll now have it's own class factory, which will abstract having a base MSurface class that will have different descendants that implement the various game specific load/display logic. That seems cleaner than littering the code with "if isM4()" checks, and having methods like "rexLoadBackground". As far as a caller is concerned, they'll just initialise a surface, and the factory will take care of giving them the right kind for their game.

With a cleaner separation of game logic into descendent classes, I'll be able to, for example, keep in place special cases we'd already figured out for Riddle of Master Lu, and make it that much easier to implement support for it later. So that should keep a certain team member who shall remain nameless happy.  :)

At the moment, I'm currently aiming for getting all the initial engine setup sorted out, and playing back the animation sequence for when you get the copy protection answer wrong. As I mentioned above, this will involve pulling some code from the old project to save time, but I'm also already doing a fair amount of refactoring and cleanup, such as changing char *'s to Common::String, using Common::Point's, properly commenting the code, and so on. Given the amount of previous work done on the old M4 engine and the mostly complete disassembly I had of Rex Nebular, we can hopefully expect rapid progress in functionality, and eventually enjoy Rex's escapdes as he quests for the vase he's been sent to find.

Thanksgiving with a Ring.

"They said it would it would never be done. They were wrong." After a slight hiatus at the start of the year, work has finally finished on our next TsAGE game, Return to Ringworld. The only remaining work still in progress is a card game available on the in-game consoles, and is completely unrelated to the main story.

This is a direct sequel to the original Ringworld game.. after saving the puppeteer race from extermination and uncovering some powerful ancient technology on the first game, Quinn, Seeker of Vengeance and Miranda Rees find themselves searched for as fugitives by all three major species, so they plan to go to Ringworld to hide. But it turns that their ship has suffered problems. And fixing them and getting to Ringworld will only be the start of their problems..

The ScummVM Team is proud to announce that Return to Ringworld is now playable in ScummVM using the latest daily builds, and ready for testing. As usual, all bugs should be reported to our bug tracker following our bug submission guidelines. While you play through the game, we would also love it if you could take some screenshots for us.