So I’ve been working on improving the efficiency of the screen scraping for JStellAI. Its been brought up to about 1/4 real time speed on my laptop, so on a powerful machine like Dice, my guess it will be somewhere between 1/2 speed to real time. This is much faster than Stella AI, but of course the object recognition here is much more primitive.
I don’t think there is any way to make the screen scraping much more efficient than it currently is. It currently just makes one copy of the video buffer, iterates over it once, from each location starting a bounding box if there isnt already one of the color at that point. The only other thing I can think of is using the boolean arrays for each color, and making threads that do the bounding boxes on all the squares, but being that I expect the other cores to be busy with other duties for Moola, I think there are diminishing returns to be found there. Aside from that, I’m going to look into other parts of the emulator and see if there is anything that is artificially slowing the clock or anything like that.
Additionally, I added some Pitfall-specific code to screen out the black border on the bottom and left side of the screen. This is necessary since otherwise a black bounding box is made over the entire screen which makes it impossible to identify holes. Aside from that there is nothing specifically for pitfall.
I’ve changed the display code to draw just the outlines of the rectangles because its the easiest way to see whats going on. Another video below:
There are still a few frames that look a little funny – I need to inspect the log to see if its the java video tearing or an actual problem in the logging. Aside from that I think this is more or less done.
As I’ve said before, I’m a bit concerned about the type of features this is producing. For example, I think that with this implementation, the ladder and the floor look like the same object, so learning the difference between the two seems like it may be tough, but I guess we can address that when/if it is actually an issue.
Next step is to get the input hacked up so I can actually control JStellAI programatically and output the proper logs that contain the actual actions. And then once that is together I have the task of getting all of this together under Moola.
That being said, there is a pretty practical question to ask here, which is “Are we going to use this instead of the other stuff?” I’d hate to keep putting time into this only to have it not being used in the end.