|
|
Acorn Arcade forums: General: A question for the emulator teams
| |
A question for the emulator teams |
| |
This is a long thread. Click here to view the threaded list. |
| |
|
Jon Abbott |
Message #120075, posted by sirbod at 23:15, 13/4/2012 |
Member
Posts: 406
|
For point 11 on the !ADFFS thread, I need a way of scripting an emulator from a file. The file will contain a list of hardware specs / OS versions which need to be systematically started up and a list keyboard / mouse clicks, with X, Y coordinates and pauses, which need to be performed and a screenshot taken at a specified time.
Can you help? We basically need to create a test suite to run 300+ games to test compatibility on a wide range on setups.
[Edited by sirbod at 23:18, 13/4/2012] |
| |
 [ Log in to reply ] |
| |
|
Jeffrey Lee |
Message #120076, posted by Phlamethrower at 23:46, 13/4/2012, in reply to message #120075 |
 Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff
Posts: 14986
|
I should be able to add that functionality to ArcEm, if you're interested.
Can you explain why you're so interested in creating a method to patch old games? In my opinion, patching is a dead-end route - as you go forwards through hardware/OS versions you'll find that creating patches gets harder and harder, until you find a hardware/OS combination where patching a game to run on it would be an almost insurmountable task. I'd say that emulation is a much better way of preserving old games. |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120077, posted by sirbod at 13:21, 14/4/2012, in reply to message #120076 |
Member
Posts: 406
|
I should be able to add that functionality to ArcEm, if you're interested. Thanks for the offer, but isn't ArcEm ARM2 only and vicariously RISC OS 3.1 and below? Sorry, I'm not familiar with either ArcEm or the hardware requirements for RISC OS.
I must recode to be ARM2 compatible to cover A4xx - best add that to the list of things to do.
Can you explain why you're so interested in creating a method to patch old games? Good question...I'm only really interested in getting everything to work on all retro hardware and official Acorn OS versions, by that I mean up to RISC PC and RISC OS 3.7. Although there's plenty of folk on newer, a lot of it is still actively maintained and updated - so, as you quite rightly point out...a moving target and can be covered by emulation.
My initial start for this project was to get software working on actual Acorn hardware, if by doing this if fixes some for newer kit then great.
I'm going to leave StrongARM out for the moment, as the split cache is going to make patching a massive task, that will need code analysers to be written and a massive amount of play testing and recoding...something I can go back too once we have most games working on 3.7 up to ARM710.
Ultimately, this all depends on the interest from the community. There's enough talent here to do this, it just needs some effort from a good organised team.
There is interest in preserving all the Arc games and I'm certain we can hit a near 100% back catalogue in a few years as there can't be more than 500 published titles. They just need tracking down imaging and covers/documents scanning.
I'm prepared to do the bulk of the core coding, if people are prepared to put a small effort in to help in collation and testing.
EDIT: Just noticed the CVS version adds ARM3, so yes, I'll take you up on that offer. I'll draft a detailed list of commands we'll need and post later to get your thoughts.
[Edited by sirbod at 13:44, 14/4/2012] |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120080, posted by sirbod at 16:04, 14/4/2012, in reply to message #120077 |
Member
Posts: 406
|
Commands I believe we'll need within the script file:
VM "<directory name containing roms>", <arm type>, <vidc chip>, <ram>, <vram>, ... others I've missed - should restart machine immediately with these settings, allowing multiple OS's etc to be tested in one script file.
KP "<string>" - key presses to insert into keyboard buffer
KD <key> - hold a key down
KU <key> - lift a previously held key
MP <x>, <y> - position mouse pointer
MC <L/M/R> - mouse click
MD <L/M/R> - hold a mouse button down
MU <L/M/R> - lift a previously held mouse button
PZ <n> -pause script for <n> milliseconds, leaving machine running
MF "<string>" - should issue SWI OS_CLI 'ADFMount "<string>" '
SS "<file name>" - save a screen snapshot
QT - immediately quit and close emulator
; - commented line
One command per line in a plain text file. Emulator needs to accept this file via command line entry, so we can front-end it into a test suite.
[Edited by sirbod at 16:10, 14/4/2012] |
| |
[ Log in to reply ] |
| |
|
Martin Bazley |
Message #120081, posted by swirlythingy at 18:45, 14/4/2012, in reply to message #120080 |
Posts: 441
|
MP <x>, <y> - position mouse pointer
MC <L/M/R> - mouse click
MD <L/M/R> - hold a mouse button down
MU <L/M/R> - lift a previously held mouse button Are you planning to use these for navigating Filer windows? Given that the position isn't guaranteed (and changes during a desktop session depending on how many you've already opened) that's dodgy in the extreme. May I suggest a more prosaic solution - a command to force the emulated desktop to process a given star command, such as "Filer_Run ADFS::0.$.!Zarch"? |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120082, posted by sirbod at 20:41, 14/4/2012, in reply to message #120081 |
Member
Posts: 406
|
They were for in-game, clicking on "next disc" type prompts etc.
I was going to start them via an F12 command line, as you suggest. |
| |
[ Log in to reply ] |
| |
|
Peter Howkins |
Message #120083, posted by flibble at 22:52, 14/4/2012, in reply to message #120075 |
Posts: 809
|
Can you help? We basically need to create a test suite to run 300+ games to test compatibility on a wide range on setups. Unfortunately accuracy issues means all this would achieve was knowing if they worked on the emulator, not on real hardware. |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120085, posted by sirbod at 01:02, 15/4/2012, in reply to message #120083 |
Member
Posts: 406
|
Can you help? We basically need to create a test suite to run 300+ games to test compatibility on a wide range on setups. Unfortunately accuracy issues means all this would achieve was knowing if they worked on the emulator, not on real hardware.
It's a starting point, what it will test is RISC OS compatibility, which is the bulk of the issue.
Hardware compatibility issues will centre around MEMC, VIDC, FDC and ARM2/3. FDC we can rule out, as ADFFS will replace it. ARM2/3 are easily covered by emulation. MEMC and VIDC issues may or may not be picked up under emulation and will need verification on hardware where appropriate.
From the 140+ I've used to test over the past three months, the bulk of the issues center around poorly implemented !Run files, which a test suite will identify immediately. |
| |
[ Log in to reply ] |
| |
|
Michael Drake |
Message #120087, posted by tlsa at 16:31, 15/4/2012, in reply to message #120085 |
Posts: 1073
|
Native support for APDs in ArcEm, like Arculator has, would be good too. Since ADFFS supports them now, and they can be used on real hardware, they aren't an Arculator specific format any more.  |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120089, posted by sirbod at 18:15, 15/4/2012, in reply to message #120087 |
Member
Posts: 406
|
Native support for APDs in ArcEm, like Arculator has, would be good too. Since ADFFS supports them now, and they can be used on real hardware, they aren't an Arculator specific format any more. That's a big coding job...I've put in over 300 hours to add it to ADFFS!
If we can fix the RISC OS 2 specific games to work on RISC OS 3.1 or 3.7, you could switch to your emulator of choice and load APD's through ADFFS. |
| |
[ Log in to reply ] |
| |
|
Michael Drake |
Message #120090, posted by tlsa at 18:17, 15/4/2012, in reply to message #120089 |
Posts: 1073
|
Hmm, aren't there any memory overheads with using ADFFS? When running say RISC OS 2, the emulated machine may only have 2 MB RAM, and perhaps the game you want to run would need most of it? |
| |
[ Log in to reply ] |
| |
|
Michael Drake |
Message #120091, posted by tlsa at 18:45, 15/4/2012, in reply to message #120077 |
Posts: 1073
|
There is interest in preserving all the Arc games and I'm certain we can hit a near 100% back catalogue in a few years as there can't be more than 500 published titles. They just need tracking down imaging and covers/documents scanning. There's a good list of games here: http://glacoras.net16.net/archimedes.html
Zap says it has 415 instances of "<TR", so there are 414 games there. |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120092, posted by sirbod at 18:50, 15/4/2012, in reply to message #120090 |
Member
Posts: 406
|
ADFFS requires just under 1MB for itself and the disc image for 880kb floppies. APD's require a little more during loading and currently have a fixed memory allocation of 900kb - I've yet to code the size detection, which will require two passes of the floppy (ie slow loading)
ADFFS requires RISC OS 3.1 or above, so memory shouldn't be an issue under emulation. I've expanded my project to getting all games working on RISC OS 3.1 upwards, which - should it succeed, RISC OS 2 will be a thing of the past so far as games go. |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120093, posted by sirbod at 18:54, 15/4/2012, in reply to message #120091 |
Member
Posts: 406
|
There's a good list of games here: http://glacoras.net16.net/archimedes.html Thanks, I've added that to the "Help wanted" section of the root post on the ADFFS thread. |
| |
[ Log in to reply ] |
| |
|
Jeffrey Lee |
Message #120105, posted by Phlamethrower at 13:50, 17/4/2012, in reply to message #120080 |
Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff
Posts: 14986
|
VM "<directory name containing roms>", <arm type>, <vidc chip>, <ram>, <vram>, ... others I've missed - should restart machine immediately with these settings, allowing multiple OS's etc to be tested in one script file. Since you mention that the command specifies a directory containing ROMs, instead of a specific ROM filename, I guess you'd want it to run N instances of the emulator, with a different ROM for each instance?
MP <x>, <y> - position mouse pointer
MF "<string>" - should issue SWI OS_CLI 'ADFMount "<string>" ' Moving the mouse to absolute coordinates, and issuing SWIs, will be tricky to do without having support code running within the emulated OS.
Does the system need to cope with games which replace/bypass the OS keyboard/mouse drivers? If not, then I think it would make sense to try implementing the script parser as a module which runs within the emulator. This way you'd only need to write the code once (instead of once for each emulator), and it could even be used for testing on real hardware. The only situations where emulator help would be needed would be for taking screenshots (if you want games which use palette splits to be captured correctly) and for killing the emulator. |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120106, posted by sirbod at 14:26, 17/4/2012, in reply to message #120105 |
Member
Posts: 406
|
VM "<directory name containing roms>", <arm type>, <vidc chip>, <ram>, <vram>, ... others I've missed - should restart machine immediately with these settings, allowing multiple OS's etc to be tested in one script file. Since you mention that the command specifies a directory containing ROMs, instead of a specific ROM filename, I guess you'd want it to run N instances of the emulator, with a different ROM for each instance?
It should be a specific OS version, I just presumed the emulator kept them in separate directories. The script file needs be processed serially, ie only one copy of the emulator running at once.
MP <x>, <y> - position mouse pointer
MF "<string>" - should issue SWI OS_CLI 'ADFMount "<string>" ' Moving the mouse to absolute coordinates, and issuing SWIs, will be tricky to do without having support code running within the emulated OS.
I can code that
Does the system need to cope with games which replace/bypass the OS keyboard/mouse drivers? If not, then I think it would make sense to try implementing the script parser as a module which runs within the emulator. This way you'd only need to write the code once (instead of once for each emulator), and it could even be used for testing on real hardware. The only situations where emulator help would be needed would be for taking screenshots (if you want games which use palette splits to be captured correctly) and for killing the emulator. The screenshot is purely for checking if the game loaded okay, after a run we'll have a screenshot for every OS and can immediately see if the game at least loads.
Sounds like it needs to be a combination of an emulator extension and a module to handle key/mouse/CLI commands. The emulator will need to pass the commands one at a time via some form of handshake, as it's going to be restarting with the different OS versions and each OS version could potentially need different commands for basic !Run patching etc.
If you can tell me how I can implement the module, I'll make a start. I presuming it will be via an interrupt or some other clean way of interrupting the machine.
I see your point about coding it all within a module, I did consider that, but it would then entail manually running the emulator multiple times. I really want an fully automated process, so 300+ games can be tested in one hit.
Thanks for you help on this and your support so far. ADFFS certainly wouldn't be where it is now without your invaluable assistance. |
| |
[ Log in to reply ] |
| |
|
Jeffrey Lee |
Message #120109, posted by Phlamethrower at 16:07, 17/4/2012, in reply to message #120106 |
Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff
Posts: 14986
|
I really want an fully automated process, so 300+ games can be tested in one hit. Don't worry; a fully automated system is what I've got in mind. I'd imagine the system to work as follows:- There's a "frontend" program which manages setting up and launching each emulator.
- Prior to launching the emulator, any required configuration changes are made. E.g. if the emulator doesn't support certain settings being specified on the command line then the relevant config files would be edited, or ROMs copied to the right place, etc.
- As part of the configuration step, the entire script file would be parsed and converted to a binary form. This will then get appended to a module which the emulator will load on boot. This will avoid the need for adding any handshaking system to the emulator (allowing closed-source emulators, or real hardware, to be used for testing). For getting the emulator to load the module, the best place is likely to be the extension ROM folder - that way it won't be dependent on the machine having a !Boot sequnce (although I guess ADFFS would require a !Boot of some kind), and won't eat into the RAM that's available for games to run.
- If accurate renditions of palette-swapped displays aren't required then the module can easily create and save screenshots itself, to somewhere on HostFS. The frontend program can then access the file and do whatever it needs to do.
- At the moment there's mixed support for allowing the emulated OS to kill the emulator. ArcEm supports it, last time I checked RPCEmu doesn't, and I'm not sure about other emulators (Red Squirrel/VirtualAcorn, Arculator, etc.). So some emulators may need updating to support that. However it's also possible to have the frontend program monitor the emulator and kill it once a certain trigger is received (e.g. a file being written to hostfs), or if the emulator looks like it's stopped working (e.g. been running for over 5 mins).
The autobuilder that ROOL are using to build the ROMs and disc images on their website is based around some code I wrote which uses a custom console (i.e. windowless) version of RPCEmu, managed by a couple of Perl and bash scripts.
It shouldn't be too hard for me to knock up something similar for this project - use a console version of RPCEmu, with an added killswitch (something ROOL's one currently lacks, I'm not sure why I didn't think to add it), a more advanced frontend script to allow it to automatically run multiple instances in parallel, and with added ArcEm support. For ArcEm I can go one step further than a simple console version and also disable the code that adjusts the speed of the hardware timers. That will mean the results of each test won't be affected by the performance of the host PC, and on a fast PC it would allow it to churn through tests many times faster than would have been possible on real hardware.
Thanks for you help on this and your support so far. ADFFS certainly wouldn't be where it is now without your invaluable assistance. No problem! |
| |
[ Log in to reply ] |
| |
|
Michael Drake |
Message #120138, posted by tlsa at 13:06, 19/4/2012, in reply to message #120105 |
Posts: 1073
|
Since you mention that the command specifies a directory containing ROMs, instead of a specific ROM filename, I guess you'd want it to run N instances of the emulator, with a different ROM for each instance? ArcEm's CMOS file works fine with RISC OS 3, but not RISC OS 2, iirc. So I guess the CMOS file needs to be switched alongside the ROM.
Also iirc, ArcEm doesn't support HostFS with RISC OS 2, but I'm not sure if that would be a problem. |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120149, posted by sirbod at 20:14, 19/4/2012, in reply to message #120138 |
Member
Posts: 406
|
ArcEm's CMOS file works fine with RISC OS 3, but not RISC OS 2, iirc. So I guess the CMOS file needs to be switched alongside the ROM.
Also iirc, ArcEm doesn't support HostFS with RISC OS 2, but I'm not sure if that would be a problem. For the purposes of this project, we wont be needing RISC OS 2. We're only interested in compatibility with RISC OS 3 up.
Why not check backward compatibility I hear you ask? Well, ADFFS requires RISC OS 3.1+ - so there's little point in testing!
That's my A310 in the skip! 
Think we need to come up with a hardware list we need to check though. |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120224, posted by sirbod at 14:26, 28/4/2012, in reply to message #120149 |
Member
Posts: 406
|
I've started buying kit, now have a few RISC PC's for testing ARM610 and StrongARM. I keep getting sniped on eBay trying to get anything with a SoC ARM or an ARM710 - which is a tad infuriating.
I kick myself now for getting rid of my A3010, not that it was a very good spec.
I have an A310, but although I circuit designed a 4mb RAM upgrade for it, I never completed the project. Very much doubt you can get the RAM chips now and it will probably be cheaper and easier to get an A440. |
| |
[ Log in to reply ] |
| |
|
Andrew Rawnsley |
Message #120230, posted by arawnsley at 13:29, 29/4/2012, in reply to message #120224 |
Member
Posts: 391
|
Wish you'd posted a list of what you were after - could have checked charity stand at Wakefield for you for bits! |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120232, posted by sirbod at 15:24, 29/4/2012, in reply to message #120230 |
Member
Posts: 406
|
Wish you'd posted a list of what you were after - could have checked charity stand at Wakefield for you for bits! Doh!
I was hoping some more knowledgable people would chip in when I mentioned needing to come up with a hardware list a few posts back.
I'm taking a stab at:
1. RISC PC with ARM610 / ARM700 / ARM710 (have)
2. RISC PC with StrongARM (have)
3. RISC PC with Kinetic (have)
4. A7000+ with an ARM7500FE (have)
5. A4x0 upgraded to 4mb with ARM2 (have ARM2 and ARM3)
6. A5000 upgraded to 4mb with ARM3 (have)
7. A4000 upgraded to 4mb with ARM250 (have)
NOTE: All need to be RISC OS 3.1+
Does that cover all hardware combinations of CPU / MEMC?
eBay is slow going, there's not a lot appearing on there and I've lost count of the amount of machines and CPU's I've lost at the last second. I refuse to pay the rip off prices I'm seeing on web sites.
Plenty of time yet though, its going to be weeks before I even think about coding a testing suite.
[Edited by sirbod at 14:17, 22/8/2012] |
| |
[ Log in to reply ] |
| |
|
Andrew Rawnsley |
Message #120233, posted by arawnsley at 16:18, 29/4/2012, in reply to message #120232 |
Member
Posts: 391
|
Probably worth stating what sort of ballpark prices you consider fair - that way if any of us spot anything, we can let you know. |
| |
[ Log in to reply ] |
| |
|
Stephen Barriball |
Message #120251, posted by z0m8ied0g at 20:15, 30/4/2012, in reply to message #120232 |
Member
Posts: 3
|
I'm taking a stab at:
1. RISC PC with ARM610 (have)
2. RISC PC with ARM710
3. RISC PC with StrongARM (have)
4. A7000 with an ARM7500FE
5. A4x0 or A3x0 upgraded to 4mb with ARM2 (I have an A310, but it's useless without a RAM upgrade)
6. A540 or A5000 upgraded to 4mb with ARM3
7. A30x0 or A4000 upgraded to 4mb with ARM250
I'm currently in the process of archiving a large quantity of games and applications (including manuals and boxes etc) so am interested in your project and would be willing to help out.
I have a large collection of Acorn machines and can supply all of the above, i'm happy to let you borrow them for testing purposes if you'd like. |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120253, posted by sirbod at 23:18, 30/4/2012, in reply to message #120251 |
Member
Posts: 406
|
I'm happy to let you borrow them for testing purposes if you'd like. That's a kind offer, but it's going to take years to collect and test everything, let alone patch them. I believe we can do the bulk of the testing virtually and then double check on physical - well, that's my aim anyway.
You're more than welcome to help when we have a test suite available (its months away). We'll need a website so we can update with results / patches / scans / screenshots etc, I'm hoping someone will come forward on that front.
Send me an eMail (it's in the help file of ADFFS) and I'll send you the imager to test, when its coded. |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120256, posted by sirbod at 00:18, 2/5/2012, in reply to message #120109 |
Member
Posts: 406
|
Jeffrey, replying to your points above...
If an emulator running on a PC cant be made to insert keys, save screenshots, move the mouse etc, it sounds like I need to code a dedicated module, separate to ADFFS, which runs the script.
That's going to be a big job, needing to hook into many vectors. Going by what I know of some of the protection methods, it's not going to work for all as a lot of methods hijack the vectors.
I was hoping one of the emulator teams could extend an emulator, as there's not enough hours in the day for me to code ADFFS, a floppy imager/GUI, design a web site to host the metadata, manage the game database, test all games on multiple platforms and code the patches!
I can't believe how few people there are that can help out on the technical aspects of what I'm aiming for. Seems there's plenty of people that want the end result (all games preserved, with metadata and patched for newer OS/hardware). Apart from yourself, nobody has come forward to help with either coding or design.
I've had quite a few offers to help image floppies, so my priority at the minute is to get that side of it done.
Moan over  |
| |
[ Log in to reply ] |
| |
|
Martin Bazley |
Message #120261, posted by swirlythingy at 15:29, 2/5/2012, in reply to message #120256 |
Posts: 441
|
I can't believe how few people there are that can help out on the technical aspects of what I'm aiming for. Seems there's plenty of people that want the end result (all games preserved, with metadata and patched for newer OS/hardware). Apart from yourself, nobody has come forward to help with either coding or design. Welcome to RISC OS...
I wouldn't have thought it would be that difficult to write a module to input a load of key and mouse presses - simply set something to be called under interrupts, read the commands with OS_BGet and stuff as much as you can into the buffers with OS_Byte 138, letting the OS buffer handlers deal with the consequences. Admittedly this might not work inside the environment of a badly behaved game, but I don't think any testing other than getting the game to load (and hence past the disc protection) should be necessary.
If you need a simple (if not terribly relevant) example, my Typer program uses OS_Byte 138 in a Wimp context to automate data entry. |
| |
[ Log in to reply ] |
| |
|
Andrew Rawnsley |
Message #120262, posted by arawnsley at 17:37, 2/5/2012, in reply to message #120256 |
Member
Posts: 391
|
Sadly this is the problem - there are very few people remaining with the low level interrupt/vector experience left within the community.
I can write you a Wimp front end or whatever (time permitting, and I'm trying to do a million and one other RISC OS things all day!), but I don't have experience at any lower level than that required for normal desktop application development. |
| |
[ Log in to reply ] |
| |
|
Simon Willcocks |
Message #120264, posted by Stoppers at 20:46, 2/5/2012, in reply to message #120256 |
Member
Posts: 264
|
Jeffrey, replying to your points above...
If an emulator running on a PC cant be made to insert keys, save screenshots, move the mouse etc, it sounds like I need to code a dedicated module, separate to ADFFS, which runs the script. I'm sure emulators could be made to do those things (Jeffrey offered to do it himself, at the top of the thread). I expect that any RO module stands a good chance of alerting the protection code; emulator code will be outside the scope of the protection code's universe and practically undetectable.
Open source emulators don't need the help of the author, of course. RPCEmu is open source... |
| |
[ Log in to reply ] |
| |
|
Jon Abbott |
Message #120265, posted by sirbod at 21:14, 2/5/2012, in reply to message #120264 |
Member
Posts: 406
|
Correct on all points. RPCEmu would certainly be the one I'd modify if I were doing it myself.
Again, I'm hoping someone will take pity on me and help! |
| |
[ Log in to reply ] |
| |
| Pages (2): 1
> >|
|
Acorn Arcade forums: General: A question for the emulator teams |
|
|
|
