Converting MSX ROM to BIN

One bright day I was trying to run "Mashou no Yakata Gabalin" 64K ROM on real MSX with 128K RAM.
128K RAM is more than enough to play a 64K ROM, and I was sure that either LOADROM or SofaROM (which supports LinearC mode) can do it.
Strangely, none of them was able to run this ROM, and there was no disk version of the game on the internet.

I said "Challenge accepted!" and decided to create disk version of "Mashou no Yakata Gabalin" myself.
Below I am going to show all steps on my way to a working multi-bin conversion.
I found no guide for "Converting MSX ROM to BIN" on the net, so maybe this text can (partially) serve as such.

I am pretty sure that most of msx.org visitors are very experienced MSX hackers and developers who won't find anything new or useful here.
I also anticipate that MSX gurus will find mistakes or suggest better ways of doing the same thing -
and I am more than happy to hear your feedback, so please don't hesitate to comment.

Desired result of conversion
Cartridge games, when run outside of cartridge, may come in many formats:

1. Just original ROMs that we can run in emulators or on real MSX using special tools.
   Pros:
   • On real MSX it's easy for ROMs up to 48K.
   • If you have 256K RAM or more on your MSX, you can run most of the ROMs.

   Cons:

   • There is no software (yet) that can run 64K or bigger ROM as-is on MSX with 128K RAM.
   • ROM loading tools require MSX-DOS2 and/or additional hardware such as MegaRAM, special Konami cartridges, etc.
2. ROMs converted to .COM so they can be run from MSX-DOS.
   This approach has similar pros and cons.
3. ROMs converted to BLOAD-able format, so they can be run from MSX-BASIC using one or more BLOAD"filename",r commands.
   This way seems to me to be the most universal:
   - Works just fine from floppy with simple loader written in MSX-BASIC.
   - BLOAD-able ROM can be loaded on diskless MSX using tape interface.
   - Works perfectly via MSX-Link on networked MSX machines.
4. Same BLOAD-able ROM conversion, but accompanied with special loader written in MSX-BASIC (which plays with POKEs, ports etc.)
   I'd like to avoid this approach, as it makes loading ROM into diskless/networked machines more complicated.
5. Specially crafted disk versions - may come as disk with special boot sector and ROM data placing,
   or as ROM-special loader that can load-on-demand ROM overlays.
   This allows running 256K or bigger ROMs on MSX with just 128K of RAM.

So - I chose No.3 - a set of BLOAD-able files that can be loaded and executed by just BLOAD"filename",r them one after another.

ROM in RAM layout, or Making it fit

Before we start trying to run ROM in RAM, let's try to understand how MSX runs the cartridge.
This requires knowledge of slots and memory mapper.

Complexity of running ROM in RAM depends on the ROMs size, so let's go in ascending order from the simplest to the hardest.

Disclaimer:
Some ROMs have copy protection and will refuse to run in RAM unless patched.

16K ROMs

Let's assume we've inserted our ROM of size <=16K into Cartridge A hole.
It will appear in Slot 1, starting from address 4000h.

When rebooted, MSX will check all its slots - regardless of them holding ROM or RAM - if at address 4000h there are 2 ASCII letters "AB" (41h 42h).
If "AB" signature is found, the next 2 bytes at 4002h are used as address to start, and that's how ROM execution begins.

So, in most cases (e.g. if the ROM has no copy protection), it's enough to place 16K ROM into RAM at address 4000h and reboot MSX.
But how can we BLOAD into 4000h ? MSX-BASIC uses Page 1 (4000h-7FFFFh) so we cannot just BLOAD there.

I saw a lot of 16K ROMs in BIN format that BLOAD 16K into 9000h-D000h, and then use assembly routine at D000h that

• switches BASIC ROM with RAM on Page 1
• copies 16K ROM contents from D000h to 4000h using LDIR instruction
• runs the ROM

But can we BLOAD ROM contents right into 4000h directly?
Yes, on our MSX with 128K RAM we can do it, because it has Memory Mapper.
Let's use 16K RAM segment that normally belongs to 4000h and make it visible at more convenient address - 8000h !
By "belongs" I mean the default memory mapper configuration:

• Port FCh=3 (RAM Segment 3 at 0000h)
• Port FDh=2 (RAM Segment 2 at 4000h)
• Port FEh=1 (RAM Segment 1 at 8000h)
• Port FFh=0 (RAM Segment 0 at C000h)

So, to place RAM Segment 2 at 8000h we should write value 2 to port FEh.

Let's check this theory with e.g. Time Pilot ROM (PILOT.ROM).
To make it BLOAD-able, rename PILOT.ROM file to PILOT.BIN and add in the beginning of the file 7 following bytes:

FE 00 80 00 C0 00 00

where

FE: Signature of BLOAD-able file
00 80: Start loading from 8000h
00 C0: Stop loading at C000h
00 00: address to start execution from when BLOAD is used with ",R"
Going to address 0000h reboots MSX.

For updating binary files use hex editor of your choice (when on Windows, I use HxD).

Now let's see it working by typing the following commands in MSX-BASIC:

OUT &hFE,2
BLOAD"PILOT.BIN",R

Now MSX restarts, and - magic happens - the ROM starts running !

32K ROMs

It's about time to explain memory layout MSX has when it starts ROM execution.

• Page 0 (0000h-3FFFh) is occupied by MSX-BIOS
• Page 1 (4000h-7FFFh) first 16K of ROM
• Page 2 (8000h-BFFFh) second 16K of ROM
• Page 3 (C000h-FFFFh) contains RAM with various system variables, z80 stack etc.

Note for advanced reader:
it can be that after boot Page 2 does not contain our ROM, and ROM startup code brings its second part to Page 2, but for our purpose it can be ignored

So nice - 32K ROM fully fits in address space.
But, if we want to run 32K ROM in RAM, a little bit more work is needed:
- split 32K ROM into 2 16K BIN files - myrom.001 and myrom.002
(e.g. in most hex editors you can do it by copy-pasting 16K blocks into new file, or just create 2 copies and delete the extra data in each file)
- in the end of myrom.002 add the following 5 bytes:

3E 02 D3 FE C9

These bytes are a small program in z80 assembly that will OUT &hFE,2 and return to MSX-BASIC
after BLOAD-ing last 16K of the ROM, in order to prepare Page 2 for BLOAD-ing the first 16K of the ROM:

3E 02	; ld	a, #02
D3 FE	; out	(#fe),a
C9	; ret 

- add 7-byte headers (as explained above) to make both binary files BLOAD-able:
FE 00 80 00 C0 00 00 in myrom.001 (this part is loaded last and reboots MSX by jumping to address 0000)
FE 00 80 05 C0 00 C0 in myrom.002 (our small z80 program sits at C000h where 16K ROM data ends, takes 5 bytes and runs from C000h)

Knowing that after MSX booting our Page 2 (8000h) already exposes RAM segment 1 needed for last 16K of ROM,
let's run them as follows:

BLOAD"myrom.002",R
BLOAD"myrom.001",R

Voila !

• Note 1: If you want files to be loaded in ascending order - 001 and then 002 - just rename them.
• Note 2: The way to execute ROM without rebooting MSX will be shown later in this guide.

48K ROMs

Things start getting complicated, but not too much.

48K ROM, when iserted e.g. in Cartridge A hole, appears in Slot 1 from address 0000h, takes three 16K pages, and ends at BFFFh.
If we split it into 16K files - myrom.001, myrom.002 and myrom.003 - the "AB" signature will appear in myrom.002.
When ROM starts, its first 16K are not visible to z80 - because Page 0 at 0000h is taken by MSX-BIOS.
To access it's own first 16K, ROM code performs slot switching when needed.
If our ROM was inserted into Slot 1, then the ROM code will know to find Page 0 in the same Slot 1.
The same is correct for any other slot where the ROM could be inserted.

Can we use this fact to run ROM in RAM ? Yes !
We'll put myrom.001 contents in RAM Segment 3, so when MSX reboots and starts running ROM code it will find it in RAM slot at 0000h.

BLOAD"myrom.002"
OUT &hFE,3
BLOAD"myrom.001"
OUT &hFE,2
BLOAD"myrom.003",R

After reboot the ROM should be running.

Note: Of course, to make the conversion cleaner and easier to run

• OUT instructions can be attached to binary files like in the example of 32K ROMs above
• files should be renamed to appear in ascending order

Big ROMs

Starting from 64K, most ROMs use special switching mechanism called MegaROM Mapper.

Converting ROMs for each of these mapper types is a different story, which I am not going to touch here.

Generally speaking, there is no universal way to run big ROM in RAM.
Success depends on amount of available RAM/VRAM in your MSX, amount of RAM/VRAM used by ROM,
and the way ROM pages are used and switched, switching frequency, etc.

I am going to talk about conversion of one specific 64K ROM.
Most of the steps in this task should be common for any big ROM conversion, so stay tuned.

There is a small group of so-called "Plain 64K ROMs".
They take all 64K of address space, from 0000h to FFFFh, in the slot they were inserted.

I am going to convert to BLOAD-able format "Mashou no Yakata Gabalin" ROM that belongs to this category.

Initial analysis

Some knowledge of z80 instructions and MSX slot switching is required, because we are going to reverse engineer how this ROM handles its data.
Let's load the ROM into disassembler - e.g. openMSX debugger, IDA Pro, Ghidra - whatever you are familiar with.
For our purpose web-based ODA (Online Disassembler) is good enough.

Normally ROM code starts with memory setup, so let's see where it all begins.
In this ROM "AB" can be found at offset 4000h, so we can guess that ROM does not have it's own mapper and appears in address space of its slot from 0000h to FFFFh.
And when there is no additional mapper, we can assume that ROM page switching is done using standard means:

1. BIOS call ENASLT: call #0024
2. BIOS call RDSLT: call #000c
   (Full BIOS calls documentation can be found here)
3. Writing to port A8h: out (#a8),a
4. Writing to address FFFFh: ld (#ffff),a

Of course, other z80 instructions also can be used for slot switching.
It's only an example of what normally you can find in the code.

The 2 bytes at address 4002h - "10 40" - contain address where ROM execution starts - 4010h.
Let's go there.

4024:
di
call	#0138	; RSLREG MSX BIOS call: reads the primary slot register
rrca
rrca
and	#03	; ....bb.. -> 000000bb

The code finds out in which primary slot it is by itself, knowing that it runs on Page 1 (4000h-7FFFh).

Now we will need some knowledge on MSX System Variables, namely EXPTBL (FCC1h-FCC4h) and SLTTBL (FCC5h-FCC8h):

402C:
ld	c,a
ld	b,#00
ld	hl,#fcc1	
add	hl,bc
ld	c,a
ld	a,(hl)
and	#80	; 80h if primary slot expanded, 00h if not
or	c
ld	c,a
inc	hl
inc	hl
inc	hl
inc	hl
ld	a,(hl)
and	#0c	; secondary slot value
or	c
ld	(#dae5),a

At this moment register A and RAM address DAE5h contain primary/secondary slot value to be used as input to ENASLT/RDSLT BIOS calls.
With this value ROM code can bring to visibility its other parts.
Which it immediately uses to bring ROM Page 2 data to address 8000h:

4044:
ld	h,#80
call	#0024	; ENASLT MSX BIOS call: switches indicated slot on indicated page

After that, there is a code that does the same primary/secondary detection, but for RAM on Page 3 (C000h-FFFFh) now, and puts result in DAE6h:

4049:
di
call	#0138	; RSLREG MSX BIOS call: reads the primary slot register
rlca
rlca
and	#03 ; bb...... -> 000000bb
...
ld	a,(hl)
and	#0c
or	c
ld	(#dae6),a

Let's analyze what comes after that:

4073:
in	a,(#a8)
and	#3f		; Take primary slot settings for pages 0,1,2
ld	b,a
ld	a,(#dae5)	
ld	c,a
and	#03		; Primary slot bits
rrca
rrca
or	b		; Use on page 3 primary slot where the ROM is
out	(#a8),a
ld	h,a		; Keep in register H

ld	a,(#ffff)
cpl
and	#3f		; Take secondary slot settings for pages 0,1,2
ld	b,a
ld	a,c
and	#0c		; Secondary slot bits
rrca
rrca
rrca
rrca
or	b		; Use on page 3 secondary slot where the ROM is
ld	(#ffff),a
ld	l,a		; Keep in register L

...

Numbers of primary and secondary slots of where ROM is placed (taken from pre-calculated value at adress DAE5h) are used to for Page 3 (C000h-FFFFh).
This is how ROM brings to visibility its last 16K block of data.
After that similar code is used to bring RAM on Page 3 back.
This time BIOS is not called - because the RAM on Page 3, where z80 stack and system variables are, is not there during page switching !
How would you return from subroutine call, if the call stack has disappeared?
For this reason, instead of BIOS calls, port A8h and address FFFFh are used directly.
Their values - before switching Page 3 from RAM to ROM, and after switching back - are stored at DAE7h-DAEAh.
This is done to perform quick slot switching later in code.

It seems all slot-switching related initialization is done at this point.

Now let's check if we have more slot-switching BIOS calls, like ENASLT ("CD 24 00") or RDSLT ("CD 0C 00").
Depending on the debugger/disassembler, it can be done by searching mnemonics or instruction opcodes.

Here is the one we haven't seen yet:

53e1:
di
push	bc
push	de
ld	a,(#dae5)
call	#000c	; RDSLT MSX BIOS call: reads the value of an address in another slot
pop	de
pop	bc
ei
ret

This code reads byte at address HL from slot/subslot stored at DAE5h.
This means, if we run our ROM in RAM, this code will continue working as is, without change.
At least as long as HL < C000h - but it is a safe assumption, because for addresses in Page 3 there is a different code, as we will see soon.

Let's check if we have more direct slot-switching code - out (#a8),a ("D3 A8") or ld (#ffff),a ("32 FF FF")
I found it in 2 subroutines.

Subroutine No.1:

4d3c:
di
ex	de,hl
...
ld	a,(#0007)	; VDP input port
ld	c,a
exx
ld	a,(#dae7)
ld	b,a
ld	a,(#dae8)
ld	c,a
ld	a,(#daea)
ld	d,a
exx
ld	a,(#dae9)
out	(#a8),a		; switch primary slot
...
ld	a,d
ld	(#ffff),a	; switch secondary slot
...
ld	a,(hl)
out	(c),a		; send data from ROM to VDP
...
ld	a,b
out	(#a8),a		; switch primary slot back
...
ld	a,c
ld	(#ffff),a	; switch secondary slot back
...
ret

This subroutine takes BC bytes from address DE and puts them into VDP port.
It's intended for Page 3 that requires special treatment, and we'll have to patch this code.

Subroutine No.2:

53ee:
di
...
out	(#a8),a	
...
ld	(#ffff),a
...
ld	d,(hl)
...
out	(#a8),a	
...
ld	(#ffff),a
...
ld	a,d
...
ei
ret

This subroutine is an inter-slot version of "ld a,(hl)" for Page 3 (HL >= C000h).
We'll patch this one as well.

Layout planning

Analysis above shows we can put first three 16K blocks of ROM in pages 0,1,2 of RAM slot.
Existing ROM code will find them where they are and will use them properly.

The bad thing is we cannot do the same for the Page 3.
Our Page 3 RAM cannot contain ROM data, because it's used by MSX BIOS and ROM code itself (z80 stack etc).

The good thing is we have 128K of RAM (half of which is not used) and Memory Mapper.
16K sections are numbered 0 to 7, and by default 4 of them are mapped.

• RAM Section 3 at 0000h will contain 1st 16K of ROM.
• RAM Section 2 at 4000h will contain 2nd 16K of ROM.
• RAM Section 1 at 8000h will contain 3rd 16K of ROM.
• RAM Section 0 at C000h contains z80 stack, MSX system variables, etc.

So we can pick e.g. RAM Section 4 to hold last 16K of ROM.

When ROM needs to use its last 16K, we will use port FFh to select RAM section 4, and when the data is fetched - switch back to RAM Section 0.
As the analysis above shows, there are just 2 places in the code where it should be done.

Splitting ROM

Let's split our 64K ROM file into 4 files as follows:

• 1st 16K: gabalin.003 (loads to RAM Segment 3/default address 0000h)
• 2nd 16K: gabalin.004 (loads to RAM Segment 2/default address 4000h)
• 3rd 16K: gabalin.001 (goes to default address 8000h, so this part will be loaded first)
• 4th 16K: gabalin.002 (loads to RAM Segment 4, as decided in layout planning)

Now add 7-byte header (as explained in small ROM conversion part of the guide)

FE 00 80 05 C0 00 C0	

in the beginning of files gabalin.001, gabalin.002, gabalin.003 to make them BLOAD-able.

Each of BLOAD-ed files should switch memory mapper using port FEh so it becomes ready for the next BLOAD.
For this to happen, in the end of files add the following 5 bytes:

• in the end of gabalin.001 add: 3E 04 D3 FE C9 (prepare RAM Segment 4 for BLOAD-ing gabalin.002)
• in the end of gabalin.002 add: 3E 03 D3 FE C9 (prepare RAM Segment 3 for BLOAD-ing gabalin.003)
• in the end of gabalin.003 add: 3E 02 D3 FE C9 (prepare RAM Segment 2 for BLOAD-ing gabalin.004)

gabalin.004 runs last and needs more patches - here we go.

Patching ROM code

As we see from analysis, 2 places in ROM code need our intervention.
Each time ROM code switches to last 16K of ROM on Page 3 (C000h-FFFFh) using "out (#a8),a" and "ld (#ffff),a" we'd like to switch to RAM Section 4 instead.
We also have to switch back to previous slot configuration when needed.

As we know, Page 3 is occupied all the time by RAM Section 0.
This means our life is easy: we just have to switch from Section 0 to Section 4 and back using Memory Mapper.
No additional slot switching is required !

How we are going to switch to RAM Section 4:

"3E 04 D3 FF" (4 bytes code sequence):

ld	a,#04
out	(#ff),a

Switching back to RAM Section 0 is even shorter:

"AF D3 FF" (3 bytes code sequence):

xor	a	; a=0
out	(#ff),a

Original code writes to port A8h and to memory address FFFFh, we should only perform one writing to port FFh.
This means we have enough bytes to replace original code with our own, and even more.
In this case the spare bytes remained from removed old code should be replaced with "nop" instructions (opcode 00).

Patch Subroutine No.1:

Address | Offset in   |  Original bytes | Replace with
in ROM  | gabalin.004 |  and code       | bytes / code

4D52h     0D52h         3A E9 DA D3 A8    3E 04 D3 FF 00
                        ld a,(#dae9)      ld a,#04
                        out (#a8),a       out (#ff),a
                                          nop

4D5Ch     0D5Ch         32 FF FF          00 00 00
                        ld (#ffff),a      nop/nop/nop

4D6Fh     0D6Fh         D3 A8             00 00
                        out (#a8),a       nop/nop

4D75h     0D75h         32 FF FF          AF D3 FF
                        ld (#ffff),a      xor a
                                          out (#ff),a

Patch Subroutine No.2:

53FDh     13FDh         3A E9 DA D3 A8    3E 04 D3 FF 00
                        ld a,(#dae9)      ld a,#04
                        out (#a8),a       out (#ff),a
                                          nop

5406h     1406h         32 FF FF          00 00 00
                        ld (#ffff),a      nop/nop/nop

540Eh     140Eh         D3 A8             00 00
                        out (#a8),a       nop/nop

5414h     1414h         32 FF FF          AF D3 FF
                        ld (#ffff),a      xor a
                                          out (#ff),a

Executing the ROM

Unfortunately, loading all parts and rebooting MSX leads to boot loop.
That's where I found I can't proceed without openMSX debugger.
Comparing execution flow of original ROM and my work-in-progress I found out that the behavior begins to differ
in subroutine (mentioned above) that starts at 53E1h, right after interrupt enabling instruction "ei".

I am not sure why there is such a difference, but after patching "ei" with "nop" MSX rebooted itself twice and started the game !

Hurray !? Should we now celebrate the victory ?
Actually, it's too early - and here are the reasons:

• leaving "AB" in RAM at address 4000h is a bad taste: the game will restart after reset, and to erase it from RAM we'll have to switch MSX power off
• waiting for 2 additional reboots is too much, there should be a better way

So it's about time to learn how to run ROM in RAM without rebooting MSX.
Let's add the following bytes to the end of gabalin.004 file, while seeing what the underlying ROM execution code does.

3E 01	; ld	a,#01 
D3 FE	; out	(#fe),a		; We should recover correct RAM Segment on Page 3

06 80	; ld	b,#80
FB	; ei
76	; halt
10 FD	; djnz	#c007		; Serve interrupts until disk stops spinning
3E C9	; ld	a,#c9
32 9F FD; ld	(#fd9f),a	; Disable interrupt hook

DB A8	; in	a,(#a8)
E6 F3	; and	#f3		; Let Page 0 use slot 0 (BIOS)
47	; ld	b,a 
0F	; rrca
0F	; rrca
E6 0C	; and	#0c		; Get Page 3 primary slot
4F	; ld	c,a
B0	; or	b		; Set it for use on Page 2 too
D3 A8	; out	(#a8),a

				; Do the same switching for secondary slot register
CB 09	; rrc c			
CB 09	; rrc c
21 C5 FC; ld	hl,#fcc5
06 00	; ld	b,#00
09	; add	hl,bc
3A ff ff; ld	a,(#ffff)
2F	; cpl
E6 F3	; and	#f3
47	; ld	b,a 
0F	; rrca
0F	; rrca
E6 0C	; and	#0c
B0	; or	b
32 FF FF; ld	(ffff),a
77	; ld	(hl),a		; Keep SLTTBL secondary slot mirror table updated

2A 02 40; ld	hl,(#4002)	; ROM execution address
AF	; xor a
32 00 40; ld	(#4000),a	; Erase "AB" signature in RAM
E9	; jp	(hl)		; And... here we go !

After adding the startup sequence to gabalin.004 file, we should measure the size of the added code - 3Eh bytes -
and add the corresponding 7-byte header in the beginning of the file:

FE 00 80 3E C0 00 C0

Note: when executed like this, the patch of "ei"->"nop" mentioned above is not needed anymore.

Loading from MSX-BASIC

Is the ROM conversion finished now?
Yes - if you manually BLOAD four files or use disk/tape loader or network BIN file sender that does not occupy addresses 8000h-Cxxxh.
No - if you want to use simple MSX-BASIC program with 4 BLOAD operators.
It won't work, because BASIC program by default starts at 8000h and will be overwritten by the very first BLOAD.

The solution is to create BASIC program that runs from address above BLOAD-able data, e.g. D000h.

The following gabalin.bas program runs itself from 8000h but does no BLOAD-ing.
Instead, it sets BASIC system variables TXTTAB and VARTAB (pointers to BASIC program and variables) to D000h and above,
and creates another BASIC program there on the fly:

10 N$="gabalin .00":FOR I%=&HD000 TO &HD036:READ A%:POKEI%,A%:NEXT:FORI%=1TO11:POKE&HD012+I%,ASC(MID$(N$,I%,1)):NEXT:POKE&HF677,&HD0:POKE&HF6C3,&HD0:POKE&HF6C2,&H2D:RUN
20 DATA 0,43,208,30,0,130,65,37,239,18,217,25,58,145,65,37,58,207,34,49,50,51,52,53,54,55,56,46,48,48,34,241,255,155,40,65,37,41,44,82,58,131,0,0,0,8,73,0,197,18,35,112,0,0,0,0

The dynamically created program (that hides itself in DATA operator in line 20) does the actual BLOAD-ing and runs the converted ROM:

30 FORA%=1TO8:PRINTA%:BLOAD"gabalin .00"+HEX$(A%),R:NEXT

P.S. The reason loop goes to 8 is that the loader is universal and works for any multi-BIN set of up to 8 files, just change the name in line 10.

=== T H E === E N D ===