Gunzip for MSX 1.0 released

Yesterday Laurens Holst released gunzip, a high-performance gzip (.gz) archive decompressor. Gzip is a relatively common file format to compress single files, with compression ratios comparable to most other archive formats.

You can download it from the project page.

This gunzip implementation is at least 2x as fast as other decompression tools for msx (excluding special-purpose formats like BitBuster). Tests showed that a 720K disk image can be decompressed in 26 seconds on turboR, and 133 seconds on MSX2. Full performance test results are presented in the forum thread, which also contains some technical details about the techniques used to achieve this high performance.

Gunzip is open source and liberally licensed, so you can reuse the code in any way you want. Gzip’s deflate compression algorithm is also used by various other file formats such as zip and png, so hopefully it will come in useful.

The project was started because VGM files for VGMPlay are commonly compressed with gzip. Until now, to play back these .vgz files you had to extract them on PC, but now you can use gunzip to decompress these files on the MSX. Later it will be integrated into VGMPlay.

Relevant link: Gunzip for MSX

Comments (16)

By Louthrax

Prophet (2465)

18-10-2015, 18:59

This looks like a major contribution to the Z80 scene, and it started on MSX Thanks Grauw !

By snout

Ascended (15187)

18-10-2015, 20:58

I cannot put into words how epic this is, particularly when looking at the sheer speed of the decompressor. First ZIP, now gzip. Not a bad year for MSX I'd say.

By syn

Prophet (2123)

18-10-2015, 21:04

The guy who makes runrar will be a hero for ever!

By Grauw

Ascended (10768)

18-10-2015, 21:34

The problem with rar and 7zip is that the dictionary sizes are too big (megabytes / gigabytes). Effectively on MSX it means you have to decompress the entire file into memory before writing to disk. Whereas the dictionary size for gzip and zip is just 32K which will fit in main memory just fine without even dealing with memory mapping.

So although they achieve a little higher compression ratios, I don’t think rar and 7z are suitable for MSX due to the memory requirements. There’s no need for them either, as gzip and zip are still extremely popular formats with built-in support in nearly every modern desktop OS.

18-10-2015, 22:02

Hehe I was joking, as a play on the names of the msx unpackers released this year

18-10-2015, 23:59

Hehe yeah, I know, but it was worth mentioning anyway .

By cbsfox

Champion (429)

19-10-2015, 06:28

Congratulations for your great unzip tool.
Is it possible to create a zip tool for MSX or it would take one day to zip a 16k file?

Regards
Marcos

19-10-2015, 09:55

I don’t know how long compression would take but probably not a day . I think it could be done at comparable speeds to lharc (which is pretty quick). There’s a trade-off between compression ratio and speed and memory consumption, which could be controlled by a command line option. Atm I don’t have plans to work on a that though...

19-10-2015, 16:43

OK. But is it difficult to implement a zip tool?

19-10-2015, 16:58

Assuming that a gzip compressor is a project comparable to gunzip;

Technically it’s quite challenging. Time-wise, looking at the gunzip commit history I worked on it for 14 days over the span of 6 months (excluding the test implementation in JavaScript).

Though now that I’ve implemented gunzip I could do gzip a little quicker, it’d probably still be a full holiday of work.

By Parn

Paladin (837)