Other C64-related stuff -> http://www.iki.fi/a1bert/Dev/

GunZip.c64

GunZip.c64 version 1.11 (10.11.2002) - a GZIP/ZIP decompressor for C64.
Handles Deflate, Implode, Shrink, Reduce, Store methods!
gunzip-1.c64 also writes non-compressed D64 files!
Sources: gunzip.asm.gz, gunzip.c

Errol Smith's unzip64 can also dissolve .zip and .gz files. (Deflate/Store only.)

See also: PuZip -- C64/C128 ZIP archiver/compressor.
See also: Burst -- C64 burst modification.
See also: geoZip -- ZIP file extractor/creator for the Wheels (GEOS) operating systems


Features


Introduction

Surprisingly, the whole thing started with C64 emulators. At some point their emulation had gotten so good that it became possible to run almost any program and some of them needed multiple files to work correctly or at all. The emulators started supporting 1541 disk images, thus allowing distribution of whole disks instead of single files or awkward .tar, .lzh, or .zip packets which needed complicated depacking and transfer procedures.

After a while the disk image distribution became so popular that users with real C64 machines were left at disadvantage. Because a disk image contains a whole disk, 683 disk blocks, a disk image is a 689-block file. A 1541 disk can't hold a file this large, so unless you have a 1571/81 drive or a ramlink/REU, you can't convert the disk image back into a disk because you don't have enough space to save the disk image anywhere in the first place.

A solution stared us right into our faces when the disk images (.d64) got gzipped (compressed with the deflate algorithm) for storage in e.g. ftp.funet.fi. A compressed disk image almost always fits even into a 1541 disk. Then you only need a C64 program to decompress the image and simultaneously create a disk from the disk image.

At this point I had studied the compression field fairly extensively for pucrunch (see http://www.iki.fi/a1bert/Dev/pucrunch/). I searched and found gzip source code and documentation and the deflate algorithm specification and started "cleaning up" the decompression C-code. The C-code had to be written in a way that made it possible to make a more or less direct translation into C64 assembly code.

I had already rolled up my own huffman routines for C64 before work on pucrunch, so that part at least went pretty effortlessly. I represent the Huffman decode tree as an array and this makes the decode routine as fast as possible for a one-bit-at-a-time decode routine. Only a routine which decodes multiple bits at a time would be perceptively faster, but would need more memory than can be spared.

Somewhere along the line I heard about Errol Smith's UnZip64, which decompressed ZIP packets. At that point it didn't understand gzip files, so I didn't abandon my project. Also, I hoped that my program would be faster than unzip64. Still, lack of free time and multiple other projects interfered and many months passed.

Then the idea of compressed disk images came up again and I decided to give it another go. A from-file-to-file version was working a weekend's work later. I then proceeded to add things: disk image write (with standard block write routine at first), CRC32 checking, ZIP support, one-drive disk image handling (custom read and write routines), directory listing, and some error checking for corrupted input files.

Later I have also added some of the older PKZIP algorithms (Implode and Shrink) so that you don't need to have another program to handle the older .zip files. After some time I got ahold of an old pkzip version (092) which generates Reduced files. This enabled me to test the decompressor for this method and include it into gunzip.c64.

And bit by bit I have included useful features. And I will continue to do so if new useful features are requested. Upto a point, of course. See the history section for further developments and general monologue and thoughts.

Usage

First the program asks for an input drive. Press 8 or 9 for drives 8 and 9, respectively. Use 0-7 for drives 10-17.

Then give the input filename or an empty name to select a file from the directory listing.

Output drive is asked next. 8-17 can be used.

If the input drive is the same as the output drive, and a 1541 format disk is detected, you can next select whether you want to dissolve a disk image or files. In disk image mode no other selections are possible.

Unless one-drive disk image mode was selected, several selections are possible for each file in a packet.

The last selections are only available if the input drive is different than the output drive. D64 handles both D64 and D71 images. The target disk must have been formatted in the right mode.

If you want to create SEQ files, you can do this by editing the filename to include ",s,w", e.g. "file.doc,s,w". The same procedure can be used for USR files.

The border is flashed when reading or writing data. The screen is blanked during writes.

If a disk error is encountered in disk image write with the one-drive version (gunzip-1.c64), the drive will flash its led multiple times in succession:

  1. sector not found
  2. no sync
  3. data block not found
  4. data block checksum error
  5. format error
  6. verify error
  7. write protect error
  8. header block checksum error
  9. data extends into next block
  10. disk ID mismatch / disk change
  11. syntax error
  12. no drive present
The only error I have received is 2:sector not found. It is usually a sign of a faulty format or failed writes and can be cured by first reformatting the disk in question.

The file size is displayed in hexadecimal after a successful decompression. If the final output size does not match with the size indicated in the packet, an error is displayed. Also, if the CRC32 does not match, both the expected and the resulted CRC are displayed.

If the file is not detected to be either Zip or GZip, the user is asked if the file is a plain non-compressed D64. Answering yes allows you to answer yes to the next question and write the file to a 1541 disk. These questions are not asked if one-drive D64 dissolve has been selected or the source disk is in 1541 format (it can not contain a non-compressed disk image, so there is no need to ask). Note that it generally takes longer to write a non-compressed D64 to disk than it takes to write a compressed image, because there is more data to read.

Speed

The decompression speed itself is not the dominating part of the executing time; the disk input/output is. The following lists the execution times for a test disk image (97275 bytes compressed, 174848 bytes uncompressed). The tests show from-file-to-file times, the creation of 1541 disks with two drives and one drive. The one-drive test includes disk changes.

History

1.3.1999
The first public release.

14.3.1999
One-drive D64 dissolve (154x/7x) added.

2.5.1999
Now handles both Deflate (Gzip and PKZIP method 8) and Implode (PKZIP method 6) methods!

3.5.1999
A small bug in the Implode code fixed. Fortunately the older version also has CRC checking which catches this bug.

5.5.1999
Added decompression for the Shrink method (PKZIP method 1). Because the LZW tables take 32kB, there is only 8kB left for the output buffer. That's why saving is done more frequently.

3.6.1999
Removed the counting sort routine stage from the Huffman tree create by using slightly different algorithm. This reduced the code size from 397 to 202 bytes and removed 576 bytes of work memory, and probably didn't even slow down the routine.

6.6.1999
A bug in the deflate fixed Huffman decode fixed.

8.6.1999
Updated .gz handling to correspond to RFC1952 instead of gzip1.2.4.

4.7.1999
Can also write plain (noncompressed) D64's.

7.7.1999
Now includes Reduce1-4 methods.

2.8.1999
Drive ready detection changed. Now you have a chance that your REU, ramlink and/or HD will be detected. Still, no special support for these peripherals is implemented.

15.9.1999
Source code is now available for interested parties. The C-language version is a very simple, yet not so slow decoder you can try to understand instead of the bloated source code of the gzip release.

13.12.1999
The $CF00 page is now reserved for REU/ramdos.

Compiled a standard-io version (gunzip-std.c64) to be used with strange devices (e.g. IEEE488-connected drives) or jiffydos-equipped systems. This version only uses KERNEL calls. Note that one-drive D64 dissolve is not possible with the standard-io version.

Oh how long did I try to find an error when the only thing wrong was a missing call to CLRCHN (which C64 doesn't need, btw) when taking my first steps programming a C128. Anyway, the first beta of a native C128 gunzip is now reality. Note that one-drive D64 dissolve is not possible with the C128 version.

It is true: if nobody wants anything, nobody gets anything. No requests and suggestions means no updates.

19.12.1999
Burst read and write are now used in the C128 version, if possible. D71 disk images should also work, provided you have already formatted the target disk to 1571 format. However, as I don't have an 1571 drive and my 1570 drive has seek problems, I have not tested it.

26.12.1999
Burst read and write are now tested with 1581 and 1571 (and the burst detection is also tested with 1541). Compressed D81 images are supported in the C64 standard-I/O and C128 versions. Burst reads and writes really fly, so I really recommend using the C128 version.

15.1.2000
Made the IDE64 fixes and cleaned up the code a bit.

Fooled by the docs again. Appnote.txt omitted the information that the data descriptor also has a header: $50,$4b,$07,$08. Gunzip now allows a data descriptor with or without the header, but this of course means that gunzip has a 0.0000000232% chance of detecting it incorrectly.

Btw, the only non-seekable device I could use to check what the Amiga zip program generated was my c1581-handler. (Seeking is possible if a file is opened for reading, but not if it is opened for writing.) If the file is not seekable, the zip program can not back up in the file and fix the CRC32 and length values in the file header and it has to use a data descriptor instead.

Both C64 and C128 versions now use burst read and write if possible. The C64 burst modification is also detected automatically. See http://www.iki.fi/a1bert/Dev/burst/ for details. Note that you can not have the cassette drive connected while using burst-capable devices with the C64 burst modification.

29.2.2000
Started working on a limited VIC20 version. Only stored and deflate handled, no D64 dissolve, and the history buffer is of course quite limited. Currently limited to 11.5kB, but PuZip only uses a 2kB window (and only 1kB on VIC20)... At least there is some way to archive and dearchive stuff on a VIC20.

The next thing to do is a dynamic output buffer sizing, a simpler version of directory selector or none at all, and of course history reference checks that inform you if the compressor used too big a window instead of just getting a CRC error.

28.8.2000
The implode method was inadvertedly disabled and contained a bug also. 8k/4k and 2/3 huffman tree flags were mixed up. Thanks to Todd Elliott for noticing there is something wrong.

23.2.2001
IDE64 block read and write added. Thanks to Kajtar Zsolt (Soci/Singular).

16.4.2001
A ROM switching bug introduced to the 25.2.2001/C64 versions fixed.

3.7.2001
The raw D64 write tried to write one too many blocks.
Overwrite? prompt added, if a file to dissolve already exists. For All overwrite is automatic.
RUN/STOP now works in more places.

25.8.2001
D64 write was not allowed because the RUN/STOP check was added to the wrong place.

12.4.2002
Overwrite prompt now has its own "All" selector, i.e. you are prompted if a file exists even if you selected to dissolve all files.

"PK00" string is now skipped in WinZIPv8-compressed files.

10.11.2002
"Zip central directory reached" is now displayed for ZIP files after all files have been processed.

IDE64 detection interfered with Action Replay. Tries to detect AR first, and only if no AR found checks for IDE64.

Automatically skips directories in zip files.


To the homepage of a1bert@iki.fi