This page is a wiki. Please login or create an account to begin editing.


3 posts / 0 new
Last post
Arthegall's picture
Offline
Joined: 2011 Dec 31
Proper Floppy Archive Format (long & nerdy post)

Goal: Define a single standard for archiving floppy disk images that addresses all anticipated use cases and requirements.

Disk Formats to be Supported: MFS, HFS
Disk Sizes to be Supported: 400k, 800k, 1.44 MB

Main Requirement: Preserve software exactly as it was for use and access into the foreseeable future.
Sub-requirement: Archive should support all 6 use cases below
Sub-requirement: Disk images used in the archive must be able to be created on a vintage Mac with a floppy drive--the kind of machine necessary to read and preserve the old software

Secondary Requirement: The archive must include full documentation including provenance and method

Use Case 1: Using an image to create a physical floppy that will work on a vintage Mac.
Use Case 2: Running the software in emulation on a modern Intel Mac
Use Case 3: Running the software in emulation on a Windows machine
Use Case 4: Running the software in emulation on a Linux machine
Use Case 5: Running the software in emulation on an unknown future architecture
Use Case 6: Accessing the image contents directly in a modern or future OS for study or maintenance purposes (checking file hashes, doing some magical bit-level decomposition and reverse engineering in a more advanced or historically-minded future)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Requirements:
Use Case 1 Narrative:
In order to create the physical floppy, the image must be transported to a computer that can write it to a floppy drive capable of reading and writing to 400k and 800k floppy disks. Presumably, only vintage Macs can do this (I personally use a Wallstreet G3 laptop with a floppy drive module and a USB expansion card. I transport to the G3 via USB and write to floppy with the floppy drive module). Most (all?) such computers run some flavor of a classic Mac OS. So the image must be usable in a classic Mac OS. Many such computers run a version of Mac System 7 (say for example a Quadra that you feed via CD or direct network connection) and may not be able to use Disk Copy 6.3.3. So the image must be usable with Disk Copy 4.2 (Disk Copy 4.2 cannot use .img files created by Disk Copy 6.3.3). Disk Copy 4.2 images have a resource fork and will not function properly if that resource fork is not preserved while the image is transported to the computer with the floppy drive. So the disk image should be compressed/archived in a format that 1) preserves the resource fork, and 2) can be read/uncompressed on a vintage Mac running some flavor of Mac System 7. This is where I get stuck. I've been using DropStuff 5.5. are .sit files backwards compatible to early versions of StuffIt Expander?

Use Case 1 Summary:
a) Disk Copy 4.2 image
b) Compressed with . . . something that preserves the resource fork and can be uncompressed on a Mac System 7 machine. DropStuff 5.5? Something else?

Use Case 2 Narrative:
In order to run under current 68k emulation (Mini vMac, Basilisk II, else?), a disk image needs to be dragged and dropped onto the emulator. It then mounts within the emulator and the software can be used normally. However if the disk image is locked (either via some configuration of the image itself or via access or read/write permissions on the host OS), the emulator will not be able to unlock it. In some cases the software will object. If the image cannot be unlocked, and the software will not run if it is locked, the user will be at a dead end. Yet locked images are preferable from an archiving perspective to minimize the possibility of unwanted change. So ideally there are two images, a locked and an unlocked one. Modern Macs respect resource forks, but modern emulators do not seem to always require them to mount an image. In addition, there is no guarantee that in the future software will be available on a modern Mac that can decompress an old .sit file. Therefore it would be preferable to have the raw image itself. A beautiful, perfect image trapped in a .sit file cannot be used in the emulator.

Use Case 2 Summary:
a) Raw disk image compatible with an emulator (Disk Copy 4.2, Disk Copy 6.3.3, DiskDup+ 2.9.2, and random .dsk files all work in my experience)
b) Not compressed in a .sit

Use Case 3 Narrative: Similar to Use Case 2, except resource forks are positively undesirable. If they are required for the image to function, the image will not function. Simple as that. My understanding is that DiskDup+ 2.9.2 disk images do not utilize resource forks and will act and operate on a Windows machine the same as on any Mac, and that they can be moved back and forth without deleterious effect. That's what I've heard. In addition, an image compressed in a .sit is wholly undesirable because expanding .sit files on Windows machines can be a high barrier to entry.

Use Case 3 Summary:
a) Raw disk image, preferably with no resource fork (so not Disk Copy 4.2 or Disk Copy 6.3.3)
b) Not compressed in a .sit or .hqx or other old, Mac type format

Use Case 4 Narrative and Summary:
Same as Use Case 3.

Use Case 5 Narrative and Summary:
Same as Use Case 3.

Use Case 6 Narrative:
All or virtually all software and files on vintage floppy images will have resource forks. If those files are handled directly on a system that doesn't respect and preserve resource forks, they will be ruined. Still, there may be some value (now or in the future) for users being able to inspect the files directly in a modern OS--if even to simply see what files where on the image or to check file hashes or whatnot. Modern Macs can still mount Disk Copy 6.3.3 .img files, but who knows how long that will last. Non-Mac OSes cannot (to my knowledge). And in the future, there's no telling what systems will be able to mount what, so for simplicity, it would be much easier if the contents of the image were simply directly available to the user and not stored in a disk image at all.

Use Case 6 Summary:
a) Image contents not stored in an image but simply dropped in a folder for easy inspection

- - - - - - - - - - - - - - - - - - - - -
Summary of Requirements for a Proper Floppy Archive:

1) Disk Copy 4.2 image compressed in a format that will preserve the resource fork and be easy to decompress on a Mac System 7 machine (WHAT IS THIS? DropStuff 5.5 .sit file?)
2) Disk image without a resource fork (what's the best way of creating a non-resource-fork disk image on a vintage Mac? This image must work with emulators. Is this DiskDup+ 2.9.2?)
3) Locked and unlocked images
4) Raw image contents in a folder

My current approach is to include:
1) Disk Copy 4.2 raw image
2) Disk Copy 4.2 image in a .sit created with DropStuff 5.5
3) Disk Copy 4.2 image in a .sit.hqx created by DropStuff 5.5
4) Disk Copy 6.3.3 locked raw image
5) Disk Copy 6.3.3 locked image in a .sit created with DropStuff 5.5
6) Disk Copy 6.3.3 locked image in a .sit.hqx created by DropStuff 5.5
7) Disk Copy 6.3.3 unlocked raw image
8) Disk Copy 6.3.3 unlocked image in a .sit created with DropStuff 5.5
9) Disk Copy 6.3.3 unlocked image in a .sit.hqx created by DropStuff 5.5
10) DiskDup+ 2.9.2 raw image
11) DiskDup+ 2.9.2 image in a .sit created with DropStuff 5.5
12) DiskDup+ 2.9.9 image in a .sit.hqx created by DropStuff 5.5
13) Raw image contents in a folder
14) All of the above zipped in a single archive that includes ReadMe .txt with explanation of provenance and MD5 hashes for all the raw image files

This feels like overkill. But it's overkill out of fear of missing something.
I want one archive that I can put in the cloud (OneDrive or DropBox or whatnot) and share online that will cover all foreseeable use cases and be properly documented.

Help me figure out what that should be.

Are MD5 hashes enough for file verification? Should I use SHA1 too? That feels like super overkill. Laughing out loud

Comments

Arthegall's picture
Offline
Joined: 2011 Dec 31

Doing some field testing today based on information about the Disk Copy 4.2 format published by the 68kMLA here: https://wiki.68kmla.org/DiskCopy_4.2_format_specification.

I zipped three 800k HFS disk images I made myself from floppies (Disk Copy 4.2, Disk Copy 6.3.3, and DiskDup+ 2.9.2) and dropped them in the cloud (I use Tresor).

I then downloaded the .zip with the three images to my Lenovo Windows 10 laptop.

I then unzipped the .zip archive, dumping all three images into my Windows 10 Downloads folder.

I then e-mailed all three images as free attachments from my Gmail account to my Office365 account.

I then opened the e-mail on my MacMini, and downloaded the three images from the e-mail.

Test 1: Emulation
All three images mounted fine in Mini vMac. The 6.3.3 .img file was recognized by OSX as an "NDIF Image" and had an extra "desktop folder" in it. (That was strange). The other two were just "documents" as far as OSX was concerned. But all three mounted fine.

Test 2: Mounting and opening in OS 9.2.2
I dropped the three images on the USB stick I use for the purpose and copied them over to my Wallstreet G3. It had no clue what any of them were. None of the original disk imaging apps (Disk Copy 4.2, Disk Copy 6.3.3, or DiskDup+ 2.9.2) would recognize, load, or mount any of them.

Adding Type and Creator with ResEdit 2.1.3.
Following the advice in the linked 68kMLA resource, I added the type "dImg" and the creator "dCpy" to the Disk Copy 4.2 image using ResEdit and the "Get Info" command.
After that, all three imaging programs recognized it, and it mounted fine.

I tried the same with the DiskDup+ image. I added "DDim" as the type and "DDp+" as the creator. After that, it loaded and mounted without issue in DiskDup+ 2.9.2.

For reference, DiskDup+ images I create do not have resource forks. If I create a new DiskDup+ format image and open it in ResEdit, it pops the "file has no resource fork" alert.

So takeaways:
1) ResEdit 2.1.3 is more important to have than StuffIt or DropStuff. If I could have only one, I'd have ResEdit.

2) My approach above is massive overkill.
I'm thinking Disk Copy 4.2 in a .sit file, raw Disk Copy 4.2 image, raw DiskDup+ image, and . . . do I even need a DiskCopy 6.3.3 .img? Is the only upside that a modern MacOS can mount and convert it?

Duality's picture
Offline
Joined: 2014 Mar 1

do I even need a DiskCopy 6.3.3 .img? Is the only upside that a modern MacOS can mount and convert it?

Not even. If you add ".img" as a suffix to any file, Mac OS X will recognize it as a disk image. Open it through the Finder, and OS X attempt to open it with DiskImageMounter.app, which is a GUI wrapper for hdiutil's "attach" feature.

hdiutil doesn't care about the suffix, it reads the disk image header and resource fork where applicable to figure out what the image really is.

The "Desktop Folder" you saw is a hidden folder in System 7 through Mac OS 9.2.2, which exists to preserve the contents of the Desktop for that mounted device. To spot bits like this, it's a good idea to have a tool around that can see hidden files in the classic Mac OS, even a Finder alternative like Greg's Browser.

While you can mount a Disk Copy 4.2 image stripped of its resource fork by recreating the fork with DC42 creator and type codes, that still does leave out the checksum that's part of that fork. Even though it's easy to recreate the fork, I'd rather preserve it through resource fork hostile file systems with BinHex, StuffIt, or any equivalent.

One tip; Mac OS X has command line "binhex", "macbinary", "applesingle" etc tools that also help with resource fork preservation. Just "man binhex" from Terminal.app to learn how to use them!

You can't mount Disk Copy 6 images stripped of a resource fork, even an uncompressed NDIF (Disk Copy 6) image. The fork there has more intelligence required to mount the disk than DC42's.