This page is a wiki. Please login or create an account to begin editing.


14 posts / 0 new
Last post
SkyCapt's picture
Offline
Joined: 2017 Jan 11
"Opening this archive may damage your system" what?

Absurd 'warning' started appearing new when using Mac OS X Tiger 10.4.9 and higher. As one example, an image I create of my Mac's own original install disc causes this warning. I can use any Mac OS including 10.4.9 to create images that cause 10.4.9+ to freak out. The precise language it uses is this:

The disk image you are opening may be damaged and could damage your system.
Are you sure you want to open this disk image? / [Open] / [Don't Open]

Damage how? How can this not be the fault of the OS and be the fault of 'documents'?? Is the damage permanent or can I log out then back in or can I reboot? It seems this warning affects images containing older software and in the .img/.dmg formats. Now I've just identified an image of old software that I confirm does "damage my system" severely when I mount it and I eject it, but this particular image also gives NO WARNING. What is this?

I'm noticing this is a problem now that I've started doing the work of 'software librarian'. I now have specific tests whose results I can replicate, and it's shocking. I can mount (then eject) images I create myself containing nothing other than an old "Mac OS ROM" file, I do NOT receive that warning when mounting the image in 10.4.9+, but my 'system' gets 'damage' anyway so badly I believe this is exactly what's being talked about in the pop-up.

I've tested logging out then back in does not stop nor repair the "damage". Rebooting, however, does. And it isnt only OSX 10.4.9+ being damaged. My esteemed 10.4.8 is corrupting too when I open/mount various images. The corruption hits the diskimagemounter and Finder file system engine, and IS causing occasional files to be copied erroneously. I'm so lucky I caught this before corrupting my software libraries.

First I'll hunt for which point version of Mac OS X that this corruption first takes place in.

Comments

Maximum R.I.S.C.'s picture
Joined: 2017 Oct 18

It just means that the HFS of the image isn't of the newer BSD-friendly variety. Lots of old ISOs pop this warning from the OS8.6 era - Often when when you make a DMG of an OS9 system that was setup by OS9, rather than drag-copied from OSX it will do the same thing.

Tiger came out during Apple's grandest approach to cross-system compatibility (Python, Pear, Ruby, Java, Flash), and it was when they started realizing there might be a security price to pay for all the compatibility. They were just covering their butts.

I was curious about this one time when making an image of a Classic system and while it would kick the warning from the first image, after running OSX Disk Utility over the Classic system and re-imaging, the error no longer appeared.

SkyCapt's picture
Offline
Joined: 2017 Jan 11

Ironic that Apple criticised windows for doing a similar thing in the apple tv commercial "Mac & PC : Security". There are two big concerns going on here. First is the panicky warning certain images generate when being opened within OSX 10.4.9+ and the second concern is real system damage caused by opening certain images within any OSX. I can't pretend this isn't a four alarm fire. The corrupted systems are letting corrupted copies of files to be created. I've tested *all* OSX on PowerPC 10.5.8 down to 10.2.8 are corruptible the same way by opening some images. There may be none, or some, but not total, overlap among these two concerns.

Ok let's look at what's causing the pop-up with OSX 10.4.9+. I've made many .dmg of older discs (using 10.4.8 but all kinds of .img/.dmg behave similarly). One I know is the image of the game CD "SimCity 3000", and I agree, to filecopy all from the disliked image to a freshly built image does prevent the newer image from causing the popup, a filesystem complaint.

An older version of file system is Not always like what's causing the pop-up in OSX. I'm using OSX 10.4.11 (Server) now to look at my archives. A .dmg of my genuine OS 9.2.1 install CD does not make the alert. But the content of the CD's installer written to a harddrive partition using Tiger Classic mode, then, archived to .dmg using Tiger : this carries the damage popup (and houses a "Mac OS ROM" file) oh, the FULL install data is warning me but the .dmg of "recommended install" (not the full install) hasn't become an image which warns. It also is the case the FULL install freezes booting my computer whereas the recommended install boots to desktop. Maybe there's some "AI" happening in OSX deciding whether to warn or not to.

ISO (.toast) can trigger the popup, you're right there. Many old .toast things I downloaded from here are causing the warning. Wow, my own .dmg scans of Luxor and Zuma's Revenge CDs cause warning, they're older games but well into the OSX era. Here's one, "Broken Age" a big new year-2016 game, its .dmg scan of disc causes the warning.

And this is weird too: the scan of my MDD2003 10.3.0 disc #1 has *stopped* causing the warning. I know 100% fact it was warning but that might have been in OSX 10.4.9 or 10.4.11 Client. Now on "professional" 10.4.11 Server it is deciding to not freak me out by saying my system's "key disc" image is damaged. No popup for it, though --> my image of the AHT segment included in the overall .dmg does cause the warning. And disc #2 carries the warning.

How messed up. OSX picks 'n chooses over a variety of criteria for what and why to present this warning. I suspect it can and has been used as a form of copy protection too. In the least, a deterrent, which Apple was aware of long before v10.4.9 - such as the 10.3.0 discs and AHT for my MDD2003. In the case of these specific discs, not the entire file system is out but I feel Apple "tagged" these discs with some marker that has set off the warning which now appears years later. Here I'm making up off the top of my head, the tag might be attaching a resource 'fork' to a folder instead of a file. Won't interfere in operation, isn't normally done, and could produce a warning about file system damage.

SkyCapt's picture
Offline
Joined: 2017 Jan 11

I can't stress enough how crucial it is to double check important copy operations. I'm using Terminal "cmp" (if verifying one file) and "diff -r" (if verifying a folder or bundle) and while these aren't totally foolproof because Terminal commands do not operate on resource forks, OSX has been phasing out resource forks anyway. Doing this is providing a good sense about whether data corruption is taking place during copying.

If I copy a massive .zip or .toast or .dmg file for example, after the copy has completed I can verify it by telling Terminal to "cmp -l " then drag the icon of my source file to Terminal and then drag the icon of my newly generated copied file to Terminal, press Return in Terminal and wait for the results. The -l flag tells cmp to List file differences, otherwise if the files differ there'll be a single line of output that says so.

To compare two folders or bundles, use "diff -r" not "cmp -l". Don't forget the -r which means Recursive checking into subfolders or else you might misinterpret whether the results mean pass/fail. Some comparisons do kick out warnings often in pairs one warning for the first operand and a matching warning for the second operand, with practice you'll get used to filtering out what're warnings not errors. The clean way to interrupt the diff command while it is running is press cntl^c.

What's helping me confirm data corruption on my system now is that I've actually started looking systematically only recently, and in becoming a frequent user here, my data sizes have grown manyfold. I rule out defective storage drives and drive cables, this is happening on multiple drives and cables. I rule out defective PCI-SATA cards in my G4 because if I were getting random read errors as frequently as Terminal can show them, then I'd be having trouble booting and trouble running apps - that's not happening, this is different. Here's what I'm finding out:

-----

Experiencing "system damage" is a big deal. I've found all OSX tested 10.2.8 thru 10.5.8 are doing the same bad things.
I now learned I can crash my OSXes, for example i'm using a Fresh install of Tiger 10.4.0 so rule out all add-on software and any modifications. Here's how I crash it.

I have a read-only .dmg of my OSX Games collection, 16.5 (modest) uncompressed GB containing about 50,000 files. If I compare each file without "expanding" them individually, simply by mounting the image and then pretending the resulting volume is "real" not condensed, the read operations must pass thru the diskimagemounter software layer, and it's increasing the tendency for failure. I often find errors come only in the second half of verifying so if I had "only" 25000 files and about 8GB total then I might not see anything wrong. What makes OSX blow up is when BOTH images being compared are passed thru this file system emulation. Here's how.

I copy my archive to one place for testing and I copy it again to a second place for testing. I give the two .dmg files different names so I can mount them both at the same time. I double click the 1st to mount it, I double click the 2nd to mount it, then I use Terminal diff -r to compare them now both are using file systems emulated and I get failed comparisons quickly and even crashes and kernel panics. This comes without mounting+ejecting any images of MacOSROM which seems to just amplify the trouble. A "sad realization" to quote the aforementioned apple tv commercial.

This crashing remains intermittent. Sometimes this Tiger 10.4.0 test settles into an "anti-funk" where it passes the large diff comparisons, again and again passes. I find all I have to do to make it fail again is reboot once or twice, rerunning the diff comparisons after each reboot. If I compare against a real hard drive volume, it has corrupted files and folders on that volume. Always have backup archives and always operate upon copies of them, don't do risky things with your originals.

adespoton's picture
Offline
Joined: 2015 Feb 15

> Terminal commands do not operate on resource forks

If you want to apply terminal commands to resource forks and know that you're checking the resource fork, enter the file name and add "/..namedfork/rsrc" (no quotes) to operate on the resource fork.

I've written a bunch of shell wrappers that take the regular commands, add an r, and then attempt to do the operation on both the data and resource fork of a file.

So I have a cmpr command that does "cmp $1" && "cmp $1/..namedfork/rsrc".

SkyCapt's picture
Offline
Joined: 2017 Jan 11

Oh cool, thanks. A quick test case of files for this is in "Game Doctor" here. Its "Prescriptions" file has no data fork, all resource fork. It comes with two different versions of Prescriptions but if you "cmp" them it says they're the same, and they'll carry the same "md5" (another Terminal command) same md5 as an Empty file. I'm finding (in Tiger 10.4.8 right now) it's not necessary to type "/..namedfork" so just adding "/rsrc" works. I don't suppose there's a way to blend both forks so that "md5" aka "md5r" returns a single number for both forks, like OS9 has with the program "Integrity 2.0"?

I forgot to stress, for the sake of being exact, that the 16.5 GB image mentioned is *installed* Games, not a collection of installers (like mg is). Trouble shows up while comparing the two mounted volumes, not their .dmg files. When two little files we already know are the same are said to "differ" then this is how a corrupted copy is produced, when that file at that moment were to be copied from within an image. Once the "system" is damaged enough, regular copying of files (not involving mounted images) produces occasional corrupt results too, which is why this is so bad, why I'm so fumin'.

Following more thorough testing I conclude those "Mac OS ROM" files are Not causing nor amplifying the "system damage" I'm experiencing. The ROM files do have at least one previously known issue relating to diskimagemounter which had made them (where I use/bypass one in my startup items) the usual suspect, I'll recount in my next msg. On the other hand, Perhaps running the "diff"(s) is an amplifier, so maybe using other software not Apple's to compare file copies could help?

Anybody know a good file/folder Compare Utilities for PowerPC OS X ?

When this "system damage" reveals itself, which affects all OS X tested 10.5.8 down to 10.2.6 then so far the only solution I can say is reboot and continue comparing files, identifying and recopying when they're found bad - meaning not the false alarm bad, bad meaning fails every compare not just once.

adespoton's picture
Offline
Joined: 2015 Feb 15

I'm not sure of a good way to virtually flatten a file for comparison, other than to cat both to output and redirect that output to md5 -- that way, it'll spit out the data and then the rsrc fork to md5 which will then sum across both.

As for a good file/folder compare utility: look no further than rsync. Apple created a resource-aware version, and it does an excellent job of comparing two filesets for differences. You don't even have to actually sync; you can tell it to just output the xsum data of the different files.

SkyCapt's picture
Offline
Joined: 2017 Jan 11

rsync, the Terminal command?

Tried your cat-md5 suggestion and it works. Picking any file with two big forks, I tried the "Crop Circles" game app file. In OS9 Integrity says the data md5 is 'xxx' and the rsrc md5 is 'yyy' but the combined md5 is not xxx+yyy nor xxx-yyy it's a unique formula so OS9 says the whole md5 checksum value is 'zzz'.

Back in OS X Terminal, command "md5 {file}" has the xxx checksum and command "md5 {file}/rsrc" has the yyy checksum. Command "cat {file} {file}/rsrc | md5" produces the correct 'zzz' checksum. That's better, since OSX calculates md5 checksums manyfold faster than OS9 "Integrity 2.0" - especially Integrity running in OS X Classic Mode for obtaining a sum md5, yuck that's so slow, I believe speed discrepancies this severe between native OS 9 and Classic Mode are caused by the program written with malloc/dealloc nested inside looping repetition - the update for Memory Protection brings about this type slowdown.

adespoton's picture
Offline
Joined: 2015 Feb 15

It's likely that running it in Classic is fully in software with no hardware optimizations, due to so many layers of APIs being involved. The MD5 command on OS X is the same one used in BSD and Linux, and takes full advantage of PPC hardware acceleration and extended registers.

But due to the memory protection and drive abstraction in Classic, anything with high Disk I/O is going to be painfully slow compared to its BSD userland equivalent.

SkyCapt's picture
Offline
Joined: 2017 Jan 11

125MB file, tested:

00:02 .. (2 seconds) md5 calculated using OSX Terminal
01:05 .. (65 seconds) md5 calculated by Integrity 2.0 in OS9
21:20 .. (1280 seconds) md5 calculated by Integrity 2.0 in OSX Classicmode

"Integrity" isn't low-level code like Terminal, Integrity was all coded in some big scripting language. I should get a better OS9-md5 eh?

Anybody know a better OS9 md5 util? Imagine if the file 2b checked is CD length ...zzz

adespoton's picture
Offline
Joined: 2015 Feb 15

Hmm... a few things to try (I haven't investigated yet):

https://www.info-mac.org/viewtopic.php?f=195&t=7901&p=7902&hilit=MD5#p7902
https://www.info-mac.org/viewtopic.php?f=158&t=6149&p=6150&hilit=MD5#p6150

SkyCapt's picture
Offline
Joined: 2017 Jan 11

The strange case of "Mac OS ROM" meets diskimagemounter can be found a ways down into the page in this link, and confirms I've been fighting with this for many months already :

http://macintoshgarden.org/forum/bizarre-file-behavior-in-ppc-tiger

In a weird twist it seems ALL files that're being corrupted and false flagged as corrupt (I've seen thousands now) all filenames contain a dot '.' and extension wording. I know that if you grab any one OS X file at random, it is more likely than not to have an extended filename, but after seeing thousands of these error filenames always extended, i'm starting to think it is part of this problem. Also, I add the information I've tested Mac OS X 10.2.6 (pre-Smeagol!) and find the same file failures described already. Tested 10.5.8 down to 10.2.6 now.

SkyCapt's picture
Offline
Joined: 2017 Jan 11

I just did a bunch of HD imaging/syncing/backing-up and again confirm the failures begin happening only after mounting my Games-OSX-PPC .dmg 16.5+ GB big. I reverified copy after copy without fault until I mounted that image, then immediately got failures on additional images. Previously when I tried mounting other images and I got checksum-invalid errors at least 3 times in a row, I assumed the copy was made bad so I would recopy it. This time, after I got checksum-invalid on mounting some big image, I switched to md5'ing the image file and received three more wrong md5s in a row BUT all different from one another. Then I rebooted, and the image ok'ed and mounted fine. It wasn't a bad copy, so, bad writes might not be happening as bad as I thought. When the "system is damaged" however, bad reads come incessantly.

What could be going on here? Not all images could trigger system damage or I'd have seen this sooner. I think it has to be related to the newly huge sizes of some of my images. Ever notice that OS X 10.3.9 limits the size of images to about 10GB or DVD-DL (it's one good reason to use 10.3.8/10.4 instead). It looks like Apple was aware of this back then, they put a bandaid on it in 10.3.9 and soon later another bandaid on it with 10.4.9 where it warns us that certain images may "damage the system" but it looks like they didn't even get the stupid criteria right, images still "damage the system" without warning and images that do give warning are false alarms.

Fast fwd even more, I heard new macOS systems refuse to open "old" images until you "convert" them. I'll put forth that they continued to have trouble with this and continued to try addressing it. Did they get it right this last time?? Can I use a new macOS to "convert" my image and then continue to open the converted image IN TIGER and not have this trouble anymore??

I'll continue running tests on other (large) images to see if I possess other image files which "cause" (sarcasm) all this "system damage".

SkyCapt's picture
Offline
Joined: 2017 Jan 11

Major progress!
I shouldn't've assumed there's only one intermittent problem. I actually have two intermittent problems, one is the tendency for Terminal commands to experience read-only errors when manipulating files, it can happen on many volumes independent of mounting anything, it's almost harmless because I can always boot and run every app without an issue. It may've been lurking in the shadows a long time and perhaps I'm just now noticing it...

The second, other, intermittent problem I've been having is my "Games OSX-PPC" volume standing out like a sore thumb that all hell breaks loose just by mounting this disk image. This definitely was worsening relative to the image growing in size. I've fixed this! Grade Crown Beer It was being caused by the presence of "Indy.app" : the file belonging to game named "Indiana Jones & the Emperor's Tomb". I've built a new disk image which doesn't have "Indy.app" in it, this trouble has stopped! How can I be certain, let's look at the evidence ~ 3 2 1...

3) I got system freezes, intermittently, just by doing this: reboot, mount "Games OSX-PPC", try copying everything out from the mounted image to a physical storage drive. That's it! Dead computer during file copy!, intermittent! But it became sooo bad, I tested my latest 21 GB image made 5 failures in 10 tries and ended in fail twice in a row. Now with the new 19 GB image post-Indy-removal I've repeated these tests on the same partitions more than twenty times now and still running more copies ... all without a hitch.

2) ClamAntiVirus scanner, supplied by Apple in the latest version of Tiger Server OS X - freaks out when it sees "Indy.app" : not reported as "infected" with anything, just internal cryptic engine warnings, over 200 of the same warning so you can't not notice this! Older versions of Clam didn't hate on Indy.app it's only after just recently upgrading Clam to its highest Tiger Server commandline version (0.94.0 - 0.94.2) do I see this. Lucky coincidence!

1) The SIZE of "Indy.app" is its problem. It's the largest .app anywhere in my system, it could probably be even larger and not be problematic, nobody has ever said there's a size limit for a .app file. What makes it agitate some nasty bug in the OS is its PARTICULAR size. Indy.app's outer shell is about 2.01 GB and inside it has a folder with about 1.99 GB. 2.00 GB is one of the critical capacity divisions between 32-bit only processing and the 32-bit afterlife. By subtly dancing a tiny amount around both sides of the 2.00 GB marker, Indy.app is triggering some nasty bug! The specific event which triggers "system damage" via "opening this image" is **recursively traversing Indy.app's internal file system, period. Happens automatically when mounting Games. Maybe I could stuff a few "filler" or "padding" files into Indy.app and get it to stop?? Right now I'm just happy to be done with this.