News Group Posts

tape backup script errors

From: Mike Brown <mike@tkg.ca>
Subject: Re: tar backup
References: <rUys9.30959$La5.68639@rwcrnsc52.ops.asp.att.net>
<3DB407D7.A3CE454E@tkg.ca>
<H4CB4s.pCA@wjv.com>
<3DB5A916.962330F2@tkg.ca>
<H4GrGG.1LCH@wjv.com> 
Date: Thu, 24 Oct 2002 22:58:02 GMT

Bill Vermillion wrote:
> 
> In article <3DB5A916.962330F2@tkg.ca>, Mike Brown  <mike@tkg.ca> wrote:
> >Bill Vermillion wrote:
> 
> >> In article <3DB407D7.A3CE454E@tkg.ca>, Mike Brown  <mike@tkg.ca> wrote:
> >> >Norbi wrote:
> 
> >> >> The situation is the following: I am writing a script to do
> >> >> a backup, in the case the backup failed (bad tape, or write
> >> >> protected tape, etc etc), then it should warn the operator
> >> >> that the backup was not successful, and ask if they want
> >> >> to retry or not.  ...
> 
> [Lucretia Deletia weild her knife here]
> 
> >> >There is no reasonable way to trust a tar backup without a way
> >> >of checking it.
> 
> [snip snip]
> 
> >> I have seen cases where the backup proceeded with no
> >> >apparent error, but the verify fails.
> 
> >> >You could set up the tar command to place a "check" file at the
> >> >end of the tape,  ....
> 
> >> That wont guarantee anything except that the 'check' file is
> >> correct.  ...
> 
> >I agree completely.  I suspect the trouble likely comes between
> >the tape head and the tape, not from the hard drive to the tape
> >drive.
> 
> I don't agree with that.  In all my past experiences, broadcast,
> recording and computers, I've been working with magnetic tape
> longer than many people on this list have been alive.
> 
> As CE of a recording studio we were in the $50,000/year column on
> purchases from Ampex, Scotch and Agfa [none of which exist any
> more]
> 
> > With such high tape density any misalignment causes
> >problems, which the tape drive will try to correct by ECC or
> >rereading the tape.
> 
> Today's tape are marvels of engineering with such things as metal
> particles that are ceramic coated to resist oxidation and make them
> stronger than a pure particle.
> 
> The newer machines are pretty much self-aligning.
> 
> You can have bad tape - but when a bit does not match on the tape
> when compared to the HD I'd bet on some electrical discturbance
> between the time it left the HD head and arrived at tape
> electronics.   I think I mentioned that if this occurs you will get
> a proper checksum on tape as it is computer against the corrupt
> data.  And a tape verify - a read from end to end - will show the
> tape is OK.
> 
> If the error occured between the head and the tape then the
> checksums on tape would not match or you would get a tape read
> error.
> 
> > Some drives will flash a LED to indicate
> >it is using ECC for correction.
> 
> And also when they need cleaing.
> 
> >Since the drives do read after write checking they can mark
> >a section of the tape bad, then continue on with the backup
> >further down. The end result is that the drive will continue to
> >heal and retry with no outward indication until it really can't
> >use that tape anymore.
> 
> That's more like a HD than tape drive.
> 
> >The supertars tend to help out with this issue
> >by tracking number of uses, average backup rate and the bit
> >level verify.  If you notice you backup rate decreasing by
> >10% or more, try a new tape.
> 
> The number of passes on tape and a rejection of the tape when NNNN
> passes have been made goes back much further than any of the
> supertar programs.
> 
> If backup rate decreases it can also be caused by a drive that has
> become fragmented badly by creation of lots of small files and then
> having the files scattered in pieces.  That is more likely to
> happen on older file system than on modern file systems - many
> based on the FFS which does it's best to allocate files all over
> the disk to keep this from happening.   I say this because slow
> backup does not always mean tape problems.
> 
> I always like to use something like Iozone 2 [not the 3 version
> with it's overwhelming output] to benchmark the hd file syste
> read/write when intalling a new system.  Then if things slow down
> you can run it again.  The nice thing about iozone is that it
> writes through the file-system and you will see how the fs works in
> real life.  Many hd benchmarks show only raw throughput.  I see
> many using dd if=<drive> to find out how fast the system is.  But
> that only show HW capability and you'll find filesystem throughput
> to be just a fraction of that.
> 
> On an older system a few years ago I had both an original S51 file
> sysem [the old Xenix and early Unix FS] and a modern FFS.
> I could get speeds up to 5 to 10 times faster on the FFS side than
> I could on the S51 side.  This was just two different FSes on the
> same drive.
> 
> IOW I'm trying to point out that slow backup is not neccesarily an
> indication of a tape problem.
> 
> >The trick with placing a check file at the end assumes a good
> >tape drive and tape media. I have found it useful when I need
> >to send someone a tape copy and they do not have a supertar.
> 
> Ah - but the supertars can be read on a normal system.  The only
> gotcha is that if you compress the files before backup on a
> supertar they automatically decompress on retreival and on a normal
> system you would have to do this yourself.
> 
> The only time I've seen an anomoly was several years ago when
> I extracted via a normal tar on a Pentium based BSD system that was
> created on an SGI Indy - with a MIPS 4440 chip and using LoneTar.
> 
> The files appeared to extract properly but upon further examination
> they were all compressed files.  A rename of the files to give them
> a .Z extension and an uncompress fixed that.  If you had forced
> decompression you would have a namespace collision.
> 
> Never investigated that one further but it was interesting.  The
> 'file' command can come in handy at times like that.
> 
> Bill
> 
> --
> Bill Vermillion - bv @ wjv . com

In reference to DAT drives, check out

http://www.seagate.com/support/kb/tape/datmdtch.html

The section on RAW ( Read - After - Write ) is what I was hinting at.
Some of the DAT drives do flash for a tape cleaning, but flash a different
pattern or LED when correcting RAW failures or using ECC for correction.

I have some DAT tapes that 'work', but have bad sections on them.  If
you use a utility like Lone Tar's tape sizer you find that the tape
holds less data then spec'd for.  A backup to it takes longer because
the drive is rewriting the data.  The RAW procedure, I think, makes its
decisions based on analog SNR rather than a hard read error.  As the tape
media degrades, you end up with more rewrites, and lower write speeds.
The marketing/engineer who was describing the process claims this to
be a GREAT advantage, because it 'safely' increases the life span of
the media.  If you do not use 100% of the tape for the backup, then
the area near the beginning of the tape experiences wear while the end
of the tape may never be reached.  By rewriting and 'skipping' over
a few bad sections at the beginning of the tape the backup data gets
moved further on to new area on the tape.

The rough rule of thumb I used at a 10% decrease in backup speed assumed
the problem of disk fragementation was cancelled out because I compare
one night backup from a clients HD to another nights backup from the
same HD.  Also, most backups are done at night when there should be
no other programs running to cause different system loads.

The last assumption I make is that the data travelling on the SCSI
bus does not undergo random errors because ( at least on the Compaq
servers I use ) they have parity protection.  The data on the drive
is ECC detected and corrected, and the transfer mechanism has
parity or checksums at every point along the way.

In an attempt to test some of these common issues, I have have picked
up tapes from customers and backed up to the tape from one of my
servers.  The test directories are on a raid 1+0 array, and are
not fragemented much.  In any case the data is available from the
drives at a rate much higher than the tape can receive it.  Some
of the tapes reduce the write speeds by over 25%, yet back up and
restore without error ( well, any noticable error ).  A size test
also indicates reduced capacity, such as turning off HW compression
and only getting 10.8 Gbytes on a 12 Gbyte tape.

With so much magic going on, I really insist on a product like
Backup Edge.

Any DAT experts out there that can comment on my theories?

Mike  


-- 
Michael Brown

The Kingsway Group
 

Comments?

Jump to Comments

Many of the products and books I review are things I purchased for my own use. Some were given to me specifically for the purpose of reviewing them. I resell or can earn commissions from the sale of some of these items. Links within these pages may be affiliate links that pay me for referring you to them. That's mostly insignificant amounts of money; whenever it is not I have made my relationship plain. I also may own stock in companies mentioned here. If you have any question, please do feel free to contact me.

Specific links that take you to pages that allow you to purchase the item I reviewed are very likely to pay me a commission. Many of the books I review were given to me by the publishers specifically for the purpose of writing a review. These gifts and referral fees do not affect my opinions; I often give bad reviews anyway.

We use Google third-party advertising companies to serve ads when you visit our website. These companies may use information (not including your name, address, email address, or telephone number) about your visits to this and other websites in order to provide advertisements about goods and services of interest to you. If you would like more information about this practice and to know your choices about not having this information used by these companies, click here.