News Group Posts

sparse files

See Sparse files also.

From: Tony Lawrence <tony@aplawrence.com>
Subject: Re: Problems with backups/restores

Bill Vermillion wrote:

> But sparse files don't have to be assocaited with hashed files - at
> least not as I understand them - so feel free to point me in the
> proper direction if I'm wrong.

.. stuff deleted..

> Let's take something like a file created with something as simple
> as a BASIC program [yes - you used to see that on Unix systems].
> 
> Using randon I/O - and for easy of math computation lets say the
> data is a 100 byte record.   Write record number 1 and you start at
> the front.  Then just call the identify the second record as record
> number 1000.  

But there would be absolutely no reason whatsoever for you to choose
1000 as the position of the second record *unless* you were using a
hashing algorithm.  Logically, the second record would be, well, the
second record.


> That is correct and this assumes something like a database as good
> design requires this.  I was only commenting on the fact that
> the hashing method is not the only way to create sparse files.

No, but it's generally the only logical reason one would arbitrarily
jump out to some high byte position.  Why else would you do such a
thing?  Maybe I'm just too tired (I had a 22 hour day a couple of days
back), but I can't think of any other reason.


The purpose of hashing is ultrafast access- no other access method can
even begin to approach it for random keys- it's really not a "database"
thing (unless you consider any access to data a database),  Back in the
days of slow media I wrote programs that used hashed storage for no
other reason than that- it let me get my data back quickly.

A side effect of hashing is the sparse file if the file system supports
it.  I really cannot think of any other reason that one would create
data in this manner.. someone less burned out than I can feel free to
remind me of such a reason..


I suppose there is the case (which I've certainly used where I could)
where some sequentially assigned number is part of the data- in that
case you can use that number in the same manner as a hash, and it is, in
effect, a degenerate hash- the function is just times 1 or plus zero-
but to my mind that's a hash even though you don't actually do any
math..  purists can disagree, of course, but I ignore purists anyway :-)

I'm pretty sure that the rationale for sparse files was hashing- I'm not
sure whether the driving force was efficiency in creating and reading or
to save space, but certainly both space and media speed were much more
critical then than now.

--
Tony Lawrence
SCO/Linux Support Tips, How-To's, Tests and more: 
 

Comments?

More Articles by Tony Lawrence - Find me on Google+

Jump to Comments

Many of the products and books I review are things I purchased for my own use. Some were given to me specifically for the purpose of reviewing them. I resell or can earn commissions from the sale of some of these items. Links within these pages may be affiliate links that pay me for referring you to them. That's mostly insignificant amounts of money; whenever it is not I have made my relationship plain. I also may own stock in companies mentioned here. If you have any question, please do feel free to contact me.

Specific links that take you to pages that allow you to purchase the item I reviewed are very likely to pay me a commission. Many of the books I review were given to me by the publishers specifically for the purpose of writing a review. These gifts and referral fees do not affect my opinions; I often give bad reviews anyway.

We use Google third-party advertising companies to serve ads when you visit our website. These companies may use information (not including your name, address, email address, or telephone number) about your visits to this and other websites in order to provide advertisements about goods and services of interest to you. If you would like more information about this practice and to know your choices about not having this information used by these companies, click here.