News Group Posts

linux swap space allocation

From: Floyd Davidson <floyd@ptialaska.net>
Newsgroups: comp.os.linux.misc
Subject: Re: Why Does the EXT2 filesystem not need defragmentation.
Date: 16 Sep 2000 03:32:55 -0800
Message-ID: <87snr0vafs.fld@barrow.com> 
References: <39BFE479.446835B2@escape.com>
<87r96nzo91.fld@barrow.com>
<slrn8s0ibq.1d4.cbbrowne@knuth.brownes.org>
<87vgvzxxpr.fld@barrow.com>
<8FB06AB90ogo1mynxwowaustcom@203.2.194.51>
<87lmwuwgho.fld@barrow.com>
<39C21644.BAB72B83@exit109.com> 

Jean-David Beyer-valinux <jdbeyer@exit109.com> wrote:
>Floyd Davidson wrote (in part):
>
>> There are some other considerations.  Linux buffers disk reads,
>> and uses otherwise unused RAM for that purpose.  Hence the more
>> RAM you have available for disk buffering, the faster your
>> programs will generally run.
>
>I suppose so, but it depends a lot on your access patterns. I
>suspect that most of the stuff in my buffers and cache are not
>useful: I am rummaging around in a large database, running a 2-hour
>populate of the database. All the data coming in are new, and I
>read precious little of the data that are already there. The
>indices are in the cache (I hope), and that can help, but since I
>am mostly inserting new records, I imagine that the cache is mostly
>full of useless stuff.

Your particular userid may not, but then again it might.  Shell
scripts call binaries in /bin, /usr/bin, and /usr/local/bin
often, and various data files and configuration files are also
accessed often.

Hence the quantity of re-usable data may be low, but it is also
possible even though small it might account for a much higher
percentage of disk activity than you might expect.
Additionally, the other data that you are reading might well be
available in sequential reads...  which means the others being
accessed from memory speeds up access to those that are read
from disk by not mixing non-sequential disk reads in the
process.

Suffice it to say that there is no doubt at least some benefit,
and it is essentially free because it uses a resource that would
otherwise be idle.

>> Linux will also swap out unused
>> processes, making the space in RAM they would otherwise take up
>> available for disk buffering.  Hence you do want _some_ swap
>> space allocated.
>>
>> I have a system similar to yours, in that it has 256Mb of RAM
>> and often is lightly loaded.  Here is what free says right now:
>>
>>          total        used     free   shared  buffers   cached
>>  Mem:   257972      242560    15412    30872    62268    62000
>>  -/+ buffers/cache: 118292   139680
>>  Swap:  315048       15928   299120
>>
>> Which indicates that 16Mb has been swapped out, and is available
>> for disk buffering.  (You can also get an idea how much swap I
>> might recommend. :-)
>
>Mine is:
>
>             total:     used:     free:  shared:  buffers:
>cached:
>Mem:  529330176 526209024   3121152 89923584 207638528 190623744
>Swap: 279642112  17080320 262561792
>
>This seems to support my view that no matter what the amount of RAM
>(iff there is "enough"), Linux uses about 16 Megabytes of swap
>space. ;-)

Not really.  It does mean that you and I have just about the
same amount of memory allocated to processes that are sleeping
virtually all of the time.  Right now, on my box that happens to
be  kflushd, kupdate, kpiod, kswapd, bash (five instances),
agetty (two instances), rpciod, lockd, startx, xinit, gnuserv,
and pppd.

Obviously I could run either more or fewer processes that would
result in the use of swap space. :-)

>> The actual virtual memory (RAM + swap) you *must* have amounts
>> to "more than the need for virtual memory will ever be".  And
>> remember that if your vm use exceeds the RAM + swap total, your
>> system will crash.
>
>Is this true of Linux? In the old days (around mid 1980s), a
>UNIX system would not crash when you ran out of memory.  The
>fork(2) routine would just give an error return. If a shell got
>that, it would just write "cannot fork", but the system would
>remain up. Of course, the sysadmin would start looking around
>for the memory hog.  Somewhere around that time, they reserved
>a little memory and a process table slot for the super-user so
>that something could be done.

Sure the system "remains up", but it is in a worthless state
until the sysadmin determines what has been arbitrarily killed
off and what has not.  The only sensible response is to reboot
immediately after discovering which process is hogging memory.
I may be a bit more liberal about the definition of "crash" than
most, but clearly a forced reboot even if it is at my leisure, is
a "crash".

>> (And if you use Netscrape, at some point the amount of vm it
>> uses will be all there is, no matter how much you have.  In that
>> case you want enough that you'll notice it before the crash...)
>
>On my systems, Netscape uses around 24 Megabytes, but it does not
>grow unbounded, even when I open multiple windows with it. Others
>report memory leaks, but I have never seen any. Does the memory
>leak without appearing in the Netscape line of the top command?

I don't remember the specifics now, but it has been positively
identified.  If I remember right it has something to do with
Java applets, which means if you access only the right web pages
it will never hit you at all.  I can often go for a week or two
and never have a problem.  Then I'll start searching for some
odd thing and after I've accessed a couple dozen odd sites I
notice that everything is slow to start up... and what I find is
that I have 200Mb of swap in use!

-- 
Floyd L. Davidson                          floyd@barrow.com
Ukpeagvik (Barrow, Alaska)

Comments?

Jump to Comments

Many of the products and books I review are things I purchased for my own use. Some were given to me specifically for the purpose of reviewing them. I resell or can earn commissions from the sale of some of these items. Links within these pages may be affiliate links that pay me for referring you to them. That's mostly insignificant amounts of money; whenever it is not I have made my relationship plain. I also may own stock in companies mentioned here. If you have any question, please do feel free to contact me.

Specific links that take you to pages that allow you to purchase the item I reviewed are very likely to pay me a commission. Many of the books I review were given to me by the publishers specifically for the purpose of writing a review. These gifts and referral fees do not affect my opinions; I often give bad reviews anyway.

We use Google third-party advertising companies to serve ads when you visit our website. These companies may use information (not including your name, address, email address, or telephone number) about your visits to this and other websites in order to provide advertisements about goods and services of interest to you. If you would like more information about this practice and to know your choices about not having this information used by these companies, click here.