drop_caches
Submitted by admin on Wed, 2006-05-31 16:12
Writing to this will cause the kernel to drop clean caches, dentries and inodes from memory, causing that memory to become free.
To free pagecache:
- echo 1 > /proc/sys/vm/drop_caches
To free dentries and inodes:
- echo 2 > /proc/sys/vm/drop_caches
To free pagecache, dentries and inodes:
- echo 3 > /proc/sys/vm/drop_caches
As this is a non-destructive operation, and dirty objects are not freeable, the user should run "sync" first in order to make sure all cached objects are freed.
This tunable was added in 2.6.16.
»
- Add new comment
- 79940 reads
"This tunable was added in
"This tunable was added in 2.6.16."
Thanks, but what shall we do in kernel 2.4 ?
Unfortunately, 2.4 kernels
Unfortunately, 2.4 kernels don't have any such neat mechanism to get rid of cached data.
Why not upgrade, btw? 2.4 is ancient history by now..
never change a running
never change a running system
this system is an embedded arm based platform
But, is it then really so
But, is it then really so important to clear those caches? Being an embedded platform, I guess your system doesn't do much disk based I/O, so it doesn't really need a "clear cache" mechanism, right?
Honestly, drop_caches mechanism is most useful for testing/benchmarking purposes. If it's needed in production somewhere, it's almost certain that something else needs to be fixed / tuned.
Linux page/buffer/inode/dentry caches are quite efficient mechanisms of caching and in 99% cases you want to leave them alone doing their job, which is speeding your computer by using unused RAM for caching.
Agreed! In my case I am
Agreed!
In my case I am copying files from usb to local. Then I want to verify them.
I want to be absolutely shure not to verify against files in cache.
OK, that really makes
OK, that really makes sense.
You could always try to allocate larger and larger chunks of memory until your caches are _mostly_ cleared (monitor with vmstat 1). It'll typically happen at about the same time when your allocator starts pushing applications to swap (if you have it). That's how I did it in the old glory days of 2.4. ;)
But, doing such procedure as explained above is definitely not certain to clear all your caches, and it's also slightly dangerous (you can easily hit OOM condition, if not careful). So proceed at your own risk. :)
You'd also need to write a simple app to allocate memory as specified on the command line, of course.
sh -c "echo 1 > /proc/sys/vm/drop_caches"
how about this one, this helps the cached memory and buffers
sh -c "echo 1 > /proc/sys/vm/drop_caches"
as root. this should be safe to use, for exsample sudo sh -c "echo 1 > /proc/sys/vm/drop_caches"
regards mikedk
Memory cache issue
Hi,
I am using memcached if i use sync;echo 3 > /proc/sys/vm/drop_caches will it clear my memcache content.
Thanks in advance.
-_Sai.
Proposed addition
I'd suggest adding the following to the second last sentence about 'sync': "...in order to make sure all cached objects are freed". Just to make it clear why the user 'should' run sync.
freeing cache
This is needed if Linux is aggressively swap thrashing, which normally does not occur. Also syncing and freeing cache will not help too much, as the root of the issue is something else. let linux manage memory in its own fashion, u try to compile programs from vanilla source with optimization flags suitable for your machine architecture.
drop_caches vs swappiness
You're right, but this tunable was never supposed to be used on production systems as an additional optimization, but rather as a tool for people doing various benchmarks and tests.
Linux kernel is really doing a good job managing memory, but if you prefer to push it in specific direction, there's another tunable with which you can achieve the desired effect: /proc/sys/vm/swappiness
Snapshot on SAN
Hi,
I was wondering if this whould be enought to garanty that everything is written to disk before making a snapshot (block level) of a volume on a SAN (equllogic) or do I have to consider other thing?
Thank you for any comment.
Andr.
No, this tunable doesn't
No, this tunable doesn't write anything out to disk. It only discards clean pages from memory, leaving dirty ones intact.
I don't think there is functionality like the one you need in the linux kernel/fs.
If you call fsync(2) your file is guaranteed to be on disk before the system call returns. But, you would need to call that for all open files/directories, and just before you're done, something else is dirtied, of course.
OTOH, if you call sync(2), kernel schedules all dirty pages for writing, but it by no means guarantees they will be safe on plates before the system call returns.
But, I believe that your snapshot functionality surely takes care of all this, right? You can always check its manual. Although, one sync(2) before doing it can't hurt.
What about echo 3 >/proc/sys/vm/drop_caches
Hi,
I was going thru the blog and I was wondering if your answer only concern the swappiness parameter or did it also include the "echo 3 >/proc/sys/vm/drop_caches" command after doing a sync?
Would this do the job?
André
I'm 100% positive that none
I'm 100% positive that none of those two parameters have any effect on the dirty data whatsoever. drop_caches only does business with clean pagecache pages (those that don't need to be written out to disk before cleaning them from memory safely). And the swappiness parameter controls how happy kernel is to swap pages out, instead of trying harder to shrink pagecache. Those are two different subsystems in the kernel, with completely different goals.
San does not know of the OS activity
Thank for the info.
The snapshot is taken at the SAN level so it doesn't have any knowledge of what happening at the os level. My SAN is a Dell Equallogic and the only support Dell seem to give is for windows OS. For those os (server, xp and vista) they recommand using VSS (volume shadow copy service) at the os level before doing the snap at the SAN level.
I will continue looking for a solution and post it here if I find one.
Thank
André
Snapshotting Linux partitions
Ah, I believe I know what bothers you. You would like to make a consistent snapshot of Linux file system that is on a SAN device. I don't think you'll be able to do that, unless, of course, you can stop services running on the partition, umount it and only then snapshot it.
If you can't stop services, typically you would run sync on the Linux machine (just in case), then snapshot live partition. If you're using a modern journaling file system like ext3 or ext4, such filesystem is easily mounted, even though it was "dirty" at the time it was snapshotted. FS journal would be replayed on the mount and everything should work. You'll probably lose only seconds worth of data. In the worst case, e2fsck should make it mountable, if you somehow get stuck at the "journal replay" stage (not probable).
Think about it, regular backup procedures easily take hours to backup the whole filesystem, now THAT is what I call inconsistent backup. But, 98% of backup procedures are just like that, inefficient and inconsistent.
The only real problem with snapshotting is when you have raw database files on the FS. But, if you use a high-end commercial database (like Oracle), they typically have a command to "freeze" the tablespace files so you can backup them consistently. Opensource databases like PostgreSQL have dump commands which can be run on the live system and produce compact and consistent backup of all the data in database, that is how I backup my pgsql.
Good luck!
Thank for the info
Thank for the info.
We use ext3 filesytem but I never give it much attention. I will look at it more closely and see if it answer my interrogation.
As for the database (Oracle) we do use rman to backup it up and we copy the backup to tape. We also have the flashback function activate on the database so we can return to any point in time in the last 2 days very quickly.
If we are looking at snapshot at the SAN level, it because the database is on one volume of the SAN and the report create by the program that access the database on another volume. With the snapshot of the two volumes at the same time we would be assure that we can go backup at the same point on both volume without having to delete report manually.
We are also looking to use the replication function for our disaster recovery plan. That why we need to be assured that when we do a snap or replication, that the data is consistent. If the ext3 journal system give us that assurance it will be great.
André
This is tricky -- get some expert advice.
Ext3 does not give you assurance that your data is consistent. It gives you assurance that your filesystem is consistent. In general if you are making backup snapshots of a database while it is running then the database itself must notified of this or it must be able to recover from an inconsistent volume. In either case you would loose data at least up to the time the snapshot started, but that's all you could hope for anyway.
I would not feel confident at all in attempting to create two separate snapshots of two different volumes with the requirement that they be in sync up to the same point snapshots were started.