drop_caches

Writing to this will cause the kernel to drop clean caches, dentries and inodes from memory, causing that memory to become free.

To free pagecache:

  • echo 1 > /proc/sys/vm/drop_caches

To free dentries and inodes:

  • echo 2 > /proc/sys/vm/drop_caches

To free pagecache, dentries and inodes:

  • echo 3 > /proc/sys/vm/drop_caches

As this is a non-destructive operation, and dirty objects are not freeable, the user should run "sync" first in order to make sure all cached objects are freed.

This tunable was added in 2.6.16.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

APC PHP eat my memory

For my performance, I must enable APC but within 2 hours memory cache is 8Gs, :(
So I add your command to crontab.
thank

Strange ftp remote listing appear / disappear behavior post memc

I was using memcached and apc on a debian squeeze server with linux 3.0.2 and the aegir (for drupal) hosting platform, when I had to reinstall the aegir platform that the majority of ther server sites were running on. SO I did that and have disabled and uninstalled and purged memcacehd and apc - which I no longer use. I put the sites back up - and I have a big problem:

Sometiles when I sftp in to the server it showh me the current configuration, and sometiles it shows me the old configuration with the previous aegir hostmaster platform - completely different directories with files in them, etc... I set the new system up diferently.

WHen I use a program that gives me ssh and sftp at the same time, ssh does not see what sftp is showing me!

I am hoping this is just a cache / memory problem and that I need to remove the old apc and /or memcached caches? Could this be right? I am new at this and would be thankful for advice as to where I might find those caches.

Reards,

Ed

Haven't used memcached at

Haven't used memcached at all, but it is probably using application (process) memory to cache stuff, so I'm pretty sure that drop_caches mechanism can't help you there. Refer to the memcached manual and/or appropriate forum to resolve your problem.

How do you confirm?

How do you check, after running this command that the cache is actually cleared?

thanks!

Run free(1) before and after.

Run free(1) before and after. Check the difference between buffers/cache(d) values before and after.

Thank you! That worked great!

Thank you! That worked great!

Don't echo, use sysctl

Better use sysctl instead of echoing:

/sbin/sysctl vm.drop_caches=3

Need Help with this command

I'm having issues running this command

sync; echo 3 > /proc/sys/vm/drop_caches

I'm getting bash: echo: write error: Operation not permitted

I have change the owner and group to a regular user and it's set to rwx across the board. I have to have SELinux enabled.

Any suggestions??

sudo

You have to use root privs. Try

sudo sh -c "sync; echo 3 > /proc/sys/vm/drop_caches"

I've always found this form

I've always found this form of the sudo command neater, since it doesn't create a new shell (and so doesn't need to be quoted):

echo 3 | sudo tee /proc/sys/vm/drop_caches

After I used command sync;

After I used command sync; echo 3 > /proc/sys/vm/drop_caches it cleared my cache as expected, but it also stopped using cache - how do I enable it back?

Don't worry, by using

Don't worry, by using drop_caches you're not disabling cache, you're just clearing it. So, as your apps start to pull stuff from disk (or write to it) it will be cached. Just give it some time, or if you don't believe me, copy or read a big file and you will see cache utilized once again.

Hi, thx for reply ... it took

Hi, thx for reply ... it took couple of hours when I notified in toolbar monitor that cache is increasing again ;) So everything is fine again :)

echo 3 > /proc/sys/vm/drop_caches

echo 3 > /proc/sys/vm/drop_caches

after using this cammand my server get hang after some time ...and i have to hard reboot it

any cleue for this ...plz

"This tunable was added in

"This tunable was added in 2.6.16."

Thanks, but what shall we do in kernel 2.4 ?

Unfortunately, 2.4 kernels

Unfortunately, 2.4 kernels don't have any such neat mechanism to get rid of cached data.

Why not upgrade, btw? 2.4 is ancient history by now..

never change a running

never change a running system

this system is an embedded arm based platform

But, is it then really so

But, is it then really so important to clear those caches? Being an embedded platform, I guess your system doesn't do much disk based I/O, so it doesn't really need a "clear cache" mechanism, right?

Honestly, drop_caches mechanism is most useful for testing/benchmarking purposes. If it's needed in production somewhere, it's almost certain that something else needs to be fixed / tuned.

Linux page/buffer/inode/dentry caches are quite efficient mechanisms of caching and in 99% cases you want to leave them alone doing their job, which is speeding your computer by using unused RAM for caching.

Agreed! In my case I am

Agreed!

In my case I am copying files from usb to local. Then I want to verify them.
I want to be absolutely shure not to verify against files in cache.

Aren't there ways to read and

Aren't there ways to read and write files without using the cache? Though you might need to write the copying and verifying code yourself, if there is no utility that can do the job with by-passing the cache.

OK, that really makes

OK, that really makes sense.

You could always try to allocate larger and larger chunks of memory until your caches are _mostly_ cleared (monitor with vmstat 1). It'll typically happen at about the same time when your allocator starts pushing applications to swap (if you have it). That's how I did it in the old glory days of 2.4. ;)

But, doing such procedure as explained above is definitely not certain to clear all your caches, and it's also slightly dangerous (you can easily hit OOM condition, if not careful). So proceed at your own risk. :)

You'd also need to write a simple app to allocate memory as specified on the command line, of course.

sh -c "echo 1 > /proc/sys/vm/drop_caches"

how about this one, this helps the cached memory and buffers

sh -c "echo 1 > /proc/sys/vm/drop_caches"
as root. this should be safe to use, for exsample sudo sh -c "echo 1 > /proc/sys/vm/drop_caches"

regards mikedk

Memory cache issue

Hi,
I am using memcached if i use sync;echo 3 > /proc/sys/vm/drop_caches will it clear my memcache content.

Thanks in advance.
-_Sai.

Of course not ! Memcache

Of course not ! Memcache memory is, from the kernel point of view, not cached memory but application memory.

Proposed addition

I'd suggest adding the following to the second last sentence about 'sync': "...in order to make sure all cached objects are freed". Just to make it clear why the user 'should' run sync.

freeing cache

This is needed if Linux is aggressively swap thrashing, which normally does not occur. Also syncing and freeing cache will not help too much, as the root of the issue is something else. let linux manage memory in its own fashion, u try to compile programs from vanilla source with optimization flags suitable for your machine architecture.

drop_caches vs swappiness

You're right, but this tunable was never supposed to be used on production systems as an additional optimization, but rather as a tool for people doing various benchmarks and tests.

Linux kernel is really doing a good job managing memory, but if you prefer to push it in specific direction, there's another tunable with which you can achieve the desired effect: /proc/sys/vm/swappiness

/proc/sys/vm/drop_caches

Hi,
Please correct me if i am wrong.

"sync; echo 3 > /proc/sys/vm/drop_caches " can finish current process and then clears cache right?

Because , in my application, i am writing data into 2 files which is read by some other source. so i am getting kernel paging failure and when i observed "vmstat" output it gives very less free memory.
once i saw this post and i gave a try, to clear cache periodically say every 30Secs , then my application is not crashing.

if this tuning is not a production use.
can you suggest me which one can i use for production purpose?

Thanks in Advance

The tunable certainly can be

The tunable certainly can be used in production, it shouldn't have any serious side effects, so it's safe. Although in your case, it would probably be better to first see what is using memory and try to optimize that aspect.

I suppose you're getting those kernel paging failures because of memory fragmentation, drop_caches could help somewhat to keep more memory free for longer periods of time and thus keep the memory less fragmented. So, if you observe that it helps, by all means, yes, go for it.

Snapshot on SAN

Hi,

I was wondering if this whould be enought to garanty that everything is written to disk before making a snapshot (block level) of a volume on a SAN (equllogic) or do I have to consider other thing?

Thank you for any comment.

Andr.

No, this tunable doesn't

No, this tunable doesn't write anything out to disk. It only discards clean pages from memory, leaving dirty ones intact.

I don't think there is functionality like the one you need in the linux kernel/fs.

If you call fsync(2) your file is guaranteed to be on disk before the system call returns. But, you would need to call that for all open files/directories, and just before you're done, something else is dirtied, of course.

OTOH, if you call sync(2), kernel schedules all dirty pages for writing, but it by no means guarantees they will be safe on plates before the system call returns.

But, I believe that your snapshot functionality surely takes care of all this, right? You can always check its manual. Although, one sync(2) before doing it can't hurt.

What about echo 3 >/proc/sys/vm/drop_caches

Hi,

I was going thru the blog and I was wondering if your answer only concern the swappiness parameter or did it also include the "echo 3 >/proc/sys/vm/drop_caches" command after doing a sync?

Would this do the job?

André

I'm 100% positive that none

I'm 100% positive that none of those two parameters have any effect on the dirty data whatsoever. drop_caches only does business with clean pagecache pages (those that don't need to be written out to disk before cleaning them from memory safely). And the swappiness parameter controls how happy kernel is to swap pages out, instead of trying harder to shrink pagecache. Those are two different subsystems in the kernel, with completely different goals.

San does not know of the OS activity

Thank for the info.

The snapshot is taken at the SAN level so it doesn't have any knowledge of what happening at the os level. My SAN is a Dell Equallogic and the only support Dell seem to give is for windows OS. For those os (server, xp and vista) they recommand using VSS (volume shadow copy service) at the os level before doing the snap at the SAN level.

I will continue looking for a solution and post it here if I find one.

Thank

André

Snapshotting Linux partitions

Ah, I believe I know what bothers you. You would like to make a consistent snapshot of Linux file system that is on a SAN device. I don't think you'll be able to do that, unless, of course, you can stop services running on the partition, umount it and only then snapshot it.

If you can't stop services, typically you would run sync on the Linux machine (just in case), then snapshot live partition. If you're using a modern journaling file system like ext3 or ext4, such filesystem is easily mounted, even though it was "dirty" at the time it was snapshotted. FS journal would be replayed on the mount and everything should work. You'll probably lose only seconds worth of data. In the worst case, e2fsck should make it mountable, if you somehow get stuck at the "journal replay" stage (not probable).

Think about it, regular backup procedures easily take hours to backup the whole filesystem, now THAT is what I call inconsistent backup. But, 98% of backup procedures are just like that, inefficient and inconsistent.

The only real problem with snapshotting is when you have raw database files on the FS. But, if you use a high-end commercial database (like Oracle), they typically have a command to "freeze" the tablespace files so you can backup them consistently. Opensource databases like PostgreSQL have dump commands which can be run on the live system and produce compact and consistent backup of all the data in database, that is how I backup my pgsql.

Good luck!

Fwiw Oracle and Postgres

Fwiw Oracle and Postgres support hot backups but they don't do it by freezing the data. They use transaction "write-ahead" logs so that when the backup is restored the changes done while taking the backup are redone which corrects any inconsistencies. You do need to tell the databases that you're about to take a backup which does something slightly different in the two but the main thing it does is mark when the beginning and end of the backup was so that the restore command knows which parts of the log need to be replayed.

Thank for the info

Thank for the info.

We use ext3 filesytem but I never give it much attention. I will look at it more closely and see if it answer my interrogation.

As for the database (Oracle) we do use rman to backup it up and we copy the backup to tape. We also have the flashback function activate on the database so we can return to any point in time in the last 2 days very quickly.

If we are looking at snapshot at the SAN level, it because the database is on one volume of the SAN and the report create by the program that access the database on another volume. With the snapshot of the two volumes at the same time we would be assure that we can go backup at the same point on both volume without having to delete report manually.

We are also looking to use the replication function for our disaster recovery plan. That why we need to be assured that when we do a snap or replication, that the data is consistent. If the ext3 journal system give us that assurance it will be great.

André

This is tricky -- get some expert advice.

Ext3 does not give you assurance that your data is consistent. It gives you assurance that your filesystem is consistent. In general if you are making backup snapshots of a database while it is running then the database itself must notified of this or it must be able to recover from an inconsistent volume. In either case you would loose data at least up to the time the snapshot started, but that's all you could hope for anyway.

I would not feel confident at all in attempting to create two separate snapshots of two different volumes with the requirement that they be in sync up to the same point snapshots were started.

vfs_cache_pressure

I should avoid dropping unflushed items from the cache! Seems a recipe for potential disaster.

Instead, one may try changing the default setting of 100 to something large like 10000 in the kernel parametre /proc/sys/vm/vfs_cache_pressure (adjust accordingly)
e.g echo 10000 > /proc/sys/vm/vfs_cache_pressure

Monitor the data in the cache in the column of sar : kbcached
e.g sar -r 1

Regards, Simon.