sshfs is a very slick way to access your remote filesystems, securely and transparently. It is based on the great FUSE (Filesystem in Userspace) framework, which has been integrated in the linux kernel since version 2.6.14. While sshfs may not be as fast and featureful as other full-blown network filesystems such as NFS or Samba, it still has some great features:
- very easy to use, on the server side there's nothing to do, on the client side mounting the filesystem is as easy as logging into the server with ssh
- provides secure (encrypted) access to remote files
- has decent performance (multithreaded, caching directory contents and allowing large reads)
- should work well even over slow and/or unstable links (think dialup), knows how to reconnect to the server when the connection is broken
PostgreSQL is a powerful, open source relational database system. It has more than 15 years of active development and a proven architecture that has earned it a strong reputation for reliability, data integrity, and correctness.
One of the PostgreSQL's most sophisticated features is so called Multi-Version Concurrency Control (MVCC), a standard technique for avoiding conflicts between reads and writes of the same object in database. MVCC guarantees that each transaction sees a consistent view of the database by reading non-current data for objects modified by concurrent transactions. Thanks to MVCC, PostgreSQL has great scalability, a robust hot backup tool and many other nice features comparable to the most advanced commercial databases.
The /proc filesystem acts as an interface to internal data structures in the kernel. It can be used to obtain information about the system and to change certain kernel parameters at runtime (sysctl).
The effort of the /proc filesystem is to provide an easy way to view kernel and information about currently running processes. As a result, some commands (ps for example) read /proc directly to get information about the state of the system. The premise behind /proc is to provide such information in a readable manner instead of having to invoke difficult to understand system calls.
One of the more interesting patches for the linux kernel lately has been Wu Fengguang's adaptive readahead patchset, currently at version 12. Talking about its performance benefits Wu says: "besides file servers and desktops, it is recently found to benefit postgresql databases a lot.".
So I decided to do a simple benchmark to see what difference would adaptive readahead make in my case. The idea was to test a very simple database query (random select) to the PostgreSQL database and see how it performs through time (while the memory is being primed with data from disk).
If you are lucky to have fresh X11R7 on your desktop with all its new features and nice filesystem layout you might have noticed that some things have compatibility problems with it. Namely, if you have flash plugin installed you might not see text in flash content displayed properly, depending on how your Linux distribution handled the upgrade.
Having already had trouble with that (for other reasons I'll explain later) this time fix for the problem was a no brainer. The reason for problems is that the official flash plugin does some font handling of it's own, not using system font file server or X server itself. But it still depends on some filesystem paths and configuration to be in place so it can find font files it needs. By doing
strings libflashplayer.so and carefully skipping over lots of uninteresting text, you can find that flash plugin looks for
/usr/X11R6/lib/X11/fs/config file which is the configuration file of the font server. Next it parses
"catalogue = " line in it to find all available fonts on the system. The trouble is that new release of X.Org server got rid of
/usr/X11R6 system path in favour of putting binaries and other files in the more appropriate places in the filesystem, just like the other linux applications do.
Is running Oracle10g on Debian Linux possible? Oh yes, definitely! And it runs great, really. It's even easier to install than the older versions of Oracle as there are no problems with incompatible libc library & other bugs. You need to make just two simple preparations before you can enjoy your new development database.
Important note: I tested this only on the Debian unstable distribution and only with the 2.6 kernel, as that's what I'm running. I believe that most of you that are running Debian unstable are also running the newest stable kernel, so that shouldn't be a problem, right? Let's go step by step...
Both runs have been done on a single 120GB 7200rpm IDE disk. Command used: iostat -x 5 (extended statistics, sample every 5 seconds).
iostat is a command line I/O performance monitoring utility. It is present in almost every major Unix flavor in use today, and here you can get the version for Linux.
It works on both 2.4 & 2.6. What makes it different from other Linux utilities, which mostly show only I/O transfer rates, is that you finally can get important information about disk utilization, number of requests, average queue size and disk & queue wait times. No serious sysadmin should be without it. Check the screenshots to learn more.
All screenshots were taken on my home workstation (Dual PIII 1GHz, 768MB RAM). Brown color represents kernel memory usage (miscellaneous caches), red is for active memory (page/buffer cache in use and mapped pages - applications), yellow is inactive page/buffer cache, green is free memory and finally orange color represents swap usage.
On the first picture you can see what happens when cron runs updatedb process, refreshing file name database (used by locate command). Rather quickly memory gets consumed by various kernel caches as updatedb traverses filesystems. Those are mostly inode & dentry caches, used by the kernel to find and access a file. Obviously, I have lots of files as more than half of my memory gets used by the caches. If you had enough memory so that you can cache and keep all this valuable information in memory, second updatedb run would finish almost instantly, instead of few minutes of disk crunching.