BLAG

BLAG Forums
It is currently Tue Dec 23, 2014 12:09 am

All times are UTC




Post new topic Reply to topic  [ 1 post ] 
Author Message
PostPosted: Fri Jul 30, 2004 11:11 pm 
Offline
Site Admin

Joined: Sun Mar 14, 2004 3:17 pm
Posts: 4492
Location: Loveland, Colorado, USA
This was posted on the audacity-devel mailing list about how to speed up audacity with huge files. It is true outside of audacity as well. Understand what you're doing before jacking around with your filesystem, of course... Here's the email from monty:

==================begin==================
BTW, to folks running Linux: There are several things you can do to
tune your filesystem to give *much* better performance in Audacity
with medium and large projects:

1) Turn on the 'dir_index' option to ext2fs and ext3fs, using tune2fs
and -O dir_index. This causes ext2fs and ext3fs to use btrees instead
of linear search for metadata in any new directories, drastically
reducing CPU overhead accessing files in large directories.
Drawbacks: This option is unsupported in older kernels, and as such,
some rescue disks will not be able to mount a filesystem using this
option. For example, the stock Debian/Woody install/rescue can't even
mount an FS using dir_index, but the bf2.4 rescue/install disks can.
Other drawbacks: None; this is safe and recommended on an existing
filesystem, and will improve all access times on *new* directories
made since changing the setting. It's not more widespread only
because the feature is new.

2) If using ext3, set the journal to 'data_ordered' mode, using
tune2fs and -o journal_data_ordered on an *unmounted* filesystem.
This tells the journal to keep the filesystem (metadata) consistent at
all times, but be slightly less paranoid about flushing new file data
out to the disk; in a crash, an open, actively-modified file will be
in-tact, but the end of the file is not guaranteed to be exactly
correct up to the moment of the crash (unlike full journalling mode,
where the end of the file is truncated at exactly the last write to
finish). The result is a filesystem more than 4x faster when
writing.

3) If Audacity is a primary use of your workstation, consider using a
filesystem like XFS, which has about 60-100% more write throughput
than even a well-tuned ext3fs. This is not a knock against ext3fs
which is phenomenally simple and stable, but when it comes to raw
numbers, XFS is just about always faster by a large margin, mostly in
writes. Again, the XFS filesystem is new to Linux (Available in stock
2.4.26 and 2.6.somethingorother), but there are no other serious
drawbacks to using it aside from potential rescue floppy/ISO
compatability. I also find XFS gets along a bit better with hardware
and software RAID although, again, a well tuned ext3fs is much better
on RAID than a default-settings ext3fs. XFS simply deals
automatically where the ext3fs must be hand-tuned for a RAID.

4) The Linux buffercache is tuned to be interactive rather than to
have high continuous disk bandwidth; most notably it tries to buffer
far too much disk output in-cache and when it runs out of machine
memory to do so, it can cause the machine to go *completely
unresponsive* for minutes to hours (depending on the amount of RAM
being used to cache) while catching up. Chances are it won't even
ping; it's not crashed, it's simply way behind on disk writes and has
punched the panic button to catch up. The problem is worst in 2.6
exactly because the buffercache is so much faster (but the disk isn't
;-) Mark Eichin of Metacarta refers to this as the 'wings stay on /
wings fall off' switch.

The 'fix' is to tell the buffercache that getting behind on writes is
unacceptable; your writes will feel like they're slower, but in fact
you're eliminating all those occasions where the machine freezes while
trying to catch up during big I/O. Actual disk throughput will be
higher.

Eichin recommends a line like:

echo "0 5000 0 0 100 3000 0 1 0" > /proc/sys/vm/bdflush

This directs bdflush to always maximize bandwidth-to-disk during
writes. Note that if you use hardware RAID or a caching controller
(like a 3ware Escalade, which is what I have), your vendor may have
their own recommended /proc/sys/vm/bdflush settings; if they do, it's
probably best to use theirs instead.

Similarly, you want the buffercache to be *more* aggressive when
reading. Something like:

echo "512" > /proc/sys/vm/max-readahead
echo "512" > /proc/sys/vm/min-readahead

...will greatly improve sequential read throughput as well as mildly
improving writes (which usually involve some reading as well).
Audacity does alot of sequential reading and writing. This setting is
perfectly safe for other software, you're simply telling the kernel to
trade off an amount random disk access performance for greater linear
throughput.

BTW, changes made using 'tune2fs' are persistent. The /proc/sys/vm
settings are not; they will need to be set after each reboot. To make
sure this happens aty boot time, put the lines into
/etc/init.d/bootmisc.sh (for Debian, or the equivalent init file on
your distribution).

Monty
===================end===========

Cool :)


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 1 post ] 

All times are UTC


Who is online

Users browsing this forum: No registered users and 0 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group