[ic] Performance hit from having many images in one directory?

[Mike Heins]

Quoting Jeff Fearn (jefffearn at gmail.com):
> On 8/23/05, John1 <list_subscriber at yahoo.co.uk> wrote:
> > At the moment we have all our item images in a single directory (and all the
> > thumbnails in one directory).  There are currently only about 1000 items,
> > but this could increase significantly as time goes on (especially as we tend
> > to leave any images of old discontinued products there just in case search
> > engines etc still have them catalogued).
> > 
> > We run the Linux ext3 filing system.  Is there a performance hit from
> > storing loads of files in a single directory?  I am sure I read somewhere
> > once that there *is* a performance hit, but I have no idea how many image
> > files we would need to store in one directory before the performance
> > implications became significant, bearing in mind that each page loads
> > between 10 and 20 images from the thumbnails directory and 1 image from the
> > main items directory?
> > 
> > Please can anyone enlighten me on this?  Thanks.
> 
> Good discussion at http://answers.google.com/answers/threadview?id=122241

There are two major options once you hit 10,000 files or so:

    1. Switch filesystems to XFS or ReiserFS for their tree-based
       scheme. They don't degrade like ext2.

    2. Use a hashing scheme of some type to store images
       in subdirectories. It can be as simple as manually
       by category, or something automated based on say,
       the first two letters of the image name. We do
       this for session and temporary files, and we have
       some routines which support finding the file name
       automatically.

-- 
Mike Heins
Perusion -- Expert Interchange Consulting    http://www.perusion.com/
phone +1.765.647.1295  tollfree 800-949-1889 <mike at perusion.com>

For a successful technology, reality must take precedence over public
relations, for Nature cannot be fooled. -- Dick Feynman