[ic] Performance hit from having many images in one directory?
mike at perusion.com
Tue Aug 23 08:39:19 EDT 2005
Quoting Jeff Fearn (jefffearn at gmail.com):
> On 8/23/05, John1 <list_subscriber at yahoo.co.uk> wrote:
> > At the moment we have all our item images in a single directory (and all the
> > thumbnails in one directory). There are currently only about 1000 items,
> > but this could increase significantly as time goes on (especially as we tend
> > to leave any images of old discontinued products there just in case search
> > engines etc still have them catalogued).
> > We run the Linux ext3 filing system. Is there a performance hit from
> > storing loads of files in a single directory? I am sure I read somewhere
> > once that there *is* a performance hit, but I have no idea how many image
> > files we would need to store in one directory before the performance
> > implications became significant, bearing in mind that each page loads
> > between 10 and 20 images from the thumbnails directory and 1 image from the
> > main items directory?
> > Please can anyone enlighten me on this? Thanks.
> Good discussion at http://answers.google.com/answers/threadview?id=122241
There are two major options once you hit 10,000 files or so:
1. Switch filesystems to XFS or ReiserFS for their tree-based
scheme. They don't degrade like ext2.
2. Use a hashing scheme of some type to store images
in subdirectories. It can be as simple as manually
by category, or something automated based on say,
the first two letters of the image name. We do
this for session and temporary files, and we have
some routines which support finding the file name
Perusion -- Expert Interchange Consulting http://www.perusion.com/
phone +1.765.647.1295 tollfree 800-949-1889 <mike at perusion.com>
For a successful technology, reality must take precedence over public
relations, for Nature cannot be fooled. -- Dick Feynman
More information about the interchange-users