[ic] more on scalability and st=text versus db, academic discussion

Mike Heins interchange-users@icdevgroup.org
Wed Oct 30 23:39:00 2002


Quoting Jeff Dafoe (jeff@badtz-maru.com):
> 
>     I was thinking more about interchange's incredible scalability due to
> the fact that it can do product scans in the text files instead of the SQL
> database.  Postgresql and mysql are essentially non-scalable past a single
> machine (neither currently support synchronous multimaster replication)
> without having to hack whatever software is using the dbms (IC, in our
> case).  They also both fork a backend process for each connection.  This can
> be a serious limitation for scalability, you can deploy multiple web servers
> but if they all have to connect to the same database server that server can
> rapidly become the bottleneck after about 400 concurrent connections.
>     In comes Interchange to save the day.  You can do product scans (which
> probably account for the majority of the database traffic on a typical
> ecommerce website) against the text files and other non-SQL-based indexes.
> These files can be easily replicated across machines.  By not relying on the
> SQL server for product scans Interchange solves, in advance, issues that can
> cause serious growing pains for ecommerce sites.
> 

Just how I planned it 7 years ago...oh, I forgot I don't lie anymore. Darn.

Actually, the fairly complete IC search mechanism is a byproduct of the
fact that when it was written, there really was no good free SQL database.
The best that could be found was mSQL, and it had many problems. DBI was
not really born, and Mysql had never been heard of. Postgres was so slow
that it would take minutes to import 100 records on a 166MHz Pentium.

So, I had little choice and blundered my way into one of the
little-mentioned strengths of IC, its search/loop mechanisms. Thanks for
mentioning it. 8-)

-- 
Mike Heins
Perusion -- Expert Interchange Consulting    http://www.perusion.com/
phone +1.513.523.7621      <mike@perusion.com>

How far can you open your mind before your brains fall out?