[interchange] Increase size of socket read for much better performance

Jon Jensen interchange-cvs at icdevgroup.org
Wed Jan 25 22:29:03 UTC 2012


commit f0479d191e4dffcfaec781d0e953f66d7c6f96c9
Author: Jon Jensen <jon at endpoint.com>
Date:   Wed Jan 25 15:20:36 2012 -0700

    Increase size of socket read for much better performance
    
    ddavenport found a serious performance regression in Vend::Server, such
    that each socket read was only for 1 byte, making large uploads
    painfully slow:
    
    https://github.com/interchange/interchange/pull/2
    
    I found the regression was introduced in this commit:
    
    commit 2a69743c5023ebb1198c0f2e7f7dcdce4935b5eb
    Author: Kevin Walsh <kevin at cursor.biz>
    Date:   Mon Feb 4 21:42:18 2008 +0000
    
            * Changes in the _read() subroutine:
    
            -- select() returns -1 upon error, whereas sysread() returns undef,
               so we need to allow for both.
    
            -- Catch EAGAIN as well as EINTR as soft errors to retry on.
    
            -- Read the entire available amount of data in one hit instead of
               forcing the data to be read in 512-byte chunks.
    
    Kevin clearly intended to read as much data as was waiting, but select()
    returns the number of waiting filehandles, not # of waiting bytes.
    
    Rather than reverting to the original 512 bytes per read, or
    ddavenport's suggested 1024, I used 1 MiB which is still arbitrary but
    large enough to get most normal http requests in few passes, and handles
    large uploads efficiently.
    
    Thanks, ddavenport!
    
    Original patch:
    
    commit 2a6308dac12c3d8fafcfad02cd41a32f6adfefe0
    Author: ddavenport <dances_with_peons at live.com>
    Date:   Fri Jul 15 20:50:54 2011 -0700
    
        Fixed Vend::Server so it'll read more than a single byte at a time
    (which was causing large file uploads to fail).
    
    diff --git a/lib/Vend/Server.pm b/lib/Vend/Server.pm
    index 5baf109..4a82ce8 100644
    --- a/lib/Vend/Server.pm
    +++ b/lib/Vend/Server.pm
    @@ -795,7 +795,7 @@ sub _read {
    
         do {
        if (($r = select($rin, undef, undef, $Global::SocketReadTimeout || 1)) > 0) {
    -       $r = sysread($fh, $$in, $r, length($$in));
    +       $r = sysread($fh, $$in, 1024, length($$in));
        }
         } while ((!defined($r) || $r == -1) && ($!{eintr} || $!{eagain}));

 lib/Vend/Server.pm |    8 +++++---
 1 files changed, 5 insertions(+), 3 deletions(-)
---
diff --git a/lib/Vend/Server.pm b/lib/Vend/Server.pm
index 884e724..99132dc 100644
--- a/lib/Vend/Server.pm
+++ b/lib/Vend/Server.pm
@@ -794,9 +794,11 @@ sub _read {
     vec($rin,fileno($fh),1) = 1;
 
     do {
-	if (($r = select($rin, undef, undef, $Global::SocketReadTimeout || 1)) > 0) {
-	    $r = sysread($fh, $$in, $r, length($$in));
-	}
+        if (($r = select($rin, undef, undef, $Global::SocketReadTimeout || 1)) > 0) {
+            # read up to an arbitrary 1 MiB at a time for efficiency
+            # (though the operating system may provide far less than that at a time anyway)
+            $r = sysread($fh, $$in, 1_048_576, length($$in));
+        }
     } while ((!defined($r) || $r == -1) && ($!{eintr} || $!{eagain}));
 
     die "read: $!" unless defined $r;



More information about the interchange-cvs mailing list