[ic] [PATCH] [RFC] PGP hard failure workaround

[Dan Browning]

I have been experiencing the "PGP hard failure" error:

	PGP failed with error level 12, status 12:
	PGP hard failure, command that failed: [...]

The symptom is that on a busy system, the encryption operation will 
occasionally (1 in 100 or so) fail, resulting in no credit card at all.

Has anyone else experienced this error?  If not, could you try the steps
to reproduce the problem (below)?  If you have, could you try the
patch below and let me know if it helps?

 * Software: 
   - Interchange 4.9 CVS, 05/13 and 04/04 both affected.
   - Interchange RPC and HIGH mode both affected.
   - Red Hat 7.3 and 9.0 both affected
   - GPG 1.2.1
   
 * Hardware: 
   - Athlon 2700 CPU
   - Ultra160 Cheetah HDD
   - 1GB RAM
   
 * Steps to reproduce the problem:
   - Create the following test page:
   
	<xmp>
	[calc]
		for (1..20) {
			my $results = $Tag->filter( 'encrypt', 'bob' );
			$out .= "Try $_: ";
			if ( $results !~ /^-----BEGIN PGP MESSAGE-----/ ) {
				$out .= "failed: results = $results\n";
			}
			else {
				$out .= "succeeded.\n";
			}
		}
		return $out;
	[/calc]
	</xmp>
	
    - Access that page many times concurrently.  I was able to cause the 
      problem to occur with apache bench (concurrency level 3):
	
      ab -n 5 -c 3 http://path/to/test
      
I wrote a patch to workaround the problem, by retrying up to 3 times 
(configurable).  You can get the patch to lib/Vend/Order.pm below or 
download it here:

	http://www.icdevgroup.org/~danb/pgp_hard_failure_workaround.patch

Feedback appreciated.

-- 
Dan Browning, Kavod Technologies, <db@kavod.com> 360.843.4074x217
6700 NE 162nd Ave, Ste 611-210, Vancouver, WA.    Random Fortune:
Deliberation, n.:
	The act of examining one's bread to determine which side it is
	buttered on.
		-- Ambrose Bierce, "The Devil's Dictionary"


--- Order.pm	2003-04-23 09:01:02.000000000 -0700
+++ /home/db/bin/ic49/lib/Vend/Order.pm	2003-06-02 18:35:49.000000000 -0700
@@ -911,24 +911,38 @@
 	my $fpre = $Vend::Cfg->{ScratchDir} . "/pgp.$Vend::Session->{id}.$$";
 	$cmd .= " >$fpre.out";
 	$cmd .= " 2>$fpre.err" unless $cmd =~ /2>/;
-	open(PGP, "|$cmd")
-			or die "Couldn't fork: $!";
-	print PGP $body;
-	close PGP;
-
-	if($?) {
-		my $errno = $?;
-		my $status = $errno;
-		if($status > 255) {
-			$status = $status >> 8;
-			$! = $status;
-		}
-		logError("PGP failed with error level %s, status %s: $!", $?, $status);
-		if($status) {
-			logError("PGP hard failure, command that failed: %s", $cmd);
-			return;
+	
+	my $tried;
+	my $success;
+	my $pgp_retries = ( $Vend::Cfg->{Limit}{pgp_retries} || 3 );
+	PGP: until ( $success or $tried > $pgp_retries ) {
+		$tried++;
+		open(PGP, "|$cmd")
+				or die "Couldn't fork: $!";
+		print PGP $body;
+		close PGP;
+	
+		if($?) {
+			my $errno = $?;
+			my $status = $errno;
+			if($status > 255) {
+				$status = $status >> 8;
+				$! = $status;
+			}
+			logError("PGP failed with error level %s, status %s: $!", $?, $status);
+			if($status) {
+				logError("PGP hard failure, command that failed: %s.  " .
+						 "Try number %s of %s.", $cmd, $tried, $pgp_retries - $tried );
+	 			next PGP;
+			}
 		}
+		$success = 1;
+		last;
 	}
+	
+	logError("PGP " . ( $success ? "succeeded " : "failed " ) . "after trying %s time(s).", $tried) 
+			if $tried > 1;
+		
 	$body = readfile("$fpre.out");
 	unlink "$fpre.out";
 	unlink "$fpre.err";