Posta Elettronica Certificata (Certified E-Mail)

The goal of this test is to assess the maximum load tolerated by OpenPec (not by the system on the whole) in therms of number of processed emails per time unit as a function of the processes number

In this first test session we chose to keed separated signing cases (ingoing mail from the client to delivery to the external world) from the verification cases ("waybill" emails to be delivered locally to the LMTP servers) in order to understand the incidence of each case.

Procedures for test execution
We installed the complete system (Postfix - OpenPec - Cyrus) on two servers, opec1 and opec2, then we used two additional servers to send emails. To ease the sending operations we configured Postfixs in order to relay also on the non secure SMTP.
We supplied every mailer (we used mailer.pl, included in the distribution) with 18 various 22525Kb messages with modified header (From and To fields) in order to be conformant to the envelop.
For every test we tested the number of processes for both Openpec and Postfix (transport).

Network architecture:

Software architecture on every server:

Description

1. Both servers must have Postfix up with empty queues and OpenPec down.
2. Send emails to opec2.
3. Start OpenPec on opec2, then execute the Postfix messages queue flush (postfix -f).
4. Observe every parameter (cpu, execution time, etc.).
5. Execute the point number 3 for opec1.
6. Observe opec1 parameters.

Results

Conclusions
Opec1 has execution time much greater than Opec2. This confirms that verification operation is more expensive than signing, almost an order of magnitude (even if we have to note that opec1 hard disk is worse than opec2 one).
However, the most important fact is that the more efficient configuration is with only one process per server, on monoprocessor servers.

Remarks

• How much does the signing weigh on the average elaboration time per email?
We made test again, after modifying the code in order to make multiple calls (not only one) to openssl, and we obtained:
the only signing test lasted 39'' (instead of 46''); the verification test lasted 1' 42'' (instead of 4' 46'').
• How important is the hard drive?
  We made the test againg, exchanging the servers (since opec2 hard drive is more performant) and the result is a difference of roughly a quarter of the time:
  the signing test lasted 32'' (instead of 46'').
  This means that the hard drive choice is fundamental for the perfomances enhancement.

The goal is to esteem the maximum load tolerated by a complete model of a PEC system using OpenPec, in therms of number of processed emails per time unit.

Procedures for test execution
We installed the complete system (Postfix - OpenPec - Cyrus) on two servers, opec1 and opec2, and we used a third server to send emails. To ease the sending operations we configured Postfixs in order to relay also on the non secure SMTP.
The mass sending of emails has been done by two instances of the simply emailer shipped with the distribution (mailer.pl), one for each server. A set of various emails (with From, To and Cc fields modified to be conformant to the envelop) has been delivered to each instance. In particular:

• opec1
  domain: domaincert1.openpec.org
  local users:
  user1@domaincert1.openpec.org
  user2@domaincert1.openpec.org
  
  
• opec2
  domain: domaincert2.openpec.org
  local users:
  user1@domaincert2.openpec.org
  user2@domaincert2.openpec.org
  
  
• third server
  
  • mailer 1:
    794 messages - 44.216KB total
    cmd: mailer.pl user1@domaincert1.openpec.org user2@domaincert2.openpec.org path_mail_1 1 opec1
  • mailer 2:
    1226 messages - 96.968KB total
    cmd: mailer.pl user1@domaincert2.openpec.org user2@domaincert1.openpec.org path_mail_2 1 opec2

Results

Processed messages group by kind of elaboration

• opec1 - total time - 161' 34"
  • 96.968KB of verified messages (waybills from Opec2)
  • 193.936KB of signed messages (acknoledgement of token charge and acknoledgement of receipt to Opec2)
  • 44.226KB of signed messages (acceptance to Opec1)


• opec2 - total time - 151' 03"
  • 44.226KB of verified messages (waybills from Opec1)
  • 88.452KB of signed messages (acknoledgement of token charge and acknoledgement of receipt to Opec1)
  • 96.968KB of signed messages (acceptance to Opec2)

Total processed/generated messages

• opec1
  • 794 - 44.226KB acknoledgements of receipt
  • 794 - 44.226KB waybills generated
  • 1226 - 96.968KB waybills received
  • 1226 - 96.968KB acknoledgement of token charge
  • 1226 - 96.968KB acknoledgement of receipt
  • total processed: 5266 - 379.336KB


• opec2 - total time - 151' 03"
  • 1226 - 96.968KB acceptance receipts
  • 1226 - 96.968KB waybills generated
  • 794 - 44.226KB waybills received
  • 794 - 44.226KB acknoledgement of token charge
  • 794 - 44.226KB acknoledgement of receipt
  • total processed: 4834 - 326.604KB

The goal is to put the system through a consistent load and analyse users' test mailboxes in order to verify the links between every different email typologies.
An email from PEC1 server to PEC2 server means:
a waybill
an acceptance receipt
an acknoledgement of token charge
an acknoledgement of receipt

Procedures for test execution
We installed the complete system (Postfix - OpenPec - Cyrus) on two servers, opec1 and opec2, and we used a third server to send emails and download mailboxes via pop3 from both servers. To ease the sending operations we configured Postfixs in order to relay also on the non secure SMTP.
The mass sending of emails has been done by two instances of the simply emailer shipped with the distribution (mailer.pl) with a dead time between each sending going from one to 15 sec, one for each server. A set of various emails (with From, To and Cc fields modified to be conformant to the envelop) has been delivered to each instance. In particular:

• opec1
  domain: domaincert1.openpec.org
  local users:
  user1@domaincert1.openpec.org
  user2@domaincert1.openpec.org
  
  
• opec2
  domain: domaincert2.openpec.org
  local users:
  user1@domaincert2.openpec.org
  user2@domaincert2.openpec.org
  
  
• third server
  
  • mailer 1:
    794 messages - 44.216KB total
    cmd: mailer.pl user1@domaincert1.openpec.org user2@domaincert2.openpec.org path_mail_1 1 opec1
  • mailer 2:
    1226 messages - 96.968KB total
    cmd: mailer.pl user1@domaincert2.openpec.org user2@domaincert1.openpec.org path_mail_2 1 opec2

Results

Users received messages group by kind

• opec1
  794 acknoledgements of acceptance
  794 acknoledgements of token charge
  794 acknoledgements of receipt
  1225 waybills
  


• opec2
  1225 acknoledgements of acceptance
  1225 acknoledgements of token charge
  1225 acknoledgements of receipt
  794 waybills
  1 Undeliverable Mail
  

* The "Undeliverable Mail" message was caused by a not conformant header, which has been managed according to the specifications.

** The test was done using openssl version 0.9.7d.

*** Note that both OpenPec servers were configured with the maximum log level and they both produced a 250Mb log file.

Remarks
The test had previoulsy been done using openssl 0.9.7a and it had shown some problems (not particularly weighty): an incostant number (about 20) of transport anomalies were generated, on a sample of 1226 emails. This was caused by the failure of the waybill verification, even if the waybill was correctly signed. We analysed the matter deeper and we made sure that the failure was wrong: the error code number 11 was returned, without any code or message to STDERR or STDOUT; several tests demonstrated the problem randomness.
The problem isn't (weighty), becase the sender didn't receive any acknoledgement of receipt, so in the PEC context, he knew that the email hadn't been delivered.
The problem was solved updating to openssl version 0.9.7d

This test is the "load test -session 1" copy, but done on a multiprocessor server, in order to quantify the advantage with a more expensive solution and eventually show hypothetical code improvements.

Procedures for test execution
We installed the complete system (Postfix - OpenPec - Cyrus) on two servers, opec2 and opec3, then we used two additional servers to send emails. To ease the sending operations we configured Postfixs in order to relay also on the non secure SMTP.
We supplied every mailer (we used mailer.pl, included in the distribution) with 18 various 22525Kb messages with modified header (From and To fields) in order to be conformant to the envelop.
For every test we modified the number of processes for both Openpec and Postfix (transport).

Server:

opec3

model name : 2 CPU Pentium III (Katmai) cpu MHz : 499.192 cache size : 512 KB RAM Memory : 384 MB Hard Disk SCSI HP 9.10GB A 80-SA40

prestazioni (fonte: hdparm)

Timing buffer-cache reads: 512 MB in 2.00 seconds = 255.62 MB/sec Timing buffered disk reads: 78 MB in 3.09 seconds = 25.24 MB/sec