[bip] IO tests

[Paul Davis]

fastacmd -I output:

Database: fetcher
           31,077 sequences; 9,423,040 total letters

Original fasta sequence file size: 13 M


8 Processes:

Ran this from a script that put 8 blastall instances into the
background and then waited for each to finish. Used time to give the
command, processor utilization (approximation) and ellapsed wall clock
time. Ran the script with time to get overall time and processor
utilization.

Notice the discrepancy, 8 * 99% != 692%. Not sure why that is exactly.
The script does absolutely nothing after it waits for its background
jobs.

Each fasta file is approximately the same number of sequences. All 8
are 1.4-1.7 MiB

blastall -p blastp -d fetcher -e 1e-5 -m 8 -i fp_6.fasta -o fp_6.blast
99% 549.40
blastall -p blastp -d fetcher -e 1e-5 -m 8 -i fp_3.fasta -o fp_3.blast
99% 572.45
blastall -p blastp -d fetcher -e 1e-5 -m 8 -i fp_5.fasta -o fp_5.blast
99% 575.52
blastall -p blastp -d fetcher -e 1e-5 -m 8 -i fp_7.fasta -o fp_7.blast
99% 579.57
blastall -p blastp -d fetcher -e 1e-5 -m 8 -i fp_0.fasta -o fp_0.blast
99% 590.05
blastall -p blastp -d fetcher -e 1e-5 -m 8 -i fp_4.fasta -o fp_4.blast
99% 597.35
blastall -p blastp -d fetcher -e 1e-5 -m 8 -i fp_2.fasta -o fp_2.blast
99% 633.53
blastall -p blastp -d fetcher -e 1e-5 -m 8 -i fp_1.fasta -o fp_1.blast
99% 691.82
4782.93user 6.48system 11:31.82elapsed 692%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+3983719minor)pagefaults 0swaps

8 Threads:

/usr/bin/time blastall -p blastp -d fetcher -e 1e-5 -m 8 -i
fetcher.fasta -o fetcher.blast -a 8
4794.22user 8.47system 19:33.96elapsed 409%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+4064461minor)pagefaults 0swaps

16 Threads:

/usr/bin/time blastall -p blastp -d fetcher -e 1e-5 -m 8 -i
fetcher.fasta -o fetcher.blast -a 16
4783.51user 8.31system 19:40.08elapsed 406%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+4101291minor)pagefaults 0swaps

For some reason, with this small of a database, processor utilization
never gets to 99% when using threads.

Ran the 8 process and 8 thread test on a tmpfs ramdisk:

8 processes:

blastall -p blastp -d fetcher -e 1e-5 -m 8 -i fp_6.fasta -o fp_6.blast
99% 549.39
blastall -p blastp -d fetcher -e 1e-5 -m 8 -i fp_3.fasta -o fp_3.blast
99% 572.67
blastall -p blastp -d fetcher -e 1e-5 -m 8 -i fp_5.fasta -o fp_5.blast
99% 576.13
blastall -p blastp -d fetcher -e 1e-5 -m 8 -i fp_7.fasta -o fp_7.blast
99% 579.53
blastall -p blastp -d fetcher -e 1e-5 -m 8 -i fp_0.fasta -o fp_0.blast
99% 590.65
blastall -p blastp -d fetcher -e 1e-5 -m 8 -i fp_4.fasta -o fp_4.blast
99% 597.14
blastall -p blastp -d fetcher -e 1e-5 -m 8 -i fp_2.fasta -o fp_2.blast
99% 634.84
blastall -p blastp -d fetcher -e 1e-5 -m 8 -i fp_1.fasta -o fp_1.blast
99% 692.08
4785.76user 6.60system 11:32.09elapsed 692%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+3983695minor)pagefaults 0swaps

8 threads:

/usr/bin/time blastall -p blastp -d fetcher -e 1E-5 -m 8 -a 8 -i
fetcher.fasta -o fetcher.blast
4795.48user 8.82system 19:33.90elapsed 409%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+4066324minor)pagefaults 0swaps

####

Obviously this isn't IO bound. Times and stats are similar.

Perhaps later when I really care I'll do the tests against NR.

Paul