ADDeX — Advanced Data Deduplication Engine

[Razor12911]

Some benchmarks, tests performed on ramdisk.

input: Bright Memory Infinite (zlib precompressed), 20,6 GB (22 175 074 282 bytes)

Srep

srep (encoding -m3f): Clock 51.695s (CPU 58.467s), Peak RAM usage 1569 MB, Output 15,0 GB (16 112 713 354 bytes)
srep (decoding): Clock 10.785s (CPU 10.468s), Peak RAM usage 1007 MB
lzma compression: Final output 7,26 GB (7 796 298 754 bytes)

Addex

addex (encoding -m1): Clock 85.233s (CPU 92.202s), Peak RAM usage 1797 MB, Output 15,0 GB (16 112 671 046 bytes)
addex (decoding): Clock 9.790s (CPU 9.561s), Peak RAM usage 989 MB
lzma2 compression: Final output 7,25 GB (7 796 022 132 bytes)

procprofile srep

encoding

Process ID       : 26296
Thread ID        : 19848
Process Exit Code: 0
Thread Exit Code : 0

User Time        :          52.421s
Kernel Time      :           6.046s
Process Time     :          58.467s
Clock Time       :          51.695s

Working Set      :         1606240 KB
Paged Pool       :             150 KB
Nonpaged Pool    :              11 KB
Pagefile         :         1599976 KB
Page Fault Count : 432691

IO Read          :        43310706 KB (in            5303 reads )
IO Write         :        15735101 KB (in           38937 writes)
IO Other         :              24 KB (in           31358 others)

decoding 

Process ID       : 11320
Thread ID        : 5852
Process Exit Code: 0
Thread Exit Code : 0

User Time        :           4.843s
Kernel Time      :           5.625s
Process Time     :          10.468s
Clock Time       :          10.785s

Working Set      :         1031588 KB
Paged Pool       :             149 KB
Nonpaged Pool    :             136 KB
Pagefile         :         1034640 KB
Page Fault Count : 259090

IO Read          :        15735170 KB (in            8090 reads )
IO Write         :        21655353 KB (in            9877 writes)
IO Other         :              21 KB (in            7414 others)

procprofile addex

encoding

Process ID       : 24816
Thread ID        : 20988
Process Exit Code: 0
Thread Exit Code : 0

User Time        :          86.859s
Kernel Time      :           5.343s
Process Time     :          92.202s
Clock Time       :          85.233s

Working Set      :         1840540 KB
Paged Pool       :             111 KB
Nonpaged Pool    :              12 KB
Pagefile         :         1846388 KB
Page Fault Count : 502279

IO Read          :        43310691 KB (in            5289 reads )
IO Write         :        15735033 KB (in           15455 writes)
IO Other         :              24 KB (in             156 others)

decoding

Process ID       : 23824
Thread ID        : 3120
Process Exit Code: 0
Thread Exit Code : 0

User Time        :           3.921s
Kernel Time      :           5.640s
Process Time     :           9.561s
Clock Time       :           9.790s

Working Set      :         1011880 KB
Paged Pool       :             110 KB
Nonpaged Pool    :              98 KB
Pagefile         :         1010832 KB
Page Fault Count : 1503488

IO Read          :        15735074 KB (in            7579 reads )
IO Write         :        21655346 KB (in           21158 writes)
IO Other         :               0 KB (in              49 others)