We're implementing checksum providers into DDF. Our initial benchmark testing is intended to give a brief overview into the performance of each of the checksums tested.
Assumptions and Decisions
Certain assumptions and decisions were made during this testing:
- The file sizes tested (~5mb,~50mb,~200mb) are sufficient for giving us an idea of performance for each algorithm tested.
- 5 tests for each size is a sufficient sample size for benchmarking.
CRC32 Benchmark Results (in seconds)
MD5 Benchmark Results (in seconds)
~5MB Comparison (in seconds)
~50MB Comparison (in seconds)
~200MB Comparison (in seconds)
~ Average MB/S Comparison
- There seems to be a skew in the data for > ~50MB and the first run taking significantly longer than other test runs.
- CRC32 has the lowest average approximate MB/s.
- MD5 seems to be outperformed in every metric in each of its size categories.
Based on customer requirements, added Adler32 for performance (faster than CRC32) and SHA-256 for secure checksums.