We're implementing checksum providers into DDF. Our initial benchmark testing is intended to give a brief overview into the performance of each of the checksums tested.
Certain assumptions and decisions were made during this testing:
Test # | ~5MB | ~50MB | ~200MB |
---|---|---|---|
1 | .005 | .106 | .576 |
2 | .004 | .082 | .216 |
3 | .005 | .054 | .24 |
4 | .005 | .052 | .401 |
5 | .005 | .053 | .239 |
Test # | ~5MB | ~50MB | ~200MB |
---|---|---|---|
1 | .015 | .208 | 1.038 |
2 | 015 | .208 | .731 |
3 | .016 | .214 | .959 |
4 | .016 | .21 | .705 |
5 | .015 | .258 | .963 |
Algorithm | Min | Max | Avg |
---|---|---|---|
CRC32 | .004 | .005 | .0048 |
MD5 | .015 | .016 | .0154 |
Algorithm | Min | Max | Avg |
---|---|---|---|
CRC32 | .052 | .106 | .06974 |
MD5 | .208 | .258 | .2196 |
Algorithm | Min | Max | Avg |
---|---|---|---|
CRC32 | .216 | .576 | .3344 |
MD5 | .705 | 1.038 | .8792 |
Algorithm | ~5MB | ~50MB | ~200MB |
---|---|---|---|
CRC32 | .00096 | .001388 | .001672 |
MD5 | .00308 | .004392 | .004396 |
Algorithm | ~Avg MB/s |
---|---|
CRC32 | .00134 |
MD5 | .003956 |
Based on customer requirements, added Adler32 for performance (faster than CRC32) and SHA-256 for secure checksums.
https://github.com/codice/ddf/tree/2.27.x/libs/checksum/src/main/java/org/codice/ddf/checksum/impl