I recently made some changes to the Entangle archive that required a good majority of the archive to be checksum reverified. That’s about 200 TB worth of checksums. Even with six systems running parallel, and RAIDs hitting 600 MB/s, it still takes a while.
One thing I noticed was that the I/O rate while the sums were being performed was ~210 MB/s. A respectable speed, but cp’ing a file from the RAID to to /dev/null results in ~600 MB/s. So either my CPU was too slow or the ‘sum’ operation wasn’t as efficient as it could be. Continue reading “Speeding Up Slow Sums – Faster BSD Checksums”
