Relative to a specific opinion (guess!?) to the point about additional cores. To the point of some of the other answers, you would likely have to have easily compressible data and would just have to benchmark it with some test of reasonableness type experiments and find out. Given today's relatively high data write rates in the 10's of MBytes/second this is a pretty high hurdle to get over. It comes down to this:Ĭan you compress the raw data faster than the raw write performance of your disk times the compression ratio you are achieving (or the multiple in speed you are trying to get) given the CPU bandwidth you have available to dedicate to this purpose? This depends on lots of factors and I don't think there is one correct answer. Just hand off the log buffer to that core's thread and it can take as long as it likes to compress the data since it's not doing anything else! The only tricky bit is you want to actually have a ring of buffers so that you don't have the producer thread (the one making the log) waiting on a mutex for a buffer that the consumer thread (the one writing it to disk) is holding. If you have an idle second core it's even easier. We do this all the time when reading optical media. 24 cycles per byte = 3072 cycles per 128-byte cache line, easily achieved.
#Compress files on computer hdd free#
If you had an algorithm that could compress your data by 25% for an effective 125mb/s write speed, you would have 24 cycles per byte to run it in and it would basically be free because the CPU wouldn't be doing anything else anyway while waiting for the disk to churn. In this case 3000/100 = 30 cycles per byte. That's the number of cycles that the CPU is spending idle, that you can spend per byte on compression. Divide the CPU speed by the disk write speed.
Now take your CPU speed in megahertz let's say 3Ghz = 3000mhz. (Go ahead and measure it, time a huge fwrite or something.) Let's say 100mb/s. Look at it this way: take your maximum contiguous disk write speed in megabytes per second.
0 kommentar(er)
