|
HP-UX - Block device versus raw device performance |
|
|
HP-UX - Block device versus raw device performance
From time to time, someone will do a dd(1) to a block device and compare the results to a dd(1) to a raw device. To their surprise, the block device will perform much more slowly than the raw device. This article will explain the differences in the results.
The example times below were taken on a K460 using a JBOD disk. The actual speed is not as important as the comparative times between the tests. The dd(1) utility was used to perform the tests using a dd block size of 1024 KB to read or write 100 MB of data to the device.
Operation device wall time (secs)
read raw 15.06
read block 20.60
write raw 18.40
write block 442.58
There are several factors to keep in mind when performing a dd(1) to the devices:
1. Note that in each case, there is no filesystem involved. The dd(1) is performed to the raw or block device (/dev/rdsk or /dev/dsk). Since there is no filesystem involved, there is no readahead. Each read is synchronous. For raw devices, the writes must be synchronous, but for block devices, they are asynchronous.
2. Also, there is no merging of buffers. Typically, buffers are chained together by the filesystem and the volume management layer will merge the buffers when the last buffer in the chain is received from the filesystem. A filesystem, however, is not in use here, so buffers are not chained together and thus are not merged for larger I/Os.
3. Buffers used by the block device are either 4 or 8 KB in size. Originally, the buffers were 4 KB in size, but the recent HP-UX 11.11 patch [PHKL_30992/PACHRDME/English] has increased the block device buffer size to 8 KB. HP-UX 11.23 and above also uses 8 KB buffers for the block device. Since the buffers are not merged, the physical I/O size will be the same size as the buffer size.
4. For raw disk devices, the largest physical I/O is either 256 KB or 1 MB, depending on the driver and operating system. In the test cases above, the I/O size for the raw device was 256 KB.
5. When writing to a block device, a read of each buffer must be performed first, then the buffer is written asynchronously with the new data.
6. The writing of data to a block device is asynchronous. The buffers for writes can consume a large portion of the buffer cache and can cause long I/O queues to the device. Also, when the last close of a block device occurs, all the buffers for the device must be flushed to disk and invalidated. This can consume a lot of CPU time for systems with a large buffer cache.
Given the above factors, we can make the following conclusions:
o Reading the raw device is faster than reading the block device because the raw device is able to issue 4 256 KB synchronous reads for each 1024 KB of data, whereas the block device must perform 256 4 KB synchronous reads (or 128 8 KB synchronous reads depending on the patch level).
o Writing the raw device is faster than writing the block device, in part due to the same reason that reading is faster. However, block writes must read from the device before the data is written. Not only does this cause additional overhead due to the fact that there are twice as many I/O operations, but this read-then-write pattern also causes much more head movement which results in longer I/O service times for the I/O. This may not be as prominent on intelligent disk arrays as it is on the simple JBOD disks as used in this example.
Overall, performing dd(1) tests on a block device is not a very good metric for identifying the actual performance of the device.
Due to all the factors mentioned above, it is not recommended to do dd(1) testing on a disk or volume block device.
|
March 2007 
