Knowledge Base

========
1.0 Introduction
2.0 Sample Recommended Options
3.0 Discussion
3.1 cam options
3.2 blk options
3.3 dos options
3.4 qnx6 options
3.5 qnx4 options
4.0 Conclusion

1.0 Introduction
============
There are many options which can be passed as arguments to a block driver (devb-*) which revise how the resource manager and associated libraries handle the transfer of data to or from a storage medium (flash drive, disk drive, etc). Some of these options can have a positive effect on the Read or Write performance on the system and as such, should likely be tweaked to achieve optimum performance. Given this however, there is always a trade-off between robustness and performance. Modifying some options to increase write performance for example, may weaken the robustness against corruption due to a power-failure. Additionally due to the varying host environments, as well as device parameters, there is no single set of options which will work best under all circumstances. Given this however, there is a common set of options which can be adjusted in an attempt to achieve better performance.

2.0 Sample Recommended Options
============================
As previously mentioned there is no one specific set of options which will work best under all circumstances, however a common set of options which optimise sequential read and write performance on the OMAP5432 evaluation board (eMMC and USB), for example, are the following:

devb-umass \
dos fat=always \
qnx6 snapshot=0,sync=none \
qnx4 bitmap=always \
cam cache \
blk noatime,commit=none,delwri=5:5,maxio=256,rapolicy=aggressive

devb-sdmmc-omap5evm \
dos fat=always \
qnx6 snapshot=0,sync=none \
qnx4 bitmap=always \
cam cache \
blk noatime,commit=none,delwri=5:5,maxio=256,rapolicy=aggressive,cache=100m

There are several option groups (separated by spaces) and their associated options (separated by commas) which control the behaviour of specific driver components. The options considered above are shown below:

dos
-----
fat=always

qnx6
-------
snapshot=0
sync=none

qnx4
-------
bitmap=always

cam
------
cache
bounce=value*

blk
----
noatime
commit=none
delwri=5:5
maxio=256*
rapolicy=aggressive
cache=100m*
vnode=value*

* were either omitted (for default options) or optimized (through testing) for performance
Note: 'rapolicy' and 'maxio' options require all-fs and all-cam libraries newer than Oct 8,2013

3.0 Discussion
============
The onus of maximising read/write performance is not solely on the resource manager and associated libs. Reading or Writing different sized “chunks” to/from the disk can also have a large impact. For example an extreme case might be reading a 1Gb file by only taking 1 byte at a time. The overhead associated with context switching and message passing will dominate and result in poor performance overall. Choosing the optimal size to access the disk will minimise the overhead associated with message-passing and also not exceed the maximum limit for blocks per read/write operation, as well as minimise the overhead associated with message passing with really large amounts of data. The optimal size is typically 32KB.

It is important to establish potential trade-offs when modifying filesystem/library options such that the risks associated with doing so are known. The discussion of each of the following driver options has either been paraphrased based on the official qnx documentation (http://www.qnx.com/developers/docs/6.5.0/index.jsp) or written from scratch.

3.1 cam options:
=============
cache
--------
This option will cause all I/O transfers to be done directly between the disk cache and the media. If this option isn't used, the CAM library will be using an intermediate 32K buffer (called "bounce buffer") for doing transfers to/from the media. This will add a memory copy operation and additionally may fragment I/O requests into 32k chunks. Both of these operations are negative for performance.

bounce=size
-----------------
This option specifies the size of bounce buffer to use. The bounce buffer acts as an intermediate buffer for doing transfers to/from the media. Bounce buffers are useful for devices which cannot access the full range of memory available to the CPU (32 vs 64-bit architecture etc). When used, size should be sufficiently large such that the CAM library doesn"t fragment the I/O requests from io-blk, this will show markedly better performance. This option is used when “cam cache” is not specified, and by default size=32k.

3.2 blk options (io-blk.so):
====================
cache=total[:hash]
-------------------------
This option is used to designate the total size of the disk cache. The disk cache is intermediate storage for all disk I/O, and caching of dirty blocks. When omitted the default behaviour is to use up to 15% of system RAM with a minimum of 512k and a max of 512Mb.

Limiting the disk cache means that there will be a higher probability that a requested block will not reside in memory and an expensive I/O operation will be necessary to bring it in. In the short term a small disk cache may improve performance (less mmap() calls to allocate cache space), however this may badly degrade performance during normal operation. Alternatively a larger disk cache expands into system memory and may degrade overall performance of the system. The optimum value for 'total' will differ greatly from platform to platform, however it is common to omit this option and allow for the default behaviour.

vnode=size[:max]
------------------------
This option specifies the number of virtual inode entries. When omitted, the default number of vnode entries is 1024. Allowing a larger number of vnodes to be held in memory can increase performance, as the vnode entry remains in memory even after a file is closed. Thus files can be opened, and read from more efficiently. Vnodes also need to be allocated and brought to life before use, thus it may be beneficial to have more available vnodes in the pool for use. The optimal 'size' will vary from platform to platform however the default
(1024) is usually suitable.

delwri=delay1[:delay2[:postpone]]
----------------------------------------------
This option is used to specify the delay time for write-behinds to the media. For write-behind caching, a queue of dirty blocks is held and (subject to a scheduling algorithm) written to the media at a later time. The delay for how long blocks are held in memory is determined by 'delay1', (fixed media) and 'delay2' (removable media) arguments. This behaviour is beneficial because multiple changes to a block can coalesce before being written at a later time, reducing the number of times blocks need to be written through to the media. Lengthening the delay introduces a larger risk of losing data which is held in volatile memory in case of a power failure, or similar scenario.

ra=min[:max]
------------------
This option sets the size of the read-ahead buffer. When omitted, the default behaviour sets the minimum to the system page size and the maximum to 64 times the system page size. This option will show a significant benefit for sequential read operations. Under the presumption that large blocks of data are laid out consecutively, reading ahead and caching the data saves the need for additional expensive I/O operations.

rapolicy=mode
--------------------
This option sets the read-ahead policy for accessing the ra buffer. When omitted the default is mode=traditional, which slowly increments the size of the ra buffer up until the max value (set with the ra option). When mode=aggressive the ra buffer reaches this maximum value much quicker.

maxio=level
----------------
This option allows io-blk to form larger I/O requests. This may be beneficial for higher end media which store an internal flash page. The media is optimized to use this flash page and when I/O request size perfectly matches the size of the internal media flash page, there can be a large increase in performance.

[no]atime
-------------
This option controls how io-blk handles metadata updates. Almost all filesystem operations involve some form of metadata update. With noatime set the directory entry is not modified when the only change is the access time i.e. reading a file without making any changes or modifications.

commit=level
------------------
This option controls the committing level of the filesystem. When level=none all writes to the media (Filesystem data, Metadata, and User data) are time delayed as per the delwri option. By specifying level=none however, all write ordering is lost, and recovering from a power failure scenario is difficult.

3.3 dos options:
============
fat=lazy|nonrmv|always
--------------------------------
The "fat" option for dos filesystems controls the reading of the FAT (File Allocation Table). When set to always, the entire FAT is read when the drive is mounted. This alleviates the need to have sporadic reads to determine where files are located on the volume after mount.

3.4 qnx6 options:
=============
snapshot=freq
-------------------
This option sets the frequency of automatic snapshots on qnx6 filesystems. When set to freq=0, filesystem snapshots are only made when an explicit call to sync or fsync is made. Disabling this feature makes the media less robust against power failure, however performance is slightly better as the system isn't tied up making periodic snapshot updates.

sync=mandatory|optional|none
------------------------------------------
This option specifies the disk synchronization capability. When set to always, a mount as r/w will fail if the medium doesn"t support synchronization. If a device doesn"t support the synchronization capability then power-safety cannot be guaranteed. When set to none, a sync command is never sent to the disk and dirty blocks are not drained from the filesystem cache.

3.5 qnx4 options:
=============
bitmap=when
------------------
This option determines when the .bitmap should be loaded into memory. When set to “always”, the entirety of the file is loaded when mounted. This is beneficial in terms of performance as the bitmap contains information regarding the location of used or free blocks.

4.0 Conclusion
============
Adjusting the filesystem and driver parameters can have a positive effect on the performance, however this typically does not come without trade-offs. The common trade-off is between robustness and performance, and with careful consideration, one can achieve a proper balance between the two. Since there is no “one size fits all” solution, it is recommended to establish realistic benchmarks for relevant activities on the system, and then adjust filesystem and driver parameters to achieve the optimal performance in that context.

More information regarding the "fine-tuning" of filesystem parameters can be found in the