Run IOR and mdtest - DAOS v2.4
Author: DAOS Project
Description: Distributed Asynchronous Object Storage

Skip to content

# Run IOR & mdtest

The `ior` benchmarking tool can be used to generate an HPC type I/O load. This is an MPI-parallel tool that requires MPI to start and control the IOR processes on the client nodes. For this purpose, any MPI implementation can be used. IOR also has an MPI-IO backend, and in order to use this MPI-IO backend with DAOS, an MPI stack that includes the DAOS ROMIO backend has to be used to build and run IOR. Refer to MPI-IO Support for details.

In addition to the default POSIX API, IOR also natively supports the DFS backend that directly uses DAOS File System (DFS) I/O calls instead of POSIX I/O calls. Details on the DAOS DFS backend for IOR can be found in the README_DAOS at the IOR github repository.

The `mdtest` tool to benchmark metadata performance is included in the same repository as the IOR tool. Like `IOR`, it is also an MPI-parallel application.

The Performance Tuning section of the Administration Guide contains further information on IOR and mdtest.

## Build ior and mdtest

    $ module load mpi/mpich-x86_64 # or any other MPI stack
    $ git clone https://github.com/hpc/ior.git
    $ cd ior
    $ ./bootstrap
    $ mkdir build;cd build
    $ ../configure --with-daos=/usr --prefix=<your_dir>
    $ make
    $ make install

## Run ior

This example uses the default IOR `API=POSIX`, and requires that the DAOS POSIX container is dfuse-mounted at `/tmp/daos_dfuse` on all client nodes. Refer to DFuse (DAOS FUSE) for details on dfuse mounts. The per-task performance over a dfuse mount is limited. To obtain better performance, the POSIX API can be used in conjunction with the IOIL interception library. For best performance IOR can be run with `API=DFS`, passing in the DAOS pool and container information (`ior -a DFS --dfs.pool=$DAOS_POOL --daos.cont=$DAOS_CONT ...`).

    $ module load mpi/mpich-x86_64 # or any other MPI stack

    $ mpirun -hostfile /path/to/hostfile_clients -np 30 <your_dir>/bin/ior -a POSIX -b 5G -t 1M -v -W -w -r -R -i 1 -o /tmp/daos_dfuse/testfile

    IOR-3.4.0+dev: MPI Coordinated Test of Parallel I/O
    Began               : Thu Apr 29 23:23:09 2021
    Command line        : ior -a POSIX -b 5G -t 1M -v -W -w -r -R -i 1 -o /tmp/daos_dfuse/testfile
    Machine             : Linux wolf-86.wolf.hpdd.intel.com
    Start time skew across all tasks: 0.00 sec
    TestID              : 0
    StartTime           : Thu Apr 29 23:23:09 2021
    Path                : /tmp/daos_dfuse/testfile
    FS                  : 789.8 GiB   Used FS: 16.5%   Inodes: -0.0 Mi   Used Inodes: 0.0%
    Participating tasks : 30

    Options:
    api                 : POSIX
    apiVersion          :
    test filename       : /tmp/daos_dfuse/testfile
    access              : single-shared-file
    type                : independent
    segments            : 1
    ordering in a file  : sequential
    ordering inter file : no tasks offsets
    nodes               : 3
    tasks               : 30
    clients per node    : 10
    repetitions         : 1
    xfersize            : 1 MiB
    blocksize           : 5 GiB
    aggregate filesize  : 150 GiB
    verbose             : 1

    Results:

    access    bw(MiB/s)  IOPS       Latency(s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   total(s)   iter
    ------    ---------  ----       ----------  ---------- ---------  --------   --------   --------   --------   ----
    Commencing write performance test: Thu Apr 29 23:23:09 2021
    write     1299.23    1299.84    0.022917    5242880    1024.00    10.79      118.17     0.000377   118.22     0
    Verifying contents of the file(s) just written.
    Thu Apr 29 23:25:07 2021

    Commencing read performance test: Thu Apr 29 23:25:35 2021

    read      5429       5431       0.005523    5242880    1024.00    0.012188   28.28      0.000251   28.29      0
    Max Write: 1299.23 MiB/sec (1362.35 MB/sec)
    Max Read:  5429.38 MiB/sec (5693.11 MB/sec)

    Summary of all tests:
    Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev   Max(OPs)   Min(OPs)  Mean(OPs)     StdDev    Mean(s) Stonewall(s) Stonewall(MiB) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt   blksiz    xsize aggs(MiB)   API RefNum
    write        1299.23    1299.23    1299.23       0.00    1299.23    1299.23    1299.23       0.00  118.22343         NA            NA     0     30  10    1   0     0        1         0    0      1 5368709120  1048576  153600.0 POSIX      0
    read         5429.38    5429.38    5429.38       0.00    5429.38    5429.38    5429.38       0.00   28.29054         NA            NA     0     30  10    1   0     0        1         0    0      1 5368709120  1048576  153600.0 POSIX      0
    Finished            : Thu Apr 29 23:26:03 2021

## Run mdtest

Use mdtest to create 30K files using `API=POSIX` (as for IOR, using `-a DFS` will provide much better performance):

    $ mpirun -hostfile /path/to/hostfile_clients -np 10 <your_dir>/bin/mdtest -a POSIX -z 0 -F -C -i 1 -n 3334 -e 4096 -d /tmp/daos_dfuse/ -w 4096

    -- started at 04/29/2021 23:28:11 --

    mdtest-3.4.0+dev was launched with 10 total task(s) on 3 node(s)
    Command line used: mdtest '-a' 'POSIX' '-z' '0' '-F' '-C' '-i' '1' '-n' '3334' '-e' '4096' '-d' '/tmp/daos_dfuse/' '-w' '4096'
    Path: /tmp/daos_dfuse
    FS: 36.5 GiB   Used FS: 18.8%   Inodes: 2.3 Mi   Used Inodes: 5.9%

    Nodemap: 1001001001
    10 tasks, 33340 files

    SUMMARY rate: (of 1 iterations)
       Operation                      Max            Min           Mean        Std Dev
       ---------                      ---            ---           ----        -------
       File creation             :       2943.697       2943.674       2943.686          0.006
       File stat                 :          0.000          0.000          0.000          0.000
       File read                 :          0.000          0.000          0.000          0.000
       File removal              :          0.000          0.000          0.000          0.000
       Tree creation             :       1079.858       1079.858       1079.858          0.000
       Tree removal              :          0.000          0.000          0.000          0.000
    -- finished at 04/29/2021 23:28:22 --