1. Attachments are working again! Check out this thread for more details and to report any other bugs.

Optimum RAID-5 "interlace-value" for Oracle DB?

Discussion in 'Fred's House of Pancakes' started by bwilson4web, Oct 8, 2012.

  1. bwilson4web

    bwilson4web BMW i3 and Model 3

    Joined:
    Nov 25, 2005
    27,137
    15,394
    0
    Location:
    Huntsville AL
    Vehicle:
    2018 Tesla Model 3
    Model:
    Prime Plus
    So this evening I'm creating a six volume, RAID-5 array and by accident, hit <RETURN> before entering the "interlace-value". Given it is a database application, I would normally use something small, say 64KB. But the default is 512KB . . . <woops!> I could not stop initialization short of rebooting the system, Solaris-10. Even then, it might just restart initialization so I might as well let it run.

    So the question is: What is the optimum, RAID-5, "interlace-value" for an Oracle DB?

    This is primarily a data collection system with rather modest, less than 1 MB/sec., data rates. There are reports to generate but we don't like reports and even they are just serial access.

    Thanks,
    Bob Wilson

    ps. Prius association? . . . I drive one to work.
     
  2. bwilson4web

    bwilson4web BMW i3 and Model 3

    Joined:
    Nov 25, 2005
    27,137
    15,394
    0
    Location:
    Huntsville AL
    Vehicle:
    2018 Tesla Model 3
    Model:
    Prime Plus
    Well we have some lessons learned:
    • 16 KByte ~ 820 GB, 128 KByte ~ 808 GB :: some loss of space
    • 126 GB backup ~ 40 hrs restore vs expected 24 hrs
    We'll do a clean-up reboot and start the application to see how well it works in practice. So far, RAID-5 appears 'less fast' but fast enough. I'm not one to let perfect be the enemy of 'good enough.' But most of all, I'm happy:
    • a single drive failure won't fail the system - we can keep working while scheduling and effecting a repair
    • one massive filesystem with I/O spread across six disks - equalizes the I/O (and wear) so hopefully the whole will last longer than one drive being 'beat to death'
    The disk drives rotate at 10,000 rpm and have 10 KB tracks. So we are looking at sectors that span 128 tracks. In theory, one might be able to size the sectors to reduce one cylinder, movement. However, we have no control over sector placement which means there will always be some cylinder movement. But there are some interesting geometry models:
    1. decrease the sector size reduces the probability of a cylinder change during a read or write
    2. increase the sector size to increase the probability of track-to-track switch for fastest read or write
    Based upon the single cylinder delay and rotational delay, one can calculate what should be an optimum 'within disk' sector size. But the model also needs to assume a 1/2 rotational delay going from disk-to-disk. An engineering rule of thumb says the inter-disk delays should be equal to the intra-disk delays.

    Regardless, I'm reasonably happy with the performance. It is not a 'barn burner' but it only has to be fast enough to meet our performance requirements. We've already had a failed disk down the system for nearly a month and I'm not going through that effort IF I can help it.

    What is the alternative? Create a massive RAID-0 drive and just reload the software and reload the database base . . . which is what we had to do the last time. With RAID-5, we have to option to delay this multi-week effort.

    Bob Wilson
     
  3. JimboPalmer

    JimboPalmer Tsar of all the Rushers

    Joined:
    Apr 14, 2009
    12,470
    6,862
    2
    Location:
    Greenwood MS USA
    Vehicle:
    2012 Prius v wagon
    Model:
    Three
    When you create the Oracle db you get to choose the size of the data blocks in the SGA. We found bigger blocks to be better, for our data. 32 Kb? You then get to specify db_file_multiblock_read_count tips
    for full table scans and index scans. You want your disk i/o to be the same or an even multiple of this.

    (I last worked as a DBA ten years ago, so my advice is suspect. I would ask at tek-tips.com)
     
  4. bwilson4web

    bwilson4web BMW i3 and Model 3

    Joined:
    Nov 25, 2005
    27,137
    15,394
    0
    Location:
    Huntsville AL
    Vehicle:
    2018 Tesla Model 3
    Model:
    Prime Plus
    Thanks,
    Oracle is part of another application so we're using the parameters the vendor configured. I was disappointed to find the vendor had no practical RAID-5 recommendations. But I noticed the Oracle configuration file showed a block size of 8K.

    I suspect we're in good shape but we'll do some metrics before making further changes. Sad to say, I suspect the first level, problem team may not have access to more detailed, engineering team data.

    Bob Wilson
     
  5. JimboPalmer

    JimboPalmer Tsar of all the Rushers

    Joined:
    Apr 14, 2009
    12,470
    6,862
    2
    Location:
    Greenwood MS USA
    Vehicle:
    2012 Prius v wagon
    Model:
    Three
    On the bright side if your RAID-5 has a NVRAM buffer, Oracle will see that as an infinite speed write.

    Our database was always updated by users, when they hit commit; they had a 7 second pause. Once I added an NVRAM buffer, commits fell to under a tenth of a second. If your use of Oracle does not include user writes, this 'feature' will not impress them.

    Oracle - Dirty Reads, Done Dirt Cheap!