Why Oracle Clusterware Needs Odd Number of Voting Disks – A statistics manner explanation

 

 
During Hayasaka-san and Okuma-san's business trip to OARDC, I was asked this question:

    Why we need odd number of voting disk in the cluster.


 
I didn't answer the question directly; instead I talked about the rule:

    Each node in the cluster can survive only if it can access more than half of the voting disk.

    Or the cluster can function normally even if no more than half of the voting disks fail.

And then I talked about various scenarios for network heart beat failure/disk heart failure.


 
Actually I was not satisfied with my answer. So I tried to find other possible ways to convince people that we should use odd number of VD, and I think this can be explained using overall system availability view in a statistics manner.

As you know, many hardware manufacture mark their product availability with 0.99…(many 9s). We can also get such information for the devices that are used for voting disk.


 
Let's start from a well-known mathematics equation:

Suppose:

    Disk availability A=0.9

    Disk un-availability U=0.1

    N hard disks used for voting disks

Then possibility for various scenarios of voting disk failure can be expressed by:

   


 
Under the rule from my previous answer, let's suppose: M=floor(N/2)

    That is: M is the biggest

The overall system un-availability due to voting disk failure =

Oracle can support up to 15 voting disks, let's take a look at the final result:

Voting Disks(N)	M	Value
1	0	0.1
2	1	0.19
3	1	0.028
4	2	0.0523
5	2	0.00856
6	3	0.015985
7	3	……
8	4	……
9	4	……
10	5	……
11	5	……
12	6	……
13	6	……
14	7	……
15	7	……

 
Conclusion:

    We can see that the overall system availability is reduced if adding voting disks from an odd number to the next even number. Adding a voting disk does not always provide higher availability, in some case may be harmful to the availability.

Knowledge Point from Exadata POC4

0. Environment Description

    X2-2 Quarter rack(X2-2 Technical Specifications here)

 

1. Speeding up data loading speed by splitting multiple files.

Oracle database can load external table file parallel automatically, but in some cases parallel loading cannot be achieved, here is the some potential cases that parallel loading is not available:

• Sequential data sources (such as a tape drive or pipe)
  
• Data in any multibyte character set whose character boundaries cannot be determined starting at an arbitrary byte in the middle of a string
  
• Records with the VAR format
  

In POC4 project, we are experiencing one of the situations: we use SJIS character CSV file with delimiter format (i.e. non-fixed length). The elapsed time to load data from external table into heap table is 52 seconds.

Since external table processing is also a CPU intensive operation, to speed up data loading on multiple-CPU environment, we need to use as more CPU as possible. (This is not true for a busy production system.) On the condition restricted by the above cases, work-around is to split the external files into smaller pieces and re-define the external table with file pointing to the smaller piece files.

 

The performance boosts almost linearly:

The X2-2 database server has 2 x six-core CPU, so I split the 1.6G file into 12 pieces and change the external table definition, the elapsed time reduced from 52 seconds to 5.6 seconds. At the same time, the "sar" output also indicates the CPU usage soaring.