From Vd

Test plan for destructive testing of the NorStore hardware and middleware installation in Oslo.

Links to the WAN test and functional test.

Contents 1 Introduction 2 Scenarios 2.1 Connection status 3 Expectations and results

Introduction

The testing will be done on the NorStore GPFS system under reasonable load. Two or more GPFS file servers will be connected.

The testing should be done after monitoring of the systems have been set up and the testing should be a joint effort with the support organization on SUN/HP to be able to test that error messages are propagated through the vendor support mechanisms.

The test plan is not yet finished!

Scenarios

1. [If trunking] Network interface with the lowest device-number on the public network is unavailable. Public network is used by upload/download clients to access storage. The failure is simulated by disconnecting the appropriate network cable from the node.
2. [If trunking] Network interface with the highest device-number on the public network is unavailable. Public network is used by upload/download clients to access storage. The failure is simulated by disconnecting the appropriate network cable from the node.
3. [If not trunking] The network interface on the public network is unavailable. Public network is used by upload/download clients to access storage. The failure is simulated by disconnecting the appropriate network cable from the node.
4. Switch with the lowest number on the public network fails. This failure is simulated by taking down the switch.
5. Switch with the highest number on the public network fails. This failure is simulated by taking down the switch.
6. How do we test GPFS heartbeat ?
7. Node krash. Simulated by "pressing the button".
8. FC-failure. Simulated by disconnecting both FC cables from the node.
9. FC-failover 1: Simulated by disconnecting the FC-cable that is the FC-interface with the lowest device-number.
10. FC-failover 2: Simulated by disconnecting the FC-cable that is the FC-interface with the highest device-number.
11. FC-failover 3: Simulated by powering off the FC-switch with the lowest number.
12. FC-failover 4: Simulated by powering off the FC-switch with the highest number.
13. FC-failover 5: Simulated by pulling out the FC-cable going from the FC-switch with the lowest number to the controller with the lowest number.
14. FC-failover 5: Simulated by pulling out the FC-cable going from the FC-switch with the lowest number to the controller with the highest number.
15. FC-failover 5: Simulated by pulling out the FC-cable going from the FC-switch with the highest number to the controller with the lowest number.
16. FC-failover 5: Simulated by pulling out the FC-cable going from the FC-switch with the highest number to the controller with the highest number.
17. Power: Power cables is pulled out and reconnected.
18. Disk failure 1: Failure simulated by removing a disk from a RAID-group.
19. Disk failure 2: Failure simulated by removing one more disk from a RAID-group an hour after removing the first one.

Connection status

Every scenario will be tested with the following connection status:

• Writing: A write task is performed against the storage solution
• Trying to write: A new write task is attempted after the "insident"

Expectations and results

For each of the designed tests we have an expectation of what is going to happen. The test is a success if the expectation is fulfilled. Test failures must be registered and investigated and a new test will have to be re-run at a later stage to verify that a corrective action has had its intended effect.

Test #	Scenario	Status	Expected result	Actual result	Date
1.	1	Writing	The client is not influenced. Bonding of networ interfaces will ensure that network traffic is transmitted on the alternate path.	?
2.	2	Writing	The client is not influenced. Bonding of networ interfaces will ensure that network traffic is transmitted on the alternate path.	?
3.	3	Writing	The client hangs/is disconnected. There is no redundant path to re-route the traffic to.	?
...	...	...	...	...
4.	1	Write	The client is not influenced. Bonding of networ interfaces will ensure that network traffic is transmitted on the alternate path.	?
5.	2	Write	The client is not influenced. Bonding of networ interfaces will ensure that network traffic is transmitted on the alternate path.	?
6.	3	Write	The client hangs. There is no redundant path to re-route the request to.	?