7. Product Maintenance Procedures

This chapter describes the various field-replaceable components of your VDURA V5000 system.

This chapter contains the following sections:

• V5000 Director Node Component Replacement
• V5000 Storage Node Component Replacement
• V5000 JBOD Component Replacement

7.1 V5000 Director Node Component Replacement

This section describes how to replace Director Node components and covers the following topics:

• Replace a Director Node
• Replace an Individual Drive in a Node
• Replace a Power Supply Unit

Warning: Use caution when you remove or replace system components; they may be hot to the touch.

7.1.1 Replace a Director Node

Follow the instructions below to replace a single Director Node in an enclosure.

Before You Begin

This section assumes that a replication set (repset) of three or more Directors is configured on the realm. The VDURA realm is governed by the realm president Director. If the realm contains at least three Directors, a three- or five-member repset is typically configured for redundancy. The president Director replicates its decrees and the realm database to the other members of the repset. Other repset member Directors may serve in other roles, such as the Management Console (MConsole) Director or the DHCP server. All repset member Directors also run a DNS service.

Before beginning the steps below, confirm that the Director has actually failed. If necessary, contact VDURA Technical Support for assistance.

In the example below, 10.72.4.12 is the Director that is being replaced. The Director is still online. The output of these commands would be different if 10.72.4.12 were not responding, but the commands would be the same.

Step 1: Remove target Director from repset.

If the target Director is a member of the repset, it should be removed from the repset before it is replaced.

To determine which Directors are members of the repset, run the syscontrol realmmgr:

[pancli] syscontrol realmmgr
NODE ADDRESS ROLE ON/OFFLINE
VCH-1,1 10.72.4.11 RM, MConsole, DNS Online
VCH-2,1 10.72.4.12 RM, DHCP, DNS Online
VCH-3,1 10.72.4.13 RM President, DNS Online

If the target Director is not present in this output, move to step 2. Otherwise, continue on below.

Note: If the target Director is the only Director in this output, or if this output shows two repset members, please contact VDURA Technical Support before proceeding. Additional steps will be required to replace the Director. If the target Director is the president, data will typically be unavailable for about a minute while the presidency fails over to another Director in the repset. In rare circumstances, data may be unavailable for longer than that if the new president must restart stuck services. If the target Director is the MConsole Director, the PanActive Manager and PanCLI interfaces will be unresponsive for about a minute while the MConsole service fails over to another repset member Director.

Remove the target Director from the repset with syscontrol realmmgr remove [director IP]:

[pancli] syscontrol realmmgr remove 10.72.4.12
/pan/bin/pan_rm_droprm: dropping RM 0x0101c7ef23af0004(RM)
/pan/bin/pan_rm_droprm: dropped RM 0x0101c7ef23af0004(RM).

The output may warn that the repset does not have an optimal number of members:

Warning: A replication set containing 2 members is not a stable configuration. Please add a Director to the repset immediately.

This is acceptable for a short period of time.

Use syscontrol realmmgr to verify that the target Director is no longer in the repset:

[pancli] syscontrol realmmgr
NODE ADDRESS ROLE ON/OFFLINE
VCH-1,1 10.72.4.11 RM, MConsole, DNS Online
VCH-3,1 10.72.4.13 RM President, DNS Online
Warning: A replication set containing 2 members is not a stable configuration. Please add a Director to the repset immediately.

Step 2: Move volume metadata management services off of target Director.

A Director may be responsible for managing the metadata for one or more volumes. This responsibility is shown by the assignment of a volume service to the Director. Use volservice list to see the volume service assignments.

The following output shows that 10.72.4.12 is the primary manager for volume service ID 0x0400000000000032. This Director is also the backup Director for 0x0400000000000005.

[pancli] volservice list
Service Service Primary Primary Backup Backup Volume
ID Status Director IP Director IP Count Volumes
0x0400000000000003(FM) online VCH-1,1 10.72.4.11 VCH-3,1 10.72.4.13 1 /
0x0400000000000004(FM) online VCH-1,1 10.72.4.11 VCH-3,1 10.72.4.13 1 /mdube
0x0400000000000032(FM) online VCH-2,1 10.72.4.12 No backup 1 /esmith
0x0400000000000005(FM) online VCH-3,1 10.72.4.13 VCH-2,1 10.72.4.12 1 /rroy

It is usually better to gracefully move the volume services where the target Director is the primary Director to another Director before the target Director is replaced.

To move the volume service, use volservice move [volume service ID] [new primary director IP].

For example, the command below moves the responsibility for volume service 0x0400000000000032 to Director 10.72.4.11. The president Director has also copied the pan_mgr service transaction logs from Director 10.72.4.12 to Director 10.72.4.11. If practical, choose the Director that is the primary Director for the least number of volume services or whose volume services contain the least active volumes as the new primary Director. If all Directors have about the same volume service load, then you can choose any Director as the new primary Director.

[pancli] volservice move 0x0400000000000032 10.72.4.11
Manual move of service id 0x0400000000000032 to 10.72.4.11 completed successfully.

If the metadata load on the volume service is high, it may take as long as 10 minutes for this command to complete. The target Director must gracefully close connections with DirectFlow Clients. Do not run the command again if it is slow to complete.

Be aware that if the volume service shows no backup, or if the target Director is not responding or is in distress, moving the volume service may result in the volumes in the service going offline and into mandatory FSRC. Consult with VDURA Technical Support before making the move. For example:

[pancli] volservice list
Service Service Primary Primary Backup Backup Volume
ID Status Director IP Director IP Count Volumes
0x0400000000000003(FM) online VCH-1,1 10.72.4.11 VCH-3,1 10.72.4.13 1 /
0x0400000000000004(FM) online VCH-1,1 10.72.4.11 VCH-3,1 10.72.4.13 1 /mdube
0x0400000000000032(FM) online VCH-2,1 10.72.4.12 No backup 1 /esmith
0x0400000000000005(FM) online VCH-3,1 10.72.4.13 VCH-2,1 10.72.4.12 1 /rroy

The pan_mgr transaction logs above are not being replicated to a backup Director. If the president Director cannot successfully move the transaction logs from Director 10.72.4.12 to the new primary Director, the volume /esmith will go offline and begin FSRC.

After you have moved the volume service, move the backup assignments with volservice move backup [volume service ID] [new backup director IP]:

[pancli] volservice move backup 0x0400000000000005 10.72.4.11
Manual move of backup for service id 0x0400000000000005 to 10.72.4.11 completed successfully.

In the example, there is only one choice for the new backup Director. Since 10.72.4.13 is already the primary Director, it cannot be the backup Director.

Verify that the volume service moves completed as expected. In this example, the target Director 10.72.4.12 is no longer the primary Director or backup Director for any volume service:

[pancli] volservice list
Service Service Primary Primary Backup Backup Volume
ID Status Director IP Director IP Count Volumes
0x0400000000000003(FM) online VCH-1,1 10.72.4.11 VCH-3,1 10.72.4.13 1 /
0x0400000000000004(FM) online VCH-1,1 10.72.4.11 VCH-3,1 10.72.4.13 1 /mdube
0x0400000000000032(FM) online VCH-1,1 10.72.4.11 VCH-3,1 10.72.4.13 1 /esmith
0x0400000000000005(FM) online VCH-3,1 10.72.4.13 VCH-1,1 10.72.4.11 1 /rroy

Step 3: Disable IP aliases on Director.

If IP aliases have been configured for NFS or CIFS access, see if any have been assigned to this Director using sysmap node [director IP]:

[pancli] sysmap node 10.72.4.12
Name: VCH-2,1
State: Online
Type: DB18-48
Services: Enclosure Master
IP Address: 10.72.4.12
Primary Aliases: 10.72.4.55 10.72.4.57
Secondary Aliases: <none>
10GE Secondary Only: disabled
. . .

If the Director has either primary or secondary aliases listed, disable them by using syscontrol node [director IP] alias disable:

[pancli] syscontrol node 10.72.4.12 alias disable
Successfully disabled alias assignment for node "10.72.4.12".

The president Director will move the aliases to other Directors. Use sysmap node [director IP] and sysmap nodes directors aliases to verify that the alias IP addresses have been moved to other Directors:

[pancli] sysmap node 10.72.4.12
Name: VCH-2,1
State: Online
Type: DB18-48
Services: Enclosure Master
IP Address: 10.72.4.12
Primary Aliases: disabled
Secondary Aliases: disabled
10GE Secondary Only: disabled
. . .
[pancli] sysmap nodes directors aliases
Node Type Serial IP Address State
VCH-1,1 Director 23a30338300028001 10.72.4.11 Online
Primary Aliases: 10.72.4.57
Secondary Aliases: N/A
VCH-2,1 Director 23a303c5300028001 10.72.4.12 Online
Primary Aliases: N/A
Secondary Aliases: N/A
VCH-3,1 Director 23a3039d300028001 10.72.4.13 Online
Primary Aliases: 10.72.4.55
10.72.4.56
Secondary Aliases: N/A

Step 4: Flash Director LED.

To physically identify the Director, flash its LED using [pancli] syscontrol node [director IP] led on:

[pancli] syscontrol node 10.72.4.12 led on
Now flashing LED for node "10.72.4.12"

If this command fails or times out, either skip this step or turn on the LEDs for surrounding nodes to identify the Director to be replaced.

Step 5: Halt the node.

Power off the Director using syscontrol node [director IP] halt:

[pancli] syscontrol node 10.72.4.12 halt

If the Director is not responding and the command times out, skip this step.

Once the Director becomes non-responsive, the president will move any configured NFS alias IP addresses to a different Director. NFS clients that have the alias IP mounted should be able to resume data access. Any NFS clients that have the Director’s non-alias IP address mounted will hang.

Step 6: Remove old Director record from realm database.

The president automatically removes the record of an OSD that is reconstructed out of a StorageSet and then physically replaced, but does not do this for a Director. Remove the record manually with syscontrol node [director IP] remove:

[pancli] syscontrol node 10.72.4.12 remove

Step 7: Physically replace node.

If the Director did not halt in step 5 above, power it off by pressing its front power button. If the Director still does not halt, press and hold its power button for five seconds.

• Disconnect all network cables from the back of the Director. Consider labeling the cables to make it easier to attach to the replacement node.
• NEED STEPS HERE
• Reconnect the network cables as they were connected to the old Director.
• Power on the Director using the front power button.
• A replacement node will go through the Not Responding and Clock Sync states before reaching Online.
• After about 10 minutes, check the status of the Director from a repset member:

[pancli] sysmap nodes directors
Node Type Serial IP Address State
VCH-1,1 Director 3c760004301002001 10.72.4.11 Online
VCH-2,1 Director 3c760005301002001 10.72.4.12 Online
VCH-3,1 Director 3c760006301002001 10.72.4.13 Online

If the new Director stays in Not Responding or Clock Sync for more than 20 minutes or goes into any other state for more than a few minutes, please contact VDURA Technical Support.

Step 8: Stop LED flashing.

Turn off the identifying LED using syscontrol node [director IP] led off:

[pancli] syscontrol node 10.72.4.12 led off
No longer flashing LED for node "10.72.4.12"

Step 9: Reassign old IP address to replacement Director.

If the old Director was a repset member, and if clients have the IP address of the old Director hard-coded into a mount command, you may want the replacement Director to obtain that same IP address. This procedure may only be used on a Director that is not currently a member of the repset.

Determine which Director currently serves as the DHCP server:

[pancli] syscontrol realmmgr
NODE ADDRESS ROLE ON/OFFLINE
VCH-1,1 10.72.4.11 RM, MConsole, DNS Online
VCH-3,1 10.72.4.13 RM President, DHCP, DNS Online

Here, the DHCP server is running on the 10.72.4.13 Director. SSH to the DHCP Director and run support servicemode yes and netconfig ipranges ipchange:

[pancli] support servicemode yes
[pancli_service] netconfig ipranges ipchange
This command will allow you to change the IP of a node to a new IP provided that the new IP is in the given IP ranges and is not used. Note that this will force a reboot of the node whose IP is getting updated.

Enter the IP address that the replacement Director currently has. In this example, that is 10.72.4.36:

Enter the current IP address of the node:

Enter the IP address that the customer wishes the Director to obtain. In this example, that is 10.72.4.12:

Enter the new IP address of the node:

The script will restart the DHCP server and then reboot the replacement Director so it can obtain its new IP address:

Restarting DHCP server.
Rebooting the remote node.
Node 10.72.4.36 is rebooting

Wait for about five minutes. Confirm that the replacement Director has come online and has the desired IP address:

[pancli] sysmap nodes directors
Node Type Serial IP Address State
VCH-1,1 Director 23a30338300028001 10.72.4.11 Online
VCH-2,1 Director 23a3033f300028001 10.72.4.12 Online
VCH-3,1 Director 23a3039d300028001 10.72.4.13 Online

Step 10: Add replacement Director to repset.

If the Director that was replaced was a member of the repset, use syscontrol realmmgr add [replacement director IP] to add the replacement Director to the repset. This command may take about a minute to complete.

This example assumes that the replacement Director’s IP address has been changed to match the removed Director’s IP address:

[pancli] syscontrol realmmgr add 10.72.4.12
Added Director: 10.72.4.12 (01c7ef23af0004)

Verify that the repset now has either three or five members. If the realm contains less than three Directors, the repset should only have one member:

[pancli] syscontrol realmmgr
NODE ADDRESS ROLE ON/OFFLINE
VCH-1,1 10.72.4.11 RM, MConsole, DNS Online
VCH-2,1 10.72.4.12 RM, DNS Online
VCH-3,1 10.72.4.13 RM President, DHCP, DNS Online

Step 11: Rebalance volume metadata services.

If a volume service was moved in step 2, a Director likely now has too many volume service assignments. The typical way to rebalance the volume service assignments is to move the volume services back to the replacement Director. This example assumes the original volservice move commands from step 2, and that the replacement Director has been reassigned the IP address 10.72.4.12. These commands move the same Primary and Backup assignments as the commands run in step 2.

Use volservice list to view current assignments:

[pancli] volservice list
Service Service Primary Primary Backup Backup Volume
ID Status Director IP Director IP Count Volumes
0x0400000000000003(FM) online VCH-1,1 10.72.4.11 VCH-3,1 10.72.4.13 1 /
0x0400000000000004(FM) online VCH-1,1 10.72.4.11 VCH-3,1 10.72.4.13 1 /mdube
0x0400000000000032(FM) online VCH-1,1 10.72.4.11 VCH-3,1 10.72.4.13 1 /esmith
0x0400000000000005(FM) online VCH-3,1 10.72.4.13 VCH-1,1 10.72.4.11 1 /rroy

Use volservice move [Service ID] [Primary IP] to move an assignment:

[pancli] volservice move 0x0400000000000032 10.72.4.12
Manual move of service id 0x0400000000000032 to 10.72.4.12 completed successfully.
[pancli] volservice move backup 0x0400000000000005 10.72.4.12
Manual move of backup for service id 0x0400000000000005 to 10.72.4.12 completed successfully.

Use volservice list again to verify that each Director has a primary and backup assignment:

[pancli] volservice list
Service Service Primary Primary Backup Backup Volume
ID Status Director IP Director IP Count Volumes
0x0400000000000003(FM) online VCH-1,1 10.72.4.11 VCH-3,1 10.72.4.13 1 /
0x0400000000000004(FM) online VCH-1,1 10.72.4.11 VCH-3,1 10.72.4.13 1 /mdube
0x0400000000000032(FM) online VCH-2,1 10.72.4.12 VCH-1,1 10.72.4.11 1 /esmith
0x0400000000000005(FM) online VCH-3,1 10.72.4.13 VCH-2,1 10.72.4.12 1 /rroy

Step 12: Confirm Director is eligible to receive an alias.

If aliases were disabled for the unhealthy Director in step 3, that configuration will not apply to the replacement Director. The realm president will not move a working alias to the replacement Director immediately, but the replacement Director should receive an IP alias the next time aliases are assigned, if applicable. Confirm the Director is eligible to receive an alias with [pancli] sysmap node [director IP].

[pancli] sysmap node 10.72.4.12
Name: VCH-2,1
State: Online
Type: DB18-48
Services: <none>
IP Address: 10.72.4.12
Primary Aliases: <none>
Secondary Aliases: <none>
10GE Secondary Only: disabled

7.1.2 Replace an Individual Drive in a Node

Director Nodes contain 12 SSDs located in bays in the front of the node. From the front, the SSDs are numbered from left to right:

1 2 3 4 5 6

7 8 9 10 11 12

Each drive slot has three status LEDs.

Status LED	Activity Description
Blue Drive Activity Indicator	- Solid: SSD is installed - Blinking: Drive activity detected - Off: No drive detected
Yellow Drive Failure Indicator	- Solid: Drive fault - Blinking: Drive rebuild - Off: Normal
Green Location Indicator	- Solid: Drive location activated - Off: Normal

If a drive has failed, the yellow Drive Failure Indicator LED may be lit. PanFS does not have direct control over the failure LEDs for the SSDs. The failure LED only turns on if the BIOS or BMC figures out that the SSD has failed, which may not always occur. Consult with VDURA Technical Support to determine which bay contains the failing SSD.

Contact VDURA Technical Support for detailed instructions on how to proceed. When ready, follow the steps below to physically replace the drive.

Caution: Verifying that an individual drive is the cause of a past or impending failure requires working with VDURA Technical Support. Attempting the steps below without assistance may cause system interruption or even data loss. Confirm the drive number from VDURA Technical Support to positively identify the drive number so you do not accidentally replace the wrong drive in the following step.

1. Remove the bezel from the front of the node by sliding the clips located toward each end and then pulling the bezel forward.
2. Press the ejector button for the failed drive to release its tray handle.
3. Use the tray handle to pull the drive and carrier out of its slot.
4. Insert the replacement drive in its carrier into the empty slot. Push it in and close the tray handle. At this point, the blue power LED will light.

7.1.3 Replace a Power Supply Unit

Director Nodes have two power supplies providing redundant power to the system. If either module fails, the other assumes the system load, allowing the node to continue uninterrupted operation. While the power supplies are hot swappable, one power supply must always be operating, so do not attempt to replace both power supplies at the same time unless you shut down the entire enclosure first.

Each power supply has a status LED.

Status LED	Activity Description
Green	- Solid: Powered and functioning normally - Off: No power to both PSUs - Blinking: Standby; AC connected
Red	- Solid: Critical event, PSU shutdown: failure, OCP, OVP, fan failure - Blinking: Non-critical event, PSU continues to operate: high temp, high power, high current, slow fan
Red and green	- Blinking alternate colors: Warning event; power supply continues to operate; high temp, high power, high current, slow fan

If a power supply has failed, the Information LED on the node’s control panel will slowly flash red.

To identify a failed power supply, look at the LEDs on the rear of the power supplies. If a power supply has failed, its LED will be red.

1. Disconnect the power cable from the failed power supply.
2. Push the ejector and use the handle to pull the power supply module out of the chassis.
3. Insert a new power supply into the chassis and push until it is fully seated. The LED on the rear of the module will turn blue.
4. Plug the power cable into the module. The power supply LED will turn green.

7.2 V5000 Storage Node Component Replacement

This section describes how to replace Mixed Fleet and All-Flash Storage Node components and covers the following topics:

• Replace a Storage Node
• Replace an Individual Drive in a Node
• Replace a Power Supply Unit

Warning: Use caution when you remove or replace system components; they may be hot to the touch.

7.2.1 Replace a Storage Node

Follow the instructions below to replace a Storage Node.

1. PanActive Manager will alert you to the node that needs to be replaced. For detailed information on setting up alerts, see the VDURA Data Platform User Guide.
2. If the node is in Failed state, the data residing on it will already have been reconstructed to other nodes. Continue to step 3 to physically replace the node.

If the node is in Reconstruction state, please wait until reconstruction is complete.

If the node is still online, drain the node and clear the record before removing it. In PanActive Manager, click the Hardware tab and select the node you want to remove. From the appropriate Node Management page, click Start OSD Drain, then confirm that you want to drain the node. When finished, proceed to step 3.

3. Use the button on the node’s control panel to turn off the power.
4. Remove the drives from the front. Remember which slot each drive was in.
5. Disconnect all cables from the back of the node. Failure to do this step can result in damaging the NIC or other rear ports.
6. Remove the node from the enclosure.
7. Reinstall the drives from the original node into the replacement node. Place the drives into the same slot locations as in the old node.
8. Reconnect the network cables as they were connected to the old node and turn the node’s power back on.
9. Reinitialize the node to return it to service by issuing the following commands:

[pancli] support service yes
[pancli_service] syscontrol node <VCH-#,#> reinit

You should see the following output:

Reinitializing node "<VCH-#,#>".

The reinitialization process takes approximately five minutes. Please consult with VDURA Technical Support if the node has not rejoined the realm after 15 minutes.

10. The VDURA system automatically detects the new node.

7.2.2 Replace an Individual Drive in a Node

If the system has alerted you that an individual Storage Node has failed and is being reconstructed out, or an individual drive in a node is emitting SMART errors, you may be able to replace the faulty drive rather than the entire node to correct the problem and bring the node back into service.

Storage Nodes contain 12 SSDs located in bays in the front of the node. From the front, the SSDs are numbered from left to right:

1 2 3 4 5 6

7 8 9 10 11 12

Each drive slot has three status LEDs.

Status LED	Activity Description
Blue Drive Activity Indicator	- Solid: SSD is installed - Blinking: Drive activity detected - Off: No drive detected
Yellow Drive Failure Indicator	- Solid: Drive fault - Blinking: Drive rebuild - Off: Normal
Green Location Indicator	- Solid: Drive location activated - Off: Normal

1. Drain or reconstruct out the node. If the node in question is still participating in the StorageSet, you will need to drain or reconstruct it out of the StorageSet before proceeding. VDURA Technical Support will let you know which option to use based on the situation.

Physically removing a drive before beginning reconstruction or completing a drain will cause unnecessary system interruption.

If the node is not participating in the StorageSet, proceed to step 2.

In PanActive Manager, click the Hardware tab, then select the node you want to remove. From the appropriate Node Management page, either click Start OSD Drain or Reconstruct depending on the desired action, then confirm. Wait for drain or reconstruction to fully complete, either by the step above or automatically by the system.

2. If a drive has failed, the yellow Drive Failure Indicator LED may already be lit. PanFS does not have direct control over the failure LEDs for the SSDs. The failure LED only turns on if the BIOS or BMC figures out that the SSD has failed, which may not always occur. Consult with VDURA Technical Support to determine which bay contains the failing SSD.

Contact VDURA Technical Support for detailed instructions on how to proceed. When ready, follow the steps below to physically replace the drive.

Caution: Verifying that an individual drive is the cause of a past or impending failure requires working with VDURA Technical Support. Attempting the steps below without assistance may cause system interruption or even data loss. Confirm the drive number from VDURA Technical Support to positively identify the drive number so you do not accidentally replace the wrong drive in the following step.

1. Remove the bezel from the front of the node by sliding the clips located toward each end and then pulling the bezel forward.
2. Press the ejector button for the failed drive to release its tray handle.
3. Use the tray handle to pull the drive and carrier out of its slot.
4. Insert the replacement drive in its carrier into the empty slot. Push it in and close the tray handle. At this point, the blue power LED will light.
5. While logged in as admin on one of the Directors, turn off the location LED on the drive in question with the following PanCLI command:

[pancli] syscontrol node <VCH-#,#> drive <#> led normal

In the above command, replace <VCH-#,#> with the name of the reconstructed or drained node and <#> with the drive number of the failed or failing drive. This command should yield the following output:

Successfully set LED for drive <#> on node <VCH-#,#> to locate.

6. Finally, reinitialize the node to return it to service by issuing the following commands:

[pancli] support service yes
[pancli_service] syscontrol node <VCH-#,#> reinit

You should see the following output:

Reinitializing node "<VCH-#,#>".

The reinitialization process takes approximately five minutes. Please consult with VDURA Technical Support if the node has not rejoined the realm after 15 minutes.

7.2.3 Replace a Power Supply Unit

Storage Nodes have two power supplies providing redundant power to the system. If either module fails, the other assumes the system load, allowing the node to continue uninterrupted operation. While the power supplies are hot swappable, one power supply must always be operating, so do not attempt to replace both power supplies at the same time unless you shut down the entire enclosure first.

Each power supply has a status LED.

Status LED	Activity Description
Green	- Solid: Powered and functioning normally - Off: No power to both PSUs - Blinking: Standby; AC connected
Red	- Solid: Critical event, PSU shutdown: failure, OCP, OVP, fan failure - Blinking: Non-critical event, PSU continues to operate: high temp, high power, high current, slow fan
Red and green	- Blinking alternate colors: Warning event; power supply continues to operate; high temp, high power, high current, slow fan

If a power supply has failed, the Information LED on the node’s control panel will slowly flash red.

To identify a failed power supply, look at the LEDs on the rear of the power supplies. If a power supply has failed, its LED will be red.

1. Disconnect the power cable from the failed power supply.
2. Push the ejector and use the handle to pull the power supply module out of the chassis.
3. Insert a new power supply into the chassis and push until it is fully seated. The LED on the rear of the module will turn blue.
4. Plug the power cable into the module. The power supply LED will turn green.

7.3 V5000 JBOD Component Replacement

This section describes how to replace JBOD components and covers the following topics:

• Replace a SAS Cable
• Replace a Power Supply Unit
• Replace a Fan
• Replace a Single Disk
• Remove a Drive from Its Carrier
• Replace an IOM
• Replace the JBOD Enclosure

You cannot replace an I/O module (IOM), power supply, or fans without first pulling the enclosure partially out of the rack and disengaging the Cable Management Arm (CMA). Depending on your cabling and space available, you may only need to disengage one end of the CMA and let it hang down. It may be easier to also disengage the other side to allow more freedom of movement. You must also detach the appropriate cables before you pull out either a power supply or the bottom IOM.

Note: All items below can be hot-swapped while the JBOD is powered on.

7.3.1 Replace a SAS Cable

1. Disconnect the cable from the rear of the Storage Node.
2. Remove the screws securing the JBOD to the rack, then slide the JBOD forward enough to access its rear.
3. Open the cable baskets.
4. Disconnect the cable from the rear of the JBOD.
5. Connect the new cable to both empty ports.

7.3.2 Replace a Power Supply Unit

1. Remove the screws securing the JBOD to the rack, then slide the JBOD forward enough to access its rear.
2. Unplug the power cable from the PSU.
3. Press the green tab sideways to unlock the PSU.
4. Pull the handle to remove the PSU.
5. Insert the new PSU. Be sure the green tab clicks to lock it in place.
6. Plug the power cable into the new PSU.

7.3.3 Replace a Fan

1. Remove the screws securing the JBOD to the rack, then slide the JBOD forward enough to access its rear.
2. Lift the black tab to unlock the fan unit.
3. Pull the handle to remove the fan unit.
4. Insert the new fan unit. Be sure the black tab clicks to lock it in place.

7.3.4 Replace a Single Disk

If PanActive Manager has alerted you that an individual drive in the JBOD is emitting errors or failing, you may be able to replace the faulty drive and bring the node back into service.

Caution: Verifying that an individual drive is the cause of a past or impending failure requires working with VDURA Technical Support. Attempting the steps below without assistance may cause system interruption or even data loss. Confirm the drive number from VDURA Technical Support to positively identify the drive number so you do not accidentally replace the wrong drive in the following step.

1. Drain or reconstruct out the node. If the node in question is still participating in the StorageSet, you will need to drain or reconstruct it out of the StorageSet before proceeding. VDURA Technical Support will let you know which option to use based on the situation.

Physically removing a drive before beginning reconstruction or completing a drain will cause unnecessary system interruption.

If the node is not participating in the StorageSet, proceed to step 2.

In PanActive Manager, click the Hardware tab, then select the node you want to remove. From the appropriate Node Management page, either click Start OSD Drain or Reconstruct depending on the desired action, then confirm. Wait for drain or reconstruction to fully complete, either by the step above or automatically by the system.

2. Turn on the drive’s LED to locate it.

Front of JBOD

Front of JBOD

Log in as admin on a Director and enter the following PanCLI command:

[pancli] syscontrol node <VCH-#,#> drive <#> led locate

Replace <VCH-#,#> with the name of the reconstructed or drained node and <#> with the drive number of the failed or failing drive.

This command should yield the following output:

Successfully set LED for drive <#> on node <VCH-#,#> to locate.

A blinking LED will appear by the drive to be replaced. The LEDs on the front of the JBOD will also indicate which drive slot you have located.

3. Press the button beside the drive to unlock it, then remove it from the enclosure.

Caution: To reduce the risk of shock damage to the drive, remove the drive just enough to make the location LED stop blinking. Wait at least 15 seconds before continuing to remove the drive; this will let the drive spin down completely.

4. Insert a replacement drive in a carrier into the bay.
5. Push the drive in as far as you can, then lock it in place.
6. While logged in as admin on one of the Directors, turn off the location LED on the drive in question with the following PanCLI command:

[pancli] syscontrol node <VCH-#,#> drive <#> led normal

In the above command, replace <VCH-#,#> with the name of the reconstructed or drained node and <#> with the drive number of the failed or failing drive. This command should yield the following output:

Successfully set LED for drive <#> on node <VCH-#,#> to locate.

7. Reinitialize the node to return it to service by issuing the following commands:

[pancli] support service yes
[pancli_service] syscontrol node <VCH-#,#> reinit

You should see the following output:

Reinitializing node "<VCH-#,#>".

The reinitialization process takes approximately five minutes. Consult with VDURA Technical Support if the node has not rejoined the realm after 15 minutes.

7.3.5 Remove a Drive from Its Carrier

1. The carrier is simply a piece of plastic that wraps around the drive. There are no screws to hold the carrier to the drive.
2. Find the slight plastic ridge in the middle of the carrier.
3. Grip the ridge with your fingers and pull the carrier slightly away from the drive.
4. Gently ease the drive out of the carrier.
5. Lever it up on the side opposite the latch hinge and where the button for the latch is located.
6. To put the drive back into the carrier, reverse the procedure.
7. Line up the side of the drive with the studs built into the inside of the carrier and fit the studs into the screw holes on the drive.
8. You may see that the drive hangs up on this corner, so you might want to do this corner first, then the other corner.

7.3.6 Replace an IOM

1. Disconnect all cables from the unit.
2. Loosen the two screws at the top of the module.
3. Use the handle to remove the module.
4. Insert the replacement module.
5. Tighten the two screws at the top of the module.
6. Reconnect all cables.

7.3.7 Replace the JBOD Enclosure

1. Take the realm or StorageSet offline.
2. Remove the disks. Pay attention to the slot each disk is in. You must place each disk in the same location slot in the new enclosure.

See Replace a Single Disk for tables listing slot and drive numbers.

3. Disconnect all cables from the enclosure. The cables must get connected back into the same ports on the replacement enclosure.
4. Remove the JBOD enclosure from the rack. Leave the PSUs, fans, and IOM in the enclosure; the replacement will have new ones.
5. Slide the new enclosure into the rack.
6. Reconnect all cables.
7. Insert the disks.
8. Bring the realm or StorageSet online.