# Standby Edge Failover Plan

## Edge Server Using Journalcopy Replication

---

# Overview

This document describes how to failover an edge server to the standby edge that is using journalcopy for replication. We assume the instance number for the server you are failing over to be 1 in this document.

## Conventions

- **Primary Edge:** Normally active edge server. We will refer to this machine as the primary edge throughout this document, even after failover for consistency.
- **Standby Edge:** This is the machine that is normally the standby for your edge server. We will refer to this machine as the standby edge throughout the document for consistency.

## Variables

| Variable | Description |
|----------|-------------|
| `edge_id` | Server ID for your edge server |
| `edge_standby_id` | Server ID for your edge standby server |
| `svc_edge_id` | Service user for the edge master |
| `svc_edge_standby_id` | Service user for the edge standby |

---

# Failover Procedure

To perform the failover from the Primary Edge to the Standby Edge, complete the following steps:

## 1.1 Check Replica Status (Done on Standby Edge)

Connect via SSH to the Standby Edge. Make sure you are the user running Perforce.

Check replication status with:

```bash
p4 pull -lj
```

## 1.2 Stop the Primary Edge

Connect via SSH to the Primary Edge. Make sure you are the user running Perforce.

Check replication status with:

```bash
p4 pull -lj
```

Stop the Primary Edge Server Perforce Service:

```bash
sudo systemctl stop p4d_1
```

Disable automatic startup of the Primary Edge Server Perforce Service:

```bash
sudo systemctl disable p4d_1
```

## 1.3 Promote the Standby Edge (Done on Standby Edge)

Stop the Standby Edge Server Perforce Service:

```bash
sudo systemctl stop p4d_1
```

Clear the state file:

```bash
rm /p4/1/root/state*
```

Remove replica journals:

```bash
rm -f /p4/1/logs/journal*
```

Copy primary journals to standby server:

```bash
rsync -av edge_id:/p4/1/logs/journal /p4/1/logs
```

Copy primary edge state to standby server:

```bash
rsync -av edge_id:/p4/1/root/state /p4/1/root
```

Replay primary journal against standby db:

```bash
/p4/1/bin/p4d_1 -r /p4/1/root -jr -f /p4/1/logs/journal
```

Log admin user into the commit server:

```bash
/p4/common/bin/p4master_run 1 p4 -p perforce-commit.ct.company.com:1666 login -a < /p4/common/config/.p4passwd.p4_1_admin
```

Log service user into the commit server:

```bash
/p4/common/bin/p4master_run 1 p4 -p perforce-commit.ct.company.com:1666 login svc_edge_id
```

Edit the server.id file and change the server id to `edge_id`:

```bash
vim /p4/1/root/server.id
```

Edit the server type file and change `p4d_edgerep` to `p4d_edge`:

```bash
vim /p4/1/root/sdp_server_type.txt
```

Start Perforce service on standby edge:

```bash
systemctl start p4d_1
```

## 1.4 Verify Failover

Verify the Standby Edge is now acting as Primary Edge:

```bash
# Check server id
p4 info

# Make sure edge is pulling from commit
p4 pull -lj
```

## 1.5 Update DNS

Failover complete. Have IT update the DNS record for the Primary Edge to point to the Standby Edge.

---

# Failback Procedure (Standby Edge to Primary Edge)

## 2.1 Verify Primary Edge is Stopped (Done on Primary Edge)

Connect via SSH to the Primary Edge.

Verify the Primary Edge Server Perforce Service is stopped:

```bash
sudo systemctl status p4d_1
```

## 2.2 Stop the Standby Edge

Connect via SSH to the Standby Edge.

Stop the Standby Edge Server Perforce Service:

```bash
sudo systemctl stop p4d_1
```

## 2.3 Restore the Primary Edge (Done on Primary Edge)

Connect via SSH to the Primary Edge.

Again, verify the Primary Edge Server Perforce Service is stopped:

```bash
sudo systemctl status p4d_1
```

Clear the state file:

```bash
rm /p4/1/root/state*
```

Remove replica journals:

```bash
rm -f /p4/1/logs/journal*
```

Copy standby checkpoints to primary server:

```bash
rsync -av edge_id:/p4/1/checkpoints/p4_1.jnl.* /p4/1/checkpoints/
```

Copy standby journals to primary server:

```bash
rsync -av edge_id:/p4/1/logs/journal /p4/1/logs
```

Check the Perforce journal counter:

```bash
/p4/1/bin/p4d_1 -r /p4/1/root -jd - db.counter | grep journal
```

Replay any newer numbered journals against primary db:

```bash
/p4/1/bin/p4d_1 -r /p4/1/root -jr -f /p4/1/checkpoints/p4_1.jnl.xx
```

Replay the current journal file against primary db:

```bash
/p4/1/bin/p4d_1 -r /p4/1/root -jr -f /p4/1/logs/journal
```

Log service user into the commit server:

```bash
/p4/common/bin/p4master_run 1 p4 -p perforce-commit.company.com:1666 login svc_edge_id
```

Enable Perforce service on primary edge:

```bash
sudo systemctl enable p4d_1
```

Start Perforce service on primary edge:

```bash
sudo systemctl start p4d_1
```

## 2.4 Verify Failback

Verify the Primary Edge is now acting as Primary Edge once again:

```bash
# Check server id
p4 info

# Make sure edge is pulling from commit
p4 pull -lj
```

## 2.5 Update DNS

Failback complete. Have IT update the DNS record to point back to the Primary Edge.

> **Note:** At this point, the major part of the maintenance is complete, and users should be able to resume work on the Primary Perforce Edge Server. The remaining task is to recover the standby edge server.

---

# Standby Edge Recovery

## 3.1 Verify Standby Edge is Stopped (Done on Standby Edge)

Connect via SSH to the Standby Edge.

Verify the Standby Edge Server Perforce Service is stopped:

```bash
sudo systemctl status p4d_1
```

## 3.2 Recover the Standby Edge

Clear the state file:

```bash
rm /p4/1/root/state*
```

Remove replica journals:

```bash
rm -f /p4/1/logs/journal*
```

Log admin user into the commit server:

```bash
/p4/common/bin/p4master_run 1 p4 -p edge_id.company.com:1666 login -a < /p4/common/config/.p4passwd.p4_1_admin
```

Log service user into the commit server:

```bash
/p4/common/bin/p4master_run 1 p4 -p edge_id.company.com:1666 login svc_edge_id_standby
```

Edit the server.id file and change the server id to `edge_id-standby`:

```bash
vim /p4/1/root/server.id
```

Edit the server type file and change `p4d_edge` to `p4d_edgerep`:

```bash
vim /p4/1/root/sdp_server_type.txt
```

Start Perforce service on standby edge:

```bash
sudo systemctl start p4d_1
```

## 3.3 Verify Standby Edge Recovery

Verify the Standby Edge is now acting as Standby Edge once again:

```bash
# Check server id
p4 info

# Make sure standby edge is pulling data
p4 pull -lj
```