# JIRA → P4 Jobs Sync

`sync_jira_to_p4jobs.sh` — one-way, read-only sync of JIRA issues into the
Perforce Public Depot job database.

## SDP site directory convention

This script is **not** part of the SDP package.  Per SDP convention, software
components that are deployed *in* an SDP environment but are not *part of* the
SDP live under the **site directory**:

```
/p4/common/site/
```

The SDP environment file (`p4_vars`) exposes these shell variables for the
site tree:

| Variable   | Default path              | Purpose                          |
|------------|---------------------------|----------------------------------|
| `$P4CSITE` | `/p4/common/site`         | Site root                        |
| `$P4CSBIN` | `/p4/common/site/bin`     | Site scripts and executables     |

`$P4CBIN` (`/p4/common/bin`) is reserved for scripts that **are** part of
the SDP package and should not be used for site-local additions.

All paths in this document and in the script's example config follow this
convention.  Sensitive credential files are kept under `/p4/common/site/etc/`,
which is inside the site tree and outside the SDP-managed directories.

## Architecture overview

```
                         ┌─────────────────────────┐
  JIRA Cloud             │ perforce.atlassian.net   │
  (source of truth)      │ /rest/api/3/search/jql   │
                         └────────────┬────────────┘
                                      │ HTTPS (read-only, API token)
                                      │ incremental (JQL: updated >= watermark)
                                      ▼
  Public Depot host      ┌─────────────────────────┐
                         │ sync_jira_to_p4jobs.sh   │
                         │  /p4/common/site/bin/    │
                         │  • polls JIRA            │
                         │  • maps fields           │
                         │  • upserts P4 jobs       │
                         │  • advances watermark    │
                         └────────────┬────────────┘
                                      │ p4 job -i -f (admin, local socket)
                                      ▼
                         ┌─────────────────────────┐
                         │  p4d (Public Depot)      │
                         │  workshop.perforce.com   │
                         └─────────────────────────┘
```

JIRA is **never written to**.  P4 jobs are **never the source of truth**;
they are a mirror maintained by this script.

---

## Prerequisites

| Requirement | Notes |
|---|---|
| `bash` 4.x+ | Standard on Ubuntu 20+ / RHEL 8+ |
| `python3` | Used for JSON parsing (no external libs needed) |
| `curl` | For JIRA API calls |
| `flock` | For advisory locking (part of `util-linux`) |
| `p4` CLI | Must be on `$PATH` |
| P4 admin account | Needs `admin` access in `p4 protect` for `p4 job -f` |
| JIRA API token | Read-only is sufficient |

---

## Installation

### 1. Place the script

```bash
cp sync_jira_to_p4jobs.sh /p4/common/site/bin/
chmod 755 /p4/common/site/bin/sync_jira_to_p4jobs.sh
```

### 2. Generate an example config

```bash
/p4/common/site/bin/sync_jira_to_p4jobs.sh --init
# writes /p4/common/site/bin/sync_jira_to_p4jobs.cfg
```

Edit the config file; all fields are documented inline.

### 3. Create the JIRA API token

1. Log in to Atlassian as the service account.
2. Go to **Account settings → Security → API tokens → Create API token**.
3. Save the token to the path set in `JiraTokenFile`:

```bash
mkdir -p /p4/common/site/etc
echo -n 'paste-token-here' > /p4/common/site/etc/.jira_api_token
chmod 600 /p4/common/site/etc/.jira_api_token
chown perforce:perforce /p4/common/site/etc/.jira_api_token
```

### 4. Create the P4 service account and ticket

```bash
# As a P4 super user:
p4 user -f svc-jira-sync      # fill in Name/Email, save
p4 protect                    # grant admin to svc-jira-sync

# Log in and save the ticket:
p4 -u svc-jira-sync -p ssl:workshop.perforce.com:1666 login -p \
   > /p4/common/site/etc/.p4_jira_sync_ticket
chmod 600 /p4/common/site/etc/.p4_jira_sync_ticket
chown perforce:perforce /p4/common/site/etc/.p4_jira_sync_ticket
```

> **Why `admin` access?**  The `p4 job -f` flag is required to set the
> `ReportedDate` and `ModifiedDate` fields, which are marked `once` and
> `always` respectively in the jobspec and are otherwise read-only.

### 5. Configure the JIRA project → P4 project mapping

The Public Depot jobspec `Project` field is a select (enum).  Each JIRA
project key must be mapped to its corresponding jobspec value.  Edit the
config file and add one `JiraProjectMap` line per project:

```ini
JiraProjectMap[SDP]=perforce-software-sdp
```

To discover valid jobspec Project values:

```bash
p4 jobspec -o | grep -A1 'Project$'
```

### 6. Verify the jobspec field names

```bash
p4 jobspec -o
```

The script writes these fields: `Job`, `Status`, `Project`, `Severity`,
`Type`, `ReportedBy`, `ReportedDate`, `ModifiedBy`, `ModifiedDate`,
`Description`.  Fields not written (`OwnedBy`, `DevNotes`, `Component`,
`Release`) retain their existing values on update.

### 7. Test manually

```bash
# Dry run (no changes):
/p4/common/site/bin/sync_jira_to_p4jobs.sh -n -v 5

# Real run for a single project, verbose:
/p4/common/site/bin/sync_jira_to_p4jobs.sh -p SDP -v 4
```

### 8. Schedule with cron

```cron
# Sync JIRA → P4 jobs every 30 minutes (as perforce user)
*/30 * * * * /p4/common/site/bin/sync_jira_to_p4jobs.sh -si
```

Or with a systemd timer (preferred on modern Ubuntu):

```ini
# /etc/systemd/system/jira-p4-sync.service
[Unit]
Description=Sync JIRA issues to P4 Public Depot jobs

[Service]
Type=oneshot
User=perforce
ExecStart=/p4/common/site/bin/sync_jira_to_p4jobs.sh -si
StandardOutput=append:/p4/1/logs/sync_jira_to_p4jobs.log
StandardError=append:/p4/1/logs/sync_jira_to_p4jobs.log
```

```ini
# /etc/systemd/system/jira-p4-sync.timer
[Unit]
Description=Run JIRA→P4 sync every 30 minutes

[Timer]
OnBootSec=5min
OnUnitActiveSec=30min

[Install]
WantedBy=timers.target
```

```bash
systemctl enable --now jira-p4-sync.timer
systemctl status jira-p4-sync.timer
```

> **Note on log paths:** The `$LOGS` SDP environment variable (set by
> `p4_vars`) points to `/p4/<instance>/logs`, which is the correct location
> for runtime logs regardless of where the script itself lives.  The script
> uses `$LOGS` automatically when it is set in the environment; the systemd
> unit above uses an explicit path as a fallback for when `p4_vars` has not
> been sourced.

---

## P4 job field mapping

| P4 field | Source | Notes |
|---|---|---|
| `Job` | JIRA issue key | e.g. `SDP-1234` |
| `Status` | JIRA status name + category | Mapped to jobspec values (see below) |
| `Project` | JIRA project key | Via `JiraProjectMap[]` in config |
| `Severity` | Fixed: `C` | JIRA has no direct equivalent |
| `Type` | JIRA issue type | `Bug` / `Doc` / `Feature` / `Problem` |
| `ReportedBy` | `fields.reporter` email or displayName | |
| `ReportedDate` | `fields.created` | `YYYY/MM/DD` |
| `ModifiedBy` | `fields.reporter` (best available) | JIRA updater not in search API |
| `ModifiedDate` | `fields.updated` | `YYYY/MM/DD` |
| `Description` | First line = JIRA summary; blank line; body | Standard P4 convention |

### Status mapping

| JIRA status (specific name, pattern) | P4 Status |
|---|---|
| matches `*block*` | `blocked` |
| matches `*review*` | `review` |
| matches `*duplicate*` | `duplicate` |
| matches `*obsolete*`, `*wontfix*` | `obsolete` |
| matches `*punt*` | `punted` |
| matches `*done*`, `*resolved*`, `*complete*` | `closed` |
| category = `To Do` / `New` | `open` |
| category = `In Progress` | `inprogress` |
| category = `Done` (fallback) | `closed` |
| (anything else) | `open` |

### Description convention

The first line of the P4 `Description` field doubles as the issue title
(JIRA summary).  This is a long-standing P4 Jobs convention on the Public
Depot: tools and views that show only one line of a job's description
show the summary.  The full structure is:

```
<JIRA summary line>
<blank line>
<JIRA description body>
```

---

## Resilience design

| Failure scenario | Behaviour |
|---|---|
| JIRA is down | `curl` times out / retries. Watermark NOT advanced. Next run retries from same point. |
| JIRA rate-limits (HTTP 429) | Back-off and retry within current run. |
| Single JIRA page fails | Project aborted; watermark NOT advanced. |
| P4 job upsert fails | Logged; script continues with next issue. Watermark NOT advanced if any failures. |
| P4 is down | `p4 info` preflight fails; script exits immediately. Watermark NOT advanced. |
| Script killed mid-run | flock lock released on exit. Watermark not advanced. Next run re-fetches idempotently. |
| Concurrent invocations | flock prevents overlap; second instance exits immediately. |

---

## Interaction with the JobIncrement trigger

The `JobIncrement.pl` trigger fires on `form-in` events for jobs whose
`Job:` field value is `new`.  Since this script always supplies an
explicit job name (the JIRA issue key, e.g. `SDP-1234`), the trigger
exits immediately without any action.  Other projects using `JobIncrement`
with their own prefixes are completely unaffected.

---

## State files

| File | Purpose |
|---|---|
| `$StateDir/last_sync_time` | Watermark of last successful poll. Delete to force a full re-sync. |
| `$StateDir/last_sync.lock` | flock advisory lock. Safe to delete when no instance is running. |
| `$StateDir/jira_page_cache/*.json` | Raw JIRA JSON pages; retained 2 days for diagnostics. |

`StateDir` defaults to `/p4/1/tmp/jira_sync` (instance-1 tmp area); override
in the config file if needed.

### Force a full re-sync

```bash
rm /p4/1/tmp/jira_sync/last_sync_time
/p4/common/site/bin/sync_jira_to_p4jobs.sh -v 4
```

---

## JIRA API notes

- Uses `/rest/api/3/search/jql` with `nextPageToken` pagination (the old
  `startAt`-based `/rest/api/3/search` was removed from Atlassian Cloud in 2025).
- Authentication: HTTP Basic with email + API token.
- The JQL query is bounded to specific projects and a `updated >=` watermark,
  keeping each poll small and incremental.
- `JiraPollDelay` (default 2 s) is inserted after every successful API call.

---

## Obtaining a JIRA API token

1. Log in to https://id.atlassian.com as the service account.
2. **Security** → **API tokens** → **Create API token**.
3. Label it (e.g. `p4-public-depot-sync`).
4. Copy the token immediately (shown only once).
5. Store in `JiraTokenFile` as described in step 3 of Installation.