# OpenCodeReview - GitLab CI Demo

This demo shows how to integrate OpenCodeReview into your GitLab CI/CD pipeline to automatically review Merge Requests and post review comments as inline discussions.

## How It Works

```
MR Created/Updated → GitLab Pipeline Triggered → OCR Reviews Diff → Discussions Posted on MR
```

1. When a Merge Request is opened or updated, the pipeline triggers
2. It installs OCR via npm in a `node:20` Docker image
3. Runs `ocr review --from origin/<target> --to <commit_sha> --format json --audience agent` to analyze the diff (uses commit SHA to support fork MRs)
4. Parses the JSON output and posts inline discussions on the MR using GitLab's Discussions API

## Setup

### 1. Copy the pipeline and script files

Copy **both** `.gitlab-ci.yml` and `post_review.py` to your repository root (or a subdirectory — adjust the `python3 post_review.py` path in the YAML if you place the script elsewhere):

```bash
cp .gitlab-ci.yml post_review.py /path/to/your/repo/
```

Or use GitLab's `include` feature in your existing `.gitlab-ci.yml` (the script path is relative to the pipeline file's location):

```yaml
include:
  - local: 'ci_demo/gitlab_ci/.gitlab-ci.yml'
```

### 2. Configure CI/CD Variables

Go to your project's **Settings → CI/CD → Variables** and add:

| Variable | Required | Masked | Description |
|----------|----------|--------|-------------|
| `OCR_LLM_URL` | Yes | No | LLM API endpoint URL (e.g., `https://api.openai.com/v1/chat/completions`) |
| `OCR_LLM_AUTH_TOKEN` | Yes | Yes | API authentication token |
| `OCR_LLM_MODEL` | Yes | No | Model name (e.g., `gpt-4o`) — OCR has no built-in default model and fails when this is unset |
| `GITLAB_API_TOKEN` | No | Yes | GitLab access token with `api` scope (falls back to `CI_JOB_TOKEN` if not set) |
| `OCR_VERSION` | No | No | npm version spec for `@alibaba-group/open-code-review` (default: `latest`). Pin it (e.g. `1.8.8` or `~1.8`) for reproducible reviews. |
| `OCR_LANGUAGE` | No | No | Review output language, written via `ocr config set language` (e.g. `English`, `Chinese`). Defaults to OCR's built-in default. |
| `OCR_LLM_AUTH_HEADER` | No | No | Custom auth header name, written via `ocr config set llm.auth_header` (e.g. `x-api-key` for some providers). |
| `OCR_LLM_EXTRA_HEADERS` | No | No | Extra headers `K=V,K=V`, written via `ocr config set llm.extra_headers`. |
| `OCR_LLM_TIMEOUT` | No | No | LLM request timeout in seconds (read natively by OCR from the env). Unset/empty = OCR's default. |
| `OCR_REVIEW_CONCURRENCY` | No | No | Max concurrent file reviews, passed via `ocr review --concurrency` (default: 8). |
| `OCR_BACKGROUND` | No | No | Business/requirement context, passed via `ocr review --background` (e.g. the MR title). |
| `OCR_RULE` | No | No | Path to a custom rules JSON file, passed via `ocr review --rule`. |

> **Note:** GitLab CI/CD does not support variables with values shorter than 8 characters, so `use_anthropic` cannot be set as a CI variable. The pipeline sets it to `false` by default. If you need to use Anthropic Claude models, you'll need to modify the `.gitlab-ci.yml` script directly.
>
> The pipeline also configures `llm.extra_body` to disable thinking mode for compatibility with various LLM providers.

### 3. Create a GitLab Access Token

You need a token with `api` scope to post discussions on MRs. Options:

- **Project Access Token** (recommended): Settings → Access Tokens → Create with `api` scope
- **Personal Access Token**: User Settings → Access Tokens → Create with `api` scope
- **Group Access Token**: For organization-wide usage

> **Note:** The built-in `CI_JOB_TOKEN` has limited API scope and may not support all discussion features (e.g., creating new threads on older GitLab versions). If `GITLAB_API_TOKEN` is not set, the pipeline falls back to `CI_JOB_TOKEN` automatically — but for best results, a dedicated token with `api` scope is recommended.
>
> **Tip:** For Project Access Tokens and Group Access Tokens, the token name determines the bot name shown in MR discussions. For example, naming your token `OpenCodeReview Bot` will make review comments appear as posted by `OpenCodeReview Bot`.

## Example Output

When an MR is reviewed, comments appear as:

- **Inline discussions**: Directly on the changed lines in the MR diff view, each prefixed with a `[category · severity]` badge when the LLM provided that metadata
- **Summary note**: A single note (updated in place across runs by default) summarizing the total number of issues found, with mutually-exclusive counts (inline / summary / routed / skipped / failed) and a detailed warnings list
- **Fallback notes**: Findings that could not be posted inline (no line info, routed by policy, or whose position could not be resolved against the diff) are collected into the summary note, each tagged with the reason it ended up there

### Inline Discussion Example

Comments are posted using GitLab's Discussion API with position data, so they appear directly next to the relevant code in the "Changes" tab. Each comment carries an HTML-comment id tag (invisible when rendered) that the script uses to avoid duplicate posts on retry.

## Supported LLM Providers

OCR supports both OpenAI and Anthropic API formats:

- **OpenAI-compatible APIs** (default):
  - OpenAI (GPT-4o, GPT-4, etc.)
  - Azure OpenAI
  - Self-hosted models (vLLM, Ollama, etc.)
- **Anthropic APIs** (modify `.gitlab-ci.yml` to set `use_anthropic: true`):
  - Anthropic Claude models

## Customization

Most review behavior is configurable via CI/CD Variables (see the table above) — no YAML edits required. The recipes below cover the remaining cases.

### Use a specific OCR version

Set the `OCR_VERSION` CI/CD variable (e.g. `1.8.8` or `~1.8`). Unset defaults to `latest`.

### Add custom review rules

Set the `OCR_RULE` CI/CD variable to the path of a custom rules JSON file (passed to `ocr review --rule`). For rules that live inside the repo, commit the file and point `OCR_RULE` at its path.

### Adjust retry and delay settings

When posting review discussions, the script includes rate-limit handling with exponential backoff (with jitter), `Retry-After` header support, and proactive throttling based on GitLab's `RateLimit-Remaining` response header. All API requests — including summary notes and MR version fetches — use the same retry logic. See [GitLab Rate Limits](https://docs.gitlab.com/security/rate_limits/) for details on GitLab's rate-limiting policies and recommended handling. You can configure the retry and delay behavior via **CI/CD Variables** (Settings → CI/CD → Variables):

| Variable | Default | Description |
|----------|---------|-------------|
| `OCR_RETRY_BASE_DELAY` | `2000` | Base delay (ms) for exponential backoff when a rate-limit error is hit |
| `OCR_MAX_RETRIES` | `3` | Maximum retry attempts per discussion when rate-limited |
| `OCR_MAX_RETRY_DELAY` | `60000` | Maximum delay (ms) per single retry, caps both `Retry-After` and backoff |
| `OCR_SUCCESS_DELAY` | `2000` | Delay (ms) after a successful discussion post to pace subsequent requests |
| `OCR_FAILURE_DELAY` | `1000` | Delay (ms) after a non-rate-limit failure to pace subsequent requests |
| `OCR_RATE_LIMIT_THRESHOLD` | `10` | Proactively slow down when GitLab `RateLimit-Remaining` is at/below this value (set `0` to disable). Applies to both writes (doubles the pacing delay) and reads (switches to the longer read spacing). |
| `OCR_READ_SUCCESS_DELAY` | `500` | Delay (ms) after a successful read (`list_notes`/`list_discussions`/`get_mr_diffs`) to pace read API calls, mirroring the GitHub Action's `readWithPacing`. |
| `OCR_READ_LOW_REMAINING_SPACING` | `5000` | Longer delay (ms) used after a read when the remaining quota is at/below `OCR_RATE_LIMIT_THRESHOLD`. |
| `OCR_ROUTE_SEVERITY_BELOW` | _(empty)_ | Optional severity threshold (`critical`, `high`, `medium`, `low`) that routes findings at-or-below it from inline comments to summary notes (fail-open: never drops a finding). Empty or unknown values disable severity routing. |
| `OCR_ROUTE_CATEGORIES` | _(empty)_ | Optional comma-separated categories (`bug`, `security`, `performance`, `maintainability`, `test`, `style`, `documentation`, `other`) routed from inline to summary notes. Unknown tokens are ignored. Combine with `OCR_ROUTE_SEVERITY_BELOW` to route on either condition. |

These variables are optional — if not configured, sensible defaults are used. Consider increasing delays for self-hosted GitLab instances with aggressive rate-limit configurations or for large MRs that generate numerous review comments. The `OCR_RATE_LIMIT_THRESHOLD` variable enables proactive throttling: when GitLab reports low remaining quota in the `RateLimit-Remaining` response header, the script automatically doubles the pacing delay to avoid hitting 429 errors.

When the primary rate limit is exhausted (`RateLimit-Remaining: 0`), the script additionally honors GitLab's `RateLimit-Reset` header, sleeping until the limit resets (defensively handling both epoch-seconds and seconds-until-reset formats). This mirrors the GitHub Action's `x-ratelimit-reset` handling.

### Publication behavior (aligned with the GitHub Action)

The posting script implements the same publication behaviors as the GitHub Action (`scripts/github-actions/post-review-comments.js`), exposed as optional CI/CD variables. All default to no-op / current behavior except `OCR_STICKY_SUMMARY`, which defaults to `true` (the summary note is updated in place across runs instead of accumulating one note per run).

| Variable | Default | Description |
|----------|---------|-------------|
| `OCR_STICKY_SUMMARY` | `true` | Update the summary note in place across runs (find-by-marker then `PUT /notes/:id`). Set `false` to post a fresh note each run. |
| `OCR_INCREMENTAL` | `false` | Skip comments whose line range overlaps a prior bot discussion on the same path (so re-runs only append new findings). Uses an IoU threshold for multi-line spans. |
| `OCR_INCREMENTAL_OVERLAP_THRESHOLD` | `0.6` | IoU (0–1) above which two multi-line comments are considered the same. Single-line comments match only on the exact same line; a single-line and multi-line comment never match. |
| `OCR_ROUTE_SEVERITY_BELOW` | _(empty)_ | Route findings whose severity is at-or-below this level to the summary note instead of posting them inline (e.g. `low` keeps only critical/high/medium inline). Unknown severities never match (fail-open). |
| `OCR_ROUTE_CATEGORIES` | _(empty)_ | Comma-separated categories to route to the summary (e.g. `style,documentation`). Routed findings never enter the inline write path, so they cannot be double-posted on retry. |
| `OCR_FAIL_ON_SEVERITY` | _(empty)_ | Fail the CI job (non-zero exit) when any comment's severity is at-or-above this level (e.g. `critical`). The summary note is still posted before the job fails, so visibility is preserved. |

Additional behaviors ported from the GitHub Action (always on, no variable):

- **Category/severity badge**: each inline discussion and fallback entry is prefixed with `[category · severity]` (using a middot, U+00B7) when the LLM provided that metadata, byte-matching the CLI's badge rendering.
- **Deterministic ordering**: comments are sorted by `path → start_line → end_line → original index` before posting, so identical reruns produce identical post sequences (which is what makes idempotency reconciliation reproducible).
- **Detailed warnings**: warnings are rendered as a bulleted list (`file (type): message`) rather than a bare count.
- **Backtick-safe fences**: fallback Before/After code blocks use a fence long enough to enclose any backticks in the code (the inline suggestion keeps the fixed triple-backtick `suggestion:-0+0` form so GitLab's "Apply suggestion" button keeps working).
- **Idempotent retry**: each inline discussion carries an invisible HTML-comment id tag (`<!-- ocr-<pipeline>-<job>-<hex> -->`). When a `POST /discussions` fails with a 5xx/408/network error (the request may still have landed), the script queries existing discussions for the id before retrying; if found it is treated as success (no duplicate), and if the read API is unavailable it skips the retry rather than risk a duplicate.
- **400 line-resolution fallback**: when a discussion `POST` returns 400 indicating a position problem, the script fetches the MR diff (`GET /merge_requests/:iid/diffs`), classifies the comment as valid/invalid/unknown, and drops provably-out-of-diff findings to the summary rather than blindly retrying. Unknown cases keep the existing fallback behavior.

### CI outputs and artifacts

The pipeline exposes the following to downstream jobs:

- **dotenv report** (`.ocr/ocr-stats.env`): `OCR_COMMENTS_TOTAL`, `OCR_COMMENTS_INLINE`, `OCR_COMMENTS_SUMMARY`, `OCR_COMMENTS_ROUTED`, `OCR_COMMENTS_SKIPPED`, `OCR_COMMENTS_FAILED`, and `OCR_SUMMARY_URL`. Consume them in later stages via the `dotenv` artifact.
- **Artifacts** (`when: always`, 1 week retention): `.ocr/ocr-result.json` (raw review JSON) and `.ocr/ocr-stderr.log` (OCR stderr), so you can inspect a failed review even when the job fails. Paths are project-relative (under `.ocr/`) because GitLab Runner refuses to upload artifacts outside the build directory.

### Limit concurrency

Set the `OCR_REVIEW_CONCURRENCY` CI/CD variable (passed to `ocr review --concurrency`). For large MRs, lowering this caps concurrent LLM requests:

```yaml
variables:
  OCR_REVIEW_CONCURRENCY: "5"
```

### Provide background context

Set the `OCR_BACKGROUND` CI/CD variable (passed to `ocr review --background`). A common choice is the MR title:

```yaml
variables:
  OCR_BACKGROUND: "$CI_MERGE_REQUEST_TITLE"
```

This is particularly useful when your MR titles follow semantic conventions (e.g., `feat(auth): add OAuth2 support`) that clearly summarize what the MR implements. The background information helps OCR provide more relevant and context-aware review comments.

### Change the trigger events

By default, the pipeline uses `only: [merge_requests]`, which triggers on **all** MR events (creation, updates, reopen). GitLab CI does not natively support fine-grained control to trigger **only on MR creation**.

To avoid re-reviewing on every push to an existing MR (and wasting LLM API tokens), you can check for existing OCR reviews **before** running `ocr review`. Use a wrapper script that skips the review step if OCR comments already exist:

```yaml
script:
  # Install OpenCodeReview
  - npm install -g @alibaba-group/open-code-review

  # Configure OCR
  - |
    : "${OCR_LLM_URL:?set OCR_LLM_URL in Settings -> CI/CD -> Variables}"
    : "${OCR_LLM_AUTH_TOKEN:?set OCR_LLM_AUTH_TOKEN in Settings -> CI/CD -> Variables}"
    : "${OCR_LLM_MODEL:?set OCR_LLM_MODEL in Settings -> CI/CD -> Variables}"
    ocr config set llm.url "$OCR_LLM_URL"
    ocr config set llm.auth_token "$OCR_LLM_AUTH_TOKEN"
    ocr config set llm.model "$OCR_LLM_MODEL"
    ocr config set llm.use_anthropic false
    ocr config set llm.extra_body '{"thinking": {"type": "disabled"}}'

  # Check for existing OCR reviews and run review only if not found
  - |
    python3 << 'WRAPPER_SCRIPT'
    import json
    import os
    import subprocess
    import sys
    import urllib.request

    GITLAB_URL = os.environ.get("CI_SERVER_URL", "https://gitlab.com")
    PROJECT_ID = os.environ["CI_PROJECT_ID"]
    MR_IID = os.environ["CI_MERGE_REQUEST_IID"]
    API_TOKEN = os.environ["GITLAB_API_TOKEN"]
    SOURCE_BRANCH = os.environ["CI_MERGE_REQUEST_SOURCE_BRANCH_NAME"]
    TARGET_BRANCH = os.environ["CI_MERGE_REQUEST_TARGET_BRANCH_NAME"]

    # Check for existing OCR reviews
    url = f"{GITLAB_URL}/api/v4/projects/{PROJECT_ID}/merge_requests/{MR_IID}/notes?per_page=100"
    req = urllib.request.Request(url, headers={"PRIVATE-TOKEN": API_TOKEN})
    with urllib.request.urlopen(req) as resp:
        notes = json.loads(resp.read().decode("utf-8"))

    for note in notes:
        if "OpenCodeReview" in note.get("body", ""):
            print("⏭️ OCR has already reviewed this MR. Skipping to save tokens.")
            print("Delete previous OCR comments to re-trigger review.")
            sys.exit(0)

    # No existing review found - run OCR
    print("🔍 No existing OCR review found. Running review...")
    COMMIT_SHA = os.environ["CI_COMMIT_SHA"]
    result = subprocess.run([
        "ocr", "review",
        "--from", f"origin/{TARGET_BRANCH}",
        "--to", COMMIT_SHA,
        "--format", "json",
        "--audience", "agent"
    ], capture_output=True, text=True)

    # Save output for the posting script
    os.makedirs(".ocr", exist_ok=True)
    with open(".ocr/ocr-result.json", "w") as f:
        f.write(result.stdout)
    with open(".ocr/ocr-stderr.log", "w") as f:
        f.write(result.stderr)

    print("OCR review completed.")
    WRAPPER_SCRIPT

  # Post review comments to MR
  - python3 post_review.py .ocr/ocr-result.json
```

The key logic: the Python wrapper checks for existing OCR comments before running `ocr review`. If found, it exits early with `sys.exit(0)` before consuming any LLM tokens. To re-trigger a review, users can manually delete the previous OCR comments.

### Self-hosted GitLab

The script automatically uses `CI_SERVER_URL` to determine the GitLab API base URL, so it works with self-hosted GitLab instances out of the box.

### Use a Service Account as Review Bot

By default, review comments are posted using the user who owns the access token configured in `GITLAB_API_TOKEN`. You can create a dedicated service account bot to post reviews with a custom identity, making it easier to distinguish automated reviews from human comments.

For more details about GitLab service accounts, see the [GitLab Service Accounts documentation](https://docs.gitlab.com/ee/user/profile/service_accounts.html).

#### Step 1: Create a Service Account

Create a service account in your project:

1. Go to your **Project → Settings → Service Accounts**
2. Click **New service account**
3. Fill in the following:
   - **Name**: e.g., `OpenCodeReview Bot` (this will be the bot name shown in MR discussions)
   - **Username**: Will be auto-generated based on the name
4. Click **Create service account**

#### Step 2: Invite the Service Account to Your Project

After the service account is created, invite it to your project with appropriate permissions:

1. Go to your **Project → Settings → Members**
2. Click **Invite member**
3. Search for the service account by name (e.g., `OpenCodeReview Bot`)
4. Select the service account and assign a role (`Developer` or `Maintainer` required for posting discussions)
5. Click **Invite**

#### Step 3: Create an Access Token

Generate an access token for the service account:

1. Go to your **Project → Settings → Service Accounts**
2. Click on the service account to view its details
3. Click **Add new token**
4. Configure the token:
   - **Name**: e.g., `ocr-review-token`
   - **Expiration**: As needed
   - **Scope**: Select `api` (required for Discussions API)
5. Click **Create token** and copy the token value

#### Step 4: Update CI/CD Variables

Update the `GITLAB_API_TOKEN` variable in your project's CI/CD settings:

Go to **Settings → CI/CD → Variables** and update `GITLAB_API_TOKEN` with the service account's token.

Now review comments will be posted with your service account identity (e.g., `OpenCodeReview Bot`), providing a clear and professional appearance for automated code reviews.

## Troubleshooting

### Common Issues

1. **"API error 403"**: The `GITLAB_API_TOKEN` lacks `api` scope or doesn't have access to the project
2. **"Failed to parse OCR output"**: Check that `OCR_LLM_URL` and `OCR_LLM_AUTH_TOKEN` variables are correctly set
3. **"Cannot find merge-base"**: Ensure `GIT_DEPTH: 0` is set (full clone)
4. **Inline comments on wrong lines**: GitLab requires exact SHA matching; the script fetches MR version metadata to get correct diff refs

### Debugging

Add verbose output to the review step:

```yaml
script:
  - cat .ocr/ocr-result.json
  - cat .ocr/ocr-stderr.log
```

## Testing

The posting logic is unit-tested with no network access and no wall-clock sleep cost. Tests use only the standard-library `unittest` and cover: comment formatting (badge, suggestion, fallback), the publication policy (severity/category routing), deterministic sort, warning rendering, safe-fence code blocks, the full `publish()` flow (partition → sticky summary → fallback), incremental overlap (IoU, single-vs-multi, bot detection), the GitLab transport (retry/backoff/jitter/`Retry-After`/rate-limit/auth/network errors), idempotent `post_discussion` reconciliation, the 400 line-resolution fallback classification, wait-until-reset, and the dotenv stats / severity-gating helpers.

```bash
# From the example directory
cd examples/gitlab_ci
python3 post_review_test.py

# From the repo root
python3 -m unittest discover -s examples/gitlab_ci -p '*_test.py'
```


