nginx-logtail/PLAN_FRONTEND.md

# Frontend v0 — Implementation Plan

Module path: `git.ipng.ch/ipng/nginx-logtail`

**Scope:** An HTTP server that queries a collector or aggregator and renders a drilldown TopN
dashboard with trend sparklines. Zero JavaScript. Filter state in the URL. Auto-refreshes every
30 seconds. Works with any `LogtailService` endpoint (collector or aggregator).

---

## Overview

Single page, multiple views driven entirely by URL query parameters:

```
http://frontend:8080/?target=agg:9091&w=5m&by=website&f_status=429&n=25
```

Clicking a table row drills down: it adds a filter for the clicked label and advances
`by` to the next dimension in the hierarchy (`website → prefix → uri → status`). The
breadcrumb strip shows all active filters; each token is a link that removes it.

---

## Step 1 — main.go

Flags:

| Flag | Default | Description |
|------|---------|-------------|
| `--listen` | `:8080` | HTTP listen address |
| `--target` | `localhost:9091` | Default gRPC endpoint (aggregator or collector) |
| `--n` | `25` | Default number of table rows |
| `--refresh` | `30` | `<meta refresh>` interval in seconds; 0 to disable |

Wire-up:
1. Parse flags
2. Register `http.HandleFunc("/", handler)` (single handler, all state in URL)
3. `http.ListenAndServe`
4. `signal.NotifyContext` for clean shutdown on SIGINT/SIGTERM

---

## Step 2 — client.go

```go
func dial(addr string) (*grpc.ClientConn, pb.LogtailServiceClient, error)
```

Identical to the CLI version — plain insecure dial. A new connection is opened per HTTP
request. At a 30-second page refresh rate this is negligible; pooling is not needed.

---

## Step 3 — handler.go

### URL parameters

| Param | Default | Values |
|-------|---------|--------|
| `target` | flag default | `host:port` |
| `w` | `5m` | `1m 5m 15m 60m 6h 24h` |
| `by` | `website` | `website prefix uri status` |
| `n` | flag default | positive integer |
| `f_website` | — | string |
| `f_prefix` | — | string |
| `f_uri` | — | string |
| `f_status` | — | integer string |
| `raw` | — | `1` → respond with JSON instead of HTML |

### Request flow

```
parseURLParams(r) → QueryParams
buildFilter(QueryParams) → *pb.Filter
dial(target) → client
concurrent:
  client.TopN(filter, groupBy, n, window)   → TopNResponse
  client.Trend(filter, window)              → TrendResponse
renderSparkline(TrendResponse.Points)       → template.HTML
buildTableRows(TopNResponse, QueryParams)   → []TableRow  (includes drill-down URL per row)
buildBreadcrumbs(QueryParams)               → []Crumb
execute template → w
```

TopN and Trend RPCs are issued concurrently (both have a 5 s context deadline). If Trend
fails, the sparkline is omitted silently rather than returning an error page.

### `raw=1` mode

Returns the TopN response as JSON (same format as the CLI's `--json`). Useful for scripting
and `curl` without needing the CLI binary.

### Drill-down URL construction

Dimension advance hierarchy (for row-click links):

```
website → CLIENT_PREFIX → REQUEST_URI → HTTP_RESPONSE → (no advance; all dims filtered)
```

Row-click URL: take current params, add the filter for the current `by` dimension, and set
`by` to the next dimension. If already on the last dimension (`status`), keep `by` unchanged.

### Types

```go
type QueryParams struct {
    Target   string
    Window   pb.Window
    WindowS  string   // "5m" — for display
    GroupBy  pb.GroupBy
    GroupByS string   // "website" — for display
    N        int
    Filter   filterState
}

type filterState struct {
    Website string
    Prefix  string
    URI     string
    Status  string   // string so empty means "unset"
}

type TableRow struct {
    Rank      int
    Label     string
    Count     int64
    Pct       float64   // 0–100, relative to top entry
    DrillURL  string    // href for this row
}

type Crumb struct {
    Text      string   // e.g. "website=example.com"
    RemoveURL string   // current URL with this filter removed
}

type PageData struct {
    Params      QueryParams
    Source      string
    Entries     []TableRow
    TotalCount  int64
    Sparkline   template.HTML   // "" if trend call failed
    Breadcrumbs []Crumb
    RefreshSecs int
    Error       string          // non-empty → show error banner, no table
}
```

---

## Step 4 — sparkline.go

```go
func renderSparkline(points []*pb.TrendPoint) template.HTML
```

- Fixed `viewBox="0 0 300 60"` SVG.
- X axis: evenly-spaced buckets across 300 px.
- Y axis: linear scale from 0 to max count, inverted (SVG y=0 is top).
- Rendered as a `<polyline>` with `stroke` and `fill="none"`. Minimal inline style, no classes.
- If `len(points) < 2`, returns `""` (no sparkline).
- Returns `template.HTML` (already-escaped) so the template can emit it with `{{.Sparkline}}`.

---

## Step 5 — templates/

Two files, embedded with `//go:embed templates/*.html` and parsed once at startup.

### `templates/base.html` (define "base")

Outer HTML skeleton:
- `<meta http-equiv="refresh" content="30">` (omitted if `RefreshSecs == 0`)
- Minimal inline CSS: monospace font, max-width 1000px, table styling, breadcrumb strip
- Yields a `{{template "content" .}}` block

No external CSS, no web fonts, no icons. Legible in a terminal browser (w3m, lynx).

### `templates/index.html` (define "content")

Sections in order:

**Window tabs** — `1m | 5m | 15m | 60m | 6h | 24h`; current window is bold/underlined;
each is a link that swaps only `w=` in the URL.

**Group-by tabs** — `by website | by prefix | by uri | by status`; current group highlighted;
links swap `by=`.

**Filter breadcrumb** — shown only when at least one filter is active:
```
Filters: [website=example.com ×] [status=429 ×]
```
Each `×` is a link to the URL without that filter.

**Error banner** — shown instead of table when `.Error` is non-empty.

**Trend sparkline** — the SVG returned by `renderSparkline`, inline. Labelled with window
and source. Omitted when `.Sparkline == ""`.

**TopN table**:
```
RANK  LABEL               COUNT     %      TREND
   1  example.com        18 432    62 %   ████████████
   2  other.com           4 211    14 %   ████
```
- `LABEL` column is a link (`DrillURL`).
- `%` is relative to the top entry (rank-1 always 100 %).
- `TREND` bar is an inline `<meter value="N" max="100">` tag — renders as a native browser bar,
  degrades gracefully in text browsers to `N/100`.
- Rows beyond rank 3 show the percentage bar only if it's > 5 %, to avoid noise.

**Footer** — "source: <source>  queried <timestamp>  refresh 30 s" — lets operators confirm
which endpoint they're looking at.

---

## Step 6 — Tests (`frontend_test.go`)

In-process fake gRPC server (same pattern as aggregator and CLI tests).

| Test | What it covers |
|------|----------------|
| `TestParseQueryParams` | All URL params parsed correctly; defaults applied |
| `TestParseQueryParamsInvalid` | Bad `n`, bad `w`, bad `f_status` → defaults or 400 |
| `TestBuildFilterFromParams` | Populated filter; nil when nothing set |
| `TestDrillURL` | website → prefix drill; prefix → uri drill; status → no advance |
| `TestBuildCrumbs` | One crumb per active filter; remove-URL drops just that filter |
| `TestRenderSparkline` | 5 points → valid SVG containing `<polyline`; 0 points → empty |
| `TestHandlerTopN` | Fake server; GET / returns 200 with table rows in body |
| `TestHandlerRaw` | `raw=1` returns JSON with correct entries |
| `TestHandlerBadTarget` | Unreachable target → 502 with error message |
| `TestHandlerFilter` | `f_website=x` passed through to fake server's received request |
| `TestHandlerWindow` | `w=60m` → correct `pb.Window_W60M` in fake server's received request |
| `TestPctBar` | `<meter` tag present in rendered HTML |
| `TestBreadcrumbInHTML` | Filter crumb rendered; `×` link present |

---

## Step 7 — Smoke test

```bash
# Start collector and aggregator (or use existing)
./logtail-collector --listen :9090 --logs /var/log/nginx/access.log
./logtail-aggregator --listen :9091 --collectors localhost:9090

# Start frontend
./logtail-frontend --listen :8080 --target localhost:9091

# Open in browser or curl
curl -s 'http://localhost:8080/' | grep '<tr'
curl -s 'http://localhost:8080/?w=60m&by=prefix&f_status=200&raw=1' | jq '.entries[0]'

# Drill-down link check
curl -s 'http://localhost:8080/' | grep 'f_website'
```

---

## ✓ COMPLETE — Implementation notes

### Files

| File | Role |
|------|------|
| `cmd/frontend/main.go` | Flags, template loading, HTTP server, graceful shutdown |
| `cmd/frontend/client.go` | `dial()` — plain insecure gRPC, new connection per request |
| `cmd/frontend/handler.go` | URL parsing, filter building, concurrent TopN+Trend fan-out, page data assembly |
| `cmd/frontend/sparkline.go` | `renderSparkline()` — `[]*pb.TrendPoint` → inline `<svg><polyline>` |
| `cmd/frontend/format.go` | `fmtCount()` — space-separated thousands, registered as template func |
| `cmd/frontend/templates/base.html` | Outer HTML shell, inline CSS, meta-refresh |
| `cmd/frontend/templates/index.html` | Window tabs, group-by tabs, breadcrumb, sparkline, table, footer |

### Deviations from the plan

- **`format.go` extracted**: `fmtCount` placed in its own file (not in `handler.go`) so it can
  be tested independently without loading the template.
- **`TestDialFake` added**: sanity check for the fake gRPC infrastructure used by the other tests.
- **`TestHandlerNoData` added**: verifies the "no data" message renders correctly when the server
  returns an empty entry list. Total tests: 23 (plan listed 13).
- **`% relative to rank-1`** as planned; the `<meter max="100">` shows 100% for rank-1
  and proportional bars below. Rank-1 is always the visual baseline.
- **`status → website` drill cycle**: clicking a row in the `by status` view adds `f_status`
  and resets `by=website` (cycles back to the start of the drilldown hierarchy).

### Test results

```
$ go test ./... -count=1 -race -timeout 60s
ok  git.ipng.ch/ipng/nginx-logtail/cmd/frontend   1.1s   (23 tests)
ok  git.ipng.ch/ipng/nginx-logtail/cmd/cli        1.0s   (14 tests)
ok  git.ipng.ch/ipng/nginx-logtail/cmd/aggregator 4.1s   (13 tests)
ok  git.ipng.ch/ipng/nginx-logtail/cmd/collector  9.7s   (17 tests)
```

### Test inventory

| Test | What it covers |
|------|----------------|
| `TestParseWindowString` | All 6 window strings + bad input → default |
| `TestParseGroupByString` | All 4 group-by strings + bad input → default |
| `TestParseQueryParams` | All URL params parsed correctly |
| `TestParseQueryParamsDefaults` | Empty URL → handler defaults applied |
| `TestBuildFilter` | Filter proto fields set from filterState |
| `TestBuildFilterNil` | Returns nil when no filter set |
| `TestDrillURL` | website→prefix, prefix→uri, status→website cycle |
| `TestBuildCrumbs` | Correct text and remove-URLs for active filters |
| `TestRenderSparkline` | 5 points → SVG with polyline |
| `TestRenderSparklineTooFewPoints` | nil/1 point → empty string |
| `TestRenderSparklineAllZero` | All-zero counts → empty string |
| `TestFmtCount` | Space-thousands formatting |
| `TestHandlerTopN` | Fake server; labels and formatted counts in HTML |
| `TestHandlerRaw` | `raw=1` → JSON with source/window/group_by/entries |
| `TestHandlerBadTarget` | Unreachable target → 502 + error message in body |
| `TestHandlerFilterPassedToServer` | `f_website` + `f_status` reach gRPC filter |
| `TestHandlerWindowPassedToServer` | `w=60m` → `pb.Window_W60M` in request |
| `TestHandlerBreadcrumbInHTML` | Active filter renders crumb with × link |
| `TestHandlerSparklineInHTML` | Trend points → `<svg><polyline>` in page |
| `TestHandlerPctBar` | 100% for rank-1, 50% for half-count entry |
| `TestHandlerWindowTabsInHTML` | All 6 window labels rendered as links |
| `TestHandlerNoData` | Empty entry list → "no data" message |
| `TestDialFake` | Test infrastructure sanity check |

---

## Deferred (not in v0)

- Dark mode (prefers-color-scheme media query)
- Per-row mini sparklines (one Trend RPC per table row — expensive; need batching first)
- WebSocket or SSE for live push instead of meta-refresh
- Pagination for large N
- `?format=csv` download
- OIDC/basic-auth gating
- ClickHouse-backed 7d/30d windows (tracked in README)