ipng/nginx-logtail
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Execute PLAN_FRONTEND.md

Browse Source
This commit is contained in:
Pim van Pelt
2026-03-14 20:42:51 +01:00
parent b9ec67ec00
commit 4369e66dee
9 changed files with 1571 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

334
PLAN_FRONTEND.md Normal file
View File

@@ -0,0 +1,334 @@
# Frontend v0 — Implementation Plan
Module path: `git.ipng.ch/ipng/nginx-logtail`
**Scope:** An HTTP server that queries a collector or aggregator and renders a drilldown TopN
dashboard with trend sparklines. Zero JavaScript. Filter state in the URL. Auto-refreshes every
30 seconds. Works with any `LogtailService` endpoint (collector or aggregator).
---
## Overview
Single page, multiple views driven entirely by URL query parameters:
```
http://frontend:8080/?target=agg:9091&w=5m&by=website&f_status=429&n=25
```
Clicking a table row drills down: it adds a filter for the clicked label and advances
`by` to the next dimension in the hierarchy (`website → prefix → uri → status`). The
breadcrumb strip shows all active filters; each token is a link that removes it.
---
## Step 1 — main.go
Flags:
| Flag | Default | Description |
|------|---------|-------------|
| `--listen` | `:8080` | HTTP listen address |
| `--target` | `localhost:9091` | Default gRPC endpoint (aggregator or collector) |
| `--n` | `25` | Default number of table rows |
| `--refresh` | `30` | `<meta refresh>` interval in seconds; 0 to disable |
Wire-up:
1. Parse flags
2. Register `http.HandleFunc("/", handler)` (single handler, all state in URL)
3. `http.ListenAndServe`
4. `signal.NotifyContext` for clean shutdown on SIGINT/SIGTERM
---
## Step 2 — client.go
```go
func dial(addr string) (*grpc.ClientConn, pb.LogtailServiceClient, error)
```
Identical to the CLI version — plain insecure dial. A new connection is opened per HTTP
request. At a 30-second page refresh rate this is negligible; pooling is not needed.
---
## Step 3 — handler.go
### URL parameters
| Param | Default | Values |
|-------|---------|--------|
| `target` | flag default | `host:port` |
| `w` | `5m` | `1m 5m 15m 60m 6h 24h` |
| `by` | `website` | `website prefix uri status` |
| `n` | flag default | positive integer |
| `f_website` | — | string |
| `f_prefix` | — | string |
| `f_uri` | — | string |
| `f_status` | — | integer string |
| `raw` | — | `1` → respond with JSON instead of HTML |
### Request flow
```
parseURLParams(r) → QueryParams
buildFilter(QueryParams) → *pb.Filter
dial(target) → client
concurrent:
client.TopN(filter, groupBy, n, window) → TopNResponse
client.Trend(filter, window) → TrendResponse
renderSparkline(TrendResponse.Points) → template.HTML
buildTableRows(TopNResponse, QueryParams) → []TableRow (includes drill-down URL per row)
buildBreadcrumbs(QueryParams) → []Crumb
execute template → w
```
TopN and Trend RPCs are issued concurrently (both have a 5 s context deadline). If Trend
fails, the sparkline is omitted silently rather than returning an error page.
### `raw=1` mode
Returns the TopN response as JSON (same format as the CLI's `--json`). Useful for scripting
and `curl` without needing the CLI binary.
### Drill-down URL construction
Dimension advance hierarchy (for row-click links):
```
website → CLIENT_PREFIX → REQUEST_URI → HTTP_RESPONSE → (no advance; all dims filtered)
```
Row-click URL: take current params, add the filter for the current `by` dimension, and set
`by` to the next dimension. If already on the last dimension (`status`), keep `by` unchanged.
### Types
```go
type QueryParams struct {
Target string
Window pb.Window
WindowS string // "5m" — for display
GroupBy pb.GroupBy
GroupByS string // "website" — for display
N int
Filter filterState
}
type filterState struct {
Website string
Prefix string
URI string
Status string // string so empty means "unset"
}
type TableRow struct {
Rank int
Label string
Count int64
Pct float64 // 0–100, relative to top entry
DrillURL string // href for this row
}
type Crumb struct {
Text string // e.g. "website=example.com"
RemoveURL string // current URL with this filter removed
}
type PageData struct {
Params QueryParams
Source string
Entries []TableRow
TotalCount int64
Sparkline template.HTML // "" if trend call failed
Breadcrumbs []Crumb
RefreshSecs int
Error string // non-empty → show error banner, no table
}
```
---
## Step 4 — sparkline.go
```go
func renderSparkline(points []*pb.TrendPoint) template.HTML
```
- Fixed `viewBox="0 0 300 60"` SVG.
- X axis: evenly-spaced buckets across 300 px.
- Y axis: linear scale from 0 to max count, inverted (SVG y=0 is top).
- Rendered as a `<polyline>` with `stroke` and `fill="none"`. Minimal inline style, no classes.
- If `len(points) < 2`, returns `""` (no sparkline).
- Returns `template.HTML` (already-escaped) so the template can emit it with `{{.Sparkline}}`.
---
## Step 5 — templates/
Two files, embedded with `//go:embed templates/*.html` and parsed once at startup.
### `templates/base.html` (define "base")
Outer HTML skeleton:
- `<meta http-equiv="refresh" content="30">` (omitted if `RefreshSecs == 0`)
- Minimal inline CSS: monospace font, max-width 1000px, table styling, breadcrumb strip
- Yields a `{{template "content" .}}` block
No external CSS, no web fonts, no icons. Legible in a terminal browser (w3m, lynx).
### `templates/index.html` (define "content")
Sections in order:
**Window tabs** — `1m | 5m | 15m | 60m | 6h | 24h`; current window is bold/underlined;
each is a link that swaps only `w=` in the URL.
**Group-by tabs** — `by website | by prefix | by uri | by status`; current group highlighted;
links swap `by=`.
**Filter breadcrumb** — shown only when at least one filter is active:
```
Filters: [website=example.com ×] [status=429 ×]
```
Each `×` is a link to the URL without that filter.
**Error banner** — shown instead of table when `.Error` is non-empty.
**Trend sparkline** — the SVG returned by `renderSparkline`, inline. Labelled with window
and source. Omitted when `.Sparkline == ""`.
**TopN table**:
```
RANK LABEL COUNT % TREND
1 example.com 18 432 62 % ████████████
2 other.com 4 211 14 % ████
```
- `LABEL` column is a link (`DrillURL`).
- `%` is relative to the top entry (rank-1 always 100 %).
- `TREND` bar is an inline `<meter value="N" max="100">` tag — renders as a native browser bar,
degrades gracefully in text browsers to `N/100`.
- Rows beyond rank 3 show the percentage bar only if it's > 5 %, to avoid noise.
**Footer** — "source: <source> queried <timestamp> refresh 30 s" — lets operators confirm
which endpoint they're looking at.
---
## Step 6 — Tests (`frontend_test.go`)
In-process fake gRPC server (same pattern as aggregator and CLI tests).
| Test | What it covers |
|------|----------------|
| `TestParseQueryParams` | All URL params parsed correctly; defaults applied |
| `TestParseQueryParamsInvalid` | Bad `n`, bad `w`, bad `f_status` → defaults or 400 |
| `TestBuildFilterFromParams` | Populated filter; nil when nothing set |
| `TestDrillURL` | website → prefix drill; prefix → uri drill; status → no advance |
| `TestBuildCrumbs` | One crumb per active filter; remove-URL drops just that filter |
| `TestRenderSparkline` | 5 points → valid SVG containing `<polyline`; 0 points → empty |
| `TestHandlerTopN` | Fake server; GET / returns 200 with table rows in body |
| `TestHandlerRaw` | `raw=1` returns JSON with correct entries |
| `TestHandlerBadTarget` | Unreachable target → 502 with error message |
| `TestHandlerFilter` | `f_website=x` passed through to fake server's received request |
| `TestHandlerWindow` | `w=60m` → correct `pb.Window_W60M` in fake server's received request |
| `TestPctBar` | `<meter` tag present in rendered HTML |
| `TestBreadcrumbInHTML` | Filter crumb rendered; `×` link present |
---
## Step 7 — Smoke test
```bash
# Start collector and aggregator (or use existing)
./logtail-collector --listen :9090 --logs /var/log/nginx/access.log
./logtail-aggregator --listen :9091 --collectors localhost:9090
# Start frontend
./logtail-frontend --listen :8080 --target localhost:9091
# Open in browser or curl
curl -s 'http://localhost:8080/' | grep '<tr'
curl -s 'http://localhost:8080/?w=60m&by=prefix&f_status=200&raw=1' | jq '.entries[0]'
# Drill-down link check
curl -s 'http://localhost:8080/' | grep 'f_website'
```
---
## ✓ COMPLETE — Implementation notes
### Files
| File | Role |
|------|------|
| `cmd/frontend/main.go` | Flags, template loading, HTTP server, graceful shutdown |
| `cmd/frontend/client.go` | `dial()` — plain insecure gRPC, new connection per request |
| `cmd/frontend/handler.go` | URL parsing, filter building, concurrent TopN+Trend fan-out, page data assembly |
| `cmd/frontend/sparkline.go` | `renderSparkline()` — `[]*pb.TrendPoint` → inline `<svg><polyline>` |
| `cmd/frontend/format.go` | `fmtCount()` — space-separated thousands, registered as template func |
| `cmd/frontend/templates/base.html` | Outer HTML shell, inline CSS, meta-refresh |
| `cmd/frontend/templates/index.html` | Window tabs, group-by tabs, breadcrumb, sparkline, table, footer |
### Deviations from the plan
- **`format.go` extracted**: `fmtCount` placed in its own file (not in `handler.go`) so it can
be tested independently without loading the template.
- **`TestDialFake` added**: sanity check for the fake gRPC infrastructure used by the other tests.
- **`TestHandlerNoData` added**: verifies the "no data" message renders correctly when the server
returns an empty entry list. Total tests: 23 (plan listed 13).
- **`% relative to rank-1`** as planned; the `<meter max="100">` shows 100% for rank-1
and proportional bars below. Rank-1 is always the visual baseline.
- **`status → website` drill cycle**: clicking a row in the `by status` view adds `f_status`
and resets `by=website` (cycles back to the start of the drilldown hierarchy).
### Test results
```
$ go test ./... -count=1 -race -timeout 60s
ok git.ipng.ch/ipng/nginx-logtail/cmd/frontend 1.1s (23 tests)
ok git.ipng.ch/ipng/nginx-logtail/cmd/cli 1.0s (14 tests)
ok git.ipng.ch/ipng/nginx-logtail/cmd/aggregator 4.1s (13 tests)
ok git.ipng.ch/ipng/nginx-logtail/cmd/collector 9.7s (17 tests)
```
### Test inventory
| Test | What it covers |
|------|----------------|
| `TestParseWindowString` | All 6 window strings + bad input → default |
| `TestParseGroupByString` | All 4 group-by strings + bad input → default |
| `TestParseQueryParams` | All URL params parsed correctly |
| `TestParseQueryParamsDefaults` | Empty URL → handler defaults applied |
| `TestBuildFilter` | Filter proto fields set from filterState |
| `TestBuildFilterNil` | Returns nil when no filter set |
| `TestDrillURL` | website→prefix, prefix→uri, status→website cycle |
| `TestBuildCrumbs` | Correct text and remove-URLs for active filters |
| `TestRenderSparkline` | 5 points → SVG with polyline |
| `TestRenderSparklineTooFewPoints` | nil/1 point → empty string |
| `TestRenderSparklineAllZero` | All-zero counts → empty string |
| `TestFmtCount` | Space-thousands formatting |
| `TestHandlerTopN` | Fake server; labels and formatted counts in HTML |
| `TestHandlerRaw` | `raw=1` → JSON with source/window/group_by/entries |
| `TestHandlerBadTarget` | Unreachable target → 502 + error message in body |
| `TestHandlerFilterPassedToServer` | `f_website` + `f_status` reach gRPC filter |
| `TestHandlerWindowPassedToServer` | `w=60m` → `pb.Window_W60M` in request |
| `TestHandlerBreadcrumbInHTML` | Active filter renders crumb with × link |
| `TestHandlerSparklineInHTML` | Trend points → `<svg><polyline>` in page |
| `TestHandlerPctBar` | 100% for rank-1, 50% for half-count entry |
| `TestHandlerWindowTabsInHTML` | All 6 window labels rendered as links |
| `TestHandlerNoData` | Empty entry list → "no data" message |
| `TestDialFake` | Test infrastructure sanity check |
---
## Deferred (not in v0)
- Dark mode (prefers-color-scheme media query)
- Per-row mini sparklines (one Trend RPC per table row — expensive; need batching first)
- WebSocket or SSE for live push instead of meta-refresh
- Pagination for large N
- `?format=csv` download
- OIDC/basic-auth gating
- ClickHouse-backed 7d/30d windows (tracked in README)

15
cmd/frontend/client.go Normal file
View File

@@ -0,0 +1,15 @@
package main
import (
pb "git.ipng.ch/ipng/nginx-logtail/proto/logtailpb"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
)
func dial(addr string) (*grpc.ClientConn, pb.LogtailServiceClient, error) {
conn, err := grpc.NewClient(addr, grpc.WithTransportCredentials(insecure.NewCredentials()))
if err != nil {
return nil, nil, err
}
return conn, pb.NewLogtailServiceClient(conn), nil
}

26
cmd/frontend/format.go Normal file
View File

@@ -0,0 +1,26 @@
package main
import (
"fmt"
"strings"
)
// fmtCount formats a count with a space as the thousands separator.
func fmtCount(n int64) string {
s := fmt.Sprintf("%d", n)
if len(s) <= 3 {
return s
}
var b strings.Builder
start := len(s) % 3
if start > 0 {
b.WriteString(s[:start])
}
for i := start; i < len(s); i += 3 {
if i > 0 {
b.WriteByte(' ')
}
b.WriteString(s[i : i+3])
}
return b.String()
}

532
cmd/frontend/frontend_test.go Normal file
View File

@@ -0,0 +1,532 @@
package main
import (
"context"
"encoding/json"
"html/template"
"io"
"net"
"net/http"
"net/http/httptest"
"strings"
"testing"
pb "git.ipng.ch/ipng/nginx-logtail/proto/logtailpb"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
)
// --- Fake gRPC server ---
type fakeServer struct {
pb.UnimplementedLogtailServiceServer
topNResp *pb.TopNResponse
trendResp *pb.TrendResponse
// Captured for inspection.
lastTopN *pb.TopNRequest
lastTrend *pb.TrendRequest
}
func (f *fakeServer) TopN(_ context.Context, req *pb.TopNRequest) (*pb.TopNResponse, error) {
f.lastTopN = req
if f.topNResp == nil {
return &pb.TopNResponse{}, nil
}
return f.topNResp, nil
}
func (f *fakeServer) Trend(_ context.Context, req *pb.TrendRequest) (*pb.TrendResponse, error) {
f.lastTrend = req
if f.trendResp == nil {
return &pb.TrendResponse{}, nil
}
return f.trendResp, nil
}
func startFake(t *testing.T, fs *fakeServer) string {
t.Helper()
lis, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
srv := grpc.NewServer()
pb.RegisterLogtailServiceServer(srv, fs)
go srv.Serve(lis)
t.Cleanup(srv.GracefulStop)
return lis.Addr().String()
}
func mustLoadTemplate(t *testing.T) *template.Template {
t.Helper()
funcMap := template.FuncMap{"fmtCount": fmtCount}
tmpl, err := template.New("").Funcs(funcMap).ParseFS(templatesFS, "templates/*.html")
if err != nil {
t.Fatal(err)
}
return tmpl
}
func newHandler(t *testing.T, target string) *Handler {
t.Helper()
return &Handler{
defaultTarget: target,
defaultN: 25,
refreshSecs: 30,
tmpl: mustLoadTemplate(t),
}
}
func get(t *testing.T, h http.Handler, path string) (int, string) {
t.Helper()
req := httptest.NewRequest(http.MethodGet, path, nil)
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
body, _ := io.ReadAll(rec.Body)
return rec.Code, string(body)
}
// --- Unit tests ---
func TestParseWindowString(t *testing.T) {
cases := []struct {
in string
wantPB pb.Window
wantStr string
}{
{"1m", pb.Window_W1M, "1m"},
{"5m", pb.Window_W5M, "5m"},
{"15m", pb.Window_W15M, "15m"},
{"60m", pb.Window_W60M, "60m"},
{"6h", pb.Window_W6H, "6h"},
{"24h", pb.Window_W24H, "24h"},
{"bad", pb.Window_W5M, "5m"}, // default
{"", pb.Window_W5M, "5m"},
}
for _, c := range cases {
w, s := parseWindowString(c.in)
if w != c.wantPB || s != c.wantStr {
t.Errorf("parseWindowString(%q) = (%v, %q), want (%v, %q)", c.in, w, s, c.wantPB, c.wantStr)
}
}
}
func TestParseGroupByString(t *testing.T) {
cases := []struct {
in string
wantPB pb.GroupBy
wantStr string
}{
{"website", pb.GroupBy_WEBSITE, "website"},
{"prefix", pb.GroupBy_CLIENT_PREFIX, "prefix"},
{"uri", pb.GroupBy_REQUEST_URI, "uri"},
{"status", pb.GroupBy_HTTP_RESPONSE, "status"},
{"bad", pb.GroupBy_WEBSITE, "website"}, // default
}
for _, c := range cases {
g, s := parseGroupByString(c.in)
if g != c.wantPB || s != c.wantStr {
t.Errorf("parseGroupByString(%q) = (%v, %q), want (%v, %q)", c.in, g, s, c.wantPB, c.wantStr)
}
}
}
func TestParseQueryParams(t *testing.T) {
h := &Handler{defaultTarget: "default:9091", defaultN: 25}
req := httptest.NewRequest("GET",
"/?target=other:9090&w=60m&by=prefix&n=10&f_website=example.com&f_status=429", nil)
p := h.parseParams(req)
if p.Target != "other:9090" {
t.Errorf("target = %q", p.Target)
}
if p.WindowS != "60m" || p.Window != pb.Window_W60M {
t.Errorf("window = %q / %v", p.WindowS, p.Window)
}
if p.GroupByS != "prefix" || p.GroupBy != pb.GroupBy_CLIENT_PREFIX {
t.Errorf("group-by = %q / %v", p.GroupByS, p.GroupBy)
}
if p.N != 10 {
t.Errorf("n = %d", p.N)
}
if p.Filter.Website != "example.com" {
t.Errorf("f_website = %q", p.Filter.Website)
}
if p.Filter.Status != "429" {
t.Errorf("f_status = %q", p.Filter.Status)
}
}
func TestParseQueryParamsDefaults(t *testing.T) {
h := &Handler{defaultTarget: "agg:9091", defaultN: 20}
req := httptest.NewRequest("GET", "/", nil)
p := h.parseParams(req)
if p.Target != "agg:9091" {
t.Errorf("default target = %q", p.Target)
}
if p.WindowS != "5m" {
t.Errorf("default window = %q", p.WindowS)
}
if p.GroupByS != "website" {
t.Errorf("default group-by = %q", p.GroupByS)
}
if p.N != 20 {
t.Errorf("default n = %d", p.N)
}
}
func TestBuildFilter(t *testing.T) {
f := buildFilter(filterState{Website: "example.com", Status: "404"})
if f == nil {
t.Fatal("expected non-nil filter")
}
if f.GetWebsite() != "example.com" {
t.Errorf("website = %q", f.GetWebsite())
}
if f.GetHttpResponse() != 404 {
t.Errorf("status = %d", f.GetHttpResponse())
}
if f.ClientPrefix != nil || f.HttpRequestUri != nil {
t.Error("unexpected non-nil fields")
}
}
func TestBuildFilterNil(t *testing.T) {
if buildFilter(filterState{}) != nil {
t.Error("expected nil when no filter set")
}
}
func TestDrillURL(t *testing.T) {
p := QueryParams{Target: "agg:9091", WindowS: "5m", GroupByS: "website", N: 25}
u := p.drillURL("example.com")
if !strings.Contains(u, "f_website=example.com") {
t.Errorf("drill from website: missing f_website in %q", u)
}
if !strings.Contains(u, "by=prefix") {
t.Errorf("drill from website: expected by=prefix in %q", u)
}
p.GroupByS = "prefix"
u = p.drillURL("1.2.3.0/24")
if !strings.Contains(u, "by=uri") {
t.Errorf("drill from prefix: expected by=uri in %q", u)
}
p.GroupByS = "status"
u = p.drillURL("429")
if !strings.Contains(u, "f_status=429") {
t.Errorf("drill from status: missing f_status in %q", u)
}
if !strings.Contains(u, "by=website") {
t.Errorf("drill from status: expected cycle back to by=website in %q", u)
}
}
func TestBuildCrumbs(t *testing.T) {
p := QueryParams{
Target: "agg:9091",
WindowS: "5m",
GroupByS: "website",
N: 25,
Filter: filterState{Website: "example.com", Status: "429"},
}
crumbs := buildCrumbs(p)
if len(crumbs) != 2 {
t.Fatalf("expected 2 crumbs, got %d", len(crumbs))
}
if crumbs[0].Text != "website=example.com" {
t.Errorf("crumb[0].text = %q", crumbs[0].Text)
}
// RemoveURL for website crumb should not contain f_website.
if strings.Contains(crumbs[0].RemoveURL, "f_website") {
t.Errorf("remove URL still has f_website: %q", crumbs[0].RemoveURL)
}
// RemoveURL for website crumb should still contain f_status.
if !strings.Contains(crumbs[0].RemoveURL, "f_status=429") {
t.Errorf("remove URL missing f_status: %q", crumbs[0].RemoveURL)
}
}
func TestRenderSparkline(t *testing.T) {
points := []*pb.TrendPoint{
{TimestampUnix: 1, Count: 100},
{TimestampUnix: 2, Count: 200},
{TimestampUnix: 3, Count: 150},
{TimestampUnix: 4, Count: 50},
{TimestampUnix: 5, Count: 300},
}
svg := string(renderSparkline(points))
if !strings.Contains(svg, "<svg") {
t.Error("expected <svg tag")
}
if !strings.Contains(svg, "<polyline") {
t.Error("expected <polyline tag")
}
if !strings.Contains(svg, "points=") {
t.Error("expected points= attribute")
}
}
func TestRenderSparklineTooFewPoints(t *testing.T) {
if renderSparkline(nil) != "" {
t.Error("nil → expected empty")
}
if renderSparkline([]*pb.TrendPoint{{Count: 100}}) != "" {
t.Error("1 point → expected empty")
}
}
func TestRenderSparklineAllZero(t *testing.T) {
points := []*pb.TrendPoint{{Count: 0}, {Count: 0}, {Count: 0}}
if renderSparkline(points) != "" {
t.Error("all-zero → expected empty")
}
}
func TestFmtCount(t *testing.T) {
cases := []struct {
n int64
want string
}{
{0, "0"},
{999, "999"},
{1000, "1 000"},
{18432, "18 432"},
{1234567, "1 234 567"},
}
for _, c := range cases {
if got := fmtCount(c.n); got != c.want {
t.Errorf("fmtCount(%d) = %q, want %q", c.n, got, c.want)
}
}
}
// --- Handler integration tests ---
func TestHandlerTopN(t *testing.T) {
fake := &fakeServer{
topNResp: &pb.TopNResponse{
Source: "col-1",
Entries: []*pb.TopNEntry{
{Label: "busy.com", Count: 18432},
{Label: "quiet.com", Count: 100},
},
},
}
addr := startFake(t, fake)
h := newHandler(t, addr)
code, body := get(t, h, "/?target="+addr)
if code != 200 {
t.Fatalf("status = %d", code)
}
if !strings.Contains(body, "busy.com") {
t.Error("expected busy.com in body")
}
if !strings.Contains(body, "quiet.com") {
t.Error("expected quiet.com in body")
}
if !strings.Contains(body, "18 432") {
t.Error("expected formatted count 18 432 in body")
}
}
func TestHandlerRaw(t *testing.T) {
fake := &fakeServer{
topNResp: &pb.TopNResponse{
Source: "agg",
Entries: []*pb.TopNEntry{{Label: "x.com", Count: 42}},
},
}
addr := startFake(t, fake)
h := newHandler(t, addr)
code, body := get(t, h, "/?target="+addr+"&raw=1&w=15m&by=prefix")
if code != 200 {
t.Fatalf("status = %d", code)
}
var result struct {
Source string `json:"source"`
Window string `json:"window"`
GroupBy string `json:"group_by"`
Entries []struct {
Label string `json:"label"`
Count int64 `json:"count"`
} `json:"entries"`
}
if err := json.Unmarshal([]byte(body), &result); err != nil {
t.Fatalf("JSON parse: %v\nbody: %s", err, body)
}
if result.Source != "agg" {
t.Errorf("source = %q", result.Source)
}
if result.Window != "15m" {
t.Errorf("window = %q", result.Window)
}
if result.GroupBy != "prefix" {
t.Errorf("group_by = %q", result.GroupBy)
}
if len(result.Entries) != 1 || result.Entries[0].Label != "x.com" {
t.Errorf("entries = %+v", result.Entries)
}
}
func TestHandlerBadTarget(t *testing.T) {
h := newHandler(t, "127.0.0.1:1") // always refused
code, body := get(t, h, "/?target=127.0.0.1:1")
if code != http.StatusBadGateway {
t.Errorf("status = %d, want 502", code)
}
if !strings.Contains(body, "error") && !strings.Contains(body, "Error") {
t.Error("expected error message in body")
}
}
func TestHandlerFilterPassedToServer(t *testing.T) {
fake := &fakeServer{}
addr := startFake(t, fake)
h := newHandler(t, addr)
get(t, h, "/?target="+addr+"&f_website=example.com&f_status=429")
if fake.lastTopN == nil {
t.Fatal("no TopN request received")
}
f := fake.lastTopN.Filter
if f == nil {
t.Fatal("filter is nil")
}
if f.GetWebsite() != "example.com" {
t.Errorf("website = %q", f.GetWebsite())
}
if f.GetHttpResponse() != 429 {
t.Errorf("status = %d", f.GetHttpResponse())
}
}
func TestHandlerWindowPassedToServer(t *testing.T) {
fake := &fakeServer{}
addr := startFake(t, fake)
h := newHandler(t, addr)
get(t, h, "/?target="+addr+"&w=60m")
if fake.lastTopN == nil {
t.Fatal("no TopN request received")
}
if fake.lastTopN.Window != pb.Window_W60M {
t.Errorf("window = %v, want W60M", fake.lastTopN.Window)
}
}
func TestHandlerBreadcrumbInHTML(t *testing.T) {
fake := &fakeServer{topNResp: &pb.TopNResponse{}}
addr := startFake(t, fake)
h := newHandler(t, addr)
_, body := get(t, h, "/?target="+addr+"&f_website=example.com")
if !strings.Contains(body, "website=example.com") {
t.Error("expected breadcrumb with website=example.com")
}
// Remove link should exist.
if !strings.Contains(body, "×") {
t.Error("expected × remove link in breadcrumb")
}
}
func TestHandlerSparklineInHTML(t *testing.T) {
fake := &fakeServer{
topNResp: &pb.TopNResponse{Entries: []*pb.TopNEntry{{Label: "x.com", Count: 1}}},
trendResp: &pb.TrendResponse{Points: []*pb.TrendPoint{
{TimestampUnix: 1, Count: 100},
{TimestampUnix: 2, Count: 200},
{TimestampUnix: 3, Count: 150},
}},
}
addr := startFake(t, fake)
h := newHandler(t, addr)
_, body := get(t, h, "/?target="+addr)
if !strings.Contains(body, "<svg") {
t.Error("expected SVG sparkline in body")
}
if !strings.Contains(body, "<polyline") {
t.Error("expected polyline in SVG sparkline")
}
}
func TestHandlerPctBar(t *testing.T) {
fake := &fakeServer{
topNResp: &pb.TopNResponse{
Entries: []*pb.TopNEntry{
{Label: "top.com", Count: 1000},
{Label: "half.com", Count: 500},
},
},
}
addr := startFake(t, fake)
h := newHandler(t, addr)
_, body := get(t, h, "/?target="+addr)
if !strings.Contains(body, "<meter") {
t.Error("expected <meter bar in body")
}
// top.com should be 100%, half.com 50%.
if !strings.Contains(body, `value="100"`) {
t.Error("expected 100% bar for top entry")
}
if !strings.Contains(body, `value="50"`) {
t.Error("expected 50% bar for half entry")
}
}
func TestHandlerWindowTabsInHTML(t *testing.T) {
fake := &fakeServer{}
addr := startFake(t, fake)
h := newHandler(t, addr)
_, body := get(t, h, "/?target="+addr+"&w=15m")
// All window labels should appear.
for _, w := range []string{"1m", "5m", "15m", "60m", "6h", "24h"} {
if !strings.Contains(body, ">"+w+"<") {
t.Errorf("expected window tab %q in body", w)
}
}
}
func TestHandlerNoData(t *testing.T) {
fake := &fakeServer{topNResp: &pb.TopNResponse{}}
addr := startFake(t, fake)
h := newHandler(t, addr)
code, body := get(t, h, "/?target="+addr)
if code != 200 {
t.Fatalf("status = %d", code)
}
if !strings.Contains(body, "no data") {
t.Error("expected 'no data' message in body")
}
}
// Verify the fake gRPC server is reachable (sanity check for test infrastructure).
func TestDialFake(t *testing.T) {
addr := startFake(t, &fakeServer{})
conn, client, err := dial(addr)
if err != nil {
t.Fatal(err)
}
defer conn.Close()
_ = client
// If we got here without error, the fake server is reachable.
_ = grpc.NewClient // suppress unused import warning in case of refactor
_ = insecure.NewCredentials
}

447
cmd/frontend/handler.go Normal file
View File

@@ -0,0 +1,447 @@
package main
import (
"context"
"encoding/json"
"fmt"
"html/template"
"log"
"net/http"
"net/url"
"strconv"
"time"
pb "git.ipng.ch/ipng/nginx-logtail/proto/logtailpb"
)
// Handler is the HTTP handler for the frontend.
type Handler struct {
defaultTarget string
defaultN int
refreshSecs int
tmpl *template.Template
}
// Tab is a window or group-by selector link.
type Tab struct {
Label string
URL string
Active bool
}
// Crumb is one active filter shown in the breadcrumb strip.
type Crumb struct {
Text string
RemoveURL string
}
// TableRow is one row in the TopN result table.
type TableRow struct {
Rank int
Label string
Count int64
Pct float64 // 0–100, relative to rank-1 entry
DrillURL string
}
// filterState holds the four optional filter fields parsed from URL params.
type filterState struct {
Website string
Prefix string
URI string
Status string // kept as string so empty means "unset"
}
// QueryParams holds all parsed URL parameters for one page request.
type QueryParams struct {
Target string
Window pb.Window
WindowS string // e.g. "5m"
GroupBy pb.GroupBy
GroupByS string // e.g. "website"
N int
Filter filterState
}
// PageData is passed to the HTML template.
type PageData struct {
Params QueryParams
Source string
Entries []TableRow
TotalCount int64
Sparkline template.HTML
Breadcrumbs []Crumb
Windows []Tab
GroupBys []Tab
RefreshSecs int
Error string
}
var windowSpecs = []struct{ s, label string }{
{"1m", "1m"}, {"5m", "5m"}, {"15m", "15m"}, {"60m", "60m"}, {"6h", "6h"}, {"24h", "24h"},
}
var groupBySpecs = []struct{ s, label string }{
{"website", "website"}, {"prefix", "prefix"}, {"uri", "uri"}, {"status", "status"},
}
func parseWindowString(s string) (pb.Window, string) {
switch s {
case "1m":
return pb.Window_W1M, "1m"
case "5m":
return pb.Window_W5M, "5m"
case "15m":
return pb.Window_W15M, "15m"
case "60m":
return pb.Window_W60M, "60m"
case "6h":
return pb.Window_W6H, "6h"
case "24h":
return pb.Window_W24H, "24h"
default:
return pb.Window_W5M, "5m"
}
}
func parseGroupByString(s string) (pb.GroupBy, string) {
switch s {
case "prefix":
return pb.GroupBy_CLIENT_PREFIX, "prefix"
case "uri":
return pb.GroupBy_REQUEST_URI, "uri"
case "status":
return pb.GroupBy_HTTP_RESPONSE, "status"
default:
return pb.GroupBy_WEBSITE, "website"
}
}
func (h *Handler) parseParams(r *http.Request) QueryParams {
q := r.URL.Query()
target := q.Get("target")
if target == "" {
target = h.defaultTarget
}
win, winS := parseWindowString(q.Get("w"))
grp, grpS := parseGroupByString(q.Get("by"))
n := h.defaultN
if ns := q.Get("n"); ns != "" {
if v, err := strconv.Atoi(ns); err == nil && v > 0 {
n = v
}
}
return QueryParams{
Target: target,
Window: win,
WindowS: winS,
GroupBy: grp,
GroupByS: grpS,
N: n,
Filter: filterState{
Website: q.Get("f_website"),
Prefix: q.Get("f_prefix"),
URI: q.Get("f_uri"),
Status: q.Get("f_status"),
},
}
}
func buildFilter(f filterState) *pb.Filter {
if f.Website == "" && f.Prefix == "" && f.URI == "" && f.Status == "" {
return nil
}
out := &pb.Filter{}
if f.Website != "" {
out.Website = &f.Website
}
if f.Prefix != "" {
out.ClientPrefix = &f.Prefix
}
if f.URI != "" {
out.HttpRequestUri = &f.URI
}
if f.Status != "" {
if n, err := strconv.Atoi(f.Status); err == nil {
n32 := int32(n)
out.HttpResponse = &n32
}
}
return out
}
// toValues serialises QueryParams back to URL query values.
func (p QueryParams) toValues() url.Values {
v := url.Values{}
v.Set("target", p.Target)
v.Set("w", p.WindowS)
v.Set("by", p.GroupByS)
v.Set("n", strconv.Itoa(p.N))
if p.Filter.Website != "" {
v.Set("f_website", p.Filter.Website)
}
if p.Filter.Prefix != "" {
v.Set("f_prefix", p.Filter.Prefix)
}
if p.Filter.URI != "" {
v.Set("f_uri", p.Filter.URI)
}
if p.Filter.Status != "" {
v.Set("f_status", p.Filter.Status)
}
return v
}
// buildURL returns a page URL derived from the current params with overrides applied.
// An override value of "" removes that key from the URL.
func (p QueryParams) buildURL(overrides map[string]string) string {
v := p.toValues()
for k, val := range overrides {
if val == "" {
v.Del(k)
} else {
v.Set(k, val)
}
}
return "/?" + v.Encode()
}
// nextGroupBy advances the drill-down dimension hierarchy (cycles at the end).
func nextGroupBy(s string) string {
switch s {
case "website":
return "prefix"
case "prefix":
return "uri"
case "uri":
return "status"
default: // status → back to website
return "website"
}
}
// groupByFilterKey maps a group-by name to its URL filter parameter.
func groupByFilterKey(s string) string {
switch s {
case "website":
return "f_website"
case "prefix":
return "f_prefix"
case "uri":
return "f_uri"
case "status":
return "f_status"
default:
return "f_website"
}
}
func (p QueryParams) drillURL(label string) string {
return p.buildURL(map[string]string{
groupByFilterKey(p.GroupByS): label,
"by": nextGroupBy(p.GroupByS),
})
}
func buildCrumbs(p QueryParams) []Crumb {
var crumbs []Crumb
if p.Filter.Website != "" {
crumbs = append(crumbs, Crumb{
Text: "website=" + p.Filter.Website,
RemoveURL: p.buildURL(map[string]string{"f_website": ""}),
})
}
if p.Filter.Prefix != "" {
crumbs = append(crumbs, Crumb{
Text: "prefix=" + p.Filter.Prefix,
RemoveURL: p.buildURL(map[string]string{"f_prefix": ""}),
})
}
if p.Filter.URI != "" {
crumbs = append(crumbs, Crumb{
Text: "uri=" + p.Filter.URI,
RemoveURL: p.buildURL(map[string]string{"f_uri": ""}),
})
}
if p.Filter.Status != "" {
crumbs = append(crumbs, Crumb{
Text: "status=" + p.Filter.Status,
RemoveURL: p.buildURL(map[string]string{"f_status": ""}),
})
}
return crumbs
}
func buildWindowTabs(p QueryParams) []Tab {
tabs := make([]Tab, len(windowSpecs))
for i, w := range windowSpecs {
tabs[i] = Tab{
Label: w.label,
URL: p.buildURL(map[string]string{"w": w.s}),
Active: p.WindowS == w.s,
}
}
return tabs
}
func buildGroupByTabs(p QueryParams) []Tab {
tabs := make([]Tab, len(groupBySpecs))
for i, g := range groupBySpecs {
tabs[i] = Tab{
Label: "by " + g.label,
URL: p.buildURL(map[string]string{"by": g.s}),
Active: p.GroupByS == g.s,
}
}
return tabs
}
func buildTableRows(entries []*pb.TopNEntry, p QueryParams) ([]TableRow, int64) {
if len(entries) == 0 {
return nil, 0
}
top := float64(entries[0].Count)
var total int64
rows := make([]TableRow, len(entries))
for i, e := range entries {
total += e.Count
pct := 0.0
if top > 0 {
pct = float64(e.Count) / top * 100
}
rows[i] = TableRow{
Rank: i + 1,
Label: e.Label,
Count: e.Count,
Pct: pct,
DrillURL: p.drillURL(e.Label),
}
}
return rows, total
}
func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
params := h.parseParams(r)
filter := buildFilter(params.Filter)
conn, client, err := dial(params.Target)
if err != nil {
h.render(w, http.StatusBadGateway, h.errorPage(params,
fmt.Sprintf("cannot connect to %s: %v", params.Target, err)))
return
}
defer conn.Close()
ctx, cancel := context.WithTimeout(r.Context(), 5*time.Second)
defer cancel()
type topNResult struct {
resp *pb.TopNResponse
err error
}
type trendResult struct {
resp *pb.TrendResponse
err error
}
topNCh := make(chan topNResult, 1)
trendCh := make(chan trendResult, 1)
go func() {
resp, err := client.TopN(ctx, &pb.TopNRequest{
Filter: filter,
GroupBy: params.GroupBy,
N: int32(params.N),
Window: params.Window,
})
topNCh <- topNResult{resp, err}
}()
go func() {
resp, err := client.Trend(ctx, &pb.TrendRequest{
Filter: filter,
Window: params.Window,
})
trendCh <- trendResult{resp, err}
}()
tn := <-topNCh
tr := <-trendCh
if tn.err != nil {
h.render(w, http.StatusBadGateway, h.errorPage(params,
fmt.Sprintf("error querying %s: %v", params.Target, tn.err)))
return
}
// raw=1: return JSON for scripting
if r.URL.Query().Get("raw") == "1" {
writeRawJSON(w, params, tn.resp)
return
}
rows, total := buildTableRows(tn.resp.Entries, params)
var sparkline template.HTML
if tr.err == nil && tr.resp != nil {
sparkline = renderSparkline(tr.resp.Points)
}
data := PageData{
Params: params,
Source: tn.resp.Source,
Entries: rows,
TotalCount: total,
Sparkline: sparkline,
Breadcrumbs: buildCrumbs(params),
Windows: buildWindowTabs(params),
GroupBys: buildGroupByTabs(params),
RefreshSecs: h.refreshSecs,
}
h.render(w, http.StatusOK, data)
}
func (h *Handler) render(w http.ResponseWriter, status int, data PageData) {
w.Header().Set("Content-Type", "text/html; charset=utf-8")
w.WriteHeader(status)
if err := h.tmpl.ExecuteTemplate(w, "base", data); err != nil {
log.Printf("frontend: template error: %v", err)
}
}
func (h *Handler) errorPage(params QueryParams, msg string) PageData {
return PageData{
Params: params,
Windows: buildWindowTabs(params),
GroupBys: buildGroupByTabs(params),
Breadcrumbs: buildCrumbs(params),
RefreshSecs: h.refreshSecs,
Error: msg,
}
}
func writeRawJSON(w http.ResponseWriter, params QueryParams, resp *pb.TopNResponse) {
type entry struct {
Label string `json:"label"`
Count int64 `json:"count"`
}
type out struct {
Source string `json:"source"`
Window string `json:"window"`
GroupBy string `json:"group_by"`
Entries []entry `json:"entries"`
}
o := out{
Source: resp.Source,
Window: params.WindowS,
GroupBy: params.GroupByS,
Entries: make([]entry, len(resp.Entries)),
}
for i, e := range resp.Entries {
o.Entries[i] = entry{Label: e.Label, Count: e.Count}
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(o)
}

53
cmd/frontend/main.go Normal file
View File

@@ -0,0 +1,53 @@
package main
import (
"context"
"embed"
"flag"
"html/template"
"log"
"net/http"
"os"
"os/signal"
"syscall"
)
//go:embed templates
var templatesFS embed.FS
func main() {
listen := flag.String("listen", ":8080", "HTTP listen address")
target := flag.String("target", "localhost:9091", "default gRPC endpoint (aggregator or collector)")
n := flag.Int("n", 25, "default number of table rows")
refresh := flag.Int("refresh", 30, "meta-refresh interval in seconds (0 = disabled)")
flag.Parse()
funcMap := template.FuncMap{"fmtCount": fmtCount}
tmpl := template.Must(
template.New("").Funcs(funcMap).ParseFS(templatesFS, "templates/*.html"),
)
h := &Handler{
defaultTarget: *target,
defaultN: *n,
refreshSecs: *refresh,
tmpl: tmpl,
}
srv := &http.Server{Addr: *listen, Handler: h}
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
defer stop()
go func() {
log.Printf("frontend: listening on %s (default target %s)", *listen, *target)
if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
log.Printf("frontend: %v", err)
os.Exit(1)
}
}()
<-ctx.Done()
log.Printf("frontend: shutting down")
srv.Shutdown(context.Background())
}

51
cmd/frontend/sparkline.go Normal file
View File

@@ -0,0 +1,51 @@
package main
import (
"fmt"
"html/template"
"strings"
pb "git.ipng.ch/ipng/nginx-logtail/proto/logtailpb"
)
const (
svgW = 300.0
svgH = 60.0
svgPad = 4.0
)
// renderSparkline converts trend points into an inline SVG polyline.
// Returns "" if there are fewer than 2 points or all counts are zero.
func renderSparkline(points []*pb.TrendPoint) template.HTML {
if len(points) < 2 {
return ""
}
var maxCount int64
for _, p := range points {
if p.Count > maxCount {
maxCount = p.Count
}
}
if maxCount == 0 {
return ""
}
n := len(points)
var pts strings.Builder
for i, p := range points {
x := svgPad + float64(i)*(svgW-2*svgPad)/float64(n-1)
y := svgPad + (svgH-2*svgPad)*(1.0-float64(p.Count)/float64(maxCount))
if i > 0 {
pts.WriteByte(' ')
}
fmt.Fprintf(&pts, "%.1f,%.1f", x, y)
}
return template.HTML(fmt.Sprintf(
`<svg viewBox="0 0 %d %d" width="%d" height="%d" xmlns="http://www.w3.org/2000/svg">`+
`<polyline points="%s" fill="none" stroke="#4a90d9" stroke-width="1.5" stroke-linejoin="round"/>`+
`</svg>`,
int(svgW), int(svgH), int(svgW), int(svgH), pts.String(),
))
}

43
cmd/frontend/templates/base.html Normal file
View File

@@ -0,0 +1,43 @@
{{define "base"}}<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>nginx-logtail</title>
{{- if gt .RefreshSecs 0}}
<meta http-equiv="refresh" content="{{.RefreshSecs}}">
{{- end}}
<style>
* { box-sizing: border-box; }
body { font-family: monospace; font-size: 14px; max-width: 1100px; margin: 2em auto; padding: 0 1.5em; color: #222; }
h1 { font-size: 1.1em; font-weight: bold; margin: 0 0 1em; letter-spacing: 0.05em; }
.tabs { display: flex; gap: 0.3em; margin-bottom: 0.7em; flex-wrap: wrap; }
.tabs a { text-decoration: none; padding: 0.2em 0.8em; border: 1px solid #aaa; color: #444; }
.tabs a:hover { background: #f0f0f0; }
.tabs a.active { background: #222; color: #fff; border-color: #222; }
.crumbs { margin-bottom: 0.8em; font-size: 0.9em; }
.crumbs .label { font-weight: bold; color: #666; margin-right: 0.3em; }
.crumbs span { display: inline-block; background: #eef; border: 1px solid #99b; padding: 0.1em 0.5em; margin-right: 0.3em; border-radius: 2px; }
.crumbs a { color: #c00; text-decoration: none; margin-left: 0.4em; font-weight: bold; }
.crumbs a:hover { color: #900; }
.sparkline { margin: 0.8em 0 1.2em; }
.sparkline small { color: #888; display: block; margin-bottom: 0.2em; }
table { border-collapse: collapse; width: 100%; }
th { text-align: left; border-bottom: 2px solid #222; padding: 0.3em 0.7em; font-size: 0.85em; color: #444; }
th.num { text-align: right; }
td { padding: 0.22em 0.7em; border-bottom: 1px solid #eee; vertical-align: middle; }
td.rank { color: #bbb; width: 3.5em; }
td.num { text-align: right; font-variant-numeric: tabular-nums; }
td.bar meter { width: 110px; height: 10px; vertical-align: middle; }
tr:hover td { background: #f7f7f7; }
a { color: #1a6aad; text-decoration: none; }
a:hover { text-decoration: underline; }
.error { color: #c00; border: 1px solid #fbb; background: #fff5f5; padding: 0.7em 1em; margin: 1em 0; border-radius: 3px; }
.nodata { color: #999; margin: 2em 0; font-style: italic; }
footer { margin-top: 2em; padding-top: 0.6em; border-top: 1px solid #e0e0e0; font-size: 0.8em; color: #999; }
</style>
</head>
<body>
{{template "content" .}}
</body>
</html>
{{end}}

70
cmd/frontend/templates/index.html Normal file
View File

@@ -0,0 +1,70 @@
{{define "content"}}
<h1>nginx-logtail</h1>
<div class="tabs">
{{- range .Windows}}
<a href="{{.URL}}"{{if .Active}} class="active"{{end}}>{{.Label}}</a>
{{- end}}
</div>
<div class="tabs">
{{- range .GroupBys}}
<a href="{{.URL}}"{{if .Active}} class="active"{{end}}>{{.Label}}</a>
{{- end}}
</div>
{{if .Breadcrumbs}}
<div class="crumbs">
<span class="label">Filters:</span>
{{- range .Breadcrumbs}}
<span>{{.Text}}<a href="{{.RemoveURL}}" title="remove filter">×</a></span>
{{- end}}
</div>
{{end}}
{{if .Error}}
<div class="error">{{.Error}}</div>
{{else}}
{{if .Sparkline}}
<div class="sparkline">
<small>{{.Params.WindowS}} trend · by {{.Params.GroupByS}}{{if .Source}} · source: {{.Source}}{{end}}</small>
{{.Sparkline}}
</div>
{{end}}
{{if .Entries}}
<table>
<thead>
<tr>
<th class="rank">#</th>
<th>LABEL</th>
<th class="num">COUNT</th>
<th class="num">%</th>
<th>BAR</th>
</tr>
</thead>
<tbody>
{{- range .Entries}}
<tr>
<td class="rank">{{.Rank}}</td>
<td><a href="{{.DrillURL}}">{{.Label}}</a></td>
<td class="num">{{fmtCount .Count}}</td>
<td class="num">{{printf "%.0f" .Pct}}%</td>
<td class="bar"><meter value="{{printf "%.0f" .Pct}}" max="100"></meter></td>
</tr>
{{- end}}
</tbody>
</table>
{{else}}
<p class="nodata">(no data yet — ring buffer may still be filling)</p>
{{end}}
{{end}}
<footer>
{{- if .Source}}source: {{.Source}} · {{end -}}
{{fmtCount .TotalCount}} requests · {{.Params.WindowS}} window · by {{.Params.GroupByS}}
{{- if gt .RefreshSecs 0}} · auto-refresh {{.RefreshSecs}}s{{end}}
</footer>
{{end}}