Implement target selection, autodiscovery via aggregator, implement listTargets

README.md

@@ -32,7 +32,10 @@ DESIGN
 ```
 nginx-logtail/
 ├── proto/
-│   └── logtail.proto                  # shared protobuf definitions
+│   ├── logtail.proto                  # shared protobuf definitions
+│   └── logtailpb/
+│       ├── logtail.pb.go              # generated: messages, enums
+│       └── logtail_grpc.pb.go         # generated: service stubs
 ├── internal/
 │   └── store/
 │       └── store.go                   # shared types: Tuple4, Entry, Snapshot, ring helpers
@@ -48,6 +51,7 @@ nginx-logtail/
     │   ├── subscriber.go              # one goroutine per collector; StreamSnapshots with backoff
     │   ├── merger.go                  # delta-merge: O(snapshot_size) per update
     │   ├── cache.go                   # tick-based ring buffer cache served to clients
+    │   ├── registry.go                # TargetRegistry: addr→name map updated from snapshot sources
     │   └── server.go                  # gRPC server (same surface as collector)
     ├── frontend/
     │   ├── main.go
@@ -66,7 +70,8 @@ nginx-logtail/
         ├── format.go                  # printTable, fmtCount, fmtTime, targetHeader
         ├── cmd_topn.go                # topn: concurrent fan-out, table + JSON output
         ├── cmd_trend.go               # trend: concurrent fan-out, table + JSON output
-        └── cmd_stream.go              # stream: multiplexed streams, auto-reconnect
+        ├── cmd_stream.go              # stream: multiplexed streams, auto-reconnect
+        └── cmd_targets.go             # targets: list collectors known to the endpoint
 ```
 
 ## Data Model
@@ -178,13 +183,23 @@ message Snapshot {
   repeated TopNEntry entries   = 3;  // full top-50K for this bucket
 }
 
+// Target discovery: list the collectors behind the queried endpoint
+message ListTargetsRequest {}
+message TargetInfo {
+  string name = 1;  // display name (--source value from the collector)
+  string addr = 2;  // gRPC address; empty string means "this endpoint itself"
+}
+message ListTargetsResponse { repeated TargetInfo targets = 1; }
+
 service LogtailService {
   rpc TopN(TopNRequest)                returns (TopNResponse);
   rpc Trend(TrendRequest)              returns (TrendResponse);
   rpc StreamSnapshots(SnapshotRequest) returns (stream Snapshot);
+  rpc ListTargets(ListTargetsRequest)  returns (ListTargetsResponse);
 }
 // Both collector and aggregator implement LogtailService.
 // The aggregator's StreamSnapshots re-streams the merged view.
+// ListTargets: aggregator returns all configured collectors; collector returns itself.
 ```
 
 ## Program 1 — Collector
@@ -259,10 +274,18 @@ service LogtailService {
 - Same tiered ring structure as the collector store; populated from `merger.TopK()` each tick.
 - `QueryTopN`, `QueryTrend`, `Subscribe`/`Unsubscribe` — identical interface to collector store.
 
+### registry.go
+- **`TargetRegistry`**: `sync.RWMutex`-protected `map[addr → name]`. Initialised with the
+  configured collector addresses; display names are updated from the `source` field of the first
+  snapshot received from each collector.
+- `Targets()` returns a stable sorted slice of `{name, addr}` pairs for `ListTargets` responses.
+
 ### server.go
 - Implements `LogtailService` backed by the cache (not live fan-out).
 - `StreamSnapshots` re-streams merged fine snapshots; usable by a second-tier aggregator or
   monitoring system.
+- `ListTargets` returns the current `TargetRegistry` contents — all configured collectors with
+  their display names and gRPC addresses.
 
 ## Program 3 — Frontend
 
@@ -278,8 +301,13 @@ service LogtailService {
   `store.ParseStatusExpr` into `(value, StatusOp)` for the filter protobuf.
 - **Regex filters**: `f_website_re` and `f_uri_re` hold RE2 patterns; compiled once per request
   into `store.CompiledFilter` before the query-loop iteration. Invalid regexes match nothing.
-- `TopN` and `Trend` RPCs issued **concurrently** (both with a 5 s deadline); page renders with
-  whatever completes. Trend failure suppresses the sparkline without erroring the page.
+- `TopN`, `Trend`, and `ListTargets` RPCs issued **concurrently** (all with a 5 s deadline); page
+  renders with whatever completes. Trend failure suppresses the sparkline; `ListTargets` failure
+  hides the source picker — both are non-fatal.
+- **Source picker**: `ListTargets` result drives a `source:` tab row. Clicking a collector tab
+  sets `target=` to that collector's address, querying it directly. The "all" tab resets to the
+  default aggregator. Picker is hidden when `ListTargets` returns ≤0 collectors (direct collector
+  mode).
 - **Drilldown**: clicking a table row adds the current dimension's filter and advances `by` through
   `website → prefix → uri → status → website` (cycles).
 - **`raw=1`**: returns the TopN result as JSON — same URL, no CLI needed for scripting.
@@ -305,6 +333,7 @@ service LogtailService {
 logtail-cli topn    [flags]   ranked label → count table (exits after one response)
 logtail-cli trend   [flags]   per-bucket time series (exits after one response)
 logtail-cli stream  [flags]   live snapshot feed (runs until Ctrl-C, auto-reconnects)
+logtail-cli targets [flags]   list targets known to the queried endpoint
 ```
 
 ### Flags
@@ -364,3 +393,4 @@ with a non-zero code on gRPC error.
 | Status filter as expression string (`!=200`, `>=400`) | Operator-friendly; parsed once at query boundary, encoded as `(int32, StatusOp)` in proto |
 | Regex filters compiled once per query (`CompiledFilter`) | Up to 288 × 5 000 per-entry calls — compiling per-entry would dominate query latency |
 | Filter expression box (`q=`) redirects to canonical URL | Filter state stays in individual `f_*` params; URLs remain shareable and bookmarkable |
+| `ListTargets` + frontend source picker (no Tuple5) | "Which nginx is busiest?" answered by switching `target=` to a collector; no data model changes, no extra memory |