95 lines
3.3 KiB
Markdown
95 lines
3.3 KiB
Markdown
# ctfetch
|
|
|
|
Tools for working with Certificate Transparency log tiles.
|
|
|
|
## Install
|
|
|
|
```bash
|
|
go install ./cmd/ctfetch
|
|
```
|
|
|
|
## Usage
|
|
|
|
`ctfetch` operates in two modes depending on the arguments given.
|
|
|
|
### Leaf-index mode
|
|
|
|
Fetch a specific entry (or all entries in its tile) by leaf index:
|
|
|
|
```bash
|
|
ctfetch [flags] <log-url> <leaf-index> [+sct] [+issuer] [+ctlog]
|
|
```
|
|
|
|
**Examples:**
|
|
|
|
Dump a specific entry:
|
|
```bash
|
|
ctfetch https://halloumi2026h1.mon.ct.ipng.ch 629794635
|
|
```
|
|
|
|
Dump with SCTs, issuer chain, and CT log details:
|
|
```bash
|
|
ctfetch https://halloumi2026h1.mon.ct.ipng.ch 629794635 +sct +issuer +ctlog
|
|
```
|
|
|
|
### Tile-dump mode
|
|
|
|
Fetch all entries from a tile URL or a local file. Automatically detects data tiles (log entries) and hash tiles (Merkle tree hashes).
|
|
|
|
```bash
|
|
ctfetch [flags] <tile-url-or-file> [+sct] [+issuer] [+ctlog]
|
|
```
|
|
|
|
**Examples:**
|
|
|
|
Data tile from a URL:
|
|
```bash
|
|
ctfetch https://halloumi2026h1.mon.ct.ipng.ch/tile/data/x002/x460/135
|
|
```
|
|
|
|
Data tile with SCTs and CT log details:
|
|
```bash
|
|
ctfetch https://halloumi2026h1.mon.ct.ipng.ch/tile/data/x002/x460/135 +sct +ctlog
|
|
```
|
|
|
|
Hash tile from a URL:
|
|
```bash
|
|
ctfetch https://halloumi2026h1.mon.ct.ipng.ch/tile/0/x100/999
|
|
```
|
|
|
|
Data tile from a local file (with issuer resolution):
|
|
```bash
|
|
ctfetch --monitoring-url https://halloumi2026h1.mon.ct.ipng.ch tile.bin +issuer
|
|
```
|
|
|
|
## Hash tiles vs data tiles
|
|
|
|
A Static CT log stores two kinds of tiles:
|
|
|
|
**Data tiles** (`/tile/data/...`) contain the actual log entries — DER-encoded certificates and precertificates along with their metadata (leaf index, timestamp, chain fingerprints, etc.). These are what `ctfetch` parses into structured JSON. The output modifiers `+sct`, `+issuer`, and `+ctlog` all operate on data tiles.
|
|
|
|
**Hash tiles** (`/tile/N/...`, where N is a tree level ≥ 0) contain the internal nodes of the Merkle tree — rows of raw 32-byte SHA-256 hashes used for inclusion and consistency proofs. There are no certificates in a hash tile; `ctfetch` outputs only the list of hashes. Using `+sct`, `+issuer`, or `+ctlog` with a hash tile is an error.
|
|
|
|
The tree is organised so that level 0 hashes cover individual leaves (each is `SHA-256(0x00 || MerkleTreeLeaf)`), and each higher level hashes pairs of nodes from the level below. The tile URL encodes the level: `/tile/0/...` is level 0, `/tile/1/...` is level 1, and so on.
|
|
|
|
## Output modifiers
|
|
|
|
| Modifier | Description |
|
|
|---|---|
|
|
| `+sct` | Parse and include embedded Signed Certificate Timestamps from final (non-precert) certificates |
|
|
| `+issuer` | Fetch and include issuer certificate details from the log's `/issuer/<fp>` endpoint |
|
|
| `+ctlog` | Look up each SCT's log ID in the CT log list and include operator/state details |
|
|
|
|
## Flags
|
|
|
|
| Flag | Default | Description |
|
|
|---|---|---|
|
|
| `--logs-list-url` | `https://www.gstatic.com/ct/log_list/v3/all_logs_list.json` | URL of the CT log list JSON used for `+ctlog` lookups |
|
|
| `--monitoring-url` | _(none)_ | Log root URL for issuer lookups when input is a local file |
|
|
|
|
## Notes
|
|
|
|
- In tile-dump mode with a tile URL, `+issuer` automatically derives the log root by stripping the `/tile/...` path. With a local file, `--monitoring-url` must be provided.
|
|
- Partial tiles (`.p/N` suffix) are tried first; on 404 the full tile is fetched automatically.
|
|
- The CT log list and issuer certificates are cached in memory, so each unique resource is fetched only once per invocation.
|