Add period:100 reporting
All checks were successful
continuous-integration/drone/push Build is passing
All checks were successful
continuous-integration/drone/push Build is passing
This commit is contained in:
@@ -542,10 +542,12 @@ sunlight_sqlite_update_duration_seconds{quantile="0.99"} 0.016859223
|
|||||||
|
|
||||||
Comparing the throughput at 4'400/s with that first test of 100/s, I expect and can confirm a
|
Comparing the throughput at 4'400/s with that first test of 100/s, I expect and can confirm a
|
||||||
significant increase in all of these metrics. The 99th percentile addchain is now 1889ms (up from
|
significant increase in all of these metrics. The 99th percentile addchain is now 1889ms (up from
|
||||||
207ms) and the sequencing duration is now 1111ms (up from 109ms). I fiddle a little bit with
|
207ms) and the sequencing duration is now 1111ms (up from 109ms).
|
||||||
Sunlight's configuration file, notably the `period` and `poolsize`. This does not seem to matter
|
|
||||||
much. Setting for example `period:2000` and `poolsize:15000` still yields pretty much the same
|
#### Sunlight: Effect of period
|
||||||
throughput:
|
|
||||||
|
I fiddle a little bit with Sunlight's configuration file, notably the `period` and `poolsize`.
|
||||||
|
First I set `period:2000` and `poolsize:15000`, which yields pretty much the same throughput:
|
||||||
|
|
||||||
```
|
```
|
||||||
pim@ctlog-test:/etc/sunlight$ while :; do curl -ksS https://ctlog-test.lab.ipng.ch:1443/checkpoint | grep -E '^[0-9]+$'; sleep 60; done
|
pim@ctlog-test:/etc/sunlight$ while :; do curl -ksS https://ctlog-test.lab.ipng.ch:1443/checkpoint | grep -E '^[0-9]+$'; sleep 60; done
|
||||||
@@ -572,6 +574,34 @@ sunlight_sequencing_duration_seconds{log="sunlight-test",quantile="0.99"} 1.5968
|
|||||||
sunlight_sqlite_update_duration_seconds{quantile="0.99"} 0.010847308
|
sunlight_sqlite_update_duration_seconds{quantile="0.99"} 0.010847308
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Then I also set a `period:100` and `poolsize:15000`, which does improve a bit:
|
||||||
|
|
||||||
|
```
|
||||||
|
pim@ctlog-test:/etc/sunlight$ while :; do curl -ksS https://ctlog-test.lab.ipng.ch:1443/checkpoint | grep -E '^[0-9]+$'; sleep 60; done
|
||||||
|
560654
|
||||||
|
950524
|
||||||
|
1324645
|
||||||
|
1720362
|
||||||
|
```
|
||||||
|
|
||||||
|
With the same generated load of 10'000/sec with a 10% duplication rate, I am still offering roughly
|
||||||
|
9'000/sec of unique certificates, and I'm seeing `(1720362 - 560654)/180` or about 6'440/sec come
|
||||||
|
through, which is a fair bit better, at the expense of more disk activity. At this rate and with
|
||||||
|
`period:100`, the latency tail looks like this:
|
||||||
|
|
||||||
|
```
|
||||||
|
pim@ctlog-test:/etc/sunlight$ curl -ksS https://ctlog-test.lab.ipng.ch/metrics | egrep 'seconds.*quantile=\"0.99\"'
|
||||||
|
sunlight_addchain_wait_seconds{log="sunlight-test",quantile="0.99"} 1.616046445
|
||||||
|
sunlight_cache_get_duration_seconds{log="sunlight-test",quantile="0.99"} 7.5123e-05
|
||||||
|
sunlight_cache_put_duration_seconds{log="sunlight-test",quantile="0.99"} 0.534935803
|
||||||
|
sunlight_fs_op_duration_seconds{log="sunlight-test",method="discard",quantile="0.99"} 0.000377273
|
||||||
|
sunlight_fs_op_duration_seconds{log="sunlight-test",method="fetch",quantile="0.99"} 4.8893e-05
|
||||||
|
sunlight_fs_op_duration_seconds{log="sunlight-test",method="upload",quantile="0.99"} 0.054685991
|
||||||
|
sunlight_http_request_duration_seconds{endpoint="add-chain",log="sunlight-test",quantile="0.99"} 1.946445877
|
||||||
|
sunlight_sequencing_duration_seconds{log="sunlight-test",quantile="0.99"} 0.980602185
|
||||||
|
sunlight_sqlite_update_duration_seconds{quantile="0.99"} 0.018385831
|
||||||
|
```
|
||||||
|
|
||||||
***Conclusion***: Sunlight on POSIX can reliably handle 4'400/s (with a duplicate rate of 10%) on
|
***Conclusion***: Sunlight on POSIX can reliably handle 4'400/s (with a duplicate rate of 10%) on
|
||||||
this setup.
|
this setup.
|
||||||
|
|
||||||
@@ -589,9 +619,9 @@ current write-load (which is about 250/s).
|
|||||||
solutions, taking into account that TesseraCT runs MariaDB (which my setup did not use ZFS
|
solutions, taking into account that TesseraCT runs MariaDB (which my setup did not use ZFS
|
||||||
for), while Sunlight uses sqlite3 on the ZFS pool.
|
for), while Sunlight uses sqlite3 on the ZFS pool.
|
||||||
|
|
||||||
***Notable***: Sunlight POSIX and S3 performance is roughly identical (both handle about 5'000/sec),
|
***Notable***: Sunlight POSIX and S3 performance is roughly identical (both handle about
|
||||||
while TesseraCT POSIX performance (12'000/s) is significantly better than its S3 (800/s). Some
|
5'000/sec), while TesseraCT POSIX performance (12'000/s) is significantly better than its S3
|
||||||
other observations:
|
(800/s). Some other observations:
|
||||||
|
|
||||||
* Sunlight has a very opinionated configuration, and can run multiple logs with one configuration
|
* Sunlight has a very opinionated configuration, and can run multiple logs with one configuration
|
||||||
file and one binary. Its configuration was a bit constraining though, as I could not manage to
|
file and one binary. Its configuration was a bit constraining though, as I could not manage to
|
||||||
|
Reference in New Issue
Block a user