Overview
Tracing necessarily incurs a performance cost by its very nature. We have conducted some benchmarks on our Node SDK under various scenarios both for transparency and also to guide our own improvement efforts. The benchmark scripts for the Node SDK can be found here.Methodology
The benchmarks are fairly simple. The main work is being performed by an Express server that we spin up before the benchmarks start. Some endpoints compute hashes repeatedly, some just sit and wait, and others send and receive large bodies. We have three benchmark files: SDK disabled, SDK enabled, and SDK enabled with transforms. These are run one after the other and completely segregated so we don’t have to worry about the SDK’s import hooks polluting other tests. Each benchmark file then runs the same set of tasks. We have two main scenarios to consider:- High CPU — we compute hashes in a loop to max out CPU utilization
- High I/O — we wait on a timer simulating pending I/O
- Run the benchmarks with memory monitoring off, since memory monitoring
(
process.memoryUsage) has some performance impact due to traversing pages. - Run the benchmarks with memory monitoring on, but we discard the other results it gives.
- Run the benchmarks with memory monitoring off but with transforms on to get the impact of transforms.
Memory
Memory is quite hard to measure exactly due to garbage collection and timing
issues, so take this with a grain of salt.
It was also measured with the internal Node.js
process.memoryUsage function,
which might not be what the system reports (for example in top).
CPU
It is expected that during the CPU bound test that CPU contention starts to foul
the results.
The I/O bound case might be more illuminating for real world usages.
Transforms show negligible impact when only a small number of transforms are
registered, and its impact is expected to scale linearly with the number of
transforms registered.