Fast local kernels need not form the fastest engine
TensorRT construction couples graph transformations, fusion, kernels, memory layouts and quantization. An individually fast kernel may disrupt fusion or increase traffic in the full graph; reduced precision may exceed application tolerance. The article therefore treats complete build routes as the optimization unit.
Default heuristics encode experience from representative models. Exceptions for one model can regress another and become difficult to maintain. Global Performance Tuner allows model- and environment-specific routes instead of embedding every successful choice in one universal strategy.
Reachable candidates and search are separate
Part 1 abstracts compilation into a builder that generates candidates and a search algorithm that selects among them. Perfect search cannot select an engine the builder never makes reachable. A channel-threshold heuristic that filters a convolution tactic can exclude the best complete route. Conversely, a large candidate space still disappoints if selection relies on misleading isolated timing.
Configurable routes vary passes and scheduling choices. Complete candidates are checked against reference data, an error metric and tolerance, then timed end to end. Selection occurs among acceptable candidates. “Global” describes the expanded solution space, not a proof that every possible program has been exhaustively searched. Build-time budgets constrain exploration.
Accuracy and measurement define the boundary
The mathematical model fixes the execution environment, while actual frequency, temperature, power and load introduce noise. Its discussion of repeated measurements explains the runtime abstraction; it does not make one minimum timing a stable gain.
My interpretation is that this framework helps distinguish missing candidates from incorrectly evaluated candidates when a fixed route regresses on a model. Part 1 provides conceptual and mathematical foundations, not transferable speedup numbers; implementation algorithms are deferred to later material. Validation-data coverage, tolerances and search budgets bound the result, and finite checks do not establish numerical guarantees for every input.
Source: Original engineering article