General Baggage Model for End-to-End Tracing and Its Application on Critical Path Analysis (George Hongkai Sun Masters Thesis)

Many monitoring and diagnosis systems have been proposed based on causal tracing for end-to-end executions in distributed systems as more and more modern computer applications become distributed systems. These applications are based on metadata propagation along the request’s executions, which has been closely tied to the application logic, the metadata formats, and the developer APIs. Such kind of coupling makes it impossible to reuse the metadata propagation code added by the developers to their systems and duplicates the efforts. Such duplicate efforts have been identified as the main barrier to entry for these tracing applications. In this report, we propose a general metadata container called the Baggage Model, which enables us to reuse the metadata propagation. Baggage provides necessary flexibility, extensibility, and isolation to allow different tracing applications to share the same instrumentation, which should significantly reduce the duplicate efforts for the developers. Three tracing applications are modified by us to take advantage of the general baggage model. We have also briefly analyzed the performance of our baggage implementation. Based on the baggage model, we propose a new tracing application called CPath for critical path analysis. This tool measures the overall latency, the critical execution path, and the slack of the request’s executions to help developers to figure out the slow-down factors. We also propose the idea of hypothetical speedups to simulate the optimization plans with the real request executions to see how much improvement on the overall latency the plans can work out, which helps the performance engineers to set up feasible and meaningful peformance goals for the development teams. We also talk about the lessons we have learnt and future work with both baggage model and CPath.

George Hongkai Sun MSc Thesis