Tracing

Tracing records operations, scripting parses Python

Tracing in TorchScript involves recording the operations performed by a PyTorch model during execution. This process creates a serialized representation of the model that can be run independently of the original Python code. Scripting, on the other hand, involves translating the PyTorch model into a TorchScript format that can be executed in a different environment, such as C++.

Example

Suppose you have a PyTorch model for image classification. By tracing this model with a sample input, TorchScript records the operations performed. Later, you can convert the traced model into TorchScript code, which can then be deployed in a C++ application for inference.

Understanding the difference between tracing and scripting is crucial for optimizing model deployment and ensuring compatibility across different programming environments. Tracing captures the model's behavior, while scripting transforms it for execution in other languages.

Related concepts

torch.compile does in PyTorch 2.0: traces and optimizes the computation graph

torch.compile optimizes computation graph by tracing and compiling it for efficiency

continuous batching does

Continuous batching adds new requests to a running batch without waiting

loop unrolling does: trades code size for reduced loop overhead

Loop unrolling reduces loop overhead by executing multiple iterations simultaneously, increasing code size

log-probabilities are used instead of probabilities: avoids numerical underflow

Log-probabilities convert multiplications into additions, preventing numerical underflow

consistent hashing does: minimizes remapping when nodes join/leave

Consistent hashing distributes data across nodes, minimizing remapping when nodes join/leave

Lambda calculus

Lambda calculus represents data using only functions

One email a day: 5 concepts + the 5 stories that matter →

Swipe through 100 ML concepts daily

Open TickerNews