PyTorch Bug: Corrupted Tensors After Failed Storage Resize

Hey there, PyTorch users and enthusiasts! Today, we're diving into a peculiar and potentially frustrating bug that can sneak up on you when you're working with tensors and their underlying storage. Specifically, we're going to explore how PyTorch's resize_() function can sometimes update a tensor's shape metadata, even if the actual storage resizing operation fails. This isn't just a minor glitch; it can lead to deeply corrupted tensors, causing unexpected crashes, Segmentation Faults, or tricky RuntimeErrors further down the line in your code. Understanding this issue is crucial for writing robust and reliable PyTorch applications, especially when dealing with custom memory management or integrating with libraries like NumPy. Let's unpack this problem, understand its implications, and discuss how you can safeguard your code against these "Zombie" tensors.

Unpacking the PyTorch Tensor Corruption Bug

When resize_() is called on a PyTorch tensor, it's designed to change the tensor's dimensions to accommodate new data. However, a significant problem arises when this function attempts to operate on a tensor that shares its underlying memory — its storage — with a buffer that cannot be resized. Imagine you've injected a NumPy array's memory into a PyTorch tensor using methods like set_(). NumPy arrays, once created, typically have a fixed memory allocation, meaning their storage isn't inherently resizable by PyTorch's internal mechanisms. In such a scenario, when resize_() is invoked, PyTorch correctly identifies this limitation and raises a RuntimeError, indicating, "Trying to resize storage that is not resizable." This is the expected, safe behavior. However, the core of the bug lies in the fact that this operation isn't entirely exception-safe. What happens is that the tensor's metadata, specifically its shape and stride information, gets updated to reflect the new target size (the one you requested in resize_()) before the system performs the critical check that determines if the storage can actually be resized. Because the storage resize then fails and an exception is thrown, the resize_() function should ideally revert any changes made to the tensor's metadata or, even better, not modify it at all until the storage operation is confirmed successful. Unfortunately, this doesn't happen. The tensor is left in an inconsistent and corrupted state – what we've playfully, but accurately, dubbed a "Zombie" tensor. Its tensor.shape property will now report a large, newly requested size, suggesting it's ready for a lot of data, but tensor.storage() will still point to the original, non-resizable, and often zero-byte buffer. This fundamental mismatch between what the tensor thinks it is (its shape metadata) and what it actually holds (its underlying storage) is the root cause of subsequent failures. Any attempt to access or print this