virtio_ring: fix stale descriptor flags after a failed packed add

In the Linux kernel, the following vulnerability has been resolved:

virtio_ring: fix stale descriptor flags after a failed packed add

In a packed ring the AVAIL and USED bits sit in the descriptor itself,
so writing them makes that descriptor available. Those bit combinations
flip meaning on every round of the ring, tracked by a wrap counter, so
invalidating or validating a descriptor means inverting both bits.

Commit 1ce9e6055fa0 ("virtio_ring: introduce packed ring support") has
virtqueue_add_packed() make every descriptor of a chain available as it
maps the chain, and write the head last. The device consumes the ring in
order and stops at a head that is not available yet, so it never reaches
the rest.

When vring_map_one_sg() fails partway, unmap_release unmaps the segments
and restores avail_used_flags, but the descriptors it wrote to in the
ring stay marked with AVAIL and USED bits. The head is now the only
entry that keeps the device from consuming these stale entries.

For example, the ring would look like this now.

Z - pre-previous command
A - previous command
B - aborted command
C - current command

[A1 DONE] [A2 DONE] <C1 EMPTY> [B2] [B3] [Z1 DONE]

When the driver now attempts to issue the C command, the next add starts
at the same head as B. If C spans less descriptors than B, there is no
end marker because AVAIL and USED bits were still in place. And that
means the device will start interpreting these stale entries (B2/B3) as
another command entry, which then blocks the queue.

This effect typically happens in swiotlb configurations under memory
pressure, because vring_map_one_sg() can then fail with larger I/O
requests which then leads to command abortions.

There are broadly 2 ways to avoid leaving those flags behind:

1) Defer those flags too until the chain is complete.
2) Rewrite those flags for the previous wrap counter.

Implement the second option in both packed add paths. The first option
traverses the chain a second time on every successful add. The second
option invalidates all added descriptors when any add fails.

With this patch applied, a packed virtqueue keeps completing requests
after a failed add.