It's true, if all you did was apply a few patches, you should have said so in the download, and you should have given the user the way to get the patches himself.
Now, since you also had to make some more manual changes, you shouldn't have released the QVM without your modifications according to the GPL.
Anyway, it is really isn't too important and that patch looks really promising. Just don't take too much time in giving the source code or the FSF will come after you :wink: