I think it's due to fungus gnats which seemed to be in the soil, so after getting rid of them I'm now repotting it.
I don't know if this is normal but there's these little pod kinda bits coming off the roots, about 3-8mm long. There's a load of them, some are dark brown on the outside and harder, in the middle they're a little green. The others are lighter brown, much softer, greener and wet on the inside. This makes me think that these are normal plant things, though I have absolutely no clue.
Dang that sucks. I never thought of bonsai-ing wisteria but I bet that would be beautiful. Good luck next time.
I have another one that thankfully is healthy (though it did decide to ditch its lower branches and leaves for some reason). It'll be a while before it blooms but yeah I'm looking forward to it :)