You can but the inferencing time is magnitudes slower. You can go another magnitude lower and chew through SSDs by using them as virtual ram, but even as fast as SSDs feel, they'd be almost a thousand times slower than using VRAM, which is designed for rapidly changing / Processing values, even more so than regular ram.