The issue with that is they're really difficult to run. Language generation isn't like image gen, it's a lot more complicated. You can use stable diffusion to generate 512 images with am iPhone. Just as an idea the smallest image gen model uses like 3GB vram. The smallest GPT-2 ( open source language Gen) uses 12. The best open source language Gen (GPT-J) uses 48GB. GPT-3 uses probably over 140GB. If you don't know what those numbers mean, think about it like this. The top of the line gamer card, the 4090, has about 24 GB of VRAM. So for GPT-J you're running two of those. For GPT-3 you'd need an array of them, or an array of those A100 cards which are what you really want to be using at that point, except they cost 21000$. So open sourcing the model wouldn't quite open up the floodgates to just everyone. It'll mostly make it available as an option to businesses and organizations who can put together enough resources to get something like that going.
I think ultimately the revolutionary left should approach AI much like we approach firearms. They're dangerous and something that should be tightly controlled in a socialist society, but we would be fools to abandon their usage in our war against the bourgeoise.
VRAM is what's used inside a GPU. The purpose of VRAM is to hold the information as you do operations on it. For a processing unit to load data from your regular RAM versus a specialized memory like VRAM is orders of magnitude slower, and you're sometimes better off just running it on the CPU at that point. The difficult thing is that the tech used for VRAM is more expensive to expand than regular RAM, so it's not cost effective to have gigabytes upon gigabytes of it.
You can but the inferencing time is magnitudes slower. You can go another magnitude lower and chew through SSDs by using them as virtual ram, but even as fast as SSDs feel, they'd be almost a thousand times slower than using VRAM, which is designed for rapidly changing / Processing values, even more so than regular ram.
The issue with that is they're really difficult to run. Language generation isn't like image gen, it's a lot more complicated. You can use stable diffusion to generate 512 images with am iPhone. Just as an idea the smallest image gen model uses like 3GB vram. The smallest GPT-2 ( open source language Gen) uses 12. The best open source language Gen (GPT-J) uses 48GB. GPT-3 uses probably over 140GB. If you don't know what those numbers mean, think about it like this. The top of the line gamer card, the 4090, has about 24 GB of VRAM. So for GPT-J you're running two of those. For GPT-3 you'd need an array of them, or an array of those A100 cards which are what you really want to be using at that point, except they cost 21000$. So open sourcing the model wouldn't quite open up the floodgates to just everyone. It'll mostly make it available as an option to businesses and organizations who can put together enough resources to get something like that going.
I think ultimately the revolutionary left should approach AI much like we approach firearms. They're dangerous and something that should be tightly controlled in a socialist society, but we would be fools to abandon their usage in our war against the bourgeoise.
can it be run on regular ram instead of vram? used ram even in terabytes is relatively cheap on ebay and stuff
VRAM is what's used inside a GPU. The purpose of VRAM is to hold the information as you do operations on it. For a processing unit to load data from your regular RAM versus a specialized memory like VRAM is orders of magnitude slower, and you're sometimes better off just running it on the CPU at that point. The difficult thing is that the tech used for VRAM is more expensive to expand than regular RAM, so it's not cost effective to have gigabytes upon gigabytes of it.
You can but the inferencing time is magnitudes slower. You can go another magnitude lower and chew through SSDs by using them as virtual ram, but even as fast as SSDs feel, they'd be almost a thousand times slower than using VRAM, which is designed for rapidly changing / Processing values, even more so than regular ram.