I still remember seeing the announcement of the 3090 during the keynote presentation. People that aren’t waist deep in this area and haven’t struggled to get a GPU that can actually handle large, (language in my case), models didn’t realize how great a deal the 3090 was. 24 GB of high speed memory in your desktop GPU for ~$2k is just a remarkably cheap increase to productivity. If your access to cloud resources is constrained, it doesn’t need to be the absolute fastest GPU, but you do absolutely need the model to fit in memory otherwise your training problem typically remains intractable.
Getting one for MSRP is a different story, but I was lucky enough to get one from the EVGA queue just a couple months after release for right around $2k. Interestingly it also might be my most slowly depreciating piece of computer hardware ever. It’s just so overpowered for a consumer, I probably won’t need a new GPU for the better part of a decade.
This. Same scenario here, fondly remember 1080ti, that was the workhorse for us alongside cloud training (startup with ~100 nlp prod models, ner, siamese, count models, etc). ULMFiT and language transfer was the moment when upgrades felt necessary though 2080ti’s vram unfortunately stayed the same (never had access to Titans for instance so perspective may be limited).