Post 20: DeepSeek: What just happened?
(How should I know? It's good though? I thought it was bad? Or it isn’t?! Again, how should I know.)
Taking a break today from broader issues to cover a current event, and I am actually EXTREMELY hesitant to do this. Why? One, because there are so many unknowns right now that will become apparent as this situation plays out, and two, I haven’t really had time to distill my thoughts on what happened and whether I think it is a net positive or negative much less to try to guess at its significance.
This won’t be published until tomorrow, which is Tuesday, January 28, so by the time you are reading this some more may have already changed or been revealed. (Another of the reasons I don’t frequently cover current events here and don’t plan to).
So, let’s call this an experimental post. At some point in a couple weeks or a month, I’ll come back and review this post and the lay of the land out there in the real world, and we’ll see how much of the benefits or fears from the event came to pass, sound good?
Okay, just a synopsis for those of you who weren’t paying attention:
A Chinese company, Deepseek, released a new AI model that purports to be as capable as models like ChatGPT, Gemini, Claude, and etc., and they claim they produced this model with resources so much smaller than what the other models used that it borders on absurd. ($6 million vs. the billions invested by the big tech companies). Anyway, the stock market got killed as a result of the panic as everyone ran away from big AI and everything to do with it.
I’m not going to get into the specific technologies they claim to have used that are different (for the most part) as that is beyond the purview of what I want to cover here, but I have some initial thoughts from some web scouring (and of course a chat with an LLM) based on some competing scenarios:
Scenario 1: This is real, they really developed a model that is, for all intents and purposes, as good as ChatGPT for a fraction of the price and energy usage, then:
This is a win for the environment as the LLMs have been and are beasts that consume a lot of energy.
This is great news for small start-ups and for the “democratization of AI” as ChatGPT puts it.
It’s possible that a great deal of what we hoped we might get from AI will come a lot more cheaply and without so many of the above-mentioned environmental costs and with a lot more participants due to this “democratization.”
Bad news for those who invested a ton in the big AI companies and panic and withdraw their money when stocks are down, (as of now, stocks are DOWN).
Neutral news for the big AI companies? Bad now because oops! someone just undercut them big time and they took a big hit in the market. . . but maybe long term good news (for some of them), because they have some really smart people working for them and no doubt they will figure out how to use the techniques DeepSeek used in short order and will jump right back in with a lot of savings to how they do things. . . (I have also seen several articles calling this a “Sputnik Moment” so this could lead to even MORE funding pumped into big tech as they race to compete with a Chinese company).
Privacy and safety?: Sadly, these might be losers. If it is really that cheap to make a new model like this then imagine how much easier it will be for malicious actors to generate all the things we fear like deep fakes, misinformation, and general mayhem.
Scenario 2: The model isn’t as good as DeepSeek says: Although it seems to have passed some initial reviews, what if it once gets poked at more, it turns out it isn’t nearly as capable as claimed.
A lot of people just lost a lot of money because people panicked.
People will hopefully be a little more sceptical when things like this happen and not panic like they did today. (I am pretty much joking about this, people gonna panic).
Scenario 3: The worst scenario. (And note, I am not accusing anyone of anything here, this is hypothetical, don’t sue me). I had a conversation with ChatGPT where we talked about a concept called “distillation.” In this context, distillation is the process by which someone essentially leverages a high quality existing AI to create a new one that is basically equally good, but gets to skip all the costly original training and foundational knowledge infrastructure, and also does nothing to lay the foundation for next generation AI, but gets to reap all the rewards of owning a high quality model.
Winners: In this case the only winners are the people who own the model. They can sell their product and products based on it with minimum relative investment. (There are likely some serious geopolitical winners here too. . .that should be covered everywhere else. . . open up the news.)
Losers: Basically everyone else. For a moment people can get cheaper products from this company, (and other distillers?), but all incentive to innovate is stripped away because of the expense and the fear that there will be no return on investment of developing the original highly capable model, or. . . big tech will continue developing new models, but these will be (very carefully) squirreled away to avoid any future distillation.
There is of course an elephant in the room here. If distillation did take place the company that is the “offender” is not in a jurisdiction that has a great reputation for punishing bad IP behavior. (Also, it's not even clear from what I have read that distillation itself is an offense under existing IP law).
Anyway, I will check back on this again at some point.
And just like that - big AI suddenly cares about copyright!