Everyone in the tech world, and everyone else beyond, cannot have failed to notice the ructions in the American markets because of the Chinese generative AI “DeepSeek”. As I am fascia
Steven Sinofsky argues pretty convincingly that DeepSeek was inevitable. He notes that the history of IT is innovation followed by scale up which is then broken by a model that “scales out”. That is when the bigger and faster approach is replaced by a smaller and more numerous approach. With that in mind, then, it was a matter of time before a scale out solution arrived, a solution that with different architectural approaches that use less capital to train the AI models. He thinks, as do I, the US innovators will respond with their own scale out approaches: DeepSeek does not mean that American has lost an AI “war” with China.
xxx
DeepSeek r1 is not smarter than earlier models, just trained more cheaply
It doesn’t solve hallucinations or problems with reliability.
From: Five things most people don’t seem to understand about DeepSeek.
Remember that such hallucinations are not simply an occasional error in LLM output, they are feature of the way that LLMs work. You can certainly implement a variety of mitigations, but you are not going to eradicate them, and they have real world consequences. This has been evident from thea rl
CharGPT also couldn’t seem to stop making things up, a phenomenon experts called “hallucinations.” One radio host in Georgia, US, sued OpenAl in the summer of 2023 for def-amation, claiming that ChatGPT had falsely aceused him of embezzling money, Not long after, two lawyers in New York were fined after they submitted a legal brief they’d cribbed from ChatGPT, which included fake case citations. Users were finding that sometimes, when they asked ChatGPT for sources of its information, it would make those up too.
OpenAl refused to disclose what ChatGPT’s hallucination rate was, but some Al researchers as well as regular users put it at roughly 20 percent, meaning that at least for certain users, and in about one in five instances, ChatGPT was fabrieating infor mation. The tool had been designed to be as useful as possible and to err on the side of confidence; the downside to that was it was often spewing hogwash. Not only were more people using a tool that made it easier to skip the process of hard thinking, they were often being fed misinformation that sounded persuasive and even
If you are interested in understanding more about the underlying theory, there are a number of useful papers on the topic. One is Xu, Jain and Kankanhalli’s “Hallucination is Inevitable: An Innate Limitation of Large Language Models” which shows that you cannot get rid of hallucinations in real world LLMs no matter what you do because they an inherent in the method. This limitation does not mean that there is no use for generative AI in financial services. There is plenty of work going on in Small Language Models (SLMs) right now. These are trained on fewer parameters, with weights and balances that are tailored to individual use cases. They hallucinate less (which should make mitigation more practical) and they are also faster and cheaper.
xxx
DeepSeek’s success points to an unintended outcome of the tech cold war between the US and China. US export controls have severely curtailed the ability of Chinese tech firms to compete on AI in the Western way—that is, infinitely scaling up by buying more chips and training for a longer period of time. As a result, most Chinese companies have focused on downstream applications rather than building their own models. But with its latest release, DeepSeek proves that there’s another way to win: by revamping the foundational structure of AI models and using limited resources more efficiently.
From: How Chinese AI Startup DeepSeek Made a Model that Rivals OpenAI | WIRED.
xxx
Snippings 