Will DeepSeek democratise AI?

Will DeepSeek be a game changer like ChatGPT was? The benefits and the public release of their models could level the playing field, as Mike Pound of the University of Nottingham argues in this Computerphile video. But will it be able to challenge the current quasi-monopoly, and democratise AI? And how accurate are the DeepSeek claims anyway?

Lowering the cost

A lot of work so far has gone into creating “Big Brain” general-purpose systems, but at a cost:

  • training such models has become extremely expensive, in terms of computations, but also in terms of curating training materials;
  • and even using them is expensive because of their size.

More recently, AI companies work on a Mixture of Experts approach: instead of a “Big Brain”, it works with smaller sections of the model, each with more specialised expertise. This allows for both more tailored training, as well as more resource-efficient use of the model.

DeepSeek has reportedly also managed to optimise algorithms for training and using the model.

Together, this means the cost of a training run for a model has gone down from the order of 100s of millions of dollars to millions of dollars.

However, as sketched by Martin Vechev, the Director of Bulgarian INSAIT (Institute for Computer Science, Artificial Intelligence and Technology):

Purchasing the hardware for a training run still costs in the order of 10s of millions of dollars. Multiple runs may be needed. And you still need to get properly curated training data, even if less may be needed than before.

Making it open

The cost of the technology is one aspect. Another aspect is actually having access to the details of the technology.

Existing systems are increasingly working with a Chain of Thought approach: the system breaks a question or problem down in smaller steps, to solve one after another. This creates better results for more complex questions. However, most providers keep the details of their approach still secret.

DeepSeek published their approach as open source, making it available to many more researchers and engineers. This improves transparency and accountability, and can help audit systems for unwanted bias and alignment with public interest values.

Again, as a however: it is unclear on which data DeepSeek’s models are trained, so we need to wait and see an organisation train their own high-performance model with more transparency on their training data, to verify the claims.
https://therecursive.com/martin-vechev-of-insait-deepseek-6m-cost-of-training-is-misleading/

Challenge for the incumbent business model?

The business model of venture capital-backed companies usually relies on being able to shield a particular innovation long enough with intellectual property rights and market domination to get a return on investment.

The currently dominant parties in the AI landscape seem to rely on their ability to raise billions of dollars of capital, needed to train and run the models.

A more level technical playing field allows more non-profits or government agencies to develop their own AI systems. But is that enough to democratise AI?

I am sceptical: the deep pockets of the dominant parties generally will, by itself, lead to a continued concentration of power.

Let’s also look at “Europe”, and its desire for more technical autonomy. Embracing the newly available technology is not enough. We also need to anchor our requirement for more openness and autonomy into our own strategies and policies. We need to support institutions and service providers that comply with those.

And we need to critically look at, and verify DeepSeek’s claims, just as much as we look at claims of “AGI” by the incumbents.

chevron_left

Leave a comment

Your email address will not be published. Required fields are marked *

Comment
Name
Email
Website

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Discover more from Rolf Kleef

Subscribe now to keep reading and get access to the full archive.

Continue reading