Full report here: https://www.stateof.ai/
I’m far from an AI expert, just an interested student who gets the tingly feels every time I use Stable Diffusion or see output from ChatGPT.
Snippets (copied verbatim):
–
The chasm between academia and industry in large scale AI work is potentially beyond repair: almost 0% of work is done in academia.
–
Finding faster matrix multiplication algorithms, a seemingly simple and well-studied problem, has been stale for decades. DeepMind’s approach not only helps speed up research in the field, but also boosts matrix multiplication based technology, that is AI, imaging, and essentially everything happening on our phones.
–
The authors argue that the ensuing reproducibility failures in ML-based science are systemic: they study 20 reviews across 17 science fields examining errors in ML-based science and find that data leakage errors happened in every one of the 329 papers the reviews span
–
many LLM capabilities emerge unpredictably when models reach a critical size. These acquired capabilities are exciting, but the emergence phenomenon makes evaluating model safety more difficult.
–
Alternatively, deploying LLMs on real-world tasks at larger scales is more uncertain as unsafe and undesirable abilities can emerge. Alongside the brittle nature of ML models, this is another feature practitioners will need to account for.
–
Landmark models from OpenAI and DeepMind have been implemented/cloned/improved by the open source community much faster than we’d have expected.
–
Compared to US AI research, Chinese papers focus more on surveillance related-tasks. These include autonomy, object detection, tracking, scene understanding, action and speaker recognition.
–
NVIDIA’s chips are the most popular in AI research papers…and by a massive margin
–
“We think the most benefits will go to whoever has the biggest computer” – Greg Brockman, OpenAI CTO
–
As such, the AI could reliably remove 36.4% of normal chest X-rays from a primary health care population data set with a minimal number of false negatives, leading to effectively no compromise on patient safety and a potential significant reduction of workload.
–
The US leads by the number of AI unicorns, followed by China & the UK; The US has created 292 AI unicorns, with the combined enterprise value of $4.6T.
–
The compute requirements for large-scale AI experiments has increased >300,000x in the last decade. Over the same period, the % of these projects run by academics has plummeted from ～60% to almost 0%. If the AI community is to continue scaling models, this chasm of “have” and “have nots” creates significant challenges for AI safety, pursuing diverse ideas, talent concentration, and more.
–
Decentralized research projects are gaining members, funding and momentum. They are succeeding at ambitious large-scale model and data projects that were previously thought to be only possible in large centralised technology companies – most visibly demonstrated by the public release of Stable Diffusion.