What's new in the world of AI

May 12, 2022
   One of the most consistent and important trends in AI has been the quest for increased generalization power in transformers, very large neural network architectures initially built for NLP. Without being tweaked for various other applications, in other words using exactly the same weights obtained during training, some transformers can now chat, play games, caption images or control a robot arm. The linked-in PDF describes such a system built by DeepMind: A Generalist Agent:

May 11, 2022
   I am very reluctant to refer to video clips of Elon Musk because he is being used too many times to simply draw attention. But there are also times when his interviews are simply too relevant to the future of AI, for anyone, including myself, to pretend otherwise. The WELT interview below, published three weeks ago, mixes in many themes important to our website and brings in current events, I hesitate to even add comments about it. So, without further ado, please watch it:

April 22, 2022
   Some interesting statistics about the conferences, the organizations, the individuals, and the countries with most published papers in ML and NLP in 2021: ML and NLP Publications in 2021.

April 19, 2022
   New architectures for neural networks allow them to learn entire operators, not just individual functions. Operators are mapping (infinite-dimensional) spaces of functions to other such spaces. This trend is one of the most exciting developments, as the cross fertilization between AI and mathematics continues. Many important natural processes studied in science and engineering (turbulence around airplanes for example) are described by Partial Differential Equations, notoriously difficult to solve. These new neural architectures allow for much faster solutions, practically impossible to find with older solvers. Latest Neural Nets Solve World’s Hardest Equations Faster Than Ever Before. One such architecture, the Fourier Neural Operator, is described in the video below. It's quite technical and long. I'm working on a larger note, "AI and Mathematics", in which I will attempt to explain these new developments in a more popular tone.

March 10, 2022
   The neural network architectures that make up transformers continue to be tweaked so that they can be used for data other than their original application, NLP. This Quanta article reviews the progress: Will Transformers Take Over Artificial Intelligence?.

From the article: "The success of transformers prompted the AI crowd to ask what else they could do. The answer is unfolding now, as researchers report that transformers are proving surprisingly versatile. In some vision tasks, like image classification, neural nets that use transformers have become faster and more accurate than those that don’t. Emerging work in other AI areas — like processing multiple kinds of input at once, or planning tasks — suggests transformers can handle even more."

February 1, 2022
   3D convolutional networks are good at aggregating local context while the self-attention mechanisms in transformers are good at capturing global dependencies. To better learn from video data, a combined architecture is proposed. This is one trend that is sure to strengthen in the coming months and years: UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learning.

January 25, 2022
   One of the most intriguing approaches to AI is taken by Numenta, a company started by Jeff Hawkins. Numenta's approach is described in very readable terms in the excellent blog The Path to Machine Intelligence: Classic AI vs. Deep Learning vs. Biological Approach. The blog makes references to Hawkins' Thousand Brain Theory, described in more detail in the book with that same title. One of the main ideas in all this work is that intelligence is still an exclusive and distinguished property of the human brain and it is not yet present in machine learning in general or in deep learning in particular. Numenta follows that dual track, neuroscience and machine learning, in their research. The following interview with Hawkins and Subutai Ahmad, who leads the machine learning track, covers some of these ideas. It is being conducted by Peter Norvig, formed Director of Research at Google, now a Distinguished Education Fellow at the Stanford Institute for Human-Centered AI, and the author with Berkeley's Stuart Russell of the most authoritative textbook on modern AI: Artificial Intelligence: A Modern Approach. Here it is:

November 18, 2021
   This is one of the most fascinating areas for me, I have been interested in what is now called Geometric Deep Learning for some time. And more generally, the synergy between AI and various areas of mathematics. The synergy with more applied kind of mathematics was well understood, but it wasn't obvious at all that AI can do geometry (or algebra or analysis or topology or number theory) or that these mathematical disciplines can be used in AI.

This is changing rapidly. While the mathematics used in standard deep learning is not complicated, Graph Neural Networks (GNNs) have been more difficult to tackle with elementary methods. Differential geometry and algebraic topology bring a new perspective. This Quanta article reviews the progress: Graph Neural Networks through the lens of Differential Geometry and Algebraic Topology.

November 1, 2021
   Below is yesterday's 60 MINUTES interview with Yuval Noah Harari about the future of AI. AI is being increasingly given data not just about where we go, what we do, what we buy, and how we vote, but also biometric data about our biological makeup. The notion of hacking another human being (ethically or not), which Harari says it means knowing that person better than they know themselves, is identical with the concept of a digital twin in the merged national graphs, which is a central concept on our website.

June 3, 2021
   This is an interview with Dr. Henry Kissinger in which he explains how he got interested in AI. It brings a unique perspective to our topic from an icon of US diplomacy, and it's especially relevant to the article AI in a Bipolar World. You might also want to listen to his interview at the Bush Center about the US, Russia, China, and AI, for additional insights. In more recent interviews with Dr. Kissinger you will also find support for one of our main points, namely the need to ramp up our AI efforts in the US considerably, and not just in the private sector.

May 20, 2021
   Transformers have been the most interesting new development in neural networks, especially in language applications, where in the beginning neural networks did not enjoy the same success as in computer vision applications. I've started to write a short technical note about them here: Transformers.

February 15, 2021
   Will AI replace or augment humans in the workforce? This is a difficult question and many very different answers have been given. This is a thoughtful presentation of this dilemma: Artificial Intelligence And The End Of Work. Most technologists lean towards augmentation. But the author of the article is leaning towards replacement, not augmentation, and reflects on what this might mean for humanity. From the article: "The myth of augmentation has spread far and wide in real-world contexts, too. One powerful reason why: job loss from automation is a frightening prospect and a political hot potato. Let’s unpack that. Entrepreneurs, technologists, politicians and others have much to gain by believing—and by persuading others to believe—that AI will not replace but rather will supplement humans in the workforce. Employment is one of the most basic social and political necessities in every society in the world today. To be openly job-destroying is therefore a losing proposition for any technology or business."

February 10, 2021
   AI can pose new mathematical conjectures, a first result of this kind. Posing good conjectures in mathematics seemed to be a very creative human endeavor, so the fact that AI can sift through enough math and create interesting conjectures is pretty significant. Machines Are Inventing New Math We've Never Seen, published by researchers from the Technion in Haifa and from Google's operation in Tel Aviv.

December 17, 2020
   AI has been used in many ways to fight the COVID-19 pandemic, especially for the development of new vaccines. This time it is being used after the vaccines are administered, to track the after effects. From the article: "The technology will also track issues or trends related to ethnicity, age, gender, or other demographic factors that can come into play with the vaccine" : How AI is Helping with COVID-19 Vaccine Rollout and Tracking.

November 11, 2020
   Mathematics can be looked at as a game, played according to rules specified within what we call formal systems. In theory, a computer can be programmed to produce all the theorems of such a formal system. So it could produce all the algebra, the geometry, and the calculus you have learned in school. In practice though, mathematicians still have to rely on a great deal of inventiveness to find such proofs. Nevertheless, the strive to develop proof assistants has never ceased and there have been many notable but disparate successes. Now this work is being pushed even further, and in a more organized way.

From the article The Effort to Build the Mathematical Library of the Future: "Digitizing mathematics is a longtime dream. The expected benefits range from the mundane—computers grading students’ homework—to the transcendent: using artificial intelligence to discover new mathematics and find new solutions to old problems. Mathematicians expect that proof assistants could also review journal submissions, finding errors that human reviewers occasionally miss, and handle the tedious technical work that goes into filling in all the details of a proof."

If this interests you in a more serious way, I have written a program of study for formal software development, in which these ideas are treated in more depth: Formal Software Development Program. It's in a slide presentation format, which allows a smooth gradual viewing of each page; view it in Adobe Acrobat Reader (free download from Adobe), not in your browser.

November 3, 2020
   An interesting view on the future of Artificial Intelligence, in an interview with Geoffrey Hinton, a pioneer of deep learning techniques. For his contributions, Hinton was awarded the Turing Award last year, together with Yann LeCun and Yoshua Bengio.

Interviewer: "You think deep learning will be enough to replicate all of human intelligence. What makes you so sure?"

Hinton: "I do believe deep learning is going to be able to do everything, but I do think there’s going to have to be quite a few conceptual breakthroughs. For example, in 2017 Ashish Vaswani et al. introduced transformers, which derive really good vectors representing word meanings. It was a conceptual breakthrough. It’s now used in almost all the very best natural-language processing. We’re going to need a bunch more breakthroughs like that."

October 12, 2020
   A timely Forbes article on some future directions of AI, with an informative look at the types of networks called transformers, the recent success they have had in NLP, and their potential uses in other areas, like computer vision. The Next Generation Of Artificial Intelligence. I'll probably add it to the permanent "Further Reading" list as a complement to the Main AI Concepts article.

October 6, 2020
   NVIDIA Uses AI to Slash Bandwidth on Video Calls. Obviously, this is even more meaningful given the current reliance on video calls by businesses and individuals, given the covid health crisis. From the article:
"What the researchers have achieved has remarkable results: by replacing the traditional h.264 video codec with a neural network, they have managed to reduce the required bandwidth for a video call by an order of magnitude. In one example, the required data rate fell from 97.28 KB/frame to a measly 0.1165 KB/frame – a reduction to 0.1% of required bandwidth. The mechanism behind AI-assisted video conferencing is breathtakingly simple. The technology works by replacing traditional full video frames with neural data. Typically, video calls work by sending h.264 encoded frames to the recipient, and those frames are extremely data-heavy. With AI-assisted video calls, first, the sender sends a reference image of the caller. Then, instead of sending a stream of pixel-packed images, it sends specific reference points on the image around the eyes, nose, and mouth."

September 13, 2020

September 12, 2020
   This is an article that meshes well with the theme found in our AI in a Bipolar World article, namely that US and China are leaving the world behind. Europe Feels Squeeze as Tech Competition Heats Up Between U.S. and China

September 10, 2020
   The Autocomplete feature in the Google search uses ML on the data collected from users. Quote from the article: "The company says that it will now remove any Autocomplete predictions that seem to endorse or oppose a candidate or a political party, or that make a claim about voting or the electoral process. That would mean eliminating predictions like 'you can vote by phone,'' 'you can’t vote by phone' or anything suggesting that you donate to a party or candidate.". here

August 14, 2020
  Using an unconventional method, machine learning, first-year physics students solve a 30-year old problem in astronomy. here.

July 24, 2020
  Apple has been the top buyer of AI startups, but it has not yet translated that power into visible results, as for example Siri is trailing all the other major personal assistants. here.

July 24, 2020
  The biggest multidisciplinary design company in Russia passed off an AI designer as human for more than a year, and no one caught on. here.

July 19, 2020
  Although the article has been published in April, it is still making its rounds. It contains a very good explanation of the differences between neuromorphic computing chips and AI accelerators. They DO NOT target the same AI applications. here.

June 28, 2020
  A nice review of the role of AI in combating the current COVID-19 crisis, from the Aspen Ideas Festival (it's a long video) is now on YouTube. Watch it directly on YouTube, the id of the video is 8LShJa_ovCY. You can find a fuller coverage of the festival here.

June 12, 2020
   The Elements of Statistical Learning (the bible for those of us working in this field), by Trevor Hastie, Robert Tibshirani and Jerome Friedman, is among the books released for free.

June 5, 2020
  Lawmakers introduced bicameral legislation this week aimed at keeping the United States ahead of China and other adversaries in AI efforts

May 14, 2020

April 17, 2020
  AI Uncovers a Potential Treatment for Covid-19 Patients

April 16, 2020
  MIT’s AI predicts catastrophe if social distancing restrictions relax too soon

March 17, 2020
  DeepMind’s Protein Folding AI Is Going After Coronavirus

March 12, 2020
  AI could help with the next pandemic — but not with this one

February 28, 2020
  European Commission’s White Paper on Artificial Intelligence

February 20, 2020
  How Technology Is Changing the Future of Higher Education

January 23, 2020
  China cares as deeply about AI ethics as the US, Microsoft CEO says, as he calls for global rules