Aardvark Daily

New Zealand's longest-running online daily news and commentary publication, now in its 25th year. The opinion pieces presented here are not purported to be fact but reasonable effort is made to ensure accuracy.

Content copyright © 1995 - 2019 to Bruce Simpson (aka Aardvark), the logo was kindly created for Aardvark Daily by the folks at

Please visit the sponsor!
Please visit the sponsor!

AI is now on steroids

12 Jun 2024

Yes, it's AI (artificial intelligence) time again.

Right now, large language models (LLMs) are forms of AI that require huge amounts of computing resources. In fact, the estimates are that if AI adoption continues at its present rate, a significant amount of the energy mankind presently uses will be going into the systems on which these LLMs run.

This tech also requires the use of expensive hardware such as AI processors or repurposed GPUs, due to the extraordinary number of calculations that must be performed at extremely high speed.

However, all of this may be about to change, at least a little bit.

On of the most common and numerically burdensome parts of an LLM's processing is matrix multiplication (MatMul).

A paper published by a group of researchers claims to have come up with a way to produce LLMs without the burden of MatMul operations and, if this is so, the cost, energy requirements and speed of LLMs could be about to undergo a significant change.

According to the paper, training speeds can be hiked by over 25 percent and memory requirements, after further optimisation, could be reduced by a factor of 10.

In systems that are as large as those usually required for commercial-grade LLMs, this can represent a huge saving on hardware costs to deliver the same level of performance as MatMul-based systems.

Also, because the very complex and processor-intensive MatMul processing is avoided, researchers have been able to build a custom FPGA-based accelerator that processes billion-parameter scale models using just 13W of power, thus demonstrating the potential for "brain-like efficiency" in future lightweight LLMs.

What are the implications of all this?

Well I'm no LLM/AI specialist and I have to concede that most of the contents of this paper soar miles over my head and level of understanding but it would appear that this breakthrough could lead to faster, cheaper and more energy-efficient AI processing and that will result in more AI everywhere.

Is that a good thing?

I don't know.

Because I'm now in the second year of my eighth decade on the planet, part of me is averse to the level of change that AI is bringing to our world. Fortunately however, I am aware of this age-related bias and thus do not discount the value of this new tech.

AI is just a tool (at this stage) so whether it's a good thing or a bad thing will be entirely determined by how we actually use it.

If we recklessly apply it to everything we do then some of the outcomes will be far less than optimal. However, if we're prudent and measured in our use of AI, the benefits could be huge and may produce see unprecedented progress in the areas of science and technology during the years ahead.

For those who might want to play around with AI at an experimenter's level then it seems that a new NPU (Neural Processing Unit) has been announced for the good old Raspberry Pi. The Hailo AI kit promises to bring AI capabilities to the humbe RPi for a suitably low price and is something I may pick up so that I can become a little more familiar with our soon-to-be computer-based overlords.

Carpe Diem folks!

Please visit the sponsor!
Please visit the sponsor!

PERMALINK to this column

Rank This Aardvark Page


Change Font

Sci-Tech headlines



The EZ Battery Reconditioning scam

Beware The Alternative Energy Scammers

The Great "Run Your Car On Water" Scam


Recent Columns

Parlez vous Python?
I've been cutting code for decades...

A career change for me?
When I was a kid, I was the quintessential geek...

Why I might change my mind about YouTube
Despite being a relatively successful content creator on the YouTube platform I've long been a critic of the company and many of its policies...

The history of Ms Susan Law, CEO of the SWDC
I recently made a submission on the South Waikato District Council's "Long Term Plan"...

Brute force is not the answer to CPU performance
Ever since the ubiqutous 8080 set the benchmark for 8-bit microprocessor chip design...

Forget fusion generators
It is interesting to note that China now outspends the USA in its investment into fusion reactor development...

Is digital ID too risky?
Around the world, governments are rolling out digital ID systems for their citizens...

Speed limiters coming to NZ soon?
I've already written columns on the strange paradox that sees us driving cars which are capable of more than twice the legal speed limit...

Arming the Karens
There is a real discussion going on in the drone community right now...

How far has the media fallen?
Sadly, I believe the mainstream media as reached a new low...