Sam Altman Says AI Using Too Much Energy, Will Require Breakthrough Energy Source

The problem there is the paradox of efficiency, making something more efficient ends up using more of it not less as the increase in use stimulated by the greater efficiency outweighs the reduced input used.

It's not so much the hardware as it is the software and utilisation, and by software I don't necessarily mean any specific algorithm, because I know they give much thought to optimisation strategies when it comes to implementation and design of machine learning architectures. What I mean by software is the full stack considered as a whole, and by utilisation I mean the way services advertise and make use of ill-suited architectures.

The full stack consists of general purpose computing devices with an unreasonable number of layers of abstraction between the hardware and the languages used in implementations of machine learning. A lot of this stuff is written in Python! While algorithmic complexity is naturally a major factor, how it is compiled and executed matters a lot, too.

Once AI implementations stabilise, the theoretically most energy efficient way to run it would be on custom hardware made to only run that code, and that code would be written in the lowest possible level of abstraction. The closer we get to the metal (or the closer the metal gets to our program), the more efficient we can make it go. I don't think we take bespoke hardware seriously enough; we're stuck in this mindset of everything being general-purpose.

As for utilisation: LLMs are not fit or even capable of dealing with logical problems or anything involving reasoning based on knowledge; they can't even reliably regurgitate knowledge. Yet, as far as I can tell, this constitutes a significant portion of its current use.

If the usage of LLMs was reserved for solving linguistic problems, then we wouldn't be wasting so much energy generating text and expecting it to contain wisdom. A language model should serve as a surface layer -- an interface -- on top of bespoke tools, including other domain-specific types of models. I know we're seeing this idea being iterated on, but I don't see this being pushed nearly enough.[^1]

When it comes to image generation models, I think it's wrong to focus on generating derivative art/remixes of existing works instead of on tools to help artists express themselves. All these image generation sites we have now consume so much power just so that artistically wanting people can generate 20 versions (give or take an order of magnitude) of the same generic thing. I would like to see AI technology made specifically for integration into professional workflows and tools, enabling creative people to enhance and iterate on their work through specific instructions.[^2] The AI we have now are made for people who can't tell (or don't care about) the difference between remixing and creating and just want to tell the computer to make something nice so they can use it to sell their products.

The end result in all these cases is that fewer people can live off of being creative and/or knowledgeable while energy consumption spikes as computers generate shitty substitutes. After all, capitalism is all about efficient allocation of resources. Just so happens that quality (of life; art; anything) is inefficient and exploiting the planet is cheap.

[^1]: For example, why does OpenAI gate external tool integration behind a payment plan while offering simple text generation for free? That just encourages people to rely on text generation for all kinds of tasks it's not suitable for. Other examples include companies offering AI "assistants" or even AI "teachers"(!), all of which are incapable of even remembering the topic being discussed 2 minutes into a conversation. [^2]: I get incredibly frustrated when I try to use image generation tools because I go into it with a vision, but since the models are incapable of creating anything new based on actual concepts I only ever end up with something incredibly artistically compromised and derivative. I can generate hundreds of images based on various contortions of the same prompt, reference image, masking, etc and still not get what I want. THAT is inefficient use of resources, and it's all because the tools are just not made to help me do art.

It's emulating a ridiculously simplified brain. Real brains have orders of magnitude more neurons, but beyond that they already have completely asynchronous evaluation of those neurons, as well as much more complicated connecting structure, as well as multiple methods of communicating with other neurons, some of which are incredibly subtle and hard to detect.

To really take AI to the next level I think you'd need a completely bespoke processor that can replicate those attributes in hardware, but it would be a very expensive gamble because you'd have no idea if it would work until you built it.

Some of the smartest people on the planet are working to make this profitable. It's fucking hard.

[Take a look at] hugging face. Power consumption is about the biggest topic you find with anyone in the know.

^ fair comment

I've seen that film

We use all of our brain. Well, some of us try to anyway.

I would love it (if there exists a FOSS variant of that) imagine being able to run a LLM, or even LAM in your head,

wait…

🤔

And yet we have brains. This brute force approach to machine learning is quite effective but has problems scaling. So, new energy sources or new thinking?

It'd be way easier to just grow brains instead

It's so, so, so much better. GenAI is actually useful, crypto is gambling pretending to be a solution in search of a problem.

There's no way these chatbots are capable of evolving into Ultron. That's like saying a toaster is capable of nuclear fusion.

I think we've got a bit before we have to worry about another major jump in AI and way longer for an Ultron. The ones we have now are effectively parsers for google or other existing data. I personally still don't see how we feel like we can get away with calling that AI.

Any AI that actually creates something 'new' that I've seen still requires a tremendous amount of oversight, tweaking and guidance to produce useful results. To me, they still feel like very fancy search engines.

I can already locally host a pretty decent ai chatbot on my old M1 Macbook (llama v2 7B) which writes at the same speed I can read, its probably already possible with the top of the line phones.

Almost all of our training was done without requiring burning fossil fuels. So maybe ole Sammy can put the brakes on his shit until it’s as fuel efficient as a human brain.

But the training has already been done, no?

Yeah, I'd say it was worth it. It also gave us this:

Worth every flood and fire

General artificial intelligence has the potential to be actually useful. Generative AI, like Chat GPT (Generative Pre-Trained Transformer) absolutely does not. It’s a glorified autocomplete.

In fact, the original script of The Matrix had the machines harvest humans to be used as ultra efficient compute nodes. Executive meddling led to the dumb battery idea .

not really. It's just different energy. Calories can be converted to a unit of heat, a unit of heat is directly analogous to a unit of energy. Electricity is a unit of energy as well. Thus you can compare them. It's how you compare things like electrical production efficiency of a thermal cycle generation process.

At least in Doom they had sense enough to do it on Mars

Yeah but our hell is out of power. Who cares about that other hell

It's more of a purgatory, or maybe fantasy has no part in dealing with reality

All

I bought pizza with bitcoin, haven't bought anything with AI yet.

It's simultaneously possible to realize that something is useful while also recognizing the damage that its trend is causing from a sustainability standpoint, and that neither realization particularly demonstrates a lack of understanding about AI.

wasn't the same thing said about ATM's? and then it created the need for banks to hire more employees?

iirc, technology/robots has only been able to create more jobs, right? or am I misinformed?

This is a silly take, people have benefitted hugely from all the big tech developments in the past and will do from ai also - just as you have a mobile phone that can save and improve your life in a myriad of ways so you'll have access to various forms of ai which will do similar. GPS is a good example, functionally free and making navigation far safer, faster, and better.

Here's a genuine already happened use case for ai benefitting you, an open source developer was able to add a whole load of useful features to their free software by using AI to help code - I know because it was me, among many many others.

I know people making open source ai tools too and they're all using AI coding assistants - mostly the free ones. I've seen a lot of academic researchers using AI tools also generally built using open source tools like pytorch and with help from ai coding tools. Even if you don't use ai yourself you're already benefitting from it, even if you don't use open source software the services you rely on do.

Imagine being able to implement the most advanced and newest methodologies in your design process or get answers to complex and niche questions about new technology instantly. You buy a printer for example and say to your computer 'I've plugged in a printer make it work' and it says 'ok, there isn't a driver available that'll work with your pc but I've written one based on the spec in the datasheet, do you want me to print a test page?'

Imagine being able to say 'talk me through diagnosing a fault on my washing machine' and it guides you through locating and fixing the fault, possibly by designing a replacement part and giving you fabrication options.

Or being able to say 'this website is annoying, change it so that I only see the video window' or 'make a playlist in release order of all abba songs that charted' or 'check on currently available archives to see if there's a mirror of this deleted post' or 'check all the sites and see if anyone posted a sub version of the next episode of this anime' or 'Keep an eye on this lemmy community and add any popular memes involving fish to my feed but don't bother with any meta stuff or aquatic mammals' or 'this advert says I can make free money, is it ligit?'

The use cases that will directly benefit your life are almost endless, natural language computing is a huge deal even without task based solvers and physical automation but we also have those too so the increased ability of people to make community projects and freely shared designs is huge.

It's important to keep in context who is building them, how they're being built, and with what oversight they are built.

We are in no way perfect in the west but we are easily a century ahead in insuring build quality and regulatory oversight.

Solar alone can't meet humanity's energy needs without breakthroughs in energy storage.

Most energy we use the grid for is generated on demand. That means only a few moments ago, the electricity powering your computer was just a lump of coal in a furnace.

If we don't have the means to store enough energy to meet demands when the sun isn't out or wind isn't blowing, then we need more sources of energy than just sun and wind.

There is a lot of misinformation being perpetuated by the solar industry to fool people like you into thinking all investments should be directed to it over other options.

Please educate yourself before parroting industry talking points that only exist to take people for a ride.

We have a system to distribute electricity

That requires someone in business to think

I'm not convinced that Altman has cleared this beyond meaningless buzzwords

As opposed to humans who think all the time. Florida for example.

1 GW of solar is much cheaper than 1 GW of nuclear. Solar is both cheaper to build and cheaper to run. It's the most efficient energy source e currently have.

The sun is a fusion reactor.

Circles are just marketing hype. Now ovals are a thing, and you can buy some pretty neat ones.

If you think about it, a transistor is a bit like a hardware if statement, it's if statements all the way down.

Threat to existence is nothing compared to ability to hoard more money

love a good thorium based metal cooled reactor

Massively subsidized

Nuclear energy is four times cheaper than renewables when externalities like baseline generation are imputed: https://www.sciencedirect.com/science/article/abs/pii/S0360544222018035?via%3Dihub

where do you put all the nuclear waste?

While more dangerous, the quantity of waste generated compared to all other forms of energy generation is very small. Storage is a solved problem, but you have probably read articles about a lack of storage in the U.S. This is entirely due to politicians' failure to agree on where to store waste. Despite the relative safety, no one wants nuclear waste stored in their "back yard."

And before you say it: no, nuclear energy is not green.

Nuclear energy generates zero CO2. Surely we can agree that this is the most pressing consideration in terms of climate change. If your concern is the nuclear waste, then I direct you to the growing problem of disposing of solar cells and wind turbines. Newer turbine blades, for example, are 40 meters long and weigh 2.5 tons. These cannot be recycled.

No matter how you cut the data, nuclear is an order of magnitude better than almost all other forms of energy generation. If our goal is to radically improve our environmental footprint while keeping the lights on even at night when it's not windy, then nuclear absolutely must be part of the mix.

you put the nuclear waste in a hole, deep underground, after burning most of it up. Modern gen 4 designs can burn the vast majority of existing waste products down to a much more reasonable time span.

Nuclear energy is vastly more green than, coal, gas, petro, etc... Currently arguably more sustainable than massive amounts of solar and wind energy. Wind in particular has a massive waste issue, solar, it's more complicated but there are a lot of precious metals involved and heavy refining done. It's not a zero emissions industry either. The actual production of electricity IS net zero, unlike coal, petro, and gas, which still powers the majority of our grids. Please continue to explain to me how fossil fuels are better than funny green rock.

You're also accusing me of knowing nothing about nuclear, which is funny, considering i have quite the autistic hyper-fixation on it. And know vastly more about it than the average person. Judging by your response, you're probably not in the field of nuclear energy either.

Nuclear is a technology we know how to build, understand how to operate safely, and are capable of doing correctly. The only thing we need, is more nuclear plants.

It blows my mind honestly. This is such a young technology that commercialization at this point seems ridiculous

While I’m too much of an optimist to say that we’ll never figure out viable fusion power, I do think you’re more right than wrong.

Fission power is essentially us discharging a fusion battery, where the battery was charged by a supernova. We don’t get any free help with fusion, and we have to replicate input energies only seen in nature with stellar amounts of gravitational mass. It is (IMO) an important area of research, but I don’t expect it to power our cities in my lifetime.

I would also like this

Elon will probably start taking preorders for Dyson spheres soon