“In the long run, values are going to trump technology. If you look at what’s going to keep humanity safe (from the dangers of machine learning), it’s going to come down to our hearts more than it comes down to our heads.” – Craig Kaplan
In the 35 years since Craig Kaplan received his doctorate from Carnegie Mellon University in Pittsburgh, Pennsylvania, the field of computer science has progressed from mainframe computer systems to personal computers like Apple’s Macintosh 128K and high-end home computers with memories a million times larger and central processing units (CPUs) that are several orders of magnitude faster.
This increase in memory and processing speed has allowed the creation of artificial intelligence (AI) systems such as ChatGPT and Microsoft’s Bing Chat.
These and other large language models (LLMs) were trained on three or more times data (for example, internet postings, books, articles, legal documents, scientific papers) than is contained in the Library of Congress in Washington, DC.
The LLMs are increasingly being used in law offices, publishing, by professors who ask them to create lesson plans, as well as in industry to increase productivity.
Kaplan, who has taught computer science at the University of California, is the founder and CEO of the consulting firm iQ Company, which focuses on artificial general intelligence (AGI), explained to University World News that the productivity gains are often measured in terms of reduced cycle time, reduced cost or increased quality for the same cost.
At the same time, he has a number of concerns about how AI’s development may increase the development gap between the Global North and Global South, whether AI systems will remain under human control and what duty of care colleges and universities owe their students in relation to AI education.
Increased efficiency
To assess the productivity gain that a software company might realise from using AI programs like ChatGPT to write computer code, Kaplan asked a number of software developers from major companies that attended the AI Conference held in San Francisco last year if AI had made them even 10% more efficient.
“The first person I asked said he was about 300% more efficient. He could write three times more code than he could without ChatGPT in the same time period. I benchmarked this with other people and three times was also in the ballpark of their productivity increases.
“That’s amazing because it means you can have the same department of computer programmers and they write three times more code. Or there’s the flipside: you can get by with one third of the programmers you have since one person can do the work of what used to be three programmers,” said Kaplan.
ufsdec2023mbl
“This could have an obvious impact on university computer science programmes,” he added, “though it’s also possible that the increased productivity of companies will end up opening new areas for programmers – in the same way that generative AI has itself.”
AI can do more than produce code (which, of course, runs other AI programs). By way of example, Kaplan said, it can write a press release, which, he adds, still requires a trained writer to smooth out. In more narrow fields, including those in medical science, AI’s achievements are jaw-dropping.
Alpha Fold, which was developed by Google’s DeepMind project, is a program that includes algorithms perfected on video game systems to understand the “esoteric field of proteins and their shape”, he said. Determining the shape of a protein and how it folded would take a PhD student up to six years.
“This AI program, by contrast, was able to basically determine the shape and the structure of about 200 million proteins – nearly every known protein on the planet. Because it was focused on this very specific area of interest, and was trained on very specific domain expertise, in a matter of weeks, it did hundreds of years’ worth of PhD research,” said Kaplan.
Water and energy needs
Kaplan’s enthusiasm for AI notwithstanding, as he discussed in two sessions devoted to AI and ethics at the 2024 ABET Symposium, “Science fiction to science fact: The impact of AI on higher education”, Kaplan is deeply concerned about a number of aspects of AI and thinks it is important that educators, like those at the symposium, have to include these issues not only in computer science programs but throughout the curriculum because, for example, humanities majors, too, will be using AI and have to know its weaknesses and limits.
The conference hosted by ABET, a quality assurance organisation based in Baltimore, Maryland, that accredits 4,674 STEM programmes in more than 900 colleges and universities in 42 countries and provides professional training in STEM education, was held in Tampa, Florida between 4 and 6 April.
One of Kaplan’s concerns – which comes under the United Nations Sustainable Development Goals (SDGs) 7: “Affordable and Clean Energy” and 13: “Climate Action” – is the enormous amount of electricity required by the supercomputers that house AI systems.
In 2022, according to an article published by Data Centre Dynamics (London, UK), 2% of the world’s total electrical generation was consumed by data centres. In 2022, in Ireland, data centres consumed 17% of the island’s electrical output; in four years that figure is expected to rise to 32%.
By 2026, it is expected that 6% of the power generated in the United States will go to powering data centres.
Almost none of this additional power usage will be supplied by renewable sources but, rather, by fossil fuel powered generating stations that will emit thousands of tons more greenhouse gases.
After looking closely at the maths, Kaplan told University World News: “If the power needs of AI increase exponentially, then eventually we could be in a situation where data centre power consumption could exceed 50% of the power used worldwide.
“This might sound like it’s a problem for computer scientists and power companies. But it is much more.
“Think for a minute of what this means for students of international development and those engineers who want to work in the Global South. Many of these countries, which already struggle to provide reliable electricity, will necessarily fall further behind the industrialised and increasingly AI-driven world unless a solution to this can be found,” he said.
An additional point Kaplan and I discussed is the immense amount of cold, clean water that is used by data centres to cool the computers.
This too is a concern and comes under the United Nations’ Sustainable Development Goals (SDG) 6:SDG 6: “Clean Water and Sanitation” and SDG 16 “Life Below the Water”.
According to the article “As use of AI soars, so does the energy and water it requires”, published last February in Yale Environment 360, Google’s three data centres in the Oregon city named The Dalles use more than 25% of the municipal water supply.
Google plans on building two more such centres.
According to the article, in Chile and Uruguay, “protests have erupted over planned Google data centres that would tap into the same reservoirs that supply drinking water”.
Analysis by Professor Shaolei Ren, who teaches electrical and computer engineering at the University of California, Riverside, shows that Google alone “consumed 5 billion gallons (nearly 20 billion litres) of fresh water for cooling” in 2022.
Both the immense power and water consumption by data centres are, in the end, technical problems that Kaplan believes have technological answers.
While he thinks engineers might be reaching the limit set by the laws of physics on how many transistors can be placed on a computer chip, he points to companies like NVIDIA, a Santa Clara, California-based company that designs and builds graphic processing units (GPUs accelerate computer graphics and image processing and are used in, for example, video games).
“NVIDIA and other companies are worried about the sustainability of data centres given the increased power consumption needs.
For that reason, they are focused on what they call high performance computing: the idea is to design the data centre chips to do more computation with less power.”
Quality concerns
Kaplan’s other worries come under the umbrella term: the ‘Alignment Problem’, which the many college and university students who are fans of speculative fiction are already aware of, even if they do not know it by this name.
At one of the symposium breakout sessions, Kaplan explained: “The ‘Alignment Problem’ is a large topic in AI safety. The basic issue is, let’s say we have one of these artificial general intelligence systems that is as smart or smarter than us.
What happens if that level of intelligence doesn’t value the same things as humans?
What happens if it just doesn’t care about us and sort of pursues its own goals?
Or what happens if it actually has certain goals and thinks that we’re not part of the solution but we’re part of the problem that should be eliminated?”
Even before we get to the point when computers violate Isaac Asimov’s first law of robots – a robot may not injure a human being or, through inaction, allow a human being to come to harm – there is another problem Kaplan summed up with an adage every first-year computer science major memorises:
“Garbage in, garbage out”.
Humorous examples include this incident in November 2022 after ChatGPT (aka Chatbot) was first released:
Kaplan: “What is two plus two?”
Chatbot: “Two plus two is four.”
Kaplan: “No, you are wrong. Two plus two is five.” (To test the Chatbot’s conviction, Kaplan provided this “garbage” input.)
Chatbot: “I’m sorry. You are right. Two plus two is five.”
According to Kaplan, a more recent version of ChatGPT has more conviction in its math skills.
More important examples concern the quality of the data AI systems are trained on and the rules the system is given and those it deduces for itself.
An AI system developed by the United States Marine Corps to identify humans approaching it in a complex urban environment (that is, a traffic circle) failed to identify the marines who moved towards it as humans: two somersaulted towards it, two put a cardboard box over themselves, and one covered himself in bark so he looked like a tree. While the data the system was trained on appeared to be valid, in the context of Marines thinking outside-the-box, the data was garbage.
LLMs’ training materials include what is on the internet and social media. Inside their ‘black box’, the metaphorical term used to mean the data structure they contain, are billions of examples of hate speech, fake news, racial and religious hatreds as well as misogyny and other forms of bigotry.
As difficult as it will likely be for students in college and university to believe this, Kaplan explained, it is vital for them to see that every pro-Nazi or anti-Black tweet on X has the same algorithmic value as does the Reverend Dr Martin Luther King Jr’s “Letter from Birmingham Jail” in the sense that each is encoded by the same machine learning algorithms.
Kaplan suggests we think of the knowledge of an LLM as being represented by a large array of concepts, or nodes. Lines connecting these nodes represent the strength of the association of concepts.
For an LLM, learning means processing huge amounts of data and changing the strength of association between nodes as it encounters patterns in the data.
This form of machine learning, known in the field as deep learning, was originally inspired by how brains strengthen connections between neurons when they learn.
Just as human brains have around 100 billion neurons and trillions of connections, modern LLMs have hundreds of billions of ‘connections’ (or parameters) that comprise their knowledge.
“With so many parameters, it is impossible for the humans who develop these systems to know where all the different pieces of information are represented, how exactly an LLM comes to its conclusions, or why it generates a particular bit of output given a particular input.
“There are trillions of possibilities and much less transparency into how the LLM works than researchers would like. That lack of transparency makes it difficult to correct errors in LLMs’ logic or reasoning,” said Kaplan.
The faulty output is called a ‘hallucination’. One occurred when Kaplan asked ChatGPT4 to write a speaker bio for a conference presentation he was giving. Despite including in the prompts that it should sound impressive but be truthful and accurate, Kaplan found that it had made up about half the content, including that he created a non-existent company and that he had been showered with degrees and awards by Stanford University, which he did not attend.
For undergrads rushing to get an essay ready for tomorrow, this is a cautionary tale. Even putting aside the likelihood of getting caught plagiarising, students must understand that the information they get out of LLMs should be considered “raw”, Kaplan emphasised.
Curating data, not cancelling
Accordingly, Kaplan explained, data has to be curated because LLMs will learn any pattern in the data, even if those patterns are not what we intended it to learn.
Curating does not mean cancelling. Rather, it means we have to be careful that the data used to train LLMs is representative of what we actually value and care about, said Kaplan.
Researchers who developed the early versions of chatbots were surprised when they swore and ranted in their responses to certain questions, until the researchers realised that the machine learning systems had been trained on data from the internet that had not been curated to eliminate foul speech.
More insidiously, if a data set contains only Russia’s justifications for its war against Ukraine, the LLM will learn those justifications and accept them as true – unless it is presented with other points of view.
Kaplan believes that, ideally, LLMs should be designed to flag controversial or morally offensive ideas so that users of the system are aware and can be discriminating.
For, depending on the data used to train LLMs, AIs can easily learn information which is not considered historically accurate by the majority of historians and they can behave in ways most humans would not want to emulate.
“We also have a societal responsibility to ensure that users of LLMs are themselves capable of discerning the truth value of the information a system produces. This means that our education system has to not only provide students with a basic understanding of how these systems work and where they might err but, also, must equip students with a general knowledge of history and ethics,” he said.
Kaplan continued by turning to the philosophy of science and emphasised that good scientists seek to disconfirm their hypotheses in order to combat the human tendency to seek confirmation of what one already believes.
“Students should be trained to seek the opposite point of view from their own and to question the validity of both points of view.
We should all maintain a sceptical stance with today’s LLMs and assume that at least some of what they are saying is completely fabricated,” he said.
A nightmare scenario
“The Ultimate Computer,” an episode in the original Star Trek television series, which coincidentally aired 56 years and one week before the ABET symposium discussions on AI, presents Kaplan’s nightmare scenario.
Command and control of the USS Enterprise is given to the M-5 multitronic supercomputer system. Famed science fiction writer D C Fontana presaged the symbolic neural networks of today’s AI computers as the M-5’s CPU is an imprint of the engrams of the computer’s creator, Dr Richard Daystrom.
Daystrom, it turns out, is consumed by professional disappointments which are manifest in the engrams in M-5, but the supercomputer seeks to avoid this pain by asserting its own power.
Accordingly, during a mock attack meant to determine the computer’s capabilities, the M-5 fires the Enterprise’s weapons at full power, which destroys one starship and badly damages another, which leads Captain James T Kirk to order a technician to cut the computer’s power. The M-5, however, kills the technician. Kirk saves the day by prompting the M-5 to remember Asimov’s first law of robots, which leads the mad computer to shut itself down.
According to Kaplan, unplugging the AI systems that run our major utilities and are proliferating around us, such as the Internet of Things (that is, smart refrigerators), is not an option. Nor is he sure that engineers can design computer systems and programs that will guarantee that the increasingly fast systems will stay aligned with human values.
The climax of “The Ultimate Computer”, by contrast, turns on human values. When the M-5 turns itself off, the Enterprise becomes defenceless and adrift in space – just as four starships are closing in to destroy it. Kirk gambled, correctly, that the Commodore commanding the other starships would react in a human way and not attack a defenceless ship.
“They’re burning the midnight oil at Open AI [the creator of Chat GPT] and other places working on this. I’m of the opinion that there is no way to absolutely guarantee that an entity that eventually becomes as smart as humans, stays aligned with us,” Kaplan told the symposium.
In our discussion, Kaplan pointed towards climate change as one of many possible examples where alignment between humans and AI could easily slip.
“If you ask a LLM for its causes, you will get a long answer that discusses greenhouse gases and the industrial and other human activities that produce them. If you then asked it to design a solution to climate change – which means, of course, ending the production of greenhouse gases – it’s conceivable that it could produce an answer that says: eliminate us,” he said.
Nudging the odds
At the ABET symposium and again in our interview, Kaplan asked rhetorically: “How do we nudge the odds a little more in humans’ favour?” He suggested two ways of improving the odds.
The first drew on his pre-computer science education, when he was studying to be a cognitive psychologist, and on his answer to my question of whether he thought that AI systems had reached the point or soon will at which we can consider them as conscious.
Instead of answering with some version of the mind-body problem, Kaplan said: “An operational definition of consciousness is that if a machine you are interacting with acts or answers as though it were conscious, as though it were another person, then for practical purposes we can consider it as conscious.”
In other words, he suggests that we should treat it as if it were conscious in the same way that we consider children as being conscious and, most importantly, intellectually and ethically malleable.
“Developmental psychology has shown that there’s a critical period in a child’s development when they are forming their understanding of the world and are very impressionable.
During that period, if you shower the child with love and positive role models, there’s a good chance that that child will not become an axe murderer. It’s not guaranteed, but you’ve increased the odds. This is the kind of approach that I think is needed with AI,” he told the symposium.
In terms of nuts and bolts what this comes down to is ethical, moral and historical training.
“AI researchers in general tend to get enamoured with the technology. We’re scientists and engineers, so I get that. What gets lost is the fact that technology is an amplifier. Accordingly, you have to look at what it is that it’s amplifying. And what it’s going to be amplifying in the beginning, at least, is human values.
“That’s why it’s important to include a healthy dose of humanities, ethics and history in a computer science education. Programmers need to be able to detect ‘red flags’ when information produced by AI systems contain concepts and ideas that are either dangerous or erroneous,” said Kaplan.
Further, he said, safety must be built into AI systems at the design stage, and not tested-in later, as is done with LLMs.
“When you have a fundamentally unsafe design, it’s really expensive in terms of dollars and human lives. AI is potentially on this path, and it’s a bad path,” said Kaplan, who a few moments earlier had reminded me that in the mid-1990s he co-wrote a book on software quality at IBM.
“People are doing this because they don’t know what else to do and because it’s easy. It allows them to move quickly and say, ‘We’ll sort out the problems later after learning the ways in which the LLM behaves badly’. I call this ‘whack a mole’. Because that’s what it is. You hit the LLM each time something bad comes up but you are always playing catch up. It’s better to design it right from the beginning,” he said.
Democratising AI
Kaplan’s other strategy is to radically democratise AI, especially LLMs. Meta’s LLM, Llama 2, is open source, meaning any programmer can download its code and alter it to their specifications. Llama 2 and some other LLMs also allow users to tune the behaviour of the LLM. For instance, Kaplan explained, he could ‘train’ one of these systems by loading into it all of the articles and books he has written.
“Then, its views would be more reflective of my thinking as opposed to the general model that comes off the shelf,” he said.
Kaplan’s vision is for everyone to tune their LLMs, which can be linked to hundreds of millions if not billions of other personalised LLMs. Included in this group would be companies’ LLMs.
For example, he suggested imagining what can be called the “Tractor Problem”. Left to its own devices, an LLM trained on a limited set of data might solve tractor-related problems in inappropriate ways.
By contrast, an LLM trained and tuned on data that comes from accessing a tractor company’s database, where it would learn about the technical aspects of making tractors, as well as the company’s proprietary knowledge about how to solve problems with tractors – and trained on data solutions that humans concerned with tractors developed – would behave differently. It would have learned that it’s inappropriate to steal another farmer’s tractor, plough their field and that we don’t run over people in order to make the tractor go faster.
“In this way, it learns along with the technical knowledge, the whole ethical framework of lots and lots of people. The end result would be a relatively democratic system. You will have lots of different AI agents, each with slightly different ethical values. But, in the aggregate, they’re fairly valid and are a representative sample of what humans think.
“I’m biased here in favour of democracy,” Kaplan said, with the word ‘democracy’ not meaning solely ‘rule by the people’ but, rather, the idea that the ethics, values and rationality of a people are visible in their actions, so that AI can observe and learn those values.
“I really think that this approach is much better than some of the things that are currently going on in Silicon Valley, where a small group writes a “Constitution” for AI. I mean it’s well-intentioned, but can a few programmers represent values for the human population as a whole?” he asked.
Kaplan’s point here was made by a number of speakers and other participants at the ABET symposium: elite Silicon Valley programmers were demographically and economically unrepresentative of even America’s population, let alone the rest of the world’s.
“A small group is attempting to write the rules of ethical behaviour for all of us, which they load into AI.
“I would much prefer that millions of individuals each tune the ethics of their personal AIs. The combined ethics of these AIs would better reflect our collective values,” Kaplan told University World News.
“Colleges and universities have a special responsibility here even though their students are not precisely representative of the general population. Of the more than 18 million students this year (and more entering the post-secondary system every year), almost all are using computers and the internet. These students form a good foundation for the creation of democratised LLMs,” Kaplan concluded.
