Before you build on foundation models, it helps to know where they came from. Today’s generative AI looks like a sudden arrival, but it sits on top of roughly 70 years of research, two major collapses in funding and enthusiasm, and a much older dream: machines that reason. This post walks through that history, from the philosophical ancestors of AI to expert systems and the road to modern deep learning, and ends by connecting the old ideas to the services you will meet on AWS.
1. The ancestors: ideas before computers
The wish to build artificial minds is ancient. Greek myths described Talos, a bronze guardian, and Jewish folklore told of the golem. These were stories, not science, but the real groundwork came from people who tried to turn thinking itself into something mechanical.
- Ramon Llull (13th century): designed paper wheels that combined concepts mechanically to generate statements, an early attempt to produce knowledge by procedure.
- Blaise Pascal and Gottfried Leibniz (17th century): built early calculating machines. Leibniz also imagined a universal logical language in which disputes could be settled by calculation.
- George Boole (1854): showed that logic could be expressed as algebra, the foundation of the true/false operations inside every computer.
- Charles Babbage and Ada Lovelace (1830s–1840s): Babbage designed the Analytical Engine, a programmable mechanical computer. Lovelace’s notes described how it could manipulate symbols, and she cautioned that it could not originate anything on its own, a point still debated today.
- Frege, Russell and Whitehead: formal logic became rigorous enough to describe reasoning precisely.
2. The foundations: 1930s to 1950s
Three developments turned the dream into a research field.
- Computability: Kurt Gödel (1931) showed limits of formal systems, and Alan Turing (1936) defined what a machine can compute at all. This gave us the theoretical computer.
- Artificial neurons: in 1943 Warren McCulloch and Walter Pitts proposed a simplified mathematical model of a neuron, the seed of neural networks.
- The question of machine thought: in 1950 Turing’s paper “Computing Machinery and Intelligence” proposed the imitation game, now known as the Turing test, as a practical way to ask whether machines can think. Around the same time Claude Shannon’s information theory and early work on chess programs showed that machines could attack problems once thought to need human intelligence.
3. The birth of AI: the Dartmouth workshop (1956)
The term artificial intelligence was coined by John McCarthy in the proposal for a summer workshop at Dartmouth College, held in 1956. McCarthy organised it with Marvin Minsky, Nathaniel Rochester and Claude Shannon. Allen Newell and Herbert Simon presented the Logic Theorist, a program that proved mathematical theorems. Optimism was enormous, and some pioneers predicted human-level machines within a generation.
Early work split into two camps, and the split still shapes the field:
- Symbolic AI: intelligence comes from manipulating explicit symbols and rules, such as logic, search and planning.
- Connectionist AI: intelligence emerges from many simple units connected in a network that learns from data. Frank Rosenblatt’s perceptron in the late 1950s was the first famous example.
Other landmarks of this era include the LISP language (1958), Arthur Samuel’s checkers program that learned from experience (he popularised the term “machine learning”), the chatbot ELIZA (1966), and the Shakey robot at SRI, which combined perception, planning and movement.
4. The first AI winter (1970s)
Promises outran results. In 1969 Minsky and Papert’s book Perceptrons highlighted serious limits of single-layer networks, and funding for neural network research dried up. In 1973 the UK’s Lighthill report criticised AI’s failure to deliver on its grand claims, and government funders in the UK and US cut back. Problems that looked easy in toy settings, like language and vision, exploded in difficulty in the real world. This period is called the first AI winter.
5. Expert systems: AI goes commercial
The field recovered by narrowing its ambition. Instead of general intelligence, researchers captured the knowledge of human specialists in one domain and encoded it as rules. These programs were called expert systems.
How an expert system works
- Knowledge base: a collection of facts and IF–THEN rules supplied by experts.
- Inference engine: the reasoning component that applies the rules to known facts to reach conclusions. It can work forwards from data to conclusions (forward chaining) or backwards from a goal to the evidence needed (backward chaining).
- Explanation facility: lets the system show which rules led to an answer, a big advantage for trust.
- Knowledge engineer: the person who interviews experts and turns their know-how into rules.
A typical rule looks like this: IF the engine will not start AND the battery is charged THEN suspect the starter motor.
Famous examples
| System | What it did |
|---|---|
| DENDRAL (Stanford, 1960s) | Helped chemists work out molecular structures from mass spectrometer data. Often cited as the first expert system. |
| MYCIN (Stanford, early 1970s) | Diagnosed bacterial infections and suggested antibiotics using several hundred rules and “certainty factors” to handle uncertainty. It performed well in evaluations but was never used in routine clinical care, partly because of legal and ethical questions about who is responsible for a machine’s advice. |
| XCON / R1 (Carnegie Mellon and DEC, around 1980) | Configured computer orders for Digital Equipment Corporation. Widely reported as one of the first commercially successful expert systems, saving the company large sums. |
In the 1980s, expert systems drove an AI boom. Companies built them, specialised hardware called Lisp machines sold well, and Japan launched its ambitious Fifth Generation Computer project in 1982.
Why they hit a wall
- The knowledge acquisition bottleneck: extracting expertise from people is slow, and experts often cannot articulate what they know.
- Brittleness: a system worked inside its narrow domain and failed badly just outside it.
- Maintenance: thousands of interacting rules became hard to update without breaking something.
- No learning: every improvement required a human to write more rules.
When the Lisp machine market collapsed in the late 1980s and the boom’s promises went unmet, funding fell again. This was the second AI winter, lasting into the early 1990s.
6. The shift to learning from data
Meanwhile, the connectionist idea came back. In 1986 Rumelhart, Hinton and Williams popularised backpropagation, a practical way to train multi-layer neural networks. During the 1990s and 2000s, statistical machine learning methods such as support vector machines and decision-tree ensembles took over many tasks, helped by more data and faster computers. In 1997 IBM’s Deep Blue defeated world chess champion Garry Kasparov, a victory of search and hand-crafted evaluation rather than learning.
7. Deep learning and the transformer era
- 2012: AlexNet, a deep convolutional network trained on GPUs, won the ImageNet image recognition contest by a wide margin and launched the deep learning boom.
- 2016: DeepMind’s AlphaGo defeated Go champion Lee Sedol, combining deep neural networks with reinforcement learning and search.
- 2017: the paper “Attention Is All You Need” introduced the transformer architecture, which underpins today’s large language models.
- 2018 onwards: large pre-trained models such as BERT and the GPT family showed that one model trained on huge text collections could be adapted to many tasks. These are called foundation models.
- November 2022: ChatGPT brought conversational generative AI to a mass audience, and cloud platforms, including Amazon Bedrock, made foundation models available as services.
8. Quick timeline
| Year | Milestone |
|---|---|
| 1854 | Boole formalises logic as algebra |
| 1936 | Turing defines computability |
| 1943 | McCulloch and Pitts model the artificial neuron |
| 1950 | Turing proposes the imitation game |
| 1956 | Dartmouth workshop; the term “artificial intelligence” is born |
| Late 1950s | Perceptron, LISP, early machine learning |
| 1969–1973 | Perceptrons book and Lighthill report; first AI winter |
| 1970s–1980s | DENDRAL, MYCIN, XCON; expert system boom |
| Late 1980s | Second AI winter |
| 1986 | Backpropagation popularised |
| 1997 | Deep Blue beats Kasparov |
| 2012 | AlexNet and the deep learning breakthrough |
| 2016 | AlphaGo defeats Lee Sedol |
| 2017 | The transformer architecture |
| 2022 | ChatGPT and the mainstream generative AI wave |
9. Why this history matters for your AWS exam
This is background rather than an exam topic, but the old ideas keep returning in modern form, and recognising them makes the services easier to remember:
- Knowledge bases: expert systems stored curated knowledge for reasoning. Retrieval augmented generation (RAG) and Amazon Bedrock Knowledge Bases also feed curated knowledge to a model, this time by retrieving documents instead of firing rules.
- Rules and guardrails: hand-written rules fell out of favour as the main intelligence, but they remain essential as safety boundaries. Guardrails around a language model are a modern descendant of IF–THEN rules.
- Explainability: expert systems could show their reasoning. Modern models are far harder to explain, which is why governance, evaluation and Responsible AI carry real weight in the Professional exam’s domains.
- Brittleness vs generalisation: expert systems failed outside their narrow domain, while foundation models generalise broadly but can hallucinate. Many production designs combine both: a flexible model plus deterministic rules and retrieval.
- Agents: today’s AI agents that plan and call tools echo the planning and reasoning programs of the 1950s and 1960s, now powered by a language model.
10. Quick check
- Who coined the term “artificial intelligence”, and at what event?
- Name the four main parts of a classic expert system.
- Give two reasons expert systems fell out of favour.
- Which 2017 architecture underlies modern large language models?
Answers: (1) John McCarthy, in the proposal for the 1956 Dartmouth workshop. (2) Knowledge base, inference engine, explanation facility, and the knowledge engineer who builds the rules. (3) Any two of: the knowledge acquisition bottleneck, brittleness outside a narrow domain, difficult maintenance, no ability to learn. (4) The transformer.
What’s next
In the next post we will build on this history by explaining the core vocabulary of modern AI: machine learning, deep learning, neural networks, foundation models and generative AI, and how they relate to each other.
Comments
Post a Comment