It’s Just Artificial Intelligence
This article is for those who have passively consumed press about Artificial Intelligence, Machine Learning, Deep Neural Networks, and the like over the past couple years, trying to track the changing topology of such a vast, complex landscape. So rather than focus on a particular portion of AI as data science or robotics, this article intends to build a map around the events, technology, and possibilities that continue to shape this winding narrative.
The field of artificial intelligence is too nuanced for a definitive birth date, but one event in the late 18th century is a good start. In his attempt to impress the Empress Maria Theresa, a Hungarian inventor named Wolfgang von Kempelen built a chess playing machine called The Turk in 1769. While other clockwork constructions were commonplace at the time, The Turk gained fame as a human-level intelligence machine at winning the game of chess. The secret of the mechanical Turk lay, sadly, in the fact it concealed a human inside the apparatus. It was a hoax but popularized for a brief time the idea of a thinking machine. While The Turk is von Kempelen’s bid at infamy, he was also a legitimate inventor. He spent 20 years of his life working on an artificial speaking machine and making valid contributions to our understanding of the human vocal tract.
Fast forward to 1950 when Alan Turing, WWII codebreaker and a founder of modern computer science, penned a paper for the journal Mind entitled Computing Machinery and Intelligence. The seminal work introduced a concept he called The Imitation Game, referred to in later papers simply as the Turing Test. He constructed the utilitarian notion that human level intelligence is achieved when computers successfully convince us that they are thinking. As long as humans can’t tell the difference, it doesn’t matter if the machine really has human thoughts (whatever that means); what matters is that the machine can be useful in the same way that a thinking human is.
For decades computer scientists claimed we were on the brink of artificial intelligence (AI), each time ending in disappointment for those yearning for all of the benefits of human-like AI. In 1957, the perceptron was invented, inspired by the firing of biological neurons. While the press was eager to throw its weight behind the promise of AI, little came of this early research. Proofs were published showing fundamental weaknesses behind this form of perceptron, and progress crawled to an “AI winter” in the field, which continued until the late 1980s. Other forms of AI research continued, based on symbolic logic breeding expert systems, or genetic algorithms based on evolution, or statistics based on the mathematics of fuzzy logic. Sometimes, what passed for AI was a combination of techniques, but the achievements never fit the hype. There were just too many things that AI couldn’t do that humans could.
Then something changed. In 1997, IBM’s Deep Blue beat grand champion Garry Kasparov in chess. It was a fleeting affair, quickly dismissed as “not AI”, since how generally useful is a machine that specializes in playing chess? The next few years had some wins at beating humans at increasingly sophisticated games, but still, none could pass Turing’s Test. That is, until, June 17, 2014.
Since Alan Turing first proposed his imitation game, computer scientists have battled in structured environments in an attempt to convince a panel of human judges that they were talking to another human, rather than a machine. That year a program called Eugene Goostman convinced 33% of judges at the Royal Society in London that it was a 13-year-old Ukrainian boy. In 64 years, it was the first time that a machine had passed the test. And yet, it was also dismissed as “not AI.”
Winning chess can be rejected as clever mathematical tricks with big computer hardware due to the relatively limited moves needed to be calculated to win a game. But there are games so complex that winning moves cannot be computed. One of these games is the ancient game of Go. A deceptively simple game, Go is played on a 19x19 board with black and white stones. The complexity of calculating a single optimal move can take eons with modern computing infrastructure, yet human brains can make expert moves in seconds. In March 2016, a program named AlphaGo bested the greatest living professional Go player in the world, Lee Sedol. Generalizing the technology has allowed researchers to best humans at many other games in an improved machine called AlphaZero. This wasn’t another Deep Blue story. Critics were not so quick to dismiss this as “not AI”. This time, something was different.
In May 2018, Google announced a project called Duplex. While conversational bots like Amazon’s Alexa and Apple’s Siri were getting better, they were still a far cry from a natural conversation. At Google’s yearly conference, GoogleIO, the project leads played a couple audio clips to an astonished crowd. The first was a human-like computer voice calling a salon to book a hair appointment with a human on the other end of the line. The phone rang, the woman picked up, and the computer successfully booked the appointment. The second call was the same computer bot calling to make restaurant reservations with someone else. Again, with a bit of struggle, the reservation was booked. What’s more astounding is that in both instances, it was clear that the humans at the other end of these calls had no idea that they were speaking to a machine.
So currently we have general purpose machines that can defeat humans in any number of games with fixed rules, as well as engage in open ended conversations while synthesizing human speech, while the human at the other end of the line is unaware that they’re conversing with a machine. Machines are now doing many tasks that we have historically believed only humans can do, from cognitive automation in white collar jobs to driving long-haul trucking and operating entire warehouses. Will we ever admit that we have created artificial intelligence, and stop changing the definition? Or will we realize that we need a better definition of intelligence, one that no longer holds humans as a benchmark? We need to understand that there are certain tasks that computers do better, and there will be more thinking work that have historically relied on humans in the past to be taken over by machines.
How To Use a New Brain
There are few fields that artificial intelligence won’t affect over the coming decades. Scratch that, let’s be bolder. Now that Pandora’s box of AI is being opened, we’ll never stop finding new ways to add intelligence to dumb processes or inanimate objects. Look forward to a world of airplane parts that can detect stress and alert engineers to their need to be replaced, to virtual doctors that diagnose patients from a science fiction tricorder, to gimmicks like smart coffee mugs and AI assistants that automatically order your lunch based on your preference, budget, location, and history. Unlike other emerging technologies, cheap artificial brains can get smarter, coordinate with each other at the speed of light, and be cloned for free. AI is the most disruptive technology of the century, shifting the landscape of industries, old and new.
As an example of how a simple application of AI can easily decimate an industry, consider the case of smart parking garages. This relatively recent market helps drivers recognize how occupied a parking garage is, by way of a green or red light above each space. You may have seen them at airport garages. Each of these lights is actually a self-contained computing device, with a sensor that detects the presence of a vehicle, a small computer that controls the system and shares its current state with a central network to provide an accurate count, which are displayed on LED screens on each floor. If your smart garage has two thousand parking spaces, that’s two thousand sensors that need purchased and maintained. As an experiment, a UK research firm used their parking lot’s existing security camera feed to train an AI to detect when a car is present in a space, then send that information through a cloud service to anyone with a mobile app. While it may seem complicated, once the model has been trained, building such a system is within the capabilities of any computer science student. Since enough drivers already have smart phones, this system provides the same basic capability with no hardware investment. It’s actually more flexible too, since the model can get smarter over time, perhaps even predicting when spaces will open up based upon the habits of certain drivers.
Tasks, not jobs
Something to keep in mind as the AI revolution steamrolls human cognitive and manual work is that it’s unlikely that AI will automate entire jobs, full-stop. Instead, various tasks are automated in conjunction with human counterparts. While an autonomous vehicle could certainly drive humans around who call for a car from a hotel to a conference center, those cars will still need to be maintained: filled with gas (or electrically charged), oil and filter changes, tires filled with air, patched, and rotated. Each of those tasks over time may fall prey to automation as well, but not all at once. In the interim, the tasks left for humans to do will drop or change.
As more tasks are automated, we’re in for a strange disruptive period where those who do have jobs will execute strange combinations of Balkanized tasks. We’re seeing it already. As systems for booking and managing travel become easier, the market for Travel Agents is shrinking. If you’re a white-collar professional who travels, you’re likely responsible for booking your own flights, cars, and hotels. You’re possibly both a “Director of Marketing + part-time personal Travel Agent”. That’s not necessarily a bad thing. “While some will dramatize the negative impacts of AI, cognitive computing, and robotics,” Deloitte’s Future of Work report states, “these powerful tools will also help create new jobs, boost productivity, and allow workers to focus on the human aspects of work.”
It’s hard to quantify exactly how many tasks exist in the world for AI to execute. But one way to get a handle on the changes happening now is to focus where AI is automating workloads by sectors that constitute huge portions of the economy and workforce.
Agriculture
Historically the largest industry, agriculture has long been in the crosshairs of innovative technologies. From plowshares, to cotton gins, to factory farming and GMOs, each innovation seems to increase yield and decrease the number of people required to work in this grueling sector. In 1870 more than 50% of Americans were employed in agriculture. Today that figure is under 2%. AI and the robots that it powers are expected to continue this fewer-employees-to-greater-yield trend time.
The perennial pestilences of farming have always been a combination of invasive plants (weeds), unwelcome wildlife (from insects to rabbits), unpredictable weather (droughts and floods), and large-scale monitoring (soil and crop). We are in the midst of a gold rush of AI based solutions to these issues.
Over 250 species have become resistant to chemical herbicides, not to mention the environmental impact of excess chemical runoff into our water supply. John Deere announced the acquisition of a company that leverages machine learning vision systems to automatically spray weed killer directly on plants, reducing herbicide by 90%. Several other companies, such as ecoRobotix are creating chemical-free, mechanical weed pulling robots. Many of these robots are also capable of targeted insecticide deployment, helping to stave off many of the unintended consequences of over-spraying, such as bee colony collapse. Speaking of bees, there’s now a pollinating robot called BrambleBee. As for weather prediction, this is a task tailor made for Big Data and Machine Learning. Nearly 90% of crop losses are due to weather-related events. The United States National Oceanic and Atmospheric Administration (NOAA) have made gains in increasingly accurate hail prediction, allowing farmers to better plan planting and harvesting. HydroBio is a company leveraging hyper-local data and AI prediction to help farmers know when they should irrigate. A handful of other longer-term climate systems can help agribusiness decide which parts of the world are safe to plant in over the coming decades.
Monitoring all the details of million-hectare farmlands is daunting work for humans, yet tailor made for machines. With satellite soil erosion and nutrient tracking deep learning systems like Plantix, computers can tirelessly detect soil issues at scale. For a more real-time fine-grained look, a company named SkySquirrel is paring machine learning with an army of drones to detect invasive plants and even molds at scale.
As a kid I used to help my grandfather and uncle bale hay in rural Indiana, and also picked strawberries alongside seasonal migrant laborers. It was brutal work for a gangly child who preferred drawing comics indoors. Younger me would jump for joy to find an endless array of machines, from visual systems detecting when crops are ideal to harvest, to auto-bailers and gentle strawberry picking robots. So, just maybe, my kids will get a chance to be more Tony Stark than George Milton.
Manufacturing
Walter Reuther, powerful leader of the automobile workers union, and Henry Ford II, head of the Ford Motor Company, toured a new factory filled with a line of automated robots building cars. After a brief silence, Ford asked, “Walter, how are you going to get those robots to pay your union dues?” To which Reuther replied, “Henry, how are you going to get them to buy your cars?”
This apocryphal exchange was first told in the early 1950’s. Even then, automation was a taking over manufacturing tasks, and outlines a persistent fear of automation that’s been with us ever since the Luddites of 1811 first destroyed high-tech cotton mills. Relatively few jobs in first world nations today are in manufacturing, as economies give way to the service sector. Still, over 8% of Americans earn livings in the manufacturing industry — over 11% of US GDP. Much of the technology needed to automate some of these jobs currently exists: robotic arms, logistics machinery, quality control systems, and the like. What AI changes is by turning expensive specialty robots into general purpose cobots. Rather than huge clunky welding robots, blind to the world and programmed for a narrow range of tasks, cobots can be taught many different tasks, retooling themselves automatically. Cobots are also aware of their surroundings, capable of working side-by-side with humans on complex tasks. This allows cobots to slowly ease their way into a workspace, taking over more work, limited only by an exponentially growing intelligence. The most famous cobot on the market is Baxster, and costs around $50,000 US. It may not be able to do all tasks, but enough to bend that 8% of jobs down a few points.
Outside of the factory, AI has made vast inroads in the realm of logistics. The ability to finely track and trace shipments has allowed for less waste and more mobility of products, reducing inventory on hand. Many warehouses are nearing peak automation, be it Amazon in the US or JD in China. The next step is autonomous shipping, which is being worked on by Waymo and Uber. Reduced waste, reduced cost and overhead, and faster time to market will be good for companies and for consumers. Let’s just hope there are people able to buy these goods. We live in interesting times.
Military
The Siren song of perfect knowledge of world events from governments to battlefields, paired with robots that bend your side’s casualties toward zero, is too attractive for the military to ignore. AI can support better troops by improving training systems, creating novel curricula for war games. Smart weapons and better intelligence, along with more pedestrian benefits of industry, like optimizing logistic challenges in the world’s most challenging situations or helping troops with maintenance tasks. Natural Language Processing (NLP) can take on many roles that human translators do today, and NLP at scale can sift through vast amounts of audio surveillance in real-time. Furthermore, complex associations between unrelated datasets will stitch together a single narrative. Imagine tracking the movement of a terrorism suspect through various video feeds leveraging facial recognition, cell calls with voice recognition, and travel documents cross checked through known aliases.
Autonomous weapons are increasingly augmented with AI, such as smart camera controlled tactical missiles. Even if control ultimately remains in human hands, the myriad of complexities that take humans years to learn can be partially automated to allow operation in less specialist hands, like flying attack drones. Moreover, the ability to correctly detect targets can drastically reduce collateral damage and innocent deaths.
These are only a few straightforward examples and may not even scratch the surface of the many uses for AI in the military. At the very least, increasing automation may allow countries to shrink their military budgets in favor of more civilian expenditures.
Services
Agribusiness, industry, and military are important for a functioning society, but most first world economies operate primarily in the services sector. Services account for 82% of US GDP in 2018 and employs most working adults. This is also the segment that is the most impacted by AI. Why? Because, contrary to common sense, cognitive labor is the easiest sort of work for AI to execute. While manual robots still resemble disturbingly precise drunks, automating the process of updating a series of spreadsheets is relatively easy to implement. While Robot Process Automation has only slightly automated corner cases of office environments, it’s AI cousin, Cognitive Automation, is poised to disrupt many tasks that require sitting in front of a computer on a daily basis. This ranges from white collar office jobs, to medical, finance, and even creative work like writing and music.
Office jobs
Office jobs are great. You get to be indoors, sit in a comfy chair, and the schedule tends to be pretty consistent and reliable. Perhaps I’m giving away too much about my own personality, but the weather is nice and physical exertion tends to be minimal. And despite a wide range of specialties, office work tends to have a lot of similarities no matter your title or expertise. You’ll likely work with a computer or other device like a phone; compose and answer emails, texts, or instant message; create, fill out, or adhere to a schedule; shuffle information from one location to another, even if it’s in the form of just answering coworker’s questions.
Organizing meetings is a slog. There’s the initial ask, coordination of calendars and location, rescheduling, and the mechanical work of filling out a meaningful invitation, take attendance, and take minutes. One of the first AI assistants I tried was X.ai, and I was immediately hooked. All of the coordination often done by administrative assistants was automated with an email-based chatbot that can understand nuanced human speech, converse clearly and politely with humans involved in scheduling, having knowledge of your calendar, travel times, and blackout times (like lunch from noon to 1pm). If you’re a manager, there’s an AI called Workloud which tracks the actual attendance of those meetings in terms of scheduling, AI Sense transcribes verbal conversations at the meeting into searchable text, and Allocate which tracks timesheets based on any real work done, saving your team from self-reporting.
Hopefully your day job is more than meetings. As you do actual work that interfaces with a computer, from taking and entering orders, fixing issues based on consumer requests, or other data entry work, Robotic Process Automation (e.g., Pega), and now Cognitive Automation (e.g., Corseer), can be trained to execute a wide variety of tasks automatically. Of course, this assumes you actually know what your process is. Well, there’s an AI for that too. Celonis can plug into your digital communications and processes, and map what you think your process is, what it actually is, and how close your reality meets expectations. Once you have process under control, discovering and communicating knowledge is often the role of help desks (“Is my laptop provisioned yet?”) or customer service representatives (“What discount can I get on my order?”). Lots of office work comes down to answering questions whose details are either in some system, or in an employee’s head. The ability to catalog and make this information available for those who ask in a natural way, from emails to phone calls, is well within the capability of several AI knowledge systems called chatbots, from Spoke to GrowthBot.
AI is great and all, but for the foreseeable future we’ll still need people. Surely only people can find other good people, right? Well, AI is coming for Human Resources, too, pal. From writing job descriptions (Textio), to talent searches (Koru), to screening resumes (Ideal), to on-boarding and training (Chorus), more and more tasks that fall under HR are being done cheaper, faster, and better with a cadre of well-placed machines.
A German online retailer, Otto, currently uses AI to predict whether a customer is likely to purchase a product with 90% accuracy and start the shipping process before the order is placed. This not only reduces wait times, leading to happier customers, but also saves the company millions a year in excess inventory.
We could go on, and tear through every office function. There are myriad solutions for sales (Salesforce Einstein), marketing (Albert.ai), public relations (Signal AI), legal ( iManage), finance (Squirro), technology operations (Moogsoft), risk (Exabeam), security (DeepArmor), and so on. What’s important to keep in mind is that each of these specialized solutions will continue to improve, generalize, and take on increasingly sophisticated tasks — because computers continue to exponentially improve, unlike our slow squishy human brains.
Medicine, Law, and Finance
For generations, parents hoped their kids would end up as doctors, lawyers, or bankers. They were secure, well-paying jobs, requiring intelligence and education. But it turns out that even these professional careers are not immune from the slow onslaught of the artificial intelligence revolution fueled by easy access to machine learning.
Worldwide, around 10% to 15% of a country’s GDP is spent on healthcare. In the US, 30 cents of every one dollar spent on healthcare goes to waste and administrative overhead. If every healthcare office adopted even standard office automation, some percentage of that overhead would fall, thus materially shrinking overall healthcare costs. But many wastes are specific to the medical field, from excess lab services to missed prevention opportunities. While it’s unlikely that doctors or nurses will go away anytime soon, many of the specialized tasks that require the most schooling are falling victim to AI. While IBM’s Watson began life as the greatest Jeopardy world champion in history, it has changed careers into a world-class medical diagnostician. Recently, China’s best brain cancer specialists lost in a diagnosis competition against an AI. On the paperwork side, insurance risk assessments and the tasks of billing and coding (H2O.ai) are being automated. Our overworked doctors and nurses are going to need help this as the Boomers age. On a quality note, R&D is improving as AI can stay current ahead of an endless avalanche of emerging publications. All of this automation leads to fewer errors, lower cost, and freeing up for more personalized medicine.
Despite what cop shows would have us believe, very little work lawyers do takes place in a court room. Whether it’s a district attorney building a case with police evidence, or a corporate law firm digging through client records, most legal work constitutes a process called discovery. Discovery is perfectly suited for computers in general, and AI in particular. Computers never get tired or distracted and can make meticulous connections in large swaths of data far beyond the abilities of humans. “If I was the parent of a law student, I would be concerned a bit,” says Todd Solomon, a partner at the Chicago law firm McDermott Will & Emery. “There are fewer opportunities for young lawyers to get trained, and that’s the case outside of AI already. But if you add AI onto that, there are ways that is advancement, and there are ways it is hurting us as well.”
Finally, Banking. AI has infiltrated the world of finance for years. High Frequency Trading (HFT) and risk assessments have been run by algorithm for decades, and increasingly those algorithms are being sharpened by AI. Cognitive Automation has been handling the work of acquisitions, until recently a relatively high paying and well-respected job due to its complexity. Bank branches are largely real-estate tax shelters, as anyone with a smart phone can cash a check by simply taking a photo of it that is verified by AI (Chase, BofA).
Like any sector of the economy, professional jobs are being reshaped by AI. It turns out that the jobs requiring the most training and knowledge are just as susceptible to automation than office or manual work. And maybe more so, considering the incentives to get the work right, and the great cost of employing humans in these fields.
Creativity
Are you a sui generis creative, knocking out artistic works that no computer could ever accomplish? I have some bad news. Creative works are increasingly the domain of artificial intelligence research and development, as well as rising cases in practice. By the start of 2019, over 30% of Bloomberg articles were written by AI. By 2025, estimates place 90% of major newspaper articles written by or assisted by AI. It started with a program called Quill, a sports article generator, which now reports on finance and global events for most major media outlets, unbeknownst to most of its readers. While there will always be a place for Atlantic-style think-pieces, most of the news we consume is far more mundane and merely conveying information, like recapping a State of the Union address, with an occasional quip that requires little human intervention.
But we’ve already seen that AI is really good at dealing with text. What about other arts? Cambridge Consultants created a Generative Adversarial Network (GAN) called Vincent that allows anyone to sketch a picture, and it will generate a painting based on what the user is trying to convey. There are apps that even convert speech (“I love tabby cats”) into visual art (a cat decorated with hearts). This isn’t replacing human creativity but is augmenting it and placing artistic expression within reach of every human, regardless of technical skill. And while composing music with AI has been around since Ray Kurzweil tried it in 1965, AI composed music has more recently found its way into the mainstream with the release of singer Tayern Southern’s computer generated album called I Am AI. Generating art and music has become such a common past time for researchers that Google actually created an open source AI art platform called Magenta, allowing non-specialists to experiment with generative art.
AI is taking over agriculture, industry, military, service, professional, and creative tasks. You might be forgiven for taking the stance of Elon Musk or Stephen Hawking and believe we’re scant years away from an automation fueled uprising. Before we freak out and welcome our new robot overlords, let’s take a breath and see how this AI thing actually works. Maybe then we can revisit the glaring weaknesses in the system and find some solace in the fact that there are still plenty of jobs that we humans are qualified to do for the foreseeable future.
What’s Different Now
Fueled by a boom in a branch of AI called machine learning (ML), AI has emerged from an academic curiosity to changing the world in practice. How did we get here? A handful of shifts in the technology landscape have contributed to this AI resurgence over the past decade, and these core investments are poised to keep bringing new cognitive capabilities to bear on an ever-wider array of goods and services. These improvements have risen from three coequal changes: better hardware, democratization of algorithms and data, and increased investment in the ML space by both industry and academia. The changes in the industry are symbiotic. On March 2017, Google’s CEO announced that they were now an “AI first” company, but they had invested in ML research for over a decade. Internal investment, academic partnerships and acquisitions set the stage for its modern AI renaissance. Concurrently, they invested in an open ML toolkit called TensorFlow in an attempt to attract top talent and control the narrative on new applications. This coincided with the creation of custom hardware specialized to execute tensor calculations called TPUs (Tensor Processing Units). The world’s largest collection of data didn’t hurt either.
Better hardware
While there are many ways to implement artificial intelligence, the current growth is based on machine learning in large part because machine hardware has gotten so good in comparison to other artificial means, such as, synthetic biological research. While Moore’s Law might be dead (transistor density doubling every 18 months), the kind of specialized hardware necessary for executing modern machine learning techniques continues to grow exponentially, thanks in large part to two unrelated trends: video game enthusiasts and cloud architectures.
The kind of hardware necessary for executing machine learning systems is similar to the hardware optimized for rendering video game graphics, called graphics processing units (GPUs). This is similar to the central processing unit (CPU) that has dominated the computing market for decades but optimized for the kind of math operations necessary for both use cases. With the proliferation of cheap and easily available GPUs, new life was breathed into the stagnant field of ML research. Google’s play in this space upped the ante, as they rapidly prototyped, built, and deployed their first generation of production TPUs by 2015. The race was on to create specialized machine learning hardware called AI accelerators. NVidia, the world’s leading GPU manufacturer, staged a concerted effort to dive into the greater AI market outside of Google’s ecosystem.
While the AI hardware war rages, the cloud is the primary battleground, namely, Google Cloud, Amazon Web Services, and Microsoft Azure. The cloud allows anyone access to cutting edge hardware.
Democratization
While it’s true that Google was an early mover in the ML space, most of the concepts and tools that they championed were developed elsewhere and, in many cases, better. What Google should be appreciated for was forcing other technology companies to open their IP. An open source ML library called Torch released in 2002 had been assisted by Facebook, IBM, and others, who contributed to the core project but were reticent to release how their particular sausages were made. But Google released more, not only the TensorFlow in 2015, but also a mountain of documentation, training videos, blog posts, and open sourced algorithms that worked in production environments. The rapid popularity of TensorFlow ushered in an age of open, sharable machine based on a common language.
This openness in software, coupled with newly accessible powerful hardware, democratized the machine learning ecosystem. Suddenly, any smart kid with a cool idea has access to cutting edge ML research, with source code, and an environment to run it in. Another driver behind the democratization of machine learning are the flood of easy-to-use frameworks like the point-and-click Orange or SageMaker, coupled with easy-to-grasp education opportunities like Udemyor Edx. No longer would machine learning be reserved for those with a “freakish knack for manipulating abstract symbols” (via Paige Bailey of Microsoft paraphrasing Bret Victor of Apple). But open AI algorithms are only half of the story. Machine learning requires data to train with and run against, and data is increasingly everywhere.
The proliferation of open data, from public weather data, to university psychology research data, to government census and economic data (data.gov), has given many professional and budding AI engineers a set of data against which to build their own ML models. Often, those models are themselves open sourced, allowing others to build on their work, prompting organizations to open even more datasets. This virtuous cycle of data to information has created new understandings in previously opaque industries, which prompts even more open data which democratizes machine learning even more. There is a constant stream of competition to create the best AI against a given dataset hosted on a site called Kaggle. They provide the data and the terms of the competition, and a dispersed community of data scientists compete to make the most sense of the data. These competitions often have a cash prize for the winners, and range anywhere from “Customer Revenue Prediction“ to “Using News to Predict Stock Movement” to “Human Protein Image Atlas Classification”, and those were all in the same week.
Hardware as a service, machine learning lingua franca, open sourced algorithms, open datasets, and easy to access education, all play a role in the democratization of machine learning. It’s great that this infrastructure is opening up, but why would companies like Google give everything away? Where does the money for the Kaggle prizes come from? Who pays for all of this?
Investment
Those of us who were around for the big data revolution quickly spotted a flaw in our operating model. While the emerging big data industry made it easier to quickly collect huge volumes of varied data (for example, by leveraging NoSQL datastores), it was not easy to make sense of the data. The general philosophy for the better part of a decade, starting around 2009, was, “collect all the data, we’ll figure out what to do with it later”. Turning data into information is a difficult task. Turning information into understanding at scale is nigh impossible for humans. A new, vaguely defined kind of job started popping up everywhere: Data Scientist. It was no longer good enough to hire statisticians, these unicorns also needed to be experts in large scale data management, to mine mountains of unstructured data for… something. The company made an investment in all of this data, just find something good.
Everyone had a sense that data was valuable, but there was no clear roadmap on what to do with that data. It was like a joke from the animated TV show South Park, where gnomes secretly stole underpants with no clear plan in sight: Step one collect underpants, then step two, then step three is profit. There never was a step two, but they felt very strongly that it would lead to profit. We like to believe the coterie of swells slinging real money have clear goals in mind, but sometimes it’s just worth taking a shot. It was highly unlikely that data would be worthless, and the emergence of ML as an increasingly popular sub-discipline of data science ended up being the missing “step two” needed to make sense of all this data sitting largely idle around corporate data centers. The rapid increase in industry and government spending on limited AI resources makes sense through this lens, alongside the standard concerns about being left behind.
The symbiotic nature of academic and industry research is important for emerging technologies like machine learning. Academics conduct leading edge research across a variety of topics, and some percentage of that research ends up, hopefully, being of interest to the corporations and governments of the world. They, in turn, invest in more of the kind of research that they believe will yield better results. Right now, there’s a goldmine of investment, which is only growing. Corporate investment in AI is on pace to have around 50% CAGR (compound annual growth rate) over the next decade. The general thrust of emerging AI research is maturing from machines that are better at describing the data, to predicting what’s next, to the holy grail of AI, prescribing better courses of action.
Types of Machine Learning
AI continues to evolve from atavistic statistics which merely describe the world, to making sophisticated predictions, then prescribing courses of action for humans, to eventually taking action itself. The tools required to evolve these capabilities have become increasingly human-like in the manner in which they understand the world. While the perceptron was insufficient for much of anything with the technology of the 1950s, artificial neurons are the design of much modern AI via ML, in the form of deep neural networks. In a short time, AI has moved from simple data-structures and symbolic algorithms to a complex artificial neurology, built on the rise structures of available data and hardware.
Plain old Data Science
In its most basic terms, statistics is about creating a model that estimate unknown parameters. Inference, probability, frequency, data science, machine learning, artificial intelligence, human intelligence… a rogue’s gallery of approaches to attacking a basic problem: perfect knowledge is not possible, so how do we fill in the gaps when confronted with something new?
Statistics is as old as the first humans making assumptions based on observation. The Greek historian Thucydides described a frequentist method used in the 5th century BCE. One of the first modern statistical models tracked mortality by sampling for signaling the Bubonic Plague (Graunt). Around the same time, the study of randomness was being used to calculate probabilities in games of chance (Pascal). Statistical work for demography and probability theory started to converge (Laplace) over the centuries, eventually consuming warring philosophies like frequentists and Bayesian into a general set of methods.
With the emergence of computers and the Big Data revolution, a new field of Data Science arose, who are basically statisticians that know how to handle lots of data and can code computers a bit. Like other scientists, they test hypothesis against large datasets, using their own tools of the trade, namely, software packages like R or Python’s scikit.
A good example of a useful statistic for extrapolating signals from incomplete knowledge is one of the simplest and oldest, called linear regression. Imagine you had a bunch of measures of people’s height compared to shoe size. While each of the dots in a 2D chart represent a single person, over a population of people a pattern emerges. Generally, the taller a person is, the larger his or her feet will be. Now, let’s draw a line through what appears to be the average of each value. This line is our prediction. Say, the average 60” tall person wears a shoe size of 8, while a 70” person wears 11.
But is our prediction line good? To find out, we can measure the distance from each point to the line. That distance is called the error, because our estimate is wrong compared to the real observed value. We then square the errors (to make distant dots stick out even more) and add them all up. That total is called the sum of squared errors (SSE). What we want to do is draw a prediction line that best fits the data points we have, and this, has the lowest SSE, since that means overall our line is closest to the most points. Linear Regression is a method for calculating the best prediction line to the data set.
Although we’ll never have line that exactly predicts every new measurement, we only need one that is close enough to be useful. That’s the crux of data science right there. We aren’t looking for perfect, we’re looking for useful. There’s also no law that says you have to group together data points using straight lines. We can also try curved lines, a.k.a. Nonlinear Regression, such as would be the case measuring average height by age. After people reach age 18 the correlation between height and age tends to flatten out and even curve down a bit later in life.
Sometimes we want to analyze clusters of data (K-means) or reduce the dimensions or features in play (manifold learning) or convert from one type of data to another (auto encoding). Moreover, not all data is numeric. Sometimes we want to classify things, such as, is that a picture of a cat or a dog? The tools for accomplishing this feat are varied, including Logistic Regression, K-Nearest Neighbors, Support Vector Machine (SVM), Decision Trees, Random Forests and so on. How can a data scientist possibly know which algorithms to use, let alone figure out how to fit the chosen model to the data set?
The answer is, we start letting the machines do the work for us. We educate a model with a training data set and look to reduce its errors against a validation data set. Then check how generally useful that model is with a test data set. In other words, we teach the machine how to learn by fitting a prediction curve with the common pattern of training, validation, and testing.
Deep Neural Nets ∩ Machine Learning = Deep Learning
“Excellence is an art won by training and habituation…we are we repeatedly do.” – Aristotle via Will Durant
Artificial Neural Networks (ANNs) are loosely inspired by biological neocortex neural clusters. Deep Neural Networks (DNNs) are ANNs where there are many layers of neurons between the input and output, in other words, the network is deep. Deep Learning is basically training a DNN with loads of data, until the network starts to be “shaped” by the commonalities in that data set. Say you train a DNN with many images of cats. With enough images, the DNN will start to recognize the common attributes that make up a cat. When you give it a picture it hasn’t seen before, it can pick out whether the image contains a cat with some level of confidence. Machine Learning is not magic, it’s just multidimensional curve fitting.
Consider an audio waveform. Since sound is just vibrations in the air, it was long ago discovered that unique sounds can be collapsed into one unique wave. Let’s say we have a series of waveforms of different people saying the word “hello”. Something about the waves are similar enough that any human capable of hearing it could make out the word “hello” (even if they don’t speak English). While the waves might not look exactly the same, there is some commonality that can be extracted by a crafty algorithm.
Send many samples of the audio waveform for “hello” to a DNN, and it may, through a series of weights and adjustments to its internal structure, start to recognize the pattern in any waveform. Then, as you stream in a series of waveforms of, say, a conversation, it can pick out a particular wave segment as the word “hello”. This vaguely mimics our human neurons recognizing a friend shouting “hello” across a crowded room.
It turns out the ability to recognize patterns in a set of noisy data is similar whether you’re talking about audio, images, video, electrical pulses, financial data, or many other signals. The mathematical representation of these data is called a tensor (think of a multidimensional matrix). As long as the data can be converted into a tensor, it’s a candidate for deep learning, and it turns out, most everything humans interact with is.
DNNs and Deep Learning open up a world of ML techniques for various uses. The “hello” example above was a way to classify an observation and is becoming a rather milquetoast technique in the ML space. It turns out, given the right incentives, you can teach machines to do more than observe, but also take action.
Supervised, Unsupervised, and Reinforcement
The examples we’ve covered generally fall into a category called supervised machine learning (SL). The word “supervised” here means our training data has a value (dependent variable) that represents what we’re training the AI to predict. If we want to train a convolutional neural network (CNN) to recognize images that have cats in them, we have to train it with against many other images of cats that humans have already labeled as cats. If you’ve ever run across a modern Google CAPTCHA (security) test, you may have been provided a series of images and been told something like, “select images containing a stop sign.” That’s Google using you to label its images, so it can later train a DNN to recognize stop signs automatically. You, dear human, are the machine’s supervisor.
This raises the question, is there an unsupervised machine learning (UL)? When you have a lot of data and want to detect patterns but aren’t really sure what those patterns might be yet, you can’t label it. Imagine you have a series of warehouses, where inbound products are scanned, and their geolocation is tracked. Providing only these locations, certain algorithms (isolation forest, k-means clustering, variable auto encoders, etc.) learn where your products are expected to be. Given a scan somewhere that’s unexpected, like Antarctica, can trigger an alert that a signal is an anomaly. Unsupervised machine learning is commonly used in many kinds of anomaly detection, from logistics, to bank fraud, to consumer profiling and recommendation systems… anytime you’re looking for a good representative form from the data.
Recently, ML practitioners are playing with combining SL components with UL into a type of ML called generative adversarial networks (GAN). We could call GANs semi-supervised models, because they’re trained by arming an unsupervised model to learn (and generate) against a known supervised model (the adversary). The details are deeper than we need to go here, but GANs are interestingly powerful for taking bodies of known works and generating outputs that are similar enough to be useful. Such as, providing a corpus of classical music, and asking the GAN to generate an endless supply of new music that somewhat “sounds like” the input. Well trained GANs are excellent for generating new creative endeavors, such as an endless selection of custom sneakers. In early 2019, an OpenAI GAN was claimed to be so good at generating convincing fake news articles, it was deemed too dangerous to release to the public.
In the history of psychological Behaviorism, Pavlov’s Dog tends to be the experiment that most of us would shout at a trivia night. But years before Ivan Pavlov rang his feeding bells, Edward Thorndike discovered the “law of effect”, which states that satisfying consequences tend to be repeated, giving rise to a study called Operant Conditioning. This is the crux of a third type of machine learning, called reinforcement learning (RL). Unlike supervised or unsupervised machine learning, reinforcement learning goes beyond data per se, and instead focuses on training agents to act in a given environment. We train these agents in the same way that we tend to train other humans, by rewarding desired behaviors and punishing undesirable ones, in service of some goal. Reinforcement learning, as a computer concept, is decades old. However, RL is experiencing a renaissance thanks to the emergence of deep neural networks, most famously Deep Q-Networks (DQN). RL tends to be the modern tool of choice for training machines to do things that have recently been done by people, such as training machines to beat world champions at Go (AlphaGo) with the goal of winning, or training computers to trade stock autonomously better and faster than any human could with the goal of higher profits. Of all the tools in the roboticist’s toolkit, reinforcement learning may become the most disruptive.
When you tie together RL, SL and UL, and you can get a sense of the next few decades of AI research. Consider autonomous vehicles. You could use cameras and CNN (SL) models to see and detect objects, use isolation forest (UL) to judge whether the objects together make common sense — like a snowman on a palm tree — and DQN (RL) a car to react based on the best available knowledge with the goal of driving down the road without hitting a living thing or being hit. While the cutting-edge researchers continue to build increasingly sophisticated ML models, us mortals can put them together like puzzle pieces in new and interesting ways.
The Kobayashi Maru
Despite the promise of artificial intelligence powered by machine learning, and the rapid gains over the past decade, there are a few issues that the industry must hammer out before we can safely coexist with our own personal Rosie the Robot housekeepers. AI is not only complex to implement, but unlike code, it’s not easy to check under the hood and see what’s going on, which makes many industries with tight quality controls nervous. AI rejection is exacerbated by the same institutional inertia that slows any new idea: “We’ve been doing fine for years,” “who are you to change things,” “your shibboleths are wrong, nerd,” and so on. Pile on known biases in data, along with economic and philosophical concerns, with the existential threats of general artificial intelligence, and we have a lot of work to do before unleashing this particular cat from the bag.
Complexity
“There are truths to be discovered, but truths complex and many-sided.” — Harry Clor
Deep neural nets are notoriously difficult to unravel. It’s not easy to ask the computer how it landed on a particular decision. Kevin Slavin, a research affiliate at MIT’s Media Lab said, “We are now writing algorithms we cannot read. That makes this a unique moment in history, in that we are subject to ideas and actions and efforts by a set of physics that have human origins without human comprehension.” Beyond the complexity of developing ML models, using them can be equally frustrating. There are many cases of deep image classifiers that can be fooled with a little bit of noise in a photograph that is imperceptible to humans, but can fool an AI to believe, for example, that a picture of a bus is actually an ostrich.
But as research into ML continues, these attacks are being accounted for and baked into increasingly robust models. It turns out that the kind of attack which can generate noise can also be used to mitigate it. Sort of like inoculating the AI against known viruses. In the future, we’re going to have to design a sort of immune system to recognize and inoculate against tricks of increasing sophistication.
A subtle but important issue is the emerging “reproducibility crisis”. While AI techniques help scientists break new grounds in complex fields such as protein folding (a DeepMind AI took home the gold in a 2019 international competition between the world’s top scientists), many victories are pyrrhic. A cornerstone of science is the ability to reproduce results, yet without a concrete set of steps that an AI took to reach its results (however correct they may seem), puts the frontiers of science on shaky ground. Until we’re able to peer into the brain of AI, and understand how it came to its conclusions, we’re going to have to tread carefully.
Bias
In early 2018 a study of the top gender-recognition AIs from Microsoft, IBM, and Megvii correctly identified a person’s gender 99% of the time — as long as you are a white man. Conversely, dark-skinned women were only accurately identified 35% of the time. The reason for the discrepancy is due to the images chosen to train the ML models. While there’s little reason to believe racial intent on the part of the data scientists, the unconscious bias of the dataset they chose to use came through in the model. The risk here is using these biased models for profiling humans in a production setting, potentially leading to false identifications, arrests, and punishments. All based on the testimony of a machine that we’re prone to believe could not possibly hold prejudices. This is just one of many examples of well-meaning AIs gone wild.
The good news is, that there is now a concerted effort between academia and industry to discover and account for biases in ML models and training data. The MIT-IBM Watson AI Lab is one of the groups working on improving the technical side, while advocacy groups like AI Now in New York, are warning officials of the dangers of taking AI outputs at face value, especially when the claims strongly disagree with our human intuitions.
Economic
“These new machines have a great capacity for upsetting the present basis of industry, and of reducing the economic value of the routine factory employee to a point at which he is not worth hiring at any price… we are in for an industrial revolution of unmitigated cruelty.” — Norbert Wiener, Mathematician and early computer pioneer, 1949
Many dystopian stories of artificial intelligence run amok take place in worlds of malevolent machine superintelligences, from The Matrix to Terminator. While technological singularity is possible, there are subtler economic concerns that don’t depend on the bleak collapse of civilization. If we’re not careful, we’re already charging headlong into a Technocracy, with a handful of elites with know-how and an ownership stake in the machines responsible for most productivity gains. Most new billionaires in the past two decades have grown from the ranks of the technology sector, followed by new millionaires and top end technology professionals.
While productivity has relentlessly ticked upward, average employee salaries remain flat for most but those at the top who either own or deeply understand the new technologies and shifting economy. As the physical labor requirements of farming became automated, the numbers in agriculture shrank. As the repetitive yet technical know-how of manufacturing continues to become automated, the number of people required to do that work is also shrinking. And there’s no reason to believe this trend won’t continue as AI automation for cognitive, professional, and creative work is increasingly automated. What economic sector will be left for humans to move into? While some hope that education for new jobs will be our saving grace, it’s worth noting that the education “ladder” is really a “pyramid”. There is a limit to how many data scientists or executives the world needs, and its far less than currently the demand for truck drivers. If even only half of the 4 million US transportation jobs are fully automated, there aren’t 2 million other jobs for them to do.
We need to consider the possibility that there just won’t be enough jobs to go around. A few options have been floated by economists and policy makers, from wealth distribution to universal basic income based on taxing any productivity gains made by AI. Each possibility opens up a myriad more questions. How we deal with this potentiality is far beyond our scope, but it’s a discussion that our society seems woefully unprepared to have.
Philosophical
In the Star Trek oeuvre, the Kobayashi Maru is a test of how Starfleet officers act in a no-win scenario. Humans rarely run into real situations that lack any viable options, and the concept seemed so remote and philosophical, audiences applauded when Captain Kirk found a way to beat it. But we’re reaching a point where, in a world that lets computers make decisions, humans will have to pre-program what the computer should do. Who should a self-driving car sacrifice on an icy road with a pedestrian standing in the middle with no hope of stopping in time? Should the car kill the pedestrian, or run the car off the bridge into the icy depths killing the passenger? What if the pedestrian is a kid? What if there are four passengers in the car? These answers are not easy for we humans to answer, long languishing in the realms of freshmen philosophy classes. But as we confront a world were these scenarios will be confronted, we need to tell our machines upfront how to react. For all the seeming magic of AI, it still lacks basic common sense and compassion, and therefore, must be taught by us what is right.
Furthermore, as the cognitive power of machines grow, they’ll reach a stage where our own moral code must come into question. Does an android that looks, talks, and acts like a human deserve any rights at all? This dark question was grappled with in the film A.I.. While it was ultimately Steven Spielberg who directed it into a science fiction and special effects bonanza, the short stories the movie was based on was initially acquired by the auteur Stanley Kubrik. It’s the story of an android child, programmed to love (or at least convincingly simulate it), and how he and the world interact with each other. Some treated the android as a human, projecting their values of human children onto it, while other believed that artificial meant no rights at all despite the machine’s protests.
If this story seems fantastical, consider MIT’s project Quest. It’s building increasingly sophisticated neural networks, like modeling a human child which learns over time. Other projects are unlocking how the brain turns raw signals into senses, perceptions, conceptions, and potentially consciousness itself. What’s the ethical consideration of a completely simulated human brain, one that acts entirely human in every way?
While these sorts of philosophical and literary situations have puzzled humanity for generations, we are the generation that is going to have to start having real answers for them. And those decisions will have real consequences for generations to come.
The AI Summer
Unlike some other emerging technologies, like blockchain, whether companies adopt AI is not really a choice. AI is inevitable, because the economic incentives of artificial minds are just too strong to stop. But the good news is it’s early enough that we can choose how to react.
While the downsides of AI may seem dark, there are many positives we shouldn’t overlook. If we can get the social factors figured out, we may realize the dream of the industrial revolution: more free time for us all to pursue personal interests. Who needs a 40-hour work week, anyway? I’d love to spend more time with my family, while letting my robotic mower cut the lawn or my AI assistant answers emails. We could completely automate farming while reducing dependence on chemicals, streamline the military while reducing human casualties, automated manufacture products on-shore while reducing transportation costs and thus carbon emissions; Hopefully while profit-sharing the output generated by a network of machines requiring little human intervention, finally taking the man out of The Turk. The dream of a post-scarcity society is possible for the first time in history.
What shall we do with all this free time… how about a nice game of chess?
