Chapter 1
Daughter of Art and Science I am the daughter of art and science. My mother, Frema the Akan, is the first artist I knew. As a child, I sat next to her as she filled canvas after canvas with powerful colors and made creative ideas reality. Art supplies littered our garage, mixed among drawing books, portfolios, artificial fruits, and flowers. My mother explored human conditions of the heart. Her work, she told me, was aimed at moving people to experience healing, to glimpse the divine, to be enraptured and swept into another place of awareness. I would observe her, deep in focus, considering the next stroke to apply to an evolving piece of art. Her experiments and works in progress were a constant presence for me. Seeing her sculpt, paint, draw, and etch out art was a delight to my senses. Her four-foot paintings towered over me, and the smells of charcoal and turpentine tantalized my nose. Our world was an open invitation for me to try my hand at creative expression. I soon had sketchbooks filled with whatever had recently caught my interest—ramps, skateboards, mustangs, animated characters, guitars, and amplifiers. My mother’s voice of encouragement, a constant echo, gave me the audacity to explore my capacities and my curiosity. But artistic experiments were not the only ones that peppered my childhood.
My father, Dr. John Buolamwini, is the first scientist I knew. He worked on topics that would take time for me to learn to pronounce: medical chemistry, pharmaceutical sciences, and computer- aided drug discovery using neural nets. Trips to his lab were fun and full of many things not to touch, a lesson I often learned the hard way. Chalk is not for eating. Dry ice burns. Walking to his office I would see hallways filled with scientific posters as he waved to colleagues and students. And then if I was lucky, while he worked on the latest grant, research paper, or other desk work, I got to play on one of the computers. When he walked to the freezer in his lab I trailed him like a shadow. He put on purple gloves, pulled out a tray, and placed it on a lab bench. I struggled to get the oversized gloves on my hands as my dad beamed at my efforts. Once I was protected, he placed a pipette in my grasp and gently applied pressure on my right thumb with his. Liquid drops bathed the cancer cells beneath our cradled hands, while my eyes widened with fascination and his beard tickled my head. Next to the lab bench sat more computers. He would show me machines linked to concepts like
flow cytometry. I would look at the squiggles on the computers that I would later learn to call graphs.
Like my mother, he was working on experiments that required bold curiosity to ask unexplored questions. But while my mother asked questions of colors, my father asked questions of cells. In the midst of their explorations, I began to ask questions about computers. For instance, how did the images that I saw on the scientific papers come to be? They looked like abstract paintings to me. My dad showed me the software on his huge Silicon Graphics computers that would create these images containing ringlets and rods of bright reds and blues, representing different protein structures. The goal of feeding the cells, designing medicinal drugs on the computer, running all the tests, and scrutinizing the squiggles was to help people who were struggling with different conditions, from heart disease to breast cancer. He showed me the software to introduce me to chemistry, but I found myself more and more enamored with the machines themselves. I quickly found games like Doom and Cycle that came preloaded. I listened to the whirs and beeps of a dial-up connection. In that office, I opened Netscape, my first browser experience into a portal I would later learn was the internet.
And so it was that, surrounded by art and science from a very young age, I was emboldened to explore, to ask questions, to dare to alter what seemed fixed, and also to view the artist’s and the scientist’s search for truth as common companions.
My parents taught me that the unknown was an invitation to learn, not a menacing dimension to avoid. Ignorance was a starting place to enter deeper realms of understanding. At some point, though, they would tire of my endless questions; after entertaining my curiosity for some time, my mother would sometimes bring me back down to earth with a gentle, “
Why has a long tail . . .” elongating her words as she spoke. In addition to my parents, I turned to another source of knowledge, television. As first-generation immigrants settling into Oxford, Mississippi, my parents wouldn’t let me watch commercials: They wanted to shield me from the materialism that appeared to be the backbone of American culture. “You will never find your worth in things,” they cautioned. However, they encouraged me to watch educational programs, so PBS became the television channel of choice in our home. I soon found myself anticipating shows like
Nature, National Geographic, Nova, and
Scientific American Frontier.
There was one episode in particular that left a lasting impact on me. When I was around nine years old, I watched a segment about robots. The program host visited a place called the Massachusetts Institute of Technology. He spoke to a graduate student named Cynthia Breazeal about her work on what she called social robots. Unlike the industrial robots I had seen before—hulking machinery set to tasks like stamping out sheet metal—her social robot was not focused on work but on connection and communication. She sat next to a robot she had built named Kismet, a dazzling and intricate web of metal and wires topped off with enchanting eyes, animated ears, and a cheeky smile. The moment I saw the machine appear to come to life, I was mesmerized. Could I make something like Kismet? Could I go to a place like MIT, the ever-present backdrop to so many of the science and technology shows I watched? From that moment, I decided I wanted to go to MIT and become a robotics engineer. I was blissfully unaware of any barriers or requirements. I had more questions to ask of computers, nurtured in the incubator of youthful possibilities by the belief that I could become anything I imagined.
My first step toward building robots was learning how to program machines to do what I wanted. To give machines instruction, I discovered different kinds of programming languages. I started by learning the basics of HTML and CSS to build a website. These programming languages focused on structure and formatting. HTML allowed me to define the elements I wanted to see on a webpage, like a block of text, a button, or an image. CSS let me determine what these elements would look like, from the color of the text to how much space existed between elements. Each programming language had its own rules for how to give a computer directions. Soon enough I was using these skills to code websites for my high school sports teams and make some pocket money or barter. Even if I was a benchwarmer on the basketball team, at least I did not have to pay for my uniform or shoes.
I wanted to go deeper than websites, and I was curious about how to make games like the ones I played with my brother on his Nintendo 64 or Tony Hawk Pro Skater 2, which I enjoyed on my Sony PlayStation. So I learned another programming language called Java. Here, I was introduced to the concept of an algorithm. An algorithm, at its most basic definition, is a sequence of instructions used to achieve a specific goal. To make my character move around the screen, I would write code that followed a logical sequence. For instance, if the user hits the left arrow, move the character left on the screen. Algorithms like this, as I would eventually learn, would become the basis for more powerful and dynamic systems.
I followed my desire to work on robotics into college. By my third year at the Georgia Institute of Technology in Atlanta, I was working on social robots. One of the professors I worked with, Andrea Thomaz, was a former student of Cynthia Breazeal’s. And to my delight, when I started working on Thomaz’s robot Simon, I learned that the code that was used to power it had descended from the
CREATURES code library that once animated Kismet. My assignment with Simon was to see if I could have the robot engage in a social interaction with a human. I settled on working on a project called peekaboo Simon. The aim was to have the robot participate in a simple turn-taking game with a human partner, similar to one that a parent might play with a young child. The larger aim behind this project was to see if we could have a robot play social games with young children and analyze how children responded and behaved during those interactions, thereby helping to diagnose early developmental delays or even early signs of autism. This kind of early detection could help a child receive necessary support as soon as possible.
To make this game work, I would need to get Simon the robot to detect a human face and direct its head toward the person. This was my introduction to face detection.
Copyright © 2023 by Joy Buolamwini. All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
