Personal computers have evolved in an office environment in which you sit on your butt, moving only your fingers, entering and receiving information censored by your conscious mind. That is not your whole life, and probably not even the best part.
—Dan O’Sullivan and Tom Igoe, Physical Computing:
Sensing and Controlling the Physical World with Computers
Physical computing is about computers interacting with more of the physical world, and using more of our physical bodies to interact with computers. “GUI technology allows you to drag and drop, but it won’t notice if you twist and shout.”1 In this appendix I shall explain two key ideas in physical computing, switches and transducers, and then show several interfaces for switching and transducing with Scratch. But first I would communicate some sense of the bigger picture of physical computing, because I believe it is one of the most important aspects of technomancy, especially as it is transmitted uniquely via Technomancy 101; and because it is fundamental to the connection between technomancy and robomancy.
Although the physical computing movement per se began recently, it is closely related to cybernetic ideas dating back to the early- and mid-20th century. Recall the “sensation/action paradigm of computing” I mentioned in the introduction; following are some diagrams showing how physical computing is related to the idea of how entities (whether organisms, mechanisms, or ???) interact with their environments.
In electrical engineering and electronics, a switch is a device that can interrupt the flow of electricity through a circuit or divert electricity from one part of the circuit to another. Perhaps the most common example is a light switch: in one state (“on”) it passes electricity through the lamp causing the latter to illume; in another state (“off”) it prevents electricity from reaching the lamp and so the lamp remains dark.
Digital computation involves switches in two ways. One is encoding. E.g., you could say that the switch being open (indicated by the lamp being off) represents logical falsity, and the switch being closed (indicated by the lamp being on) represents logical truth. If someone were to ask you a true-or-false (or yes-or-no) question, you could answer by switching the lamp on or off. All binary data are essentially encoded this way; e.g., the letter ‘A’ in the ASCII encoding scheme is represented by the binary value ‘01000001’, which you could express with eight switches (or one switch eight times): the first one open, the second one closed, the next five open, and the last one closed.
The other way digital computation uses switches is with logic gates, which output a value dependent on the state of one or (usually) more input values.2 E.g., a simple AND gate has two inputs and one output; if both inputs are ‘true’, the gate outputs ‘true’; if either input is ‘false’, the gate outputs ‘false’. An OR gate returns ‘true’ if either input is ‘true’, and only returns ‘false’ if both inputs are ‘false’. A NOT gate has but one input and returns ‘true’ if the input is ‘false’, and ‘false’ if the input ‘true’. Computers use combinations of logic gates to make decisions: if a and (b or c) and not d then e, &c. Computer programming statements that make choices between multiple possible states are like software switches, and some programming languages even include a switch statement that implements a specific kind of decision.
Computers and computer software allow us to make complex or unique switches more easily than by implementing such things in hardware alone.
Without a computer, you can connect a button being pressed to a light turning on. With a computer, you can make the relationship between the button and the light more complex. For example, you can make the light's turning on dependent on the number of times the button was pressed, for how long it was pressed, or whether it was pressed in conjunction with other buttons in other rooms or on other continents. You can change the relationships on the fly; for example, you can make the light come on after two button presses during the day, and only after one button press at night. To get the computer to make these relationships between events it senses and events it causes, you write computer programs.3
Switching allows us to program technomantic interactions that respond to gestures, sounds, the presence of objects (see “Electrical Materials”), or the actions performed with them. Examples of switches are observable in many Technomancy 101 projects. The AppleOfDiscord, e.g., functions quite like a light switch, but instead of a plastic toggle there is an enchanted apple, and instead of activating a lamp it selects a divinatory sign.
Transducers convert variations in one physical quantity to variations in another. The word ‘transducer’ is from the Latin transducere: to lead, bring, transport, or conduct across or over something. Transduction always occurs across a boundary, such as when the eye receives light and converts it to electrical impulses that travel to the brain in the act of seeing, or when electrical signals from the brain activate motor neurons causing hand muscles to move a pencil along a sheet of paper in the act of drawing. Here the eye and hand are interfaces between the body and the space it inhabits—interfaces across which interactions take place.
Electronic sensors and actuators do the same thing. A photoresistor, e.g., converts light into electrical resistance, whereas a loudspeaker converts an electrical audio signal into a sound. Analog-to-digital converters convert analog electrical signals to digital signals a computer can understand.
f(x) = (x − input_start) / (input_end − input_start) * (output_end − output_start) + output_start
Where x is the light sensor value; input_start and input_end are 0 and 100, respectively; and output_start and output_end are −240 and 240 respectively. Here is a Scratch project I made that interactively illustrates the mapping function.
There are many devices that can connect to Scratch for the purpose of connecting Scratch to the physical world. Following are data about the hardware featured or mentioned in Technomancy 101. These may be combined with electrical materials for a great variety of technomantic designs.
Camera & Microphone
If you are running Scratch on a recently manufactured computer, there is a good chance that computer has a webcam with microphone built into it, or you can connect external ones. The Scratch Sensing palette includes blocks for using your camera and mic as sensors. Here are a few examples:
Makey Makey is a set of touch sensors that are interpreted as keyboard or mouse events by your computer. Since Scratch has blocks for responding to keyboard and mouse events, you can use Makey Makey to make Scratch react whenever you touch or connect something that is at least a little electrically conductive. It may be easier to understand what I mean by observing Makey Makey in action:
Makey Makey GO is a USB stick that can respond as a single keyboard or mouse event (the defaults are space-bar and right-click but those can be changed). Unlike the Makey Makey Classic, the GO does not require a ground connection, which makes it even easier to use. I like the GO so much I keep one in my sling with my other everyday tools.
There are a few kinds of switches you can implement with Makey Makey. All of the following examples assume the object you are touching sends a space-bar signal to Scratch, except the fourth which assumes you are touching two objects: one interpreted as the ‘space’ key and the other as the ‘w’ key (requires Makey Makey Classic).
Makey Makey Classic allows for programming interactions where you connect objects to the computer and leave them connected for some time, by simultaneously attaching both the signal and ground wires to the object. E.g., you could make SIBOR function only when its material sigil, inscribed in electrically conductive paint, is connected to Makey Makey (you could accomplish this kind of interaction as well with a PicoBoard or Raspberry Pi).
You can use Makey Makey Classic to light an LED or power a small motor when a connected object is touched, without doing anything in Scratch.
Makey Makey resources:
- Browse all Technomancy 101 projects featuring Makey Makey
- Makey Makey Classic hookup guide and tutorial @ SparkFun
- Makey Makey GO How-To
- 20 Makey Makey Projects for the Evil Genius by Colleen Graves and Aaron Graves
The PicoBoard was developed by the Playful Invention Company (PICO), and was based on an earlier apparatus called the Scratch Sensor Board. Today, SparkFun manufactures and sells this version of the PicoBoard; its functionality is the same. It includes a button that can be in one of two states (pressed or not pressed); a slider that changes values as it moves from one side to the other; light and sound sensors; and four electrical resistance sensors.
The PicoBoard works with Scratch 2.0 online or off-line versions, and Scratch 1.4 including on the Raspberry Pi (v.i.). Its design is open-source, so you can legally modify it or manufacture your own, and you can sometimes find modified versions online.
- objects composed of electrical materials
- resistance-based sensors such as photocells (for detecting changes in light intensity), thermistors (for changes in temperature), reed switches (changes in proximity), and force-sensitive resistors (for responding to weight or impact)
- buttons, switches, and potentiometers
Here are just a few examples of things you could do with the PicoBoard:
Makey Makey is better for when you want something to happen when you touch the sigil, either with part of your body or with another conductive object connected to Makey Makey.
PicoBoard is superior when you want something to happen so long as the sigil is connected to the board. The latter can also be accomplished with Makey Makey Classic by attaching the sigil to signal and ground connections at the same time, but keep in mind that Makey Makey accomplishes this by repeatedly sending the same keyboard signal to Scratch, which may interfere if your project requires you to input text at some point; whereas PicoBoard has a dedicated reporter block within Scratch for returning the value of the sigil’s electrical resistance.
PicoBoard can also be used to detect temporary contact between two conductive objects, by attaching each object to one end of the pair of crocodile clips plugged into the board and detecting the resistance change when they touch, but it is not sensitive enough to detect you tapping an object with your finger.
- Browse all Technomancy 101 projects featuring the PicoBoard
- PicoBoard tutorial @ SparkFun — excellent intro to the kinds of things you can do with the PicoBoard (written for Scratch 1.4 but applies as well to 2.0)
- PicoBoard article on the Scratch Wiki
Here are some Sparkfun products that work well with the PicoBoard (you can get many of these from Adafruit or other vendors, too):
- big dome pushbutton (available in various colors)
- concave pushbtton (like the ones used on arcade video games; available in various colors)
- momentary button, panel mount (available in various colors)
- toggle switch (add a missile switch cover!)
- rocker switch
- mountable slide switch
- SPDT slide switch
- keylock switch (use with the toggle switch and missile switch cover!)
- LilyPad buttons, switches, and LEDs (these are made to be used with conductive thread)
- rotary potentiometer 10kΩ linear taper (add a knob to it!)
- slide potentiometer 10kΩ linear taper: small, medium, x-large (add a knob to that too: small and medium, or x-large)
- mini photocell
- thermistors 10K
- force-sensitive resistor: small, round 0.5″, square, or long
- flex sensors: 2.2″ or 4.5″ (make your own power glove!)
LEGO WeDo is a lower-primary-school STEM education platform that includes a motor, distance/motion sensor, and tilt sensor, all compatible with the LEGO construction system. The kits are not inexpensive, but you can acquire the pieces individually from eBay and other markets. The motor (LEGO part #8883) is the same one found in the LEGO Power Functions set, and the PF lights (#8870) and extension cable (#8871) also work with WeDo. The extension cable will also allow you to drive a LEGO Technics 9v mini-motor (#71427/#43362; out of production but you can still find them for resale), or perhaps a hacked block for a custom circuit of your own design.
In early 2016, LEGO released WeDo 2.0, which has similar capabilities to the previous version but features a wireless (Bluetooth) controller with built-in RGB LED and a speaker that can play tones, and it can interface with all LEGO Power Function 2.0 sensors and actuators.
Scratch 2.0 and 1.4 include built-in extensions that add blocks for controlling the WeDo motors and reading data from the WeDo sensors. Scratch 2.0 includes an extension for WeDo 2.0, adding blocks to set the light color and play notes through the speaker
LEGO is highly modular and can be used to rapidly prototype a good variety of machines and interactions, but it is expensive compared to most of the other tech on this page. If the cost is prohibitive, do not worry, for anything you can make with WeDo you can make without it; it just might require a little more work to make it.
- Browse all Technomancy 101 projects featuring LEGO WeDo
- LEGO WeDo Construction Set article on the Scratch Wiki
- Instructions for building the WeDo models
- Instructions for the WeDo 2.0 models
- The LEGO TECHNIC Idea Book Series (Tora no Maki) by Yoshihito Isogawa — an amazing collection of machines you can build with LEGO, many of which can be animated with the WeDo sensors and motors. The author also wrote two idea books for LEGO Power Functions: Machines and Mechanisms, and Cars and Contraptions.
- RoboCAMP has several educational series of models built with WeDo 1.0, including CityCAMP, SafariCAMP, and StarCAMP — I mention these for inspiration
Following is a list of LEGO part numbers for compatible parts:
- WeDo 1.0 kit: #9580
- 1.0 USB hub: #9581
- 1.0 motor: #8883 (works also with Technic motors #71427 and #43362)
- 1.0 motion/distance sensor: #9583
- 1.0 tilt sensor: #9584
- 1.0 light: #8870
- 1.0 extension wire: is 8″ (20cm) #8886, 20″ (50cm) #8871
- WeDo 2.0 kit: #45300
- 2.0 Smart Hub: #45301
- 2.0 rechargeable battery: #45302
- 2.0 motor: #45303
- 2.0 motion/distance sensor: #45304
- 2.0 tilt sensor: #45305
Arduino is an open-source prototyping platform based on the Atmel AVR family of microcontrollers. There are many different Arduino boards; the one pictured above is the UNO, the most commonly used board (at the time I wrote this).
Arduino capabilities can be extended with boards call shields that plug into the Arduino’s pin headers.
There are a few ways to interface Scratch with Arduino:
- The Scratch Arduino Extension, written for Scratch 2.0, adds blocks for LEDs, servos, buttons, knobs, and more. You can run this extension in ScratchX.
- The s2aio, also written for Scratch 2.0
- S4A is a modification of Scratch 1.4, providing new blocks for the Arduino pins. Here is a good tutorial from Instructables.
- Try mBlock, a Scratch derivative.
- By configuring Arduino as a human interface device (HID), the board can send mouse or keyboard signals to Scratch.
The Raspberry Pi is an inexpensive, credit-card-sized, single-board computer that can be run in various configurations including desktop, laptop, video game console, media center, and embedded system. Raspbian, the Pi’s official operating system, is a variant of Debian Linux optimized for the Pi’s architecture. The current version of Raspbian includes special versions of Scratch 1.4 and 2.0 modified to interact with the Pi’s general-purpose input/output (GPIO) pins. The Pi’s Scratch 2.0 also includes an extension to interface with the Sense HAT (“hardware attached on-top,” similar to Arduino’s shields; you can also work with the Sense HAT in 1.4). Scratch 1.4 is able to interface with some older extension hardware for the Pi, including PiFace and Gertboard.
The Raspberry Pi differs from the other interfaces mentioned here in that it is a personal computer with the form factor of a sensor board or microcontroller. That means you can run Scratch on the Pi itself, but operate the Pi in headless or semi-headless mode, i.e., without a monitor, keyboard, or mouse attached to it. I use RealVNC for remote access to my Pis (I have several running in various places) from my laptop, tablet, or phone.
Raspberry Pi resources:
- MagPi, the official magazine. MagPi also publishes books about RasPi essentials and projects, and everything they publish (books and magazines) may be downloaded in PDF for free.
- Scratch 2.0 on the Raspberry Pi
- Creating Scratch 2 Extensions on Your Raspberry Pi — a tutorial on how to write your own extension for Scratch 2.0 on the Raspberry Pi, which adds several features beyond the standard GPIO extension
- Scratch 1.4 GPIO
- Raspberry Pi products @ SparkFun
- Raspberry Pi products @ Adafruit
Because Scratch can respond to keyboard and mouse input, any device that can appear to your computer as a mouse or keyboard, or which has a utility for mapping the device’s data to keyboard or mouse signals, can interface with Scratch (this is exactly what Makey Makey does, and the same example scripts for it apply to these other devices as well). This includes many video game controllers, the Leap Motion controller (which also has a third-party Scratch extension), the Myo gesture control armband, and the Emotiv EPOC and Insight brain-computer interfaces. These allow you to program Scratch to respond to your hand gestures, arm movements, thoughts or emotions, &c.
- Dan O’Sullivan and Tom Igoe, Physical Computing: Sensing and Controlling the Physical World with Computers (Thomson, 2004) xvii.
- This tutorial from Electronics for Dummies illustrates the relationship between switches and logic gates by showing how to wire two DPDT switches to make AND, OR, and NOT gates.
- O’Sullivan and Igoe, op. cit., xviii.