Digital media are images, sounds, text, &c., that can be stored, transmitted, processed, and accessed by digital computers. Multiple kinds of digital media working together are called multimedia. The implementation of computers as boxes for presenting multimedia is contradistinct to physical computing, although there is always a physical component to digital media, such as the memory cells that store them, or the patterns of light or sound vibrations that manifest them to the senses. As well, there is always a digital component to physical computing. Technomancy 101 utilizes both multimedia and physical computing, often combining them such that physical actions produce changes in the images or sounds Scratch exhibits.
Due to the textual nature of the command-line interface, most computer programming languages treat data (as text) first, and images and sounds later if at all. Scratch excels at doing things straight away with images and sounds, which may appeal to those who might be inspired to take up multimedia programming but who are incurious about data processing or other aspects of computing per se (which Scratch can also do).
In this appendix, I explain some peculiarities of digital media in Scratch, and suggest sources of visual and audio media for use in your own projects. N.b., most of this information is based on Scratch 2.0 and may apply differently or not at all to other versions.
Images in Scratch
Scratch can display images as sprite costumes or stage backdrops, and generate turtle graphics with the pen feature. Scratch has a built-in image editor for creating or editing bitmap (a.k.a. raster) or vector graphics (see this article for more about the differences between them), and you may import PNG, JPG/JPEG, GIF, and SVG file types.
The Scratch screen area is 480 pixels wide × 360 pixels high. Larger bitmap images are automatically scaled down when imported; vector images are not. Scratch has a full-screen mode that enlarges the Scratch screen area while maintaining the 4:3 aspect ratio, which is great for viewing projects on a large television or projected onto a wall or other surface.
When Scratch is run in full-screen mode, vector graphics scale to fit the larger size, whereas raster (bitmap) graphics and graphics drawn by the pen are stretched and become blurred and pixelated.
Scratch supports transparency and semi-transparency in GIF, PNG, and SVG image formats, although the transparency of SVG images is sometimes lost when importing. You can import SVGs with partial transparency that were made in Inkscape (v.i.) if you do it this way. Scratch’s ghost effect can also be used to make a sprite transparent or semi-transparent.
There are some known problems with Scratch’s graphic effects in version 2.0. If you are using any effect other than ghost you can expect performance degradation or vector graphics to become pixelated (as they are transformed to bitmaps), or both.
Scratch cannot play video files but simple animations are possible by rapidly cycling through a series of images, and you can also program sprites to move across the screen at various speeds and in various directions, or rotate left or right. When you import an animated GIF into Scratch, Scratch automatically creates a new costume for each frame in the file.
While Scratch does not support three-dimensional graphics, there are some projects that simulate 3D, and an experimental extension that lets you program true 3D graphics in Scratch.
Here are a few Scratch studios related to graphics:
- 2D Pen Projects
- Amazing Art
- Blockshade Art
- Lindenmayer Systems! — fractals, recursion, &c., featuring L-systems
- PEN ONLY
Using a touchscreen allows you to interact with Scratch images in different ways:
Displaying Text in Scratch
You can work with text or string data in Scratch, but Scratch’s only intrinsic means of outputting variable text to the screen is as cartoonish dialog or thought bubbles, or by showing the value of a variable or list on screen, none of which bestows any control over typography. You can draw text into costumes and display it that way; the Scratch paint editor includes six fonts to draw with, or you may import images made outside of Scratch with whatever fonts you have installed on your computer. You can even build your own typing system (as I did for the Oraskul project).
There are Scratch 1.4 modifications, e.g., Scribble, have special blocks for displaying variable text to the screen using Scratch’s built-in fonts.
Sounds in Scratch
Scratch can play imported MP3 or WAV files, and some musical notes and drum sounds with the sound blocks. It also has a built-in sound recorder and editor with some basic editing and effects tools. All audio played through Scratch is monaural; imported stereo files are automatically mixed and any surround-sound channels are removed.
Prior to version 2.0, Scratch used MIDI sounds for playing notes and drums, but since 2.0 was developed with Flash which does not support MIDI, the Scratch team made MIDI-like sounds for playing notes and drums. Thus, the range of available instrument voices was considerably shortened, however the note numbers are still based on the MIDI architecture. See the Scratch Wiki page for play note () for () beats, for a list of numbers and their corresponding musical notes and frequencies (in hertz).
There is an experimental extension that creates sound synthesizer blocks in Scratch. Unlike regular Scratch audio, these synth tones are stereo and can be panned.
Here are some Scratch studios related to sound:
- Create Music! — an excellent selection of Scratch sound projects including Scratch Soundpad v1, MIDI Player v2.0-beta (take a minute to load—1909 assets!), and Keyboard Piano
- ElecTronic Music
- Music Studio
Making Music in Scratch
Versions of Scratch prior to 2.0 used the Squeak MIDI library and included actual MIDI voices. Since there is no MIDI support in Flash, Scratch 2.0 uses a bank of 21 instrument and 18 drum sounds. The Scratch wiki explains how the notes work. Additionally, several of the sound files bundled with Scratch are recordings of musical instruments, and you can record or import your own.
You can make your own keyboards and drum pads with Makey Makey and Scratch’s when () key pressed or key () pressed? block. Below are two keyboards I made for playing the sounds and colors of the seven Hellenistic planets; one is made with electrically conductive paint, and the other with physical keys that were ritually suffumigated and incanted over at auspicious hours.
You could also use potentiometers with a PicoBoard to make your own volume and tempo controls, instrument selectors, &c.
Web Sources of Images & Sounds
There are many online repositories of sound and image files. Some require fees, whereas others are free to use under certain licenses such as Creative Commons, or because they are in the public domain. Be mindful that any project you share on scratch.mit.edu is automatically licensed under Creative Commons Attribution Share-Alike license 2.0 (CC-BY-SA), which may conflict with the licenses or copyrights of any images or sounds you wish to include in said project. For licenses requiring attribution, you can include it in the “Notes and Credits” section of the project’s page, which is also a fine place to credit the source of any third-party media you have used.
Here are some sources of media that are free to use:
- “Finding Public Domain & Creative Commons Media” by Harvard Law School Library — teaches you how to identify media licensed for your own use, and also includes links to several image, audio, and video repositories
- The Internet Archive — my favorite online repository of digital media of all kinds, but n.b. some of these are copyrighted
- BBC Sound Effects
- Conserve the Sound
If you have cause to convert an image or sound file from one format to another, I make frequent use CloudConvert, or some of the editors I will mention next can assist with that.
Making Your Own Images & Sounds
You may obtain your own images and sounds from cameras, scanners, drawing tablets, audio recorders, &c., and you can get quite artsy with this to create weird, mind-altering multimedia, or include digital samples of your intended magical effects or targets. Many professional sound effects are made with common objects, e.g., you can make a laser sound with a rubber band (check out this guide to creating sound effects for more ideas).
In addition to Scratch’s own paint and sound editors, I use Inkscape, GIMP, and Audacity, extensively, for generating and editing images and sounds. They are open-source and free to use. Of course, if you have access to premium editors such as Adobe Photoshop, Illustrator, Audition, &c., you may rely on those.
Following are some online editors and generators of sounds and images.
- Pixlr Editor — cloud-based photo editor and bitmap drawing application
- Draw SVG — online SVG drawing tool
- svg-edit — Google’s SVG drawing editor
- Paulstretch — Stretches audio over time, turning almost any sound into an ambient drone soundscape. If you use Audacity, Paulstretch is included as an available effect.
- Soundation, Audiotool, and AudioSana — online music and sound effects studios (try AudioSauna with Makey Makey)
- Online Tone Generator — just what it says
- P22 Music Text Composition Generator — Translates text into music. You can use it to make musical “sigils” from statements of intent.
- Musicalgorithms — Tools for composing algorithmic music. Also good for making musical sigils.
- FastHub.net — online text-to-speech (TTS) synthesizer for personal/non-commercial use
- YAKiToMe — another free TTS tool free for personal/non-commercial use
- Text2Speech.org — TTS free for any purpose including commercial