Digital Media

Digital media are images, sounds, text, &c., that can be stored, transmitted, processed, and accessed by digital computers. Multiple kinds of digital media working together are called multimedia. The implementation of computers as boxes for presenting multimedia is contradistinct to physical computing, although there is always a physical component to digital media, such as the memory cells that store them, or the patterns of light or sound vibrations that manifest them to the senses. As well, there is always a digital component to physical computing. Technomancy 101 utilizes both multimedia and physical computing, often combining them such that physical actions produce changes in the images or sounds Scratch exhibits.

Due to the textual nature of the command-line interface, most computer programming languages treat data (as text) first, and images and sounds later if at all. Scratch excels at doing things straight away with images and sounds, which may appeal to those who might be inspired to take up multimedia programming but who are incurious about data processing or other aspects of computing per se (which Scratch can also do).

In this appendix, I explain some peculiarities of digital media in Scratch, and suggest sources of visual and audio media for use in your own projects. N.b., most of this information is based on Scratch 2.0 and may apply differently or not at all to other versions.

Images in Scratch

Scratch can display images as sprite costumes or stage backdrops, and generate turtle graphics with the pen feature. Scratch has a built-in image editor for creating or editing bitmap (a.k.a. raster) or vector graphics (see this article for more about the differences between them), and you may import PNG, JPG/JPEG, GIF, and SVG file types.

Scratch’s built-in paint editor lets you make or edit bitmap or vector images as costumes and backdrops. You can import images made with other editors.

The Scratch screen area is 480 pixels wide × 360 pixels high. Larger bitmap images are automatically scaled down when imported; vector images are not. Scratch has a full-screen mode that enlarges the Scratch screen area while maintaining the 4:3 aspect ratio, which is great for viewing projects on a large television or projected onto a wall or other surface.

Left: an animated magic circle and sigil projected onto the floor, with spaces for actual candles. Right: a pentagram and animated caput draconis glyph projected onto a skull.
Video projector
This mini video projector has a 1/4 inch 20 UNC thread hole to fit on tripods and similar mounts, so it may be placed in various locations including on high to project down onto the ground.

When Scratch is run in full-screen mode, vector graphics scale to fit the larger size, whereas raster (bitmap) graphics and graphics drawn by the pen are stretched and become blurred and pixelated.

You can sneak large bitmap images into Scratch by embedding them in a SVG file and using Scratch’s set size to ()% or change size by () block to shrink them back down after importing the SVG into Scratch. When running Scratch in full-screen mode, such images do not pixelate the way bitmap images do. To see what I mean, compare this project to this one, running both in full-screen mode.

Scratch supports transparency and semi-transparency in GIF, PNG, and SVG image formats, although the transparency of SVG images is sometimes lost when importing. You can import SVGs with partial transparency that were made in Inkscape (v.i.) if you do it this way. Scratch’s ghost effect can also be used to make a sprite transparent or semi-transparent.

There are some known problems with Scratch’s graphic effects in version 2.0. If you are using any effect other than ghost you can expect performance degradation or vector graphics to become pixelated (as they are transformed to bitmaps), or both.

Scratch’s graphic effects often behave differently in the online vs. offline editor. E.g, compare running Technodelic or Technodelic2 online and off; the projects show severe performance degradation in the offline editor. On the other hand, using any effect other than ghost or brightness will automatically pixelize vector images in the online editor but not in the offline editor.

Scratch cannot play video files but simple animations are possible by rapidly cycling through a series of images, and you can also program sprites to move across the screen at various speeds and in various directions, or rotate left or right. When you import an animated GIF into Scratch, Scratch automatically creates a new costume for each frame in the file.

While Scratch does not support three-dimensional graphics, there are some projects that simulate 3D, and an experimental extension that lets you program true 3D graphics in Scratch.

Here are a few Scratch studios related to graphics:

Using a touchscreen allows you to interact with Scratch images in different ways:

Displaying Text in Scratch

You can work with text or string data in Scratch, but Scratch’s only intrinsic means of outputting variable text to the screen is as cartoonish dialog or thought bubbles, or by showing the value of a variable or list on screen, none of which bestows any control over typography. You can draw text into costumes and display it that way; the Scratch paint editor includes six fonts to draw with, or you may import images made outside of Scratch with whatever fonts you have installed on your computer. You can even build your own typing system (as I did for the Oraskul project).

There are Scratch 1.4 modifications, e.g., Scribble, have special blocks for displaying variable text to the screen using Scratch’s built-in fonts.

Sounds in Scratch

Audio Input and Output Devices
External and wireless speakers can be placed in diverse locations for various effects. Field recorders can capture sounds to use in your own projects. Headset microphones let you talk to Scratch while standing away from your computer or moving around.

Scratch can play imported MP3 or WAV files, and some musical notes and drum sounds with the sound blocks. It also has a built-in sound recorder and editor with some basic editing and effects tools. All audio played through Scratch is monaural; imported stereo files are automatically mixed and any surround-sound channels are removed.

Scratch’s built-in sound editor allows you to import or record sounds, make simple edits, and add effects including fade in and out, amplify (louder) and attenuate (softer), add silence, and reverse the sound.

Prior to version 2.0, Scratch used MIDI sounds for playing notes and drums, but since 2.0 was developed with Flash which does not support MIDI, the Scratch team made MIDI-like sounds for playing notes and drums. Thus, the range of available instrument voices was considerably shortened, however the note numbers are still based on the MIDI architecture. See the Scratch Wiki page for play note () for () beats, for a list of numbers and their corresponding musical notes and frequencies (in hertz).

There is an experimental extension that creates sound synthesizer blocks in Scratch. Unlike regular Scratch audio, these synth tones are stereo and can be panned.

Here are some Scratch studios related to sound:

Making Music in Scratch

Versions of Scratch prior to 2.0 used the Squeak MIDI library and included actual MIDI voices. Since there is no MIDI support in Flash, Scratch 2.0 uses a bank of 21 instrument and 18 drum sounds. The Scratch wiki explains how the notes work. Additionally, several of the sound files bundled with Scratch are recordings of musical instruments, and you can record or import your own.

Making Music

You can make your own keyboards and drum pads with Makey Makey and Scratch’s when () key pressed or key () pressed? block. Below are two keyboards I made for playing the sounds and colors of the seven Hellenistic planets; one is made with electrically conductive paint, and the other with physical keys that were ritually suffumigated and incanted over at auspicious hours.

Planetary Key-Boards

You could also use potentiometers with a PicoBoard to make your own volume and tempo controls, instrument selectors, &c.

Web Sources of Images & Sounds

There are many online repositories of sound and image files. Some require fees, whereas others are free to use under certain licenses such as Creative Commons, or because they are in the public domain. Be mindful that any project you share on is automatically licensed under Creative Commons Attribution Share-Alike license 2.0 (CC-BY-SA), which may conflict with the licenses or copyrights of any images or sounds you wish to include in said project. For licenses requiring attribution, you can include it in the “Notes and Credits” section of the project’s page, which is also a fine place to credit the source of any third-party media you have used.

Here are some sources of media that are free to use:

If you have cause to convert an image or sound file from one format to another, I make frequent use CloudConvert, or some of the editors I will mention next can assist with that.

Making Your Own Images & Sounds

You may obtain your own images and sounds from cameras, scanners, drawing tablets, audio recorders, &c., and you can get quite artsy with this to create weird, mind-altering multimedia, or include digital samples of your intended magical effects or targets. Many professional sound effects are made with common objects, e.g., you can make a laser sound with a rubber band (check out this guide to creating sound effects for more ideas).

In addition to Scratch’s own paint and sound editors, I use Inkscape, GIMP, and Audacity, extensively, for generating and editing images and sounds. They are open-source and free to use. Of course, if you have access to premium editors such as Adobe Photoshop, Illustrator, Audition, &c., you may rely on those.

Following are some online editors and generators of sounds and images.