Panoramic Virtual Reality and Archaeology
John W. Rick and Dakin Hart
Table of Contents:
As archaeologists, we often deal with an intrinsically visual subject matter in the form of landscapes, sites, and artifacts, but until recently we have been very limited in our ability to convey to others the reality of these subjects. Only in the context of slide presentations, motion pictures, or well-illustrated and expensive books have we come close to giving a palpable sense of the stuff we work on. While it is true that our physical data can hardly speak for themselves, it is also the case that a clear visual understanding of these places and objects often should precede the more intellectual pursuits of interpretation or hypothesis testing. We need to know our material well, and we need to pass this knowledge to others, be they our students, colleagues, or a broader public.
Electronic media are becoming available to serve these purposes, including digital video and photography as well as less photo-realistic technologies such as GIS (geographic information systems), CAD (computer-aided design) and associated three-dimensional modeling and rendering. With the increase in our ability to deliver visual messages comes a new challenge of how to organize the content. For example, a CD-ROM full of color stills or video clips is less accessible than either a color picture book or an analog video. One interesting option is to organize the media in such a way that they simulate reality, so that the end user can deal with the information as if it were a version of the real world--exploring, sensing, and coming to the same understandings we do during fieldwork or analysis.
We would like to discuss one particular form of such virtual reality based on panoramic photographs. Aside from our own familiarity with this technique, we want to call it to our colleagues' attention because it is one of the easiest forms of virtual reality to produce in terms of equipment and expertise, one of the easiest to deliver, and yet a quite effective means of conveying reality.
These representational strategies give the sense of navigating through a scene or manipulating an object and thus have often been called "three-dimensional" (3-D). In truth, they are not truly 3-D, because the viewer is restricted to a single point from which the image is manipulated. Because a single landscape or object panorama is a very limited segment of reality, multiple panoramas are often linked together to form a semicontinuous spatial experience. The viewer can jump from one panorama, or "node," to the next, giving either the effect of traveling through space or of examining objects. In addition, still photographs or video clips can be linked to panoramas and displayed by clicking on specific points (usually called "hot spots"), allowing greater detail of subjects.
How wide is very wide? We have found that for 35-mm photography, a lens in the 15-18 mm range is best; the lens can be fish-eye or nondistorting rectilinear. The rectilinear lens is superior in producing images more generally useful beyond the PVR arena. Lenses of this sort are available from Tamron and Tokina as well as camera manufacturers, although the latter can be quite expensive. A good if not excellent new 17-mm lens ranges around $250-350. This significant expense may be somewhat compensated by the broad utility of the lens in other circumstances; we cannot imagine doing archaeological photography without one. Perhaps the greatest disadvantage of such a wide-angle lens is that it will pick up sun flare somewhere in the panorama, except at about midday. Lens hoods may help with this problem, but tend to crop the shot with an irregular edge, defeating some of the purpose of the wide-angle lens. In practice, we have found that lens flare can be fairly easily edited out of the stitched image, and it actually is desirable in many cases since it gives a sense of the sun's direction.
To further increase the angle of vision of the outward pan-oramas, shots are usually taken with the camera in vertical or portrait position. To facilitate this, and to allow the camera to be positioned at the lens's nodal point, a number of useful, if expensive ($200-600), specialized PVR tripod heads are available from companies such as Kaidan (http://www.kaidan.com/) or Peace River Studios (http://www.peaceriverstudios.com/). These allow the camera to be positioned and leveled independent of the tripod (Figure 2). In addition, most have click-stops that will position the camera in 8, 12, 16, 24, or other intervals for each full rotation. A 17-mm lens will take a full 360[[ordmasculine]] panorama in 12 shots in portrait position, with about 60 percent overlap between images. While it would be possible to take an eight-shot panorama, it may be more difficult to stitch, and should one shot fail or be missing, the panorama will be incomplete. Panoramas shot with "normal" wide-angle lenses of 28 mm will require 18 to 24 shots to complete. For taking large numbers of outward panoramas, such a full-featured tripod head is indispensable, increasing the speed and accuracy of the shoot.
Ambitious PVR projects will require a considerable amount of film, which makes digital photography an attractive option. Less expensive digital cameras (generally $600-1,000, still triple the cost of a film camera) suffer from relatively low resolution, small storage capacity, poor lens options, and limited control over exposure and other settings. Professional-level digital cameras are extremely expensive (generally in the $5,000-30,000 range), fall short of the resolution that film achieves, are often bulky, and require costly mass storage media or frequent uploading of images to a computer. While digital photography will soon be an excellent choice for PVR projects, the technology for it is not quite ready for reliable, economical, and practical field projects. Image resolution is perhaps the least of the problems, as quite acceptable stitched PVR images are frequently around 2,600 x 700 pixels, and many low-end digital cameras will produce resolutions of around 700 pixels on their longer dimension, which in portrait mode is the image's height. New PVR display technologies, however, will soon be increasing the usable image size, rendering low-resolution panoramas obsolete.
Conventional film, with its very high resolution, can be taken with a variety of cameras and does not require professional equipment. In fact, the use of automatic exposure and auto-focus is not advisable. A fixed focus and exposure is best, leaving scale, color, and density identical in the overlapped areas of sequential images. A motor drive or a self-winding camera is worthwhile, both to speed shooting of the multiple frames and to reduce forces exerted on the mount that might change camera position. A cable release is helpful, and a two-way flash shoe bubble level keeps the camera rotating on a horizontal plane. A strong tripod--better than the generic camera/video models--will pay off, because some designs of tripod heads will exert a strong bias on the tripod as the camera rotates, and off-plane tilting of the camera will have negative consequences.
In practice, we have generally shot three 12-shot panoramas per 36-exposure roll of film, with a leading and trailing storyboard image. It is desirable that the same film be used throughout a given lighting situation, but a shift into shady or interior situations can be coupled with a change to a different film; color shifts are expectable and even desirable for these transitions. We strongly prefer to shoot slides because of their utility in other situations and their ease of sorting and ordering prior to digitizing, but color negatives can also be digitized. Labeling film and keeping a field register of the shooting process will pay off later. Film from large-scale projects can get mixed up easily, and in many panoramas there may be problems that need to be recorded: skipped or duplicated frames, lighting changes, or other factors affecting the final image.
Having two functioning cameras while shooting panoramas allows simultaneous shooting of stills for inclusion as details within a panorama. They are ideally taken under the same lighting conditions, without interrupting the orderly progression of the panoramic sequences. Overall, a crew of up to three people can be effectively employed, although care must be taken to avoid shooting the crew--they have to become accustomed to walking clockwise ahead of the PVR camera's rotation!
We have little experience with digital cameras and will not try to give a hands-on account of the digital shoot. It should be clear, however, that the primary difference is the storage media. With digital images, the photographic outcome is immediately apparent, so corrective measures can be taken in the field. The loss of digital files, however, is perhaps greater than the likelihood of damaging or losing film; backup of the shoot is therefore a requirement. All this suggests that immediate access to computers is necessary. Some sort of large-scale storage will be required, such as writeable CD-ROMs or backup tapes. To illustrate this issue, a productive day of shooting can produce about 25 panoramas of 12 images each, for a total of about 300 images. Using a fairly high-quality size of 1,000 x 1,750 pixels, each uncompressed image will require about five mb of storage space, for a total day's storage requirement of 1,500 mb. This would fill a high-capacity backup tape or about 2.5 CDs; there are both monetary issues and time costs in storing this much information. Image compression could be used or a lower resolution could be saved, but both will compromise the long-range potential of the images. Assuredly these problems will be overcome in the future, but some of the current difficulties facing an intensive production schedule should be evident.
The stitching process is carried out by specialized programs that meld the overlapped areas of the images, bending and blending them into a seamless image when all goes well. The usual stitch is of a cylindrical image from 360[[ordmasculine]] of photographs, but panoramas can also be made from a full sphere of images, which adds shots taken up and down, or of partial panoramas when a full 360[[ordmasculine]] coverage is not available or desired. The original software created for stitching images is Apple's Quicktime Virtual Reality (http://quicktimevr.apple.com/), or QTVR, a name that is sometimes applied to the whole genre. This program is now in its second major version (2.0) and offers many features, but until recently it has been restricted to the Mac environment and requires the creation of scripts written in rather opaque code. The program is expensive ($395) and employs a keyboard "dongle" as a hardware lock, which can be clumsy if you are using various machines for development. It produces a Quicktime product that can be played back or navigated by any computer running recent Quicktime software. This format is also supported by recent versions of Netscape.
As might be anticipated, a number of competitors have entered the market, with increased user friendliness. These include:
We use PhotoVista by Live Picture on a Windows platform; here is how the stitching process proceeds in this program:
A word about the process of creating the movie from stills is necessary. The basic goal is to create a single file containing a sequence of linked images. Any software that can import images and then export a Quicktime, MPEG, or AVI file will do this (this includes Adobe Premiere, Apple's Movie Player, and other free and shareware programs). The panorama viewers are able to decode these movie files and control them frame by frame. Movement is enabled by creating a matrix of pictures following a simple formula--every row must have as many frames as the longest row. If there are 10 shots of a pot around its middle, then 10 frames of the pot from the top are also necessary, even if only one shot would have covered it (these can, of course, be copies of one image). This makes more sense if you think of the movie as a Mercator map (Figure 5); from any point in the center, one can move up and hit the North Pole. The viewer program has a similar map that records the number of rows and frames in the movie and the relationships between frames. By clicking and dragging within a viewer program, the user can scroll frame by frame in any direction for which there are stills, typically in a looping, endless sequence. Like the flip drawings in our elementary school textbook margins, this frame-by-frame scrolling creates the illusion of movement.
Practically, these guidelines lead to fairly simple standards for shooting. If at all possible, abide by the following:
Another issue to consider in the production of object movies is the possibility of blue-screening or masking. This is equivalent to the technology used to superimpose a TV weatherman on a map of the United States. The pure blue screen, because of its uniformity, is easy to replace with another scene as long as the color does not also appear on the weatherman. The same is true for object movies. If you use a background very different in color from the object, it is possible later to mask the object and replace the background. The result is an object that will float and spin in the middle of whatever scene you place it in. You could, for example, create a hot spot in a panorama of an excavation that would place a spinnable pot in the location where it was found. There are several examples at the Fine Arts Museums of San Francisco's web site (http://www.thinker.org) that show object movies within exhibit hall scenes.
First, it is important to make some decisions about platform. A Wintel-only application to be delivered via CD-ROM, for example, can be developed very differently from a web site. Once you know something about who will use the application and with what equipment, you can decide what programming and language environment to work in. Visual Basic is an environment capable of developing applications for several platforms, as are Java (Sun's language for which various companies have developed tools), Macromedia Director (a tool that has its own scripting language), Apple Media Tool (a cross-platform, programming-free solution), and mFactory's mTropolis (which combines the strengths of Director and Apple Media Tool). Using each of these environments involves a steep learning curve. A language like Java is more difficult to learn and use but allows you to do almost anything. Object-oriented programs like Director, Apple Media Tool, and mTropolis are easier to use and offer faster production times, but may not have the functionality you need.
HTML is another possible development environment. A web page can present almost any kind of content that was originally developed elsewhere and is hard to beat for simplicity and functionality. A web site also can be written to CD or other media and run locally. Most important, it can be developed and deployed competently by someone at any skill level. This makes it an ideal platform for low-budget, time-sensitive projects whose personnel lack programming expertise.
We are convinced, however, that more professional use can also be made of PVR. Research at an archaeological site can be described in great detail for the benefit of colleagues, with text accompanied by many 100s of still images, maps, graphs, and so forth. The spatial logic of PVR is unlike more formal means of archaeological reporting, but we believe that it is a very effective means for encouraging familiarity with a place while also presenting archaeological arguments and interpretations.
As a last word of advice, the Internet is a great source of information on this rapidly changing technology. Both the Kaidan (http://www.kaidan.com/) and Apple Computer QTVR (http://quicktimevr.apple.com/) web sites are good starting places for an exploration of the topic.
John W. Rick is in the Department of Anthropology of Stanford University; Dakin Hart is assistant to the director of the Fine Arts Museums of San Francisco.