NZSM Online

Get TurboNote+ desktop sticky notes

Interclue makes your browsing smarter, faster, more informative

SciTech Daily Review

Webcentre Ltd: Web solutions, Smart software, Quality graphics

Feature

New Ways of Using Old Media

Combining all manner of media could revolutionise the way we communicate and learn.

Jennifer Lennon and Hermann Maurer

Modern hypermedia systems which combine a variety of "old" media, such as computers, video and audio systems, are leading to impressive breakthroughs in the technology of information transfer. Nonetheless there are certain aspects of modern hypermedia systems which are definitely dinosaurian. Many systems are huge, cumbersome and difficult to harness. We need systems which enable us to communicate much more effectively than we do at present. We need novel ways of "writing" information that make the "reading" much more efficient.

There are many untapped ways of using old media in new ways, yielding entirely new techniques and communication facilities. One reason why such techniques have not been used before is that it was not possible to do so -- only the advent of powerful multimedia computers has placed these techniques at our disposal.

We already have the capacity within the human brain to undertake faster, more efficient processing of computer information than we do at present. We know the brain is amazingly good at parallel processing -- making telephone calls while driving the car, reading a book while watching television, keeping an ear on several conversations at once, and so on. It requires a certain expenditure of energy, but could be well worth it if we can vastly improve the way in which we take in information.

Picture Galleries and Multi-movies

Applications such as CDWriter are making extremely effective use of the idea of displaying sets of miniaturised pictures to aid in the retrieval of photos from a CD. We have seen anything from 20 to 100 miniature photos on the screen at once, and it is surprising how easily we can locate the ones we want, particularly when we are familiar with the originals.

We can imagine a particularly useful real estate system based on this. Demonstration catalogues could be set up which include multiple shots of the same house. A particularly useful system could be coupled with graphical queries and visualisation programs. The Human-Computer Interaction Laboratory at the University of Maryland has developed an interactive program for prospective house buyers. Available houses are indicated on a city map. The prospective buyer defines price ranges, number of rooms, what key amenities they would like to be close to, such as schools.

Every change in the defined parameters is immediately reflected on the map by a corresponding change in the number of houses which fit the requirements. Thus the buyer narrows down the options to a manageable number that can be viewed in greater detail using a graphical database containing two- and three-dimensional views of house exteriors and interiors. Project a little further into the future, and it will be possible to take virtual "tours" of selected houses, saving considerable leg work when house-hunting.

Suppose that for a biology course we have three movies each depicting a bird flying -- an albatross, a fantail and a sparrow, for example. Displaying the three movies simultaneously would provide an effective means of comparing and contrasting different methods of flight. It would be even more effective if we could resize the windows, zoom into specific ones or re-play all three clips simultaneously in slow motion. With the advent of video software such as Quicktime, movie demonstrations such as this will become much easier to develop.

Preparing demonstrations containing video clips is a time-consuming task, so it would be useful to find alternative ways of doing it more efficiently. Scanning two or three movies simultaneously while looking for the desired sections is one way of saving time.

Better Use of Sound

Digitised sound could be applied more usefully than at present. There is no doubt that listening to the sound of fine oratory is much more engaging than reading a transcript. As tele-teaching and distance learning become more computerised, the use of digitised sound must inevitably increase. This will encourage the development of more efficient information retrieval systems -- we will need to be able to "search" digitised speech more efficiently than we can do at present.

It is now possible for computers to play back speech at twice the recorded speed and still maintain an acceptable level of pitch and clarity. This is certainly a first step in the right direction. Another interesting alternative to traditional search techniques is to divide a recording up into (say) four parts and to play all four sections simultaneously. If, for example, we know that the recording of a 20-minute lecture contains a section on ozone holes that is of particular interest, we can divide the speech up into four five-minute sections and play them simultaneously. The word "ozone" will stand out dominantly just as names "jump out" at us from a crowd at a noisy party.

It should be possible to extend the use made of digitised speech, searching it more efficiently than we do at present. If the waveform patterns corresponding to speech are displayed on a screen, it may be possible to "read" them. We may, for example, be able to find significant passages by noting changes in volume. We may learn to recognise our own characteristics and to differentiate them from a colleague's patterns, so that we can see certain demarcations in conversations.

We could also provide signposting for users searching for particular passages by reading their waveforms. Let us suppose that the waveforms associated with a speech can be displayed. Signposts in the form of icon markers can be inserted at key intervals, for example at the beginning of new paragraphs and sections. The symbols can be text or icons and can, of course, be edited at any stage. They can be opened as needed to provide additional information which can include text, graphics and movie clips, as well as speech and other types of sound.

A Powerful New Shorthand

We believe that if traditional multimedia is augmented with a language consisting of dynamic abstract symbols, users will be given more control over the amount and type of information they have on the screen at any one time. A powerful referencing system can be developed that will enable readers to skim over sections they are not interested in, and then progressively add details as and when they are ready for them.

Since the symbols are dynamic, we use the term "abstract movies" or "MUSLI" movies, an anagram for multi-sensory language interface.

Simple symbols can have a variety of attributes and states added to them to represent further information. For example, if an open semi-circle represents something which is alive, then we can use a closed circle to represent an animal and a closed semi-circle to represent a plant. If the attribute ~ denotes intelligence, then a circle containing a tilde could denote a human. If the human is cold, then we might colour the circle blue.

We can use symbols across a variety of contexts -- if a rectangle denotes a building, then applying a medical cross to it produces a symbol for a hospital, along with symbols for a doctor or a medicinal plant.

Symbols, together with their associated attributes and states, may be condensed down to a single symbol and expanded back again at any time. Besides ensuring uncluttered screens, this provides an effective help and reference system. Forgotten information can be accessed at any time, and additional information retrieved as and when it is required.

The idea behind condensed and expanded symbols can be extended considerably further. It is not applicable only to MUSLI movies. We should be able to incorporate symbols, whether graphical, mathematical, biological or chemical, directly in our wordprocessing documents and have them expandable in place.

A system such as this will greatly aid the writing of scientific papers and perhaps revolutionise the writing of law documents, where whole phrases are extremely repetitive. Jargon will become much more bearable when we have explanations at the touch of a button. Once dictionaries of symbol abbreviations are widely available, documents written in abbreviated form will be open to expansion by other readers. Perhaps authors can be given the option of saving additional special symbols along with their documents. And it could be handy to have our own shorthand hieroglyphics and be able to expand them if necessary for printing or emailing.

The beauty of it all is that if we can't remember what a symbol means -- whether someone else's or our own -- we can get the full description back with just the click of a mouse.

Interactive and Annotated Movies

All these techniques can be incorporated into an exciting new form of computer-controlled movie. If we divide the screen up into four quadrants, we can show any sort of movie in one quadrant and use the other areas to give additional information. Let us suppose that a movie of the 1994 Commonwealth Games is shown in the first quadrant, and that the user can pause it at any stage.

The user can then:

  • look at close-ups of performers, equipment and other items
  • look up key words used by the commentator to get more detailed information
  • get symbolic representations of common routines, such as ice-skating moves
  • watch multimedia presentations about training programmes
  • receive tourist information about the city where the Games were being held

The educational and entertainment possibilities for interactive and annotated movies are endless. Coupled with the many new ways of using old media that are emerging, we can be sure that before too long we shall experience a multimedia world that will make television seem very limited.

Jennifer Lennon works in the Hypermedia Unit at Auckland University's Computer Science Department.
Hermann Maurer works in the Hypermedia Unit at Auckland University's Computer Science Department. Professor Maurer is also with the Institute for Information Processing and Computer Supported New Media at Graz University of Technology in Austria.