Transcript Comp311/Comp511 - School of Science and Technology

Comp311/Comp511
Lecture01 - Introduction
Unit Plan
• Text Book: Fundamentals of
Multimedia by Ze-Nian Li and Mark
S. Drew
• Theme - ”Platform independent
multimedia”
• What we will not do:
– How to become computer graphic
artists!
– How to drive commercial
multimedia suits!
Contents

Foundational aspects: Our study
about this is minimal but there is five
lectures covering
• Human Vision and Colour models.
• Image data representations.
• Data compression.
• Transform coding.
• Video compression.
Contents

Rest of the lectures are on programming
aspects including
• Java Media APIs (Java2D, JAI, JMF, little bit of
Java3D)
• SMIL – Synchronized Multimedia Integration
language

A XML based language.
• VRML/X3D – Virtual Reality Markup Language

X3D is the current XML based VRML specification
• Basic Java Game programming.
Assessment

Assessment: 60% Assignments and 40%
Exam
o Must obtain at least 50% from each
component to pass the unit.
o There are 3 assignments covering
programming and multimedia development.
o

The first assignment will be released today.
Lecturer: Ashoka Jayawardena
o Room 171
o Extension: 2267

Tutor: TBA
Multimedia

Multimedia involves multiple
modalities of text, audio, images,
drawings, animation, and video.
Applications






Video teleconferencing.
Distributed lectures for higher
education.
Tele-medicine.
Co-operative work environments.
Searching in (very) large video and
image databases for target visual
objects.
etc.
Hypermedia and Multimedia



A hypertext system: meant to be read
nonlinearly, by following links that point to other
parts of the document, or to other documents.
Hypermedia: not constrained to be text-based,
can include other media, e.g., graphics, images,
and especially the continuous media – sound and
video: The World Wide Web (WWW) – the best
example of a hypermedia application.
Multimedia means that computer information can
be represented through audio, graphics, images,
video, and animation in addition to traditional
media.
Multimedia Software Tools






Music Sequencing and Notation
Digital Audio
Graphics and Image Editing
Video Editing
Animation
Multimedia Authoring
Music Sequencing and Notation

Cakewalk: now called Pro Audio.
o The term sequencer comes from older devices that stored
sequences of notes (events, in MIDI).
o
o
o
MIDI – Musical Instrument Digital Interface.
MIDI is a protocol adopted by electronic music industry that
enables computers, synthesizers, keyboards, and other musical
devices to communicate with each other.
It is a scripting language that codes events that stand for the
production of certain sounds.
o It is also possible to insert WAV files and Windows MCI
commands (for animation and video) into music tracks (MCI is
a ubiquitous component of the Windows API.)


Cubase: another sequencing/editing program, with
capabilities similar to those of Cakewalk. It includes some
digital audio editing tools.
Macromedia Soundedit: mature program for creating audio
for multimedia projects and the web that integrates well
with other Macromedia products such as Flash and Director.
Digital Audio

Digital Audio tools deal with accessing and
editing the actual sampled sounds that
make up audio:
o Cool Edit: a very powerful and popular digital
audio toolkit; emulates a professional audio
studio —multi-track productions and sound file
editing including digital signal processing
effects.
o Sound Forge: a sophisticated PC-based
program for editing audio WAV files.
o Pro Tools: a high-end integrated audio
production and editing environment — MIDI
creation and manipulation; powerful audio
mixing, recording, and editing software.
Graphics and Image Editing


Adobe Illustrator: a powerful publishing tool from
Adobe. Uses vector graphics; graphics can be
exported to Web.
Adobe Photoshop: the standard in a graphics,
image processing and manipulation tool.
o Allows layers of images, graphics, and text that can be
separately manipulated for maximum flexibility.
o Filter factory permits creation of sophisticated lightingeffects filters.


Macromedia Fireworks: software for making
graphics specifically for the web.
Macromedia Freehand: a text and web graphics
editing tool that supports many bitmap formats
such as GIF, PNG, and JPEG.
Video Editing

Adobe Premiere: an intuitive, simple video editing
tool for nonlinear editing, i.e., putting video clips
into any order:
o Video and audio are arranged in ”tracks”.
o Provides a large number of video and audio tracks,
super-impositions and virtual clips.
o A large library of built-in transitions, filters and motions
for clips ) effective multimedia productions with little
effort.


Adobe After Effects: a powerful video editing tool
that enables users to add and change existing
movies. Can add many effects: lighting, shadows,
motion blurring; layers.
Final Cut Pro: a video editing tool by Apple;
Macintosh only.
Multimedia Authoring

Multimedia authoring is the creation
of multimedia productions,
sometimes called ”movies” or
”presentations”. We are mostly
interested in interactive applications.
Multimedia Authoring Metaphors






Scripting Language Metaphor: use a special language to
enable interactivity (buttons, mouse, etc.), and to allow
conditionals, jumps, loops, functions/macros etc.
Slide Show Metaphor: A linear presentation by default,
although tools exist to perform jumps in slide shows.
Hierarchical Metaphor: User-controllable elements are
organised into a tree structure – often used in menu-driven
applications.
Iconic/Flow-control Metaphor: Graphical icons are available
in a toolbox, and authoring proceeds by creating a flow
chart with icons attached.
Frames Metaphor: Like Iconic/Flow-control Metaphor;
however links between icons are more conceptual, rather
than representing the actual flow of the program.
Card/Scripting Metaphor: Uses a simple index-card
structure – easy route to producing applications that use
hypertext or hypermedia; used in schools.
Multimedia Authoring Metaphors

Cast/Score/Scripting Metaphor:
o Time is shown horizontally; like a spreadsheet: rows, or
tracks, represent instantiations of characters in a multi- media
production.
o Multimedia elements are drawn from a cast of characters, and
scripts are basically event procedures or procedures that are
triggered by timer events.
o Director, by Macromedia, is the chief example of this
metaphor. Director uses the Lingo scripting language, an
object-oriented event-driven language.


Hypermedia Linkage – The Hypermedia Linkage paradigm is
similar to the Frame paradigm in that it shows conceptual
links between elements; however, it lacks the Frame
paradigm’s visual linkage metaphor.
Tagging – The Tagging paradigm uses tags in text files (for
instance, SGML/HTML, SMIL (Synchronised Media
Integration Language), VRML) to link pages, provide
interactivity and integrate multimedia elements.
Multimedia Authoring Tools




Macromedia Flash: allows users to create
interactive movies by using the score metaphor,
i.e., a timeline arranged in parallel event
sequences.
Macromedia Director: uses a movie metaphor to
create interactive presentations—very powerful
and includes a built-in scripting language, Lingo,
that allows creation of complex interactive
movies.
Authorware: a mature, well-supported authoring
product based on the Iconic/Flow-control
metaphor.
Quest: similar to Authorware in many ways, uses
a type of flowcharting metaphor. However, the
flowchart nodes can encapsulate information in a
more abstract way (called frames) than simply
subroutine levels.
Programming vs. Authoring



Authoring involves the assembly and bringing
together of Multimedia with possibly high level
graphical interface design and some high level
scripting.
Programming involves low level assembly and
construction and control of Multimedia and
involves real languages like C++ and Java.
Later in this course we will study:
o Java media programming (Java2D, JAI, JMF (Java Media
Framework), Basic Java game programming, Java3D.
o SMIL - Synchronized Multimedia Integration Language.
o VRML/X3D - Virtual Reality Modelling Language.