Transcript intro - Duke Database Devils

Duke Systems
CPS 110 210 310
Introduction to Operating Systems
Fall 2013
Jeff Chase
Duke University
Resources to know about
• Course Web
– http://www.cs.duke.edu/~chase/cps310
– Or other links through CS department
– All powerpoints, policies, reading, schedule, lab
(project) instructions are posted there.
• Piazza
– Announcements
– Chatter. “Maybe even anonymous posting.”
• Sakai
– Minimal use, e.g., mainly for disseminating grades
Meetings
• “Lectures”:
– MW 3:05 – 4:20
– ~25 lectures total
• Recitations
– Fri 3:05, Bio Sci 111
– TA: Balakrishnan Chandrasekaran
• Two midterms
– 10/4 (recit) and 11/6 (in class)
• “Light” final exam (1.5x)
– 12/15 from 2-5 PM
“Bio Sci”
What is this course about?
• Programs
• Platforms
• Performance
• …
User Applications
Operating System
Substrate / Architecture
“The system is all the code your program uses that you didn’t have to write.”
http://steve-yegge.blogspot.com/2008/03/get-that-job-at-google.html
Steve Yegge
Course goals
• Learn to think about systems holistically.
• Sound mastery of structures and principles.
• Reinforce with practical, concrete examples.
• Minimize “unknown unknowns”.
2003
OS Platform: A Model
Applications/services.
May interact and serve
one another.
Libraries/frameworks:
packaged code used by
multiple applications
OS platform: same for all
applications on a system
E,g,, classical OS kernel
OS platform mediates access to shared resources.
[RAD Lab]
“Software Architecture”
User Applications
Software
architecture
Physics stops here.
Operating System(s)
Computer
architecture
Substrate / Architecture
Comparative architecture: what works
Reusable / recurring design patterns
• Used in OS
• Supported by OS
Platform abstractions
• Platforms provide “building blocks”…
• …and APIs to use them to construct software.
– Instantiate/create/allocate
– Manipulate/configure
– Attach/detach
– Combine in uniform ways
– Release/destroy
• Abstractions are layered.
– What to expose? What to hide?
The choice of abstractions reflects a philosophy
of how to build and organize software systems.
Managing Complexity
System
Component
Component
System
Environment
Component
Component
Systems are built from
components.
Operating systems define
styles of software
components and how they
interact.
OS maps components onto
the underlying machine.
…and makes it all work
together.
Comparative software architecture
Large, long-lived software systems are like buildings.
They are built by workers using standard design patterns.
They depend on some underlying infrastructure.
But they can evolve and are not limited by the laws of physics.
Watch it!
Computer systems
is a liberal arts
discipline.
Just because
someone will pay
you to do it doesn’t
mean it’s not liberal
arts.
Course prerequisites
• Basic data structures and programming
– Lists, stacks, queues, graphs, DAGs, trees
– Abstract data types (ADTs), classes, objects
– Dynamic data structures
• Basic architecture
– CPU context: registers
– Execution: runtime stack and frames
– Memory and L1/L2/L3 caches, DMA I/O
– Virtual addressing and memory layout
• Basic discrete math and probability
Dynamic data structures
A simple module
• A set of procedures/functions/methods.
• An interface (API) that defines a template
for how to call/invoke the procedures.
• State (data) maintained and accessed by
the procedures.
• A module may be a class that defines a
template (type) for a data structure, which
may have multiple instances (objects).
state
P1()
P2()
P3()
P4()
Abstract Data Type (ADT): the module’s state is manipulated only
through its API (Application Programming Interface).
Code: instructions in memory
_p1:
pushq
movq
movl
movq
popq
ret
%rbp
%rsp, %rbp
$1, %eax
%rdi, -8(%rbp)
%rbp
load
add
store
_x, R2
R2, 1, R2
R2, _x
; load global variable x
; increment: x = x + 1
; store global variable x
A Peek Inside a Running Program
CPU core
0
common runtime
x
your program
code library
your data
R0
heap
Rn
PC
SP
x
y
registers
y
stack
high
“memory”
address space
(virtual or physical)
e.g., a virtual memory
for a running program
(process)
Data in memory
64 bytes: 3 ways
Memory is “fungible”.
p + 0x0
0x0
int p[]
int* p
char p[]
char *p
0x1f
p
0x0
char* p[]
char** p
0x1f
Pointers (addresses) are 8
bytes on a 64-bit machine.
0x1f
Heap: dynamic memory
The “heap” is an ADT in a
runtime library: the code to
maintain the heap is a
heap manager.
It allocates a contiguous
slab of virtual memory from
the OS kernel, then “carves
it up” as needed.
It enables the programming
language environment, to
store dynamic objects.
E.g., with Unix malloc and
free library calls.
Allocated heap blocks
for structs or objects.
Align!
Free block
Read it on the course web
But Java programs are interpreted
They run on an “abstract machine” (e.g., JVM)
implemented in software.
”bytecode”
http://www.media-art-online.org/java/help/how-it-works.html
Platforms are layered/nested
Grades: CPS 210 Fall 2012
4 A+
8A
11 A-
13 B+
8B
7 B9 C* or lower
Cumulative Distribution Function
CPS 2/310 Spring 2013
Probability
that a random
student’s score
is X or below
A*
B*
C*
Total score X
Cumulative Distribution Function (CDF)
80% of the requests have
response time R with x1 < R < x2.
“Tail” of 10% of requests with
response time R > x2.
90%
quantile
What’s the
mean R?
50%
(median)
A few requests
have very long
response times.
median
value
10%
quantile
x1
x2
Understand how the mean (average) can be misleading, e.g. if tail is heavy.
What is this course about?
• Programs
• Platforms
• Sharing
• Concurrency
• Storage
• Protection and trust
• Resource management
• Virtualization
• Scale and performance
• Abstractions
User Applications
Operating System
Substrate / Architecture
Reading
• Course notes and slides
• External sources on every topic
– OS in Three Easy Pieces
– A few academic papers and web readings
– Yes, even a “comic book”
We’ll look at
these with
varying levels
of scrutiny.
New!
$10!
Web/SaaS/cloud
http://saasbook.info
New! $75!
http://csapp.cs.cmu.edu
a classic
No text, but these
may be useful.
Saltzer/Kaashoek
Very MIT
Do not buy kindle edition.
Workload: projects
1. Dynamic heap memory (malloc/free)
2. Unix shell (“Devil Shell”)
3. Java concurrency: multi-threaded
programming (“Elevator”)
4. Key/value store (“Devil Filer”)
5. Performance evaluation of storage server
Collaboration
• OK among groups:
– General discussion of course concepts and
programming environment.
– “What does this part of the handout mean?”
• Not OK among groups
– Design/writing of another’s program
– “How do I do this part of the handout?”
• Definitely not OK:
– Using code from a previous semester.
• If in doubt, ask me.
Thoughts on cheating
Cheating is a form of laziness.
Cheating happens at those other schools.
Duke students work hard and don’t cut corners.
Your work is your own: if in doubt, ask.
Listen to what shame tells you.
Extra slides
The point of the remaining slides is:
• We take a broad view of “operating systems”
encompassing a variety of application platforms.
• We start with Unix, a canonical/classical OS.
• Unix has continuing relevance: it continues to thrive
deep inside the rich platforms we use today: knowing
about the Unix kernel helps to understand how they
work.
• Hardware and application demands change rapidly.
Operating system kernels evolve slowly, but we often
add more code around them to adapt to change.
• You’ll see these slides again.
What is this course about?
• “Greater Unix”
– Classical OS abstractions and structure
– Systems programming with C and Unix
• Networked systems
– Sockets and servers, smartphones to clouds
– Elementary cryptosystems
– Distributed systems topics
• Managing concurrency
– Threads, multi-threaded Java
• Managing storage and data
– Files, caches, disks, recovery, content delivery
Some lessons of history
• At the time it was created, Unix was the “simplest
multi-user OS people could imagine.”
– It’s in the name: Unix vs. Multics
• Simple abstractions can deliver a lot of power.
– Many people have been inspired by the power of Unix.
• The community spent four decades making Unix
complex again....but the essence is unchanged.
• Unix is a simple context to study core issues for
classical OS design. “It’s in there.”
• Unix variants continue to be in wide use.
• They serve as a foundation for advances.
Virtual Machine
(JVM)
“Classical OS”
Reloaded.
[http://www.android.com]
End-to-end application delivery
Where is your application?
Where is your data?
Where is your OS?
Cloud and Software-as-a-Service (SaaS)
Rapid evolution, no user upgrade, no user data management.
Agile/elastic deployment on virtual infrastructure.
SaaS platform elements
browser
[wiki.eeng.dcu.ie]
container
“Classical OS”
OpenStack, the Cloud Operating System
Management Layer That Adds Automation & Control
[Anthony Young @ Rackspace]
EC2
The canonical public cloud
Virtual
Appliance
Image
Canonical OS Example: “Classical OS”
• Unix/Linux, Windows, OS-X
• Research systems
– Multics
– Mach
– Minix
– …
Drivers of Change
Exponential growth
Increasing diversity
User Applications
Operating System
Substrate / Architecture
Aggregation
Composition
Orchestration
Backward compatibility
Broad view: smartphones to servers, web, and cloud.
Moore’s Law and CPU Performance
Performance (vs. VAX-11/780)
10000
1000
3X
From Hennessy and Patterson,
Computer Architecture: A
Quantitative Approach, 4th edition,
2006
??%/year
52%/year
100
 Sea change in chip design: multiple
“cores” or processors per chip
10
25%/year
1
1978 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006
Uniprocessor Performance (SPECint)