Transcript Protection
Protection and OS Structure
Andrew Whitaker
CSE451
Protection
Challenge: OS must safely support multiple
protection domains
OS as “law enforcement”
Goals
Buggy application can’t crash the system
Malicious application can’t take control
User data is protected from untrusted users and
programs
The User/Kernel boundary
Implemented in
hardware
Allows the OS to
execute privileged
instructions
Applications enter
kernel mode by
executing a system
call
user
kernel
App
App
OS
App
Examples of Privileged Instructions
Manipulating I/O devices
Why?
Interrupt enable/disable flag
Why?
Halt instruction
Why?
System Call Overview
User program invokes helper procedure
e.g., read, write, gettimeofday
Helper passes control to the OS
Indicates the system call number
Packages user arguments into registers
Issues a software interrupt (or trap)
OS saves user state (registers)
OS invokes appropriate system call handler
OS returns control to the user application
A kernel crossing illustrated
Firefox: read(int fileDescriptor, void *buffer,int numBytes); )
package arguments
user mode
trap to kernel mode
kernel mode
trap handler
save registers
find sys_read( )
handler in
vector table
sys_read( ) kernel routine
restore app
state, return to
user mode,
resume
Kernel Entry Points
Interrupts
Disk, network, timer, etc.
Software interrupts (traps, exceptions)
System calls
Protection violations
e.g,. User executes a privileged instructions
Page faults
Error conditions
e.g., divide by zero, illegal opcode
Memory Protection
Problem #1: OS must protect applications
from each other
Solution: virtual memory -- each application has
its own address space, which maps to private
physical pages
Problem #2: Kernel must protect its own
code and data
Solution: Split address space in half
Kernel half requires privileged mode access
Simplified Linux Address Space
Layout
user-accessible
kernel space
0xc0000000
Other Forms of OS Protection
Disk protection:
Expressed in terms of file system access control
permissions (UNIX: read, write, execute)
users
files
drwxr-xr-x
drwxrwx--x
drwxrwxr-x
-rw------drwxrwxr-x
4
4
9
1
3
gaetano
zahorjan
levy
lazowska
beame
www
www
www
www
ctheory
4096
4096
4096
2006
4096
Mar 15 2005 sewpc
Mar 15 2005 software
Mar 16 2005 sosp16
Oct 9 1998 staff
Jun 1 2002 stoc96
CPU protection:
Must guarantee each process a fraction of the CPU
Sample Test Question: Insecure
System Call
Consider a hypothetical system call, zeroFill, which fills a
user buffer with zeroes:
zeroFill(char* buffer, int bufferSize);
The following kernel implementation of zeroFill contains a
security vulnerability. What is the vulnerability, and how
would you fix it?
void sys_zeroFill(char* buffer, int bufferSize) {
for (int i=0; i < bufferSize; i++) {
buffer[i] = 0;
}}
Solution
The user buffer pointer is untrusted, and
could point anywhere. In particular, it
could point inside the kernel address
space. This could lead to a system crash
or security breakdown.
Fix: verify the pointer is a valid user
address
Follow-up Question
Is it a security risk to execute the zeroFill
function in user-mode?
void zeroFill(char* buffer, int bufferSize) {
for (int i=0; i < bufferSize; i++) {
buffer[i] = 0;
}}
Solution
No. User-mode code does not have
permission to access the kernel’s address
space. If it tries, the hardware raises an
exception, which is safely handled by the
OS
More generally, no user mode code should
ever be a security vulnerability.
Unless the OS has a bug…
Sample Test Question
What bad thing could happen if a user
application could overwrite the interrupt
dispatch vector?
How does the OS prevent this?
Solution
An application could: 1) Prevent I/O operations
from ever completing; 2) Prevent time from
advancing, thus dominating the processor
Applications cannot modify the interrupt vector
because it lives in the kernel address space.
Any attempt to modify the interrupt vector raises
a kernel exception, which is safely handled.
Sample Test Question
What prevents an application from directly
reading from the disk, instead of passing
through file system access control
checks?
Solution
Instructions for manipulating I/O devices
are privileged. Any attempt to use them in
user mode raises a protection exception,
which the operating system gracefully
handles.