Introduction

Modern operating systems are multi-user and multi-process environments where multiple programs execute simultaneously while sharing hardware resources such as memory, CPU time, files, devices, and network connections. Without proper control mechanisms, one process could interfere with another process, access unauthorized data, corrupt memory, or even crash the entire system.

The operating system therefore requires a mechanism that controls how resources are accessed and who is allowed to use them. This mechanism is called protection.

Protection is one of the most fundamental responsibilities of an operating system because it ensures:

  • Controlled resource sharing

  • Process isolation

  • System stability

  • Secure execution

  • Reliability in multi-user systems

Protection is different from security, although the two are closely related. Protection focuses on controlling internal access to resources, while security focuses on defending the system against threats and attacks.

What is Protection?

Protection refers to the set of mechanisms that control the access of processes, users, and programs to system resources.

The operating system ensures that:

  • Authorized entities receive access

  • Unauthorized access is prevented

  • Processes cannot interfere with each other improperly

Core Idea

Who can access what resource and in what way?

Protection determines:

  • Which resources can be accessed

  • Which operations are allowed

  • Under what conditions access is permitted

Why Protection is Necessary

Consider a system without protection.

Possible problems:

  • One process overwrites another process’s memory

  • Any user can delete system files

  • Malware accesses confidential data

  • Programs monopolize hardware resources

Without protection:

  • Multi-user systems become impossible

  • System crashes become common

  • Data integrity collapses

Important Insight

Protection enables safe resource sharing in multi-user systems

Goals of Protection

Protection mechanisms aim to achieve several important objectives.

1. Prevent Unauthorized Access

Users and processes should access only permitted resources.

2. Ensure Controlled Sharing

Resources should be shared safely.

3. Improve System Reliability

Faulty programs should not damage the entire system.

4. Isolate Processes

Processes should operate independently unless explicitly allowed to communicate.

5. Enforce Privilege Levels

Critical system operations should require higher privileges.

Protection Domains

A protection domain defines the set of resources a process can access and the operations it can perform.

A domain contains:

  • Objects

  • Access rights

Objects

Objects are system resources such as:

  • Files

  • Memory

  • Devices

  • CPU

  • Processes

Rights

Rights define permitted operations:

  • Read

  • Write

  • Execute

  • Delete

  • Modify

Example

Process A:
Read → File1
Write → File2
Execute → ProgramX

Important Insight

A protection domain defines the capabilities of a process

Domain Switching

Processes may change domains temporarily.

Example:

  • User program invokes system call

  • CPU enters kernel mode

  • Privilege level increases temporarily

After completion:

  • System returns to user mode

This controlled privilege switching is critical for operating system security.

User Mode and Kernel Mode

Modern systems use dual-mode operation.

User Mode

Restricted execution mode.

Applications:

  • Cannot directly access hardware

  • Cannot execute privileged instructions

Kernel Mode

Privileged execution mode.

Operating system can:

  • Access hardware

  • Modify memory mappings

  • Execute critical operations

Protection Mechanisms

Operating systems use several mechanisms to implement protection.

1. Memory Protection

Processes must not access memory belonging to other processes.

Implemented using:

  • Base and limit registers

  • Paging

  • Segmentation

  • Virtual memory

Example

A process attempting to access invalid memory:

  • Generates exception

  • OS terminates process

Important Insight

Memory protection prevents process interference

2. CPU Protection

Processes should not monopolize CPU.

Implemented using:

  • Timer interrupts

  • Scheduling

Timer ensures:

  • CPU eventually returns control to OS

3. I/O Protection

User programs should not directly access devices.

Reason:

  • Could corrupt hardware state

  • Could bypass security

Only kernel mode executes I/O instructions.

4. File Protection

Files require controlled permissions.

Permissions may include:

  • Read

  • Write

  • Execute

Example:

  • Linux rwx permissions

Principle of Least Privilege

One of the most important security principles.

It states:

A process should receive only the minimum privileges necessary to perform its task.

Advantages:

  • Reduces damage from bugs

  • Limits malware impact

  • Improves security

Example

A text editor:

  • Needs file access

  • Does not need direct hardware control

Protection Rings

Some architectures implement hierarchical privilege levels called protection rings.

Ring 0

Highest privilege:

  • Kernel

Ring 3

Lowest privilege:

  • User applications

Intermediate rings may exist but are less commonly used.

Important Insight

Protection rings enforce privilege separation at hardware level

Hardware Support for Protection

Operating systems rely heavily on hardware mechanisms.

Examples:

  • Privileged instructions

  • Mode bits

  • Memory Management Unit (MMU)

  • Interrupt protection

Without hardware support:

  • Protection enforcement becomes weak

Access Validation

Whenever a process requests resource access:

OS checks:

  • Identity

  • Permissions

  • Requested operation

Only then:

  • Access granted

Example

User requests file write →
OS checks permission →
Allow or deny

Process Isolation

Protection ensures one process cannot:

  • Modify another process’s memory

  • Directly manipulate CPU state

  • Access private data

Isolation is critical for:

  • Stability

  • Reliability

  • Security

Protection vs Security

Students commonly confuse these concepts.

Protection

Internal control mechanism.

Focus:

  • Resource access control

Security

Defense against threats and attacks.

Focus:

  • Authentication

  • Encryption

  • Threat prevention

Important Insight

Protection controls access; security defends the system

Real-World Example

Suppose a web browser crashes.

Without protection:

  • Entire OS could crash

With protection:

  • Browser process isolated

  • Other processes remain safe

This isolation is one of the biggest reasons modern operating systems are stable.

Modern Protection Techniques

Modern systems implement advanced protection features such as:

  • Address Space Layout Randomization (ASLR)

  • Sandboxing

  • Secure boot

  • Capability systems

  • SELinux

These extend basic protection into sophisticated security architectures.