1. Why Security Matters in Operating Systems
An operating system sits between users, applications, and hardware. Every file access, memory allocation, network connection, and process execution passes through the operating system.
Because of this central role, the OS becomes a primary target for attackers.
If the operating system is compromised:
User data can be stolen
Applications can be manipulated
System resources can be abused
Entire networks can be affected
Example
Attacker gains OS access
↓
Reads sensitive files
↓
Installs malware
↓
Controls system remotely
Key Insight
The operating system forms the foundation of system security. If the OS is compromised, all applications running on it become vulnerable.
2. What is a Security Threat?
A security threat is any potential action, event, or entity capable of violating the confidentiality, integrity, or availability of a computer system.
Core Security Goals
Security threats are evaluated against three fundamental principles:
Confidentiality
Protecting information from unauthorized access.
Integrity
Preventing unauthorized modification of data.
Availability
Ensuring resources remain accessible when needed.
Example
Data Theft → Confidentiality Violation
Data Modification → Integrity Violation
System Shutdown → Availability Violation
Key Insight
Most security threats target one or more aspects of the CIA Triad (Confidentiality, Integrity, Availability).
3. Types of Security Threats
Security threats can be broadly classified into:
Malware-based threats
Network-based threats
Authentication threats
System-level threats
Human-based threats
4. Malware Threats
Malware stands for "Malicious Software."
It refers to software designed to damage systems, steal data, or gain unauthorized access.
4.1 Virus
What is a Virus?
A virus is a malicious program that attaches itself to legitimate files and spreads when those files are executed.
Working
Infected File
↓
User Executes File
↓
Virus Activates
↓
Infects Other Files
Characteristics
Requires host program
Spreads through file sharing
Can corrupt data
Example
Executable File
↓
Virus Embedded
↓
Runs When File Opens
Key Insight
Viruses require user action or program execution to spread.
4.2 Worm
What is a Worm?
A worm is a self-replicating malware that spreads automatically across networks.
Working
System A
↓
Network Spread
↓
System B
↓
System C
Characteristics
No host file required
Self-replicating
Rapid network spread
Impact
Consumes bandwidth
Slows networks
Causes large-scale infections
Key Insight
Unlike viruses, worms do not require user interaction to spread.
4.3 Trojan Horse
What is a Trojan?
A Trojan is malicious software disguised as legitimate software.
Example
Free Game Download
↓
User Installs
↓
Hidden Malware Executes
Characteristics
Appears useful
Contains hidden malicious code
Often creates backdoors
Key Insight
Trojans rely on deception rather than self-replication.
4.4 Ransomware
What is Ransomware?
Ransomware encrypts user data and demands payment for recovery.
Working
System Infection
↓
Files Encrypted
↓
Ransom Demand
Example
Documents
Photos
Databases
become inaccessible until decrypted.
Impact
Data loss
Financial damage
Business disruption
Key Insight
Ransomware attacks availability by denying access to data.
4.5 Spyware
What is Spyware?
Spyware secretly collects information about users and sends it to attackers.
Targets
Passwords
Browsing history
Personal information
Banking details
Working
User Activity
↓
Spyware Records
↓
Attacker Receives Data
Key Insight
Spyware primarily targets confidentiality.
5. Authentication Threats
Authentication threats focus on stealing or bypassing user credentials.
5.1 Password Attacks
Attackers attempt to discover passwords.
Methods
Brute Force Attack
Try every possible password combination.
Dictionary Attack
Use common password lists.
Credential Stuffing
Reuse leaked passwords from previous breaches.
Example
Username
Password Guessing
↓
Unauthorized Access
Key Insight
Weak passwords significantly increase security risks.
5.2 Phishing
What is Phishing?
Phishing tricks users into revealing sensitive information.
Example
Fake Banking Email
↓
User Clicks Link
↓
Enters Credentials
↓
Attacker Obtains Password
Characteristics
Social engineering attack
Exploits human trust
Often delivered via email
Key Insight
Many successful attacks target people rather than technology.
6. Network-Based Threats
Network threats exploit communication channels between systems.
6.1 Denial of Service (DoS)
What is DoS?
An attack that overwhelms a system with requests.
Working
Thousands of Requests
↓
Server Overloaded
↓
Legitimate Users Blocked
Goal
Reduce system availability.
Key Insight
DoS attacks target availability rather than confidentiality.
6.2 Distributed Denial of Service (DDoS)
What is DDoS?
A DDoS attack uses many compromised machines simultaneously.
Working
Botnet
↓
Thousands of Devices
↓
Single Target
Impact
Massive traffic volume
Service disruption
Website outages
Key Insight
DDoS attacks are much harder to defend against than standard DoS attacks.
6.3 Man-in-the-Middle (MITM)
What is MITM?
An attacker secretly intercepts communication between two parties.
Working
User
↓
Attacker
↓
Server
Risks
Data theft
Session hijacking
Credential interception
Key Insight
MITM attacks compromise confidentiality and integrity simultaneously.
7. System-Level Threats
These threats directly target operating system functionality.
7.1 Privilege Escalation
What is Privilege Escalation?
A user gains permissions beyond those originally granted.
Example
Normal User
↓
Exploit Vulnerability
↓
Administrator Access
Impact
Full system control
Security bypass
Data compromise
Key Insight
Privilege escalation is one of the most dangerous OS-level attacks.
7.2 Rootkits
What is a Rootkit?
A rootkit is malware designed to hide its presence while maintaining privileged access.
Characteristics
Difficult to detect
Modifies system behavior
Hides malicious processes
Example
Malware Installed
↓
Rootkit Conceals Activity
↓
Attacker Remains Hidden
Key Insight
Rootkits target the trustworthiness of the operating system itself.
7.3 Buffer Overflow
What is Buffer Overflow?
Occurs when a program writes more data than a memory buffer can hold.
Example
Buffer Size = 20 Bytes
Input = 100 Bytes
Extra data overwrites adjacent memory.
Consequences
Program crashes
Arbitrary code execution
Privilege escalation
Key Insight
Buffer overflows exploit weaknesses in memory management.
8. Insider Threats
Not all threats come from external attackers.
Insider Threat
A legitimate user intentionally or accidentally compromises security.
Examples
Employee stealing data
Administrator misuse
Accidental deletion
Working
Authorized User
↓
Abuses Privileges
↓
Security Breach
Key Insight
Insiders often possess greater access than external attackers.
9. Security Mechanisms Used by Operating Systems
Operating systems implement multiple defenses.
9.1 Authentication
Verifies user identity.
Examples:
Passwords
Biometrics
Multi-factor authentication
9.2 Access Control
Determines who can access resources.
Examples:
Read
Write
Execute
permissions.
9.3 Encryption
Protects data from unauthorized access.
Example
Plain Data
↓
Encryption
↓
Ciphertext
9.4 Firewalls
Monitor and filter network traffic.
Functions
Block malicious connections
Enforce network policies
9.5 Antivirus Software
Detects and removes malware.
Functions
Signature-based detection
Behavioral analysis
Key Insight
Security relies on multiple layers rather than a single mechanism.
10. Security Threats vs Vulnerabilities
Vulnerability
A weakness in the system.
Threat
Something capable of exploiting the weakness.
Example
Weak Password
↓
Vulnerability
Brute Force Attack
↓
Threat
Key Insight
Threats exploit vulnerabilities.
11. Real-World Example
Consider a corporate workstation.
Scenario
Employee Receives Phishing Email
↓
Downloads Trojan
↓
Trojan Installs Spyware
↓
Credentials Stolen
↓
Attacker Gains Network Access
This single incident involves:
Phishing
Trojan malware
Spyware
Credential theft
Key Insight
Real attacks often combine multiple security threats.
12. Summary of Major Security Threats
| Threat | Primary Target |
|---|---|
| Virus | Integrity |
| Worm | Availability |
| Trojan | Confidentiality |
| Ransomware | Availability |
| Spyware | Confidentiality |
| Phishing | Credentials |
| DoS/DDoS | Availability |
| MITM | Confidentiality & Integrity |
| Rootkit | System Control |
| Buffer Overflow | System Security |
| Privilege Escalation | Access Control |
Final Insight
Security threats are any actions or entities capable of compromising a system's confidentiality, integrity, or availability. Modern operating systems defend against threats such as malware, phishing, denial-of-service attacks, privilege escalation, rootkits, and insider misuse through mechanisms like authentication, access control, encryption, firewalls, and antivirus protection. Because attackers continuously evolve their techniques, operating system security must be viewed as an ongoing process rather than a one-time solution.