1. If Apps Cannot Control Hardware, How Do They Open a File?
Welcome back to Gate & Kernel, where we explore how computers work, from logic gates to operating systems.
In the previous tutorial, we explored user mode and kernel mode. Ordinary applications run with restricted privileges, while the kernel—the core of the operating system—manages protected resources.
But this raises a practical question.
If an application cannot freely control a storage device, how does a text editor open a document? How does a browser receive data from the network?
The application requests a service from the operating system. A key mechanism behind these requests is the system call.
Today, we will follow a file-reading request from application code into the kernel and back again.
2. What Is a System Call?
A system call is a controlled way for a program to request a service from the kernel.
These services include reading files, communicating over networks, creating processes, and changing memory mappings. A memory mapping connects a range of virtual addresses to memory or another resource, such as a file.
The application specifies what it wants. The kernel interprets the request, checks the conditions that apply, and performs the permitted work.
A system call does not give application code unrestricted access to the machine. It transfers execution to kernel code through an entry mechanism defined by the system.
Think of it as requesting service at a counter. You describe the operation you need; you do not receive permission to walk into every restricted room.
The Linux manual describes system calls as the fundamental interface between applications and the Linux kernel. Source: Linux system calls overview
3. Why Does a Program Need the Kernel?
Many operations need no system call.
Adding two numbers, comparing values, or rearranging data already available in application memory can happen entirely in user mode.
However, opening a file involves resources managed by the operating system. The system must interpret the file’s name, locate it, and determine whether the requested access is allowed.
A process is a running instance of a program, together with resources such as its address space and open files. An address space is the set of virtual memory addresses available to that process.
If every process could freely change storage or access another process’s private memory, one faulty program could damage unrelated work.
System calls provide defined operations across this protection boundary. Applications request access, and the kernel enforces the relevant rules.
4. A Library Function Is Not Always a System Call
Programmers usually work with functions rather than writing the instructions that enter the kernel.
A function is a named piece of code that performs a task. A library is a collection of reusable code that programs can call.
Some library functions perform all their work in user mode. Others request kernel services.
A wrapper function is a function that handles the details of calling another interface. On Linux, many familiar system-call interfaces are reached through wrappers in the C library.
Calling such a wrapper initially looks like an ordinary function call. Inside it, additional instructions arrange the transition into the kernel. Source: Linux system call introduction
There is not always a one-to-one relationship between a high-level function and a system call.
For example, a library may collect output in a buffer before sending it to the operating system. A buffer is an area of memory used to hold data temporarily.
Several library calls might therefore lead to one later system call.
5. First, Open the File
Imagine that a program wants to read a text file on Linux.
Before reading, it requests that the file be opened for reading. The request supplies the file’s path—its name and location—and the desired access mode.
The kernel resolves the path and checks whether opening the file in that mode is allowed.
If the operation succeeds, the program receives a file descriptor. This is a small, nonnegative integer that identifies an open resource within the process.
The descriptor is not the file’s contents or its memory address. It is a reference the program can use in later requests.
For example, the program might receive descriptor 3. Another process could also have descriptor 3, referring to a completely different file.
Once the file is open, the program can request data using that descriptor. Source: Linux open documentation
6. What Information Does a Read Request Need?
Consider this Linux-style call: read(fd, buffer, count).
Here, fd identifies the open file. Buffer identifies where the returned data should be placed in the application’s memory. Count specifies the maximum number of bytes requested.
A byte is a unit of data containing eight bits.
These supplied values are called arguments: the information passed to a function or system call.
Suppose the program requests up to 100 bytes. It provides a destination buffer large enough to hold them.
The kernel must handle this information carefully. An application-supplied memory address cannot simply be trusted as a safe destination.
The operation also needs a valid descriptor that supports reading.
These checks do not mean that every read repeats the entire path lookup and permission checking performed when the file was opened. Different checks belong to different stages of the operation.
7. How Does Execution Enter the Kernel?
The system-call wrapper arranges the request according to the platform’s rules.
Typically, it places a system call number and arguments into designated CPU registers. Registers are small storage locations inside the processor.
The system call number identifies the requested operation. Its value depends on the operating system and processor interface.
The code then executes an instruction that invokes the kernel entry mechanism. On 64-bit x86 Linux, this is commonly the syscall instruction. On RISC-V Linux, it is ecall.
The processor redirects execution through a controlled entry point, and kernel entry code preserves the state needed to handle the request and return safely.
Hardware and kernel software cooperate in this transition; the exact division of work differs across architectures.
The critical point is that execution now follows kernel code. The application’s own instructions do not become privileged code. Source: Linux architecture calling conventions
8. What Happens Inside the Kernel?
The kernel uses the request information to select the appropriate system-call handler. A handler is code that processes a particular kind of request or event.
For our read request, the kernel identifies the open resource and follows the appropriate file-system path. A file system provides the rules and data structures used to organize and access files.
For an ordinary buffered read from a regular file, the required data may already be in the page cache. This is an area of memory where the operating system keeps file data to avoid unnecessary storage access.
If the data is available there, the kernel can satisfy the request without reading those bytes from the storage device again.
Otherwise, it may need to arrange storage input. A device driver is software that communicates with a device or a class of devices.
Once the data is available, the normal buffered-read path copies the requested data into the application’s buffer.
So a read system call does not necessarily mean a physical disk operation. The kernel may complete it using data already in memory.
9. What If the Data Is Not Ready?
Some requests can complete immediately. Others must wait.
If a read needs storage data that has not arrived, the calling thread may block. A thread is a sequence of execution within a process. Blocking means that it cannot continue until the condition it is waiting for is satisfied.
The scheduler—the part of the operating system that chooses which runnable thread receives CPU time—can select another thread.
Switching execution from one thread to another is a context switch. The system preserves the outgoing thread’s execution state and restores the state needed by the incoming thread.
This differs from a mode switch, which changes the processor’s privilege level.
A system call enters kernel mode, but it does not necessarily switch to another thread. If the request completes promptly, the same thread can return to user mode.
If it blocks, other work can run while it waits. When the data becomes available, the waiting thread becomes eligible to run again.
10. How Does the Result Return?
For a read that completes and returns normally, the kernel prepares a result and the system returns execution to user mode.
The application then continues after the call.
A successful read reports how many bytes were actually read. That number may be smaller than the requested amount, so the program must examine the result.
When reading a regular file with a positive requested count, a return value of zero indicates that the current position is at the end of the file.
Through the usual Linux C-library interface, an error is reported as minus one, with an error code made available through errno. This code helps explain what went wrong.
The returned byte count and the file data are different things: the count is the function’s result, while the data is placed in the supplied buffer. Source: Linux read documentation
The program can now process those bytes in user mode. When it finishes using the file, it closes the descriptor to release that reference.
11. System Calls Apply to Windows Too
System calls are not exclusive to Linux.
Windows also separates ordinary application execution from privileged operating-system execution. Applications request protected services through system interfaces. Source: Microsoft user mode and kernel mode
Windows programmers commonly use the Windows API. API stands for application programming interface: a defined set of functions and rules that software uses to interact with another component.
Not every API function requires a system call. Some work can stay in user mode; other operations reach kernel services through system libraries.
Windows and Linux have different interfaces and internal implementations, but both need controlled ways for applications to request protected operations.
This also explains why the name of a library can be misleading. Despite its name, Windows kernel32.dll runs in user mode; it is not the Windows kernel itself. Source: Microsoft on kernel32.dll
12. System Calls and Hardware Interrupts
System calls and hardware interrupts can both cause kernel code to execute, but they begin for different reasons.
A system call starts because the currently executing program deliberately requests a service.
A hardware interrupt is a signal that causes the processor to respond to a hardware event. For example, a storage device may report that an operation has completed.
In our file-reading example, the application starts the request through a system call. If storage access is required, a later hardware interrupt may help the operating system learn that the data transfer has finished.
The request and the completion notification are different parts of the story.
Also, making a system call does not normally mean sending work to a separate kernel process. Kernel code often runs in the context of the calling thread, although some work may be delegated elsewhere.
13. Why System Calls Have a Cost
A system call requires more work than simply calling a small function that stays in user mode.
The system must cross the privilege boundary, preserve necessary state, interpret the request, perform checks, and eventually return.
The total cost depends heavily on what the request does. A request satisfied from memory is very different from one that waits for storage.
This is one reason libraries use buffering. Instead of requesting one byte at a time, a program can request a larger block and process it locally.
Buffering reduces the number of requests, although the best approach depends on the application.
The lesson is not to avoid system calls. It is to understand when the program needs an operating-system service and to request it sensibly.
14. Conclusion
A system call gives a program a controlled way to request a service from the operating system’s kernel.
We followed a file-reading example: open the file, obtain a descriptor, prepare a read request, enter the kernel, obtain the data, and return a result to the application.
Along the way, we saw that a library function is not always a system call, a system call does not always access a physical device, and a mode switch does not necessarily involve a context switch.
Most importantly, the application requests an operation without gaining unrestricted control of the machine.
That is how the protection boundary remains useful: programs can access the services they need while the operating system manages the resources behind them.
Thanks for watching Gate & Kernel.
Explore more computer fundamentals at https://gateandkernel.com.