Lecture
Let’s start with system calls, which can be described as follows:
Your program (in what is called user space) has to ask the operating system kernel to perform an input/output operation on your program’s behalf.
System calls — are the way a program asks the kernel to do something. The specifics of their implementation depend on the OS, but the basic principle is the same everywhere. There must be some specific instruction for transferring control from your program through the kernel (like a function call, only with a special “extra” for working in this situation). In general, system calls are blocking, i.e. the program waits until the kernel returns to your code.
Above it was said that system calls are blocking, and in general that’s true. However, some calls can be described as non-blocking. This means that the kernel accepts your request, puts it in a queue or some buffer, and then, without any waiting, immediately returns to the input/output it is currently performing. So “blocking” happens only for a very short period of time, just enough to place your request in the queue.
To make this clearer, here are some examples (Linux system calls):
read() — a blocking call: you pass it a handle telling it which file to take and which buffer to deliver the read data to; the call returns when the data ends up at its destination. Simple and clear.epoll_create(), epoll_ctl() and epoll_wait() — calls that, respectively, allow you to create a group of descriptors to listen on; add descriptors to the group / remove them from it; block until activity appears. This lets you efficiently manage a large number of input/output operations with a single thread of execution. It’s great that this functionality exists, but it’s rather complex to use.
It’s important to understand the differences in timing. If the processor core runs at 3 GHz, with no optimizations, it performs 3 billion cycles per second (3 cycles per nanosecond). A non-blocking system call might take a few dozen cycles, that is, a few nanoseconds. A call blocking on receiving information over the network can take much longer: for example, 200 milliseconds (1/5 of a second). That is, if a non-blocking call takes 20 nanoseconds, a blocking one takes 200 million nanoseconds. The process waits for the blocking call to complete 10 million times longer.
The kernel provides means for performing both blocking (“read data from the network connection and give it to me”) and non-blocking (“tell me when new data appears on these network connections”) input/output. And depending on the mechanism chosen, the duration of the calling process’s blocking will differ dramatically.
Scheduling is also extremely important if you have many threads of execution or processes that start blocking.
For our purposes the difference between a process and a thread of execution is small. In real life the most important difference between them in terms of performance is that a thread of execution uses the same memory area, while processes get their own areas. That’s why separate processes require much more memory. But when we talk about scheduling, it all comes down to how much execution time threads and processes need on the available processor cores. If you have 300 threads and eight cores, you’ll have to divide up the time so that each thread gets its share: each core briefly runs one thread, then moves on to the next. This is done via context switching, when the processor switches from one running thread/process to another.
But there are certain costs associated with these context switches — they take some time. Sometimes this can take less than 100 nanoseconds, but the switch often takes 1000 nanoseconds or more, depending on the specifics of the implementation, the speed/architecture of the processor, its cache, etc.
And the more threads of execution (or processes), the more context switches. If we’re talking about thousands of threads, when switching from each of them takes hundreds of nanoseconds, everything runs very sluggishly.
However, non-blocking calls essentially tell the kernel: “Call me only when new data or an event appears on one of these connections.” These calls are designed for efficiently handling a heavy input/output load and reducing the number of context switches.
Comments