12.1. Logging and buffering

Change logging is closely related not only to transaction management, but also to the buffering of in-memory database pages. For reasons of objectively existing difference in the speed of the processors and RAM and external memory devices (this difference in speed existed, exists and will always exist), buffering database pages in RAM is the only real way to achieve satisfactory performance of the DBMS.

If a record of a database change, which should be logged during any modification operation of a database, would actually be immediately recorded in external memory, this would lead to a significant slowdown of the system. Therefore, journal entries are also buffered: during normal operation, the next page is pushed into the external journal memory only when it is completely filled with entries.

But the real situation is more complicated. There are two kinds of buffers - the log buffer and the main memory page buffer, which contain related information. Both those buffers can be pushed into external memory. The problem is to develop some general pushing policy that would provide the possibility of restoring the state of the database after a crash.

The problem does not occur with individual rollbacks of transactions, since in these cases the contents of the RAM are not lost and you can use the contents of both the log buffer and the database page buffers. But if a mild failure has occurred, and the contents of the buffers are lost, to restore the database, you need to have some consistent state of the log and database in external memory.

The basic principle of a consistent policy of pushing out a log buffer and database page buffers is that the record about changing a database object must go into the external log memory before the changed object is in the external memory of the database. The corresponding logging protocol (and buffering management) is called Write Ahead Log (WAL) - “write to the log first,” and if you need to push a modified database object to external memory, you must ensure that the external log is pushed to external memory. records of its change.

In other words, if there is a database object in the external memory of the database with respect to which the modification operation was performed, then an entry corresponding to this operation is necessarily found in the external memory of the log. The converse is not true, i.e. If an external memory log contains a record of some operation to modify a database object, then the modified object itself may not exist in the external memory of the database.

An additional condition for ejecting buffers is imposed by the requirement that every successfully completed transaction must actually be recorded in external memory. Whatever failure occurs, the system must be able to restore the state of the database containing the results of all transactions recorded at the time of the failure.

A simple solution would be to push the log buffer, followed by a massive push of the database page buffers that were modified by this transaction. Quite often they do this, but it causes significant overhead when performing a transaction to commit a transaction.

It turns out that the minimum requirement to guarantee the possibility of restoring the last consistent state of the database is to push all transactions to change the database when the transaction is committed to the external log memory. In this case, the last entry in the log, made on behalf of this transaction, is a special record of the end of the transaction.

Let us now consider how to perform database recovery operations in various situations, if the system supports a common journal for all transactions with general buffering of records, supported in accordance with the WAL protocol.