Memory Allocation Logging
This section explains the logging features of the kernel memory allocator and how
you can employ them to debug system crashes.
Buftag Data Integrity
As explained earlier, the second half of each buftag contains extra information about
the corresponding buffer. Some of this data is debugging information, and some
is data private to the allocator. While this auxiliary data can take
several different forms, it is collectively known as “Buffer Control” or bufctl data.
However, the allocator needs to know whether a buffer's bufctl pointer is valid,
since this pointer might also have been corrupted by malfunctioning code. The
allocator confirms the integrity of its auxiliary pointer by storing the pointer and
an encoded version of that pointer, and then cross-checking the two versions.
As shown in Figure 9-5, these pointers are the bcp (buffer control pointer) and
bxstat (buffer control XOR status). The allocator arranges bcp and bxstat so
that the expression bcp XOR bxstat equals a well-known value.
Figure 9-5 Extra Debugging Data in the Buftag
In the event that one or both of these pointers becomes corrupted,
the allocator can easily detect such corruption and panic the system. When a
buffer is allocated, bcp XOR bxstat = 0xa110c8ed (“allocated”). When a buffer is free, bcp XOR bxstat = 0xf4eef4ee (“freefree”).
Note - You might find it helpful to re-examine the example provided in Freed Buffer Checking: 0xdeadbeef,
in order to confirm that the buftag pointers shown there are consistent.
In the event that the allocator finds a corrupt buftag, it panics
the system and produces a message similar to the following:
kernel memory allocator: boundary tag corrupted
bcp ^ bxstat = 0xffeef4ee, should be f4eef4ee
Remember, if bcp is corrupt, it is still possible to retrieve its
value by taking the value of bxstat XOR 0xf4eef4ee or bxstat XOR 0xa110c8ed, depending on whether the
buffer is allocated or free.
The bufctl Pointer
The buffer control (bufctl) pointer contained in the buftag region can have different
meanings, depending on the cache's kmem_flags. The behavior toggled by the
KMF_AUDIT flag is of particular interest: when the KMF_AUDIT flag is not set, the
kernel memory allocator allocates a kmem_bufctl_t structure for each buffer. This structure
contains some minimal accounting information about each buffer. When the KMF_AUDIT flag
is set, the allocator instead allocates a kmem_bufctl_audit_t, an extended version of the kmem_bufctl_t.
This section presumes the KMF_AUDIT flag is set. For caches that do not
have this bit set, the amount of available debugging information is reduced.
The kmem_bufctl_audit_t (bufctl_audit for short) contains additional information about the last transaction that
occurred on this buffer. The following example shows how to apply the bufctl_audit
macro to examine an audit record. The buffer shown is the
example buffer used in Detecting Memory Corruption:
0x70a9ae00: 5 4ef83
Using the techniques presented above, it is easy to see that 0x70ae3200
points to the bufctl_audit record: it is the first pointer following the redzone.
To examine the bufctl_audit record it points to, apply the bufctl_audit
0x70ae3200: next addr slab
70378000 70a9ae00 707c86a0
0x70ae320c: cache timestamp thread
70039928 e1bd0e26afe 70aac4e0
0x70ae321c: lastlog contents stackdepth
7011c7c0 7018a0b0 4
The 'addr' field is the address of the buffer corresponding to this bufctl_audit
record. This is the original address: 0x70a9ae00. The 'cache' field points at
the kmem_cache that allocated this buffer. You can use the ::kmem_cache dcmd to
examine it as follows:
ADDR NAME FLAG CFLAG BUFSIZE BUFTOTL
70039928 kmem_alloc_24 020f 000000 24 612
The 'timestamp' field represents the time this transaction occurred. This time is expressed
in the same manner as gethrtime(3C).
'thread' is a pointer to the thread that performed the last transaction on
this buffer. The 'lastlog' and 'contents' pointers point to locations in the
allocator's transaction logs. These logs are discussed in detail in Allocator Logging Facility.
Typically, the most useful piece of information provided by bufctl_audit is the
stack trace recorded at the point at which the transaction took place.
In this case, the transaction was an allocation called as part of executing