Custom

The function is called in privileged mode (EL1/Ring0) and using the kernel stack, but without entering the kernel. Interrupts are disabled. In general, the same rules and restrictions apply as for an ISR: keep it short, don't enable interrupts, and don't make any kernel calls.

The signature for the callout is:

uintptr_t (*custom_callout)( struct syspage_entry *syspageptr,
                             uint32_t type_id,
                             uintptr_t *regs )

The arguments are as follows:

syspageptr

A pointer to the system page; see the “System Page” chapter. The system page includes references to many data structures that store essential system information.

In the callout_entry data structure, the value for the custom field must be either:

• NULL, indicating that no custom kernel call callout is provided
• non-NULL, and pointing to valid code provided by the startup program, indicating that a custom kernel call callout is provided

type_id

The calling process's type (which can be used by the callout for permission checking). This is useful only if your system is using security policies.

regs

An array holding the register values of the calling thread. The layout of the thread registers is defined by the arch_CPU_REGISTERS structure, which can be found in the arch/context.h header. For example, the layout for x86_64 is defined by the X86_64_CPU_REGISTERS structure in x86_64/context.h, and for AArch64 by AARCH64_CPU_REGISTERS in aarch64/context.h.

Note that there's no C library function wrapping the kernel call, as the C signature of the function depends on the implementation of the callout and is at the discretion of the author of the callout. The kernel only takes care to propagate the registers of the calling thread to and from the callout. The standard ABI should be followed when writing C wrappers for the kernel call.

On a multiCPU system, it's possible that the callout may be invoked on multiple CPUs at the same time. This can be a problem if the callout accesses a resource that's shared between CPUs, because there could end up being multiple callout instances concurrently accessing and modifying the resource.

For example, suppose your custom kernel callout shares a resource and you implement a simple spinlock. If this callout is heavily used (e.g., in a burst of calls), it's possible for the callout instance running in one CPU to continually lose on the spinlock and thus, interrupts could be disabled for a long time. In this case, you would have to implement a synchronization primitive that provides fairness, implements the necessary barriers, doesn't make kernel calls, and disables interrupts.

If the callout will access only per-CPU resources (i.e., those not shared across CPUs), this isn't a problem.

Example

Here's an example of a custom callout for AArch64:

#include "callout.ah"

/*
 * -----------------------------------------------------------------------
 * Routine to patch callout code
 *
 * On entry:
 *  r0 - physical address of syspage
 *  r1 - virtual  address of syspage
 *  r2 - offset from start of syspage to start of the callout routine
 *  r3 - offset from start of syspage to read/write data used by callout
 * -----------------------------------------------------------------------
 */
patch_custom:
    ret

/*
 * -----------------------------------------------------------------------
 * Sample implementation of a custom kernel call callout.
 *
 * On entry:
 *  r0 - pointer to the system page
 *  r1 - type ID for the calling process
 *  r2 - pointer to the first position in an array of 64-bit values holding
 *           the register context of the calling thread
 * -----------------------------------------------------------------------
 */
CALLOUT_START(custom, 0, patch_custom)
    mov     x5, #0xabcd
    movk    x5, #0xabcd, lsl #16
    movk    x5, #0xabcd, lsl #32
    movk    x5, #0xabcd, lsl #48

    /*
     * Reject processes with a type ID other than 47.
     */
    cmp      x1, #47
    beq      1f
    mov      x0, #-1
    ret

1:
    /*
     * Return the sum of the values held in the caller's x0 and x1 registers.
     */
    ldp      x2, x3, [x2]
    add      x0, x2, x3
    ret
CALLOUT_END(custom)

Invoking the callout

Here's a program that uses the callout by invoking a kernel call:

#include <stdio.h>

extern unsigned long SysCustom(unsigned long a, unsigned long b);

int
main(int argc, char **argv)
{
    unsigned long   rc = SysCustom(3, 4);
    if (rc == -1) {
        perror("SysCustom");
    } else {
        printf("rc=%ld\n", rc);
    }

    return 0;
}

Here's a sample implementation of a SysCustom() kernel callout on AArch64:

.text
.align 2
.global SysCustom
SysCustom:
mov w8, #106 ; prep register with call type
svc #0x51
ret ; return
ret
.type SysCustom,function
.size SysCustom,.-SysCustom

Here's a sample implementation of a SysCustom() kernel callout on x86_64 with fewer than 4 arguments:

.text
.p2align 4,,15
.global SysCustom
.type SysCustom, @function
SysCustom:
movl    $106,%eax
syscall
ret
ret
.size SysCustom,.-SysCustom

Here's a sample implementation of a SysCustom() kernel callout on x86_64 with 4 or more arguments:

.text
.p2align 4,,15
.global SysCustom
.type SysCustom, @function
SysCustom:
mov     %rcx, %r10
movl    $106, %eax
syscall
ret
ret
.size SysCustom,.-SysCustom