Add some documentation for the kvx architecture and its Linux port. Co-developed-by: Clement Leger <clement(a)clement-leger.fr&gt; Signed-off-by: Clement Leger <clement(a)clement-leger.fr&gt; Co-developed-by: Jules Maselbas <jmaselbas(a)kalray.eu&gt; Signed-off-by: Jules Maselbas <jmaselbas(a)kalray.eu&gt; Co-developed-by: Guillaume Thouvenin <gthouvenin(a)kalray.eu&gt; Signed-off-by: Guillaume Thouvenin <gthouvenin(a)kalray.eu&gt; Signed-off-by: Yann Sionneau <ysionneau(a)kalray.eu&gt; --- Notes: V1 -> V2: - converted to .rst, typos and formatting fixes - reword few paragraphs Documentation/arch.rst | 1 + Documentation/kvx/index.rst | 17 ++ Documentation/kvx/kvx-exceptions.rst | 256 ++++++++++++++++++++++++ Documentation/kvx/kvx-iommu.rst | 191 ++++++++++++++++++ Documentation/kvx/kvx-mmu.rst | 287 +++++++++++++++++++++++++++ Documentation/kvx/kvx-smp.rst | 39 ++++ Documentation/kvx/kvx.rst | 273 +++++++++++++++++++++++++ 7 files changed, 1064 insertions(+) create mode 100644 Documentation/kvx/index.rst create mode 100644 Documentation/kvx/kvx-exceptions.rst create mode 100644 Documentation/kvx/kvx-iommu.rst create mode 100644 Documentation/kvx/kvx-mmu.rst create mode 100644 Documentation/kvx/kvx-smp.rst create mode 100644 Documentation/kvx/kvx.rst diff --git a/Documentation/arch.rst b/Documentation/arch.rst index 41a66a8b38e4..1ccda8ef6eef 100644 --- a/Documentation/arch.rst +++ b/Documentation/arch.rst @@ -13,6 +13,7 @@ implementation. arm/index arm64/index ia64/index + kvx/index loongarch/index m68k/index mips/index diff --git a/Documentation/kvx/index.rst b/Documentation/kvx/index.rst new file mode 100644 index 000000000000..0bac597b5fbe --- /dev/null +++ b/Documentation/kvx/index.rst @@ -0,0 +1,17 @@ +.. SPDX-License-Identifier: GPL-2.0 + +======================= +Kalray kvx Architecture +======================= + +.. toctree:: + :maxdepth: 2 + :numbered: + + kvx + kvx-exceptions + kvx-smp + kvx-mmu + kvx-iommu + + diff --git a/Documentation/kvx/kvx-exceptions.rst b/Documentation/kvx/kvx-exceptions.rst new file mode 100644 index 000000000000..5e01e934192f --- /dev/null +++ b/Documentation/kvx/kvx-exceptions.rst @@ -0,0 +1,256 @@ +========== +Exceptions +========== + +On kvx, handlers are set using ``$ev`` (exception vector) register which +specifies a base address. An offset is added to ``$ev`` upon exception +and the result is used as the new ``$pc``. +The offset depends on which exception vector the cpu wants to jump to: + + ============== ========= + ``$ev + 0x00`` debug + ``$ev + 0x40`` trap + ``$ev + 0x80`` interrupt + ``$ev + 0xc0`` syscall + ============== ========= + +Then, handlers are laid in the following order:: + + _____________ + | | + | Syscall | + |_____________| + | | + | Interrupts | + |_____________| + | | + | Traps | + |_____________| + | | ^ + | Debug | | Stride + BASE -> |_____________| v + + +Interrupts and traps are serviced similarly, ie: + + - Jump to handler + - Save all registers + - Prepare the call (do_IRQ or trap_handler) + - restore all registers + - return from exception + +entry.S file is (as for other architectures) the entry point into the kernel. +It contains all assembly routines related to interrupts/traps/syscall. + +Syscall handling +---------------- + +When executing a syscall, it must be done using ``scall $r6`` where ``$r6`` +contains the syscall number. This convention allows to modify and restart +a syscall from the kernel. + +Syscalls are handled differently than interrupts/exceptions. From an ABI +point of view, syscalls are like function calls: any caller-saved register +can be clobbered by the syscall. However, syscall parameters are passed +using registers r0 through r7. These registers must be preserved to avoid +clobberring them before the actual syscall function. + +On syscall from userspace (``scall`` instruction), the processor will put +the syscall number in $es.sn and switch from user to kernel privilege +mode. ``kvx_syscall_handler`` will be called in kernel mode. + +The following steps are then taken: + + 1. Switch to kernel stack + 2. Extract syscall number + 3. If the syscall number is bogus, set the syscall function to ``sys_ni_syscall`` + 4. If tracing is enabled + + - Jump to ``trace_syscall_enter`` + - Save syscall arguments (``r0`` -> ``r7``) on stack in ``pt_regs``. + - Call ``do_trace_syscall_enter`` function. + + 5. Restore syscall arguments since they have been modified by C call + 6. Call the syscall function + 7. Save ``$r0`` in ``pt_regs`` since it can be clobberred afterward + 8. If tracing is enabled, call ``trace_syscall_exit``. + 9. Call ``work_pending`` + 10. Return to user ! + +The trace call is handled out of the fast path. All slow path handling +is done in another part of code to avoid messing with the cache. + +Signals +------- + +Signals are handled when exiting kernel before returning to user. +When handling a signal, the path is the following: + + 1. User application is executing normally + Then any exception happens (syscall, interrupt, trap) + 2. The exception handling path is taken + and before returning to user, pending signals are checked + 3. Signal are handled by ``do_signal``. + Registers are saved and a special part of the stack is modified + to create a trampoline to call ``rt_sigreturn``, + ``$spc`` is modified to jump to user signal handler; + ``$ra`` is modified to jump to sigreturn trampoline directly after + returning from user signal handler. + 4. User signal handler is called after ``rfe`` from exception + when returning, ``$ra`` is retored to ``$pc``, resulting in a call + to the syscall trampoline. + 5. syscall trampoline is executed, leading to rt_sigreturn syscall + 6. ``rt_sigreturn`` syscall is executed. Previous registers are restored to + allow returning to user correctly. + 7. User application is restored at the exact point it was interrupted + before. + +:: + + +----------+ + | 1 | + | User app | @func + | (user) | + +---+------+ + | + | it/trap/scall + | + +---v-------+ + | 2 | + | exception | + | handling | + | (kernel) | + +---+-------+ + | + | Check if signal are pending, if so, handle signals + | + +---v--------+ + | 3 | + | do_signal | + | handling | + | (kernel) | + +----+-------+ + | + | Return to user signal handler + | + +----v------+ + | 4 | + | signal | + | handler | + | (user) | + +----+------+ + | + | Return to sigreturn trampoline + | + +----v-------+ + | 5 | + | syscall | + |rt_sigreturn| + | (user) | + +----+-------+ + | + | Syscall to rt_sigreturn + | + +----v-------+ + | 6 | + | sigreturn | + | handler | + | (kernel) | + +----+-------+ + | + | Modify context to return to original func + | + +----v-----+ + | 7 | + | User app | @func + | (user) | + +----------+ + + +Registers handling +------------------ + +MMU is disabled in all exceptions paths, during register save and restoration. +This will prevent from triggering MMU fault (such as TLB miss) which could +clobber the current register state. Such event can occurs when RWX mode is +enabled and the memory accessed to save register can trigger a TLB miss. +Aside from that which is common for all exceptions path, registers are saved +differently depending on the exception type. + +Interrupts and traps +-------------------- + +When interrupt and traps are triggered, only caller-saved registers are saved. +Indeed, we rely on the fact that C code will save and restore callee-saved and +hence, there is no need to save them. The path is:: + + +------------+ +-----------+ +---------------+ + IT | Save caller| C Call | Execute C | Ret | Restore caller| Ret from IT + +--->+ saved +--------->+ handler +------->+ saved +-----> + | registers | +-----------+ | registers | + +------------+ +---------------+ + +However, when returning to user, we check if there is work_pending. If a signal +is pending and there is a signal handler to be called, then all registers are +saved on the stack in ``pt_regs`` before executing the signal handler and +restored after that. Since only caller-saved registers have been saved before +checking for pending work, callee-saved registers also need to be saved to +restore everything correctly when before returning to user. +This path is the following (a bit more complicated !):: + + +------------+ + | Save caller| +-----------+ Ret +------------+ + IT | saved | C Call | Execute C | to asm | Check work | + +--->+ registers +--------->+ handler +------->+ pending | + | to pt_regs | +-----------+ +--+---+-----+ + +------------+ | | + Work pending | | No work pending + +--------------------------------------------+ | + | | + | +------------+ + v | + +------+------+ v + | Save callee | +-------+-------+ + | saved | | Restore caller| RFE from IT + | registers | | saved +-------> + | to pt_regs | | registers | + +--+-------+--+ | from pt_regs | + | | +-------+-------+ + | | +---------+ ^ + | | | Execute | | + | +-------->+ needed +-----------+ + | | work | + | +---------+ + |Signal handler ? + v + +----+----------+ RFE to user +-------------+ +--------------+ + | Copy all | handler | Execute | ret | rt_sigreturn | + | registers +------------>+ user signal +------>+ trampoline | + | from pt_regs | | handler | | to kernel | + | to user stack | +-------------+ +------+-------+ + +---------------+ | + syscall rt_sigreturn | + +-------------------------------------------------+ + | + v + +--------+-------+ +-------------+ + | Recopy all | | Restore all | RFE + | registers from +--------------------->+ saved +-------> + | user stack | Return | registers | + | to pt_regs | from sigreturn |from pt_regs | + +----------------+ (via ret_from_fork) +-------------+ + + +Syscalls +-------- + +As explained before, for syscalls, we can use whatever callee-saved registers +we want since syscall are seen as a "classic" call from ABI pov. +Only different path is the one for clone. For this path, since the child expects +to find same callee-registers content than his parent, we must save them before +executing the clone syscall and restore them after that for the child. This is +done via a redefinition of __sys_clone in assembly which will be called in place +of the standard sys_clone. This new call will save callee saved registers +in pt_regs. Parent will return using the syscall standard path. Freshly spawned +child however will be woken up via ret_from_fork which will restore all +registers (even if caller saved are not needed). diff --git a/Documentation/kvx/kvx-iommu.rst b/Documentation/kvx/kvx-iommu.rst new file mode 100644 index 000000000000..240995d315ce --- /dev/null +++ b/Documentation/kvx/kvx-iommu.rst @@ -0,0 +1,191 @@ +.. SPDX-License-Identifier: GPL-2.0 + +===== +IOMMU +===== + +General Overview +---------------- + +To exchange data between device and users through memory, the driver +has to set up a buffer by doing some kernel allocation. The buffer uses +virtual address and the physical address is obtained through the MMU. +When the device wants to access the same physical memory space it uses +the bus address which is obtained by using the DMA mapping API. The +Coolidge SoC includes several IOMMUs for clusters, PCIe peripherals, +SoC peripherals, and more; that will translate this "bus address" into +a physical one for DMA operations. + +Bus addresses are IOVA (I/O Virtual Address) or DMA addresses. These +addresses can be obtained by calling the allocation functions of the DMA APIs. +It can also be obtained through classical kernel allocation of physical +contiguous memory and then calling mapping functions of the DMA API. + +In order to be able to use the kvx IOMMU we have implemented the IOMMU DMA +interface in arch/kvx/mm/dma-mapping.c. DMA functions are registered by +implementing arch_setup_dma_ops() and generic IOMMU functions. Generic IOMMU +are calling our specific IOMMU functions that adds or remove mappings between +DMA addresses and physical addresses in the IOMMU TLB. + +Specific IOMMU functions are defined in the kvx IOMMU driver. The kvx IOMMU +driver manage two physical hardware IOMMU: one used for TX and one for RX. +In the next section we described the HW IOMMUs. + +Cluster IOMMUs +-------------- + +IOMMUs on cluster are used for DMA and cryptographic accelerators. +There are six IOMMUs connected to the: + + - cluster DMA tx + - cluster DMA rx + - first non secure cryptographic accelerator + - second non secure cryptographic accelerator + - first secure cryptographic accelerator + - second secure cryptographic accelerator + +SoC peripherals IOMMUs +---------------------- + +Since SoC peripherals are connected to an AXI bus, two IOMMUs are used: one for +each AXI channel (read and write). These two IOMMUs are shared between all master +devices and DMA. These two IOMMUs will have the same entries but need to be configured +independently. + +PCIe IOMMUs +----------- + +There is a slave IOMMU (read and write from the MPPA to the PCIe endpoint) +and a master IOMMU (read and write from a PCIe endpoint to system DDR). +The PCIe root complex and the MSI/MSI-X controller have been designed to use +the IOMMU feature when enabled. (For example for supporting endpoint that +support only 32 bits addresses and allow them to access any memory in a +64 bits address space). For security reason it is highly recommended to +activate the IOMMU for PCIe. + +IOMMU implementation +-------------------- + +The kvx is providing several IOMMUs. Here is a simplified view of all IOMMUs +and translations that occurs between memory and devices:: + + +---------------------------------------------------------------------+ + | +------------+ +---------+ | CLUSTER X | + | | Cores 0-15 +---->+ Crypto | +-----------| + | +-----+------+ +----+----+ | + | | | | + | v v | + | +-------+ +------------------------------+ | + | | MMU | +----+ IOMMU x4 (secure + insecure) | | + | +---+---+ | +------------------------------+ | + | | | | + +--------------------+ | + | | | | + v v | | + +---+--------+-+ | | + | MEMORY | | +----------+ +--------+ +-------+ | + | +<-|-----+ IOMMU Rx |<----+ DMA Rx |<----+ | | + | | | +----------+ +--------+ | | | + | | | | NoC | | + | | | +----------+ +--------+ | | | + | +--|---->| IOMMU Tx +---->| DMA Tx +---->+ | | + | | | +----------+ +--------+ +-------+ | + | | +------------------------------------------------+ + | | + | | +--------------+ +------+ + | |<--->+ IOMMU Rx/Tx +<--->+ PCIe + + | | +--------------+ +------+ + | | + | | +--------------+ +------------------------+ + | |<--->+ IOMMU Rx/Tx +<--->+ master Soc Peripherals | + | | +--------------+ +------------------------+ + +--------------+ + + +There is also an IOMMU dedicated to the crypto module but this module will not +be accessed by the operating system. + +We will provide one driver to manage IOMMUs RX/TX. All of them will be +described in the device tree to be able to get their particularities. See +the example below that describes the relation between IOMMU, DMA and NoC in +the cluster. + +IOMMU is related to a specific bus like PCIe we will be able to specify that +all peripherals will go through this IOMMU. + +IOMMU Page table +~~~~~~~~~~~~~~~~ + +We need to be able to know which IO virtual addresses (IOVA) are mapped in the +TLB in order to be able to remove entries when a device finishes a transfer and +release memory. This information could be extracted when needed by computing all +sets used by the memory and then reads all sixteen ways and compare them to the +IOVA but it won't be efficient. We also need to be able to translate an IOVA +to a physical address as required by the iova_to_phys IOMMU ops that is used +by DMA. Like previously it can be done by extracting the set from the address +and comparing the IOVA to each sixteen entries of the given set. + +A solution is to keep a page table for the IOMMU. But this method is not +efficient for reloading an entry of the TLB without the help of an hardware +page table. So to prevent the need of a refill we will update the TLB when a +device request access to memory and if there is no more slot available in the +TLB we will just fail and the device will have to try again later. It is not +efficient but at least we won't need to manage the refill of the TLB. + +This limits the total amount of memory that can be used for transfer between +device and memory (see Limitations section below). +To be able to manage bigger transfer we can implement the huge page table in +the Linux kernel and use a page table that match the size of huge page table +for a given IOMMU (typically the PCIe IOMMU). + +As we won't refill the TLB we know that we won't have more than 128*16 entries. +In this case we can simply keep a table with all possible entries. + +Maintenance interface +~~~~~~~~~~~~~~~~~~~~~ + +It is possible to have several "maintainers" for the same IOMMU. The driver is +using two of them. One that writes the TLB and another interface reads TLB. For +debug purpose it is possible to display the content of the tlb by using the +following command in gdb:: + + gdb> p kvx_iommu_dump_tlb( <iommu addr>, 0) + +Since different management interface are used for read and write it is safe to +execute the above command at any moment. + +Interrupts +~~~~~~~~~~ + +IOMMU can have 3 kind of interrupts that corresponds to 3 different types of +errors (no mapping. protection, parity). When the IOMMU is shared between +clusters (SoC periph and PCIe) then fifteen IRQs are generated according to the +configuration of an association table. The association table is indexed by the +ASN number (9 bits) and the entry of the table is a subscription mask with one +bit per destination. Currently this is not managed by the driver. + +The driver is only managing interrupts for the cluster. The mode used is the +stall one. So when an interrupt occurs it is managed by the driver. All others +interrupts that occurs are stored and the IOMMU is stalled. When driver cleans +the first interrupt others will be managed one by one. + +ASN (Address Space Number) +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This is also know as ASID in some other architecture. Each device will have a +given ASN that will be given through the device tree. As address space is +managed at the IOMMU domain level we will use one group and one domain per ID. +ASN are coded on 9 bits. + +Device tree +----------- + +Relationships between devices, DMAs and IOMMUs are described in the +device tree (see `Documentation/devicetree/bindings/iommu/kalray,kvx-iommu.txt` +for more details). + +Limitations +----------- + +Only supporting 4KB page size will limit the size of mapped memory to 8MB +because the IOMMU TLB can have at most 128*16 entries. diff --git a/Documentation/kvx/kvx-mmu.rst b/Documentation/kvx/kvx-mmu.rst new file mode 100644 index 000000000000..b7186331396c --- /dev/null +++ b/Documentation/kvx/kvx-mmu.rst @@ -0,0 +1,287 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=== +MMU +=== + +Virtual addresses are on 41 bits for kvx when using 64-bit mode. +To differentiate kernel from user space, we use the high order bit +(bit 40). When bit 40 is set, then the higher remaining bits must also be +set to 1. The virtual address must be extended with 1 when the bit 40 is set, +if not the address must be zero extended. Bit 40 is set for kernel space +mappings and not set for user space mappings. + +Memory Map +---------- + +In Linux physical memories are arranged into banks according to the cost of an +access in term of distance to a memory. As we are UMA architecture we only have +one bank and thus one node. + +A node is divided into several kind of zone. For example if DMA can only access +a specific area in the physical memory we will define a ZONE_DMA for this purpose. +In our case we are considering that DMA can access all DDR so we don't have a specific +zone for this. On 64 bit architecture all DDR can be mapped in virtual kernel space +so there is no need for a ZONE_HIGHMEM. That means that in our case there is +only one ZONE_NORMAL. This will be updated if DMA cannot access all memory. + +Currently, the memory mapping is the following for 4KB page: + + ======================== ======================= ====== ======= ============== + Start End Attr Size Name + ======================== ======================= ====== ======= ============== + 0000 0000 0000 0000 0000 003F FFFF FFFF --- 256GB User + 0000 0040 0000 0000 0000 007F FFFF FFFF --- 256GB MMAP + 0000 0080 0000 0000 FFFF FF7F FFFF FFFF --- --- Gap + FFFF FF80 0000 0000 FFFF FFFF FFFF FFFF --- 512GB Kernel + FFFF FF80 0000 0000 FFFF FF8F FFFF FFFF RWX 64GB Direct Map + FFFF FF90 0000 0000 FFFF FF90 3FFF FFFF RWX 1GB Vmalloc + FFFF FF90 4000 0000 FFFF FFFF FFFF FFFF RW 447GB Free area + ======================== ======================= ====== ======= ============== + +Enable the MMU +-------------- + +All kernel functions and symbols are in virtual memory except for kvx_start() +function which is loaded at 0x0 in physical memory. +To be able to switch from physical addresses to virtual addresses we choose to +setup the TLB at the very beginning of the boot process to be able to map both +pieces of code. For this we added two entries in the LTLB. The first one, +LTLB[0], contains the mapping between virtual memory and DDR. Its size is 512MB. +The second entry, LTLB[1], contains a flat mapping of the first 2MB of the SMEM. +Once those two entries are present we can enable the MMU. LTLB[1] will be +removed during paging_init() because once we are really running in virtual space +it will not be used anymore. +In order to access more than 512MB DDR memory, the remaining memory (> 512MB) is +refill using a comparison in kernel_perf_refill that does not walk the kernel +page table, thus having a faster refill time for kernel. These entries are +inserted into the LTLB for easier computation (4 LTLB entries). The drawback of +this approach is that mapped entries are using RWX protection attributes, +leading to no protection at all. + +Kernel strict RWX +----------------- + +``CONFIG_STRICT_KERNEL_RWX`` is enabled by default in defconfig. +Once booted, if ``CONFIG_STRICT_KERNEL_RWX`` is enable, the kernel text and memory +will be mapped in the init_mm page table. Once mapped, the refill routine for +the kernel is patched to always do a page table walk, bypassing the faster +comparison but enforcing page protection attributes when refilling. +Finally, the LTLB[0] entry is replaced by a 4K one, mapping only exceptions with +RX protection. It allows us to never trigger nomapping on nomapping refill +routine which would (obviously) not work... Once this is done, we can flush the +4 LTLB entries for kernel refill in order to be sure there is no stalled +entries and that new entries inserted in JTLB will apply. + +By default, the following policy is applied on vmlinux sections: + + - init_data: RW + - init_text: RX (or RWX if parameter rodata=off) + - text: RX (or RWX if parameter rodata=off) + - rodata: RW before init, RO after init + - sdata: RW + +Kernel RWX mode can then be switched on/off using /sys/kvx/kernel_rwx file. + +Privilege Level +--------------- + +Since we are using privilege levels on kvx, we make use of the virtual +spaces to be in the same space as the user. The kernel will have the +$ps.mmup set in kernel (PL1) and unset for user (PL2). +As said in kvx documentation, we have two cases when the kernel is +booted: + + - Either we have been booted by someone (bootloader, hypervisor, etc) + - Or we are alone (boot from flash) + +In both cases, we will use the virtual space 0. Indeed, if we are alone +on the core, then it means nobody is using the MMU and we can take the +first virtual space. If not alone, then when writing an entry to the tlb +using writetlb instruction, the hypervisor will catch it and change the +virtual space accordingly. + +Memblock +======== + +When the kernel starts there is no memory allocator available. One of the first +step in the kernel is to detect the amount of DDR available by getting this +information in the device tree and initialize the low-level "memblock" allocator. + +We start by reserving memory for the whole kernel. For instance with a device +tree containing 512MB of DDR you could see the following boot messages:: + + setup_bootmem: Memory : 0x100000000 - 0x120000000 + setup_bootmem: Reserved: 0x10001f000 - 0x1002d1bc0 + +During the paging init we need to set: + + - min_low_pfn that is the lowest PFN available in the system + - max_low_pfn that indicates the end if NORMAL zone + - max_pfn that is the number of pages in the system + +This setting is used for dividing memory into pages and for configuring the +zone. See the memory map section for more information about ZONE. + +Zones are configured in free_area_init_core(). During start_kernel() other +allocations are done for command line, cpu areas, PID hash table, different +caches for VFS. This allocator is used until mem_init() is called. + +mem_init() is provided by the architecture. For MPPA we just call +free_all_bootmem() that will go through all pages that are not used by the +low level allocator and mark them as not used. So physical pages that are +reserved for the kernel are still used and remain in physical memory. All pages +released will now be used by the buddy allocator. + +Peripherals +----------- + +Peripherals are mapped using standard ioremap infrastructure, therefore +mapped addresses are located in the vmalloc space. + +LTLB Usage +---------- + +LTLB is used to add resident mapping which allows for faster MMU lookup. +Currently, the LTLB is used to map some mandatory kernel pages and to allow fast +accesses to l2 cache (mailbox and registers). +When CONFIG_STRICT_KERNEL_RWX is disabled, 4 entries are reserved for kernel +TLB refill using 512MB pages. When CONFIG_STRICT_KERNEL_RWX is enabled, these +entries are unused since kernel is paginated using the same mecanism than for +user (page walking and entries in JTLB) + +Page Table +========== + +We only support three levels for the page table and 4KB for page size. + +3 levels page table +------------------- + +:: + + ...-----+--------+--------+--------+--------+--------+ + 40|39 32|31 24|23 16|15 8|7 0| + ...-----++-------+--+-----+---+----+----+---+--------+ + | | | | + | | | +---> [11:0] Offset (12 bits) + | | +-------------> [20:12] PTE offset (9 bits) + | +-----------------------> [29:21] PMD offset (9 bits) + +----------------------------------> [39:30] PGD offset (10 bits) + +Bits 40 to 64 are signed extended according to bit 39. If bit 39 is equal to 1 +we are in kernel space. + +As 10 bits are used for PGD we need to allocate 2 pages. + +PTE format +========== + +About the format of the PTE entry, as we are not forced by hardware for choices, +we choose to follow the format described in the RiscV implementation as a +starting point:: + + +---------+--------+----+--------+---+---+---+---+---+---+------+---+---+ + | 63..23 | 22..13 | 12 | 11..10 | 9 | 8 | 7 | 6 | 5 | 4 | 3..2 | 1 | 0 | + +---------+--------+----+--------+---+---+---+---+---+---+------+---+---+ + PFN Unused S PageSZ H G X W R D CP A P + where: + P: Present + A: Accessed + CP: Cache policy + D: Dirty + R: Read + W: Write + X: Executable + G: Global + H: Huge page + PageSZ: Page size as set in TLB format (0:4KB, 1:64KB, 2:2MB, 3:512MB) + S: Soft/Special + PFN: Page frame number (depends on page size) + +Huge bit must be somewhere in the first 12 bits to be able to detect it +when reading the PMD entry. + +PageSZ must be on bit 10 and 11 because it matches the TEL.PS bits. And +by doing that it is easier in assembly to set the TEL.PS to PageSZ. + +Fast TLB refill +=============== + +kvx core does not feature a hardware page walker. This work must be done +by the core in software. In order to optimize TLB refill, a special fast +path is taken when entering in kernel space. +In order to speed up the process, the following actions are taken: + + 1. Save some registers in a per process scratchpad + 2. If the trap is a nomapping then try the fastpath + 3. Save some more registers for this fastpath + 4. Check if faulting address is a memory direct mapping one. + + * If entry is a direct mapping one and RWX is not enabled, add an entry into LTLB + * If not, continue + + 5. Try to walk the page table + + * If entry is not present, take the slowpath (do_page_fault) + + 6. Refill the tlb properly + 7. Exit by restoring only a few registers + +ASN Handling +============ + +Disclaimer: Some part of this are taken from ARC architecture. + +kvx MMU provides 9-bit ASN (Address Space Number) in order to tag TLB entries. +It allows for multiple process with the same virtual space to cohabit without +the need to flush TLB everytime we context switch. +kvx implementation to use them is based on other architectures (such as arc +or xtensa) and uses a wrapping ASN counter containing both cycle/generation and +asn. + +:: + + +---------+--------+ + |63 10|9 0| + +---------+--------+ + Cycle ASN + +This ASN counter is incremented monotonously to allocate new ASNs. When the +counter reaches 511 (9 bit), TLB is completely flushed and a new cycle is +started. A new allocation cycle, post rollover, could potentially reassign an +ASN to a different task. Thus the rule is to reassign an ASN when the current +context cycles does not match the allocation cycle. +The 64 bit @cpu_asn_cache (and mm->asn) have 9 bits MMU ASN and rest 55 bits +serve as cycle/generation indicator and natural 64 bit unsigned math +automagically increments the generation when lower 9 bits rollover. +When the counter completely wraps, we reset the counter to first cycle value +(ie cycle = 1). This allows to distinguish context without any ASN and old cycle +generated value with the same operation (XOR on cycle). + +Huge page +========= + +Currently only 3 level page table has been implemented for 4KB base page size. +So the page shift is 12 bits, the pmd shift is 21 and the pgdir shift is 30 bits. +This choice implies that for 4KB base page size if we use a PMD as a huge +page the size will be 2MB and if we use a PUD as a huge page it will be 1GB. + +To support other huge page sizes (64KB and 512MB) we need to use several +contiguous entries in the page table. For huge page of 64KB we will need to +use 16 entries in the PTE and for a huge page of 512MB it means that 256 +entries in PMD will be used. + +Debug +===== + +In order to debug the page table and tlb entries, gdb scripts contains commands +which allows to dump the page table: + +:``lx-kvx-page-table-walk``: Display the current process page table by default +:``lx-kvx-tlb-decode``: Display the content of $tel and $teh into something readable + +Other commands available in kvx-gdb are the following: + +:``mppa-dump-tlb``: Display the content of TLBs (JTLB and LTLB) +:``mppa-lookup-addr``: Find physical address matching a virtual one diff --git a/Documentation/kvx/kvx-smp.rst b/Documentation/kvx/kvx-smp.rst new file mode 100644 index 000000000000..12efddbfd1e0 --- /dev/null +++ b/Documentation/kvx/kvx-smp.rst @@ -0,0 +1,39 @@ +=== +SMP +=== + +The Coolidge SoC is comprised of 5 clusters, each organized as a group +of 17 cores: 16 application core (PE) and 1 secure core (RM). +These 17 cores have their L1 cache coherent with the local Tightly +Coupled Memory (TCM or SMEM). The L2 cache is necessary for SMP support +is and implemented with a mix of HW support and SW firmware. The L2 cache +data and meta-data are stored in the TCM. +The RM core is not meant to run Linux and is reserved for implementing +hypervisor services, thus only 16 processors are available for SMP. + +Booting +------- + +When booting the kvx processor, only the RM is woken up. This RM will +execute a portion of code located in the section named ``.rm_firmware``. +By default, a simple power off code is embedded in this section. +To avoid embedding the firmware in kernel sources, the section is patched +using external tools to add the L2 firmware (and replace the default firmware). +Before executing this firmware, the RM boots the PE0. PE0 will then enable L2 +coherency and request will be stalled until RM boots the L2 firmware. + +Locking primitives +------------------ + +spinlock/rwlock are using the kernel standard queued spinlock/rwlocks. +These primitives are based on cmpxch and xchg. More particularly, it uses xchg16 +which is implemented as a read modify write with acswap on 32bit word since +kvx does not have atomic cmpxchg instructions for less than 32 bits. + +IPI +--- + +An IPI controller allows to communicate between CPUs using a simple +memory mapped register. This register can simply be written using a +mask to trigger interrupts directly to the cores matching the mask. + diff --git a/Documentation/kvx/kvx.rst b/Documentation/kvx/kvx.rst new file mode 100644 index 000000000000..c222328aa4b2 --- /dev/null +++ b/Documentation/kvx/kvx.rst @@ -0,0 +1,273 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========= +kvx Linux +========= + +This documents will try to explain any architecture choice for the kvx +Linux port. + +Regarding the peripheral, we MUST use device tree to describe ALL +peripherals. The bindings should always start with "kalray,kvx" for all +core related peripherals (watchdog, timer, etc) + +System Architecture +------------------- + +On kvx, we have 4 levels of privilege level starting from 0 (most +privileged one) to 3 (less privilege one). A system of owners allows +to delegate ownership of resources by using specials system registers. + +The 2 main software stacks for Linux Kernel are the following:: + + +-------------+ +-------------+ + | PL0: Debug | | PL0: Debug | + +-------------+ +-------------+ + | PL1: Linux | | PL1: HyperV | + +-------------+ +-------------+ + | PL2: User | | PL2: Linux | + +-------------+ +-------------+ + | | | PL3: User | + +-------------+ +-------------+ + +In both cases, the kvx support for privileges has been designed using +only relative PL and thus should work on both configurations without +any modifications. + +When booting, the CPU is executing in PL0 and owns all the privileges. +This level is almost dedicated to the debug routines for the debugguer. +It only needs to own few privileges (breakpoint 0 and watchpoint 0) to +be able to debug a system executing in PL1 to PL3. +Debug routines are not always there for instance when the kernel is +executing alone (booted from flash). +In order to ease the load of debug routines, software convention is to +jump directly to PL1 and let PL0 for the debug. +When the kernel boots, it checks if the current privilege level is 0 +(`$ps.pl` is the only absolute value). If so, then it will delegate +almost all resources to PL1 and use a RFE to lower its execution +privilege level (see asm_delegate_pl in head.S). +If the current PL is already different from 0, then it means somebody +is above us and we need to request resource to inform it we need them. It will +then either delegate them to us directly or virtualize the delegation. +All privileges levels have their set of banked registers (ps, ea, sps, +sr, etc) which contain privilege level specific values. +`$sr` (system reserved) is banked and will hold the current task_struct. +This register is reserved and should not be touched by any other code. +For more information, refer to the kvx system level architecture manual. + +Boot +---- + +On kvx, the RM (Secure Core) of Cluster 0 will boot first. It will then be able +to boot a firmware. This firmware is stored in the rm_firmware section. +The first argument ($r0) of this firmware will be a pointer to a function with +the following prototype: void firmware_init_done(uint64_t features). This +function is responsible of describing the features supported by the firmware and +will start the first PE after that. +By default, the rm_firmware function act as the "default" firmware. This +function does nothing except calling firmware_init_done and then goes to sleep. +In order to add another firmware, the rm_firmware section is patched using +objcopy. The content of this section is then replaced by the provided firmware. +This firmware will do an init and then call firmware_init_done before running +the main loop. +When the PE boots, it will check for the firmware features to enable or disable +specific core features (L2 for instance). + +When entering the C (kvx_lowlevel_start) the kernel will look for a special +magic (``0x494C314B``) in ``$r0``. This magic tells the kernel if there is arguments +passed by a bootloader. +Currently, the following values are passed through registers: + + :``$r1``: pointer to command line setup by bootloader + :``$r2``: device tree + +If this magic is not set, then, the command line will be the one +provided in the device tree (see bootargs). The default device tree is +not builtin but will be patched by the runner used (simulator or jtag) in the +dtb section. + +A default stdout-path is desirable to allow early printk. + +Boot Memory Allocator +--------------------- + +The boot memory allocator is used to allocate memory before paging is enabled. +It is initialized with DDR and also with the shared memory. This first one is +initialized during the setup_bootmem() and the second one when calling +early_init_fdt_scan_reserved_mem(). + + +Virtual and physical memory +--------------------------- + +The mapping used and the memory management is described in +Documentation/kvx/kvx-mmu.rst. +Our Kernel is compiled using virtual addresses that starts at ``0xffffff0000000000``. +But when it is started the kernel uses physical addresses. +Before calling the first function arch_low_level_start() we configure 2 entries +of the LTLB. + +The first entry will map the first 1G of virtual address space to the first +1G of DDR:: + + TLB[0]: 0xffffff0000000000 -> 0x100000000 (size 512Mo) + +The second entry will be a flat mapping of the first 512 Ko of the SMEM. It +is required to have this flat mapping because there is still code located at +this address that needs to be executed:: + + TLB[1]: 0x0 -> 0x0 (size 512Ko) + +Once virtual space reached the second entry is removed. + +To be able to set breakpoints when MMU is enabled we added a label called +gdb_mmu_enabled. If you try to set a breakpoint on a function that is in +virtual memory before the activation of the MMU this address as no signification +for GDB. So, for example, if you want to break on the function start_kernel() +you will need to run:: + + kvx-gdb -silent path_to/vmlinux \ + -ex 'tbreak gdb_mmu_enabled' -ex 'run' \ + -ex 'break start_kernel' \ + -ex 'continue' + +We will also add an option to kvx-gdb to simplify this step. + +Timers +------ + +The free-running clock (clocksource) is based on the DSU. This clock is +not interruptible and never stops even if core go into idle. + +Regarding the tick (clockevent), we use the timer 0 available on the core. +This timer allows to set a periodic tick which will be used as the main +tick for each core. Note that this clock is percpu. + +The get_cycles implementation is based on performance counter. One of them +is used to count cycles. Note that since this is used only when the core +is running, there is no need to worry about core sleeping (which will +stop the cycle counter) + +Context switching +----------------- + +Context switching is done in entry.S. When spawning a fresh thread, +copy_thread is called. During this call, we setup callee saved register +``r20`` and ``r21`` to special values containing the function to call. + +The normal path for a kernel thread will be the following: + +1. Enter copy_thread_tls and setup callee saved registers which will + be restored in __switch_to. +2. set r20 and r21 (in thread_struct) to function and argument and + ra to ret_from_kernel_thread. + These callee saved will be restored in switch_to. +3. Call _switch_to at some point. +4. Save all callee saved register since switch_to is seen as a + standard function call by the caller. +5. Change stack pointer to the new stack +6. At the end of switch to, set sr0 to the new task and use ret to + jump to ret_from_kernel_thread (address restored from ra). +7. In ret_from_kernel_thread, execute the function with arguments by + using r20, r21 and we are done + +For more explanations, you can refer to https://lwn.net/Articles/520227/ + +User thread creation +-------------------- + +We are using almost the same path as copy_thread to create it. +The detailed path is the following: + + 1. Call start_thread which will setup user pc and stack pointer in + task regs. We also set sps and clear privilege mode bit. + When returning from exception, it will "flip" to user mode. + 2. Enter copy_thread_tls and setup callee saved registers which will + be restored in __switch_to. Also, set the "return" function to be + ret_from_fork which will be called at end of switch_to + 3. set r20 (in thread_struct) with tracing information. + (simply by lazyness to avoid computing it in assembly...) + 4. Call _switch_to at some point. + 5. The current pc will then be restored to be ret_from fork. + 6. Ret from fork calls schedule_tail and then check if tracing is + enabled. If so call syscall_trace_exit + 7. finally, instead of returning to kernel, we restore all registers + that have been setup by start_thread by restoring regs stored on + stack + +L2 handling +----------- + +On kvx, the L2 is handled by a firmware running on the RM. This firmware +needs various information to be aware of its configuration and communicate +with the kernel. In order to do that, when firmware is starting, the device +tree is given as parameter along with the "registers" zone. This zone is +simply a memory area where data are exchanged between kernel <-> L2. When +some commands are written to it, the kernel sends an interrupt using a mailbox. +If the L2 node is not present in the device tree, then, the RM will directly go +into sleeping. + +Boot diagram:: + + RM PE 0 + + + +---------+ | + | Boot | | + +----+----+ | + | | + v | + +-----+-----+ | + | Prepare | | + | L2 shared | | + | memory | | + |(registers)| | + +-----+-----+ | + | | +-----------+ + +------------------->+ Boot | + | | +-----+-----+ + v | | + +--------+---------+ | | + | L2 firmware | | | + | parameters: | | | + | r0 = registers | | | + | r1 = DTB | | | + +--------+---------+ | | + | | | + v | | + +-------+--------+ | +------+------+ + | L2 firmware | | | Wait for L2 | + | execution | | | to be ready | + +-------+--------+ | +------+------+ + | | | + +------v-------+ | v + | L2 requests | | +------+------+ + +--->+ handling | | | Enable | + | +-------+------+ | | L2 caching | + | | | +------+------+ + | | | | + +------------+ + v + + +Since this driver is started early (before SMP boot), A lot of drivers +are not yet probed (mailboxes, IOMMU, etc) and thus can not be used. + +Building +-------- + +In order to build the kernel, you will need a complete kvx toolchain. +First, setup the config using the following command line:: + + $ make ARCH=kvx O=your_directory defconfig + +Adjust any configuration option you may need and then, build the kernel:: + + $ make ARCH=kvx O=your_directory -j12 + +You will finally have a vmlinux image ready to be run:: + + $ kvx-mppa -- vmlinux + +Additionally, you may want to debug it. To do so, use kvx-gdb:: + + $ kvx-gdb vmlinux + -- 2.37.2

Hi, While reviewing kvx-mmu.rst I've spotted several places where the documentation does not match the code, please make sure that the documentation in this patch actually reflects what the code is doing. Also, sectioning looked a bit strange to me, make sure to take a look at the generated html and see if it is rendered as intended. On Fri, Jan 20, 2023 at 03:09:32PM +0100, Yann Sionneau wrote: ... There is only 64-bit support at the moment, so no need to mention it. I'd reorder this paragraph to first mention that bit 40 is set for kernel space and then write about sign extension. These two paragraphs have nothing to do with the virtual memory map. Please drop them. This is not what I see in the current patchset. Please re-add this section along with CONFIG_STRICT_KERNEL_RWX support, otherwise it's misleading. I'm failing to parse this. What will be in the same space as user? There is nothing kvx-specific in the memblock section. Please strive to keep Docimentaion/arch/kvx about architecture specific details. Please remove CONFIG_STRICT_KERNEL_RWX documentation for now. This should be a part of a patchset than enables CONFIG_STRICT_KERNEL_RWX. Previously you've mentioned that bit 40 is used to differentiate kernel and user space. Which one is correct? Shouldn't PTE format section be the same level as "3 levels page table"? -- Sincerely yours, Mike.

Hi Krzysztof, On Sun, Jan 22, 2023 at 12:44:46PM +0100, Krzysztof Kozlowski wrote: This will be fixed, sorry for the inconvenience. Ack This (per CPU) interrupt controller is not memory mapped at all, it is controlled and configured through system registers. I do not have found existing .yaml bindings for such devices, only the file snps,archs-intc.txt has something similar. I do not know what is the best way to represent such devices in the device-tree. Any suggestions are welcome. This property is not even used in our device-tree, this will be removed from the documentation and from the driver as well. I am starting over from the example-schema. As said above, this is not a memory mapped device, but is accessed through system registers. Thanks, -- Jules

From: Jules Maselbas <jmaselbas(a)kalray.eu&gt; Add documentation for `kalray,kv3-1-apic-gic` binding. Co-developed-by: Jules Maselbas <jmaselbas(a)kalray.eu&gt; Signed-off-by: Jules Maselbas <jmaselbas(a)kalray.eu&gt; Signed-off-by: Yann Sionneau <ysionneau(a)kalray.eu&gt; --- Notes: V1 -> V2: new patch .../kalray,kv3-1-apic-gic.yaml | 66 +++++++++++++++++++ 1 file changed, 66 insertions(+) create mode 100644 Documentation/devicetree/bindings/interrupt-controller/kalray,kv3-1-apic-gic.yaml diff --git a/Documentation/devicetree/bindings/interrupt-controller/kalray,kv3-1-apic-gic.yaml b/Documentation/devicetree/bindings/interrupt-controller/kalray,kv3-1-apic-gic.yaml new file mode 100644 index 000000000000..7a37f19db2fb --- /dev/null +++ b/Documentation/devicetree/bindings/interrupt-controller/kalray,kv3-1-apic-gic.yaml @@ -0,0 +1,66 @@ +# SPDX-License-Identifier: GPL-2.0 +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/interrupt-controller/kalray,kv3-1-apic-gic# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Kalray kv3-1 APIC-GIC + +description: | + Each cluster in the Coolidge SoC includes an Advanced Programmable Interrupt + Controller (APIC) which is split in two part: + - a Generic Interrupt Controller (referred as APIC-GIC) + - a Mailbox Controller (referred as APIC-Mailbox) + The APIC-GIC acts as an intermediary interrupt controller, muxing/routing + incoming interrupts to output interrupts connected to kvx cores interrupts lines. + The 139 possible input interrupt lines are organized as follow: + - 128 from the mailbox controller (one it per mailboxes) + - 1 from the NoC router + - 5 from IOMMUs + - 1 from L2 cache DMA job FIFO + - 1 from cluster watchdog + - 2 for SECC, DECC + - 1 from Data NoC + The 72 possible output interrupt lines: + - 68 : 4 interrupts per cores (17 cores) + - 1 for L2 cache controller + - 3 extra that are for padding + +allOf: + - $ref: /schemas/interrupt-controller.yaml# + +properties: + compatible: + const: kalray,kv3-1-apic-gic + "#interrupt-cells": + const: 1 + description: + The IRQ number. + interrupt-controller: true + interrupt-parent: true + interrupts: + maxItems: 4 + description: | + Specifies the interrupt line(s) in the interrupt-parent controller node; + valid values depend on the type of parent interrupt controller + +required: + - compatible + - reg + - "#interrupt-cells" + - interrupt-controller + - interrupt-parent + - interrupts + +examples: + - | + apic_gic: interrupt-controller@a20000 { + compatible = "kalray,kv3-1-apic-gic"; + reg = <0 0xa20000 0 0x12000>; + #interrupt-cells = <1>; + interrupt-controller; + interrupt-parent = <&intc>; + interrups = <4 5 6 7>; + }; + +... -- 2.37.2

From: Jules Maselbas <jmaselbas(a)kalray.eu&gt; Add documentation for `kalray,kv3-1-core-intc` binding. Co-developed-by: Jules Maselbas <jmaselbas(a)kalray.eu&gt; Signed-off-by: Jules Maselbas <jmaselbas(a)kalray.eu&gt; Signed-off-by: Yann Sionneau <ysionneau(a)kalray.eu&gt; --- Notes: V1 -> V2: new patch .../kalray,kv3-1-apic-mailbox.yaml | 75 +++++++++++++++++++ 1 file changed, 75 insertions(+) create mode 100644 Documentation/devicetree/bindings/interrupt-controller/kalray,kv3-1-apic-mailbox.yaml diff --git a/Documentation/devicetree/bindings/interrupt-controller/kalray,kv3-1-apic-mailbox.yaml b/Documentation/devicetree/bindings/interrupt-controller/kalray,kv3-1-apic-mailbox.yaml new file mode 100644 index 000000000000..e1eb1c9fda0d --- /dev/null +++ b/Documentation/devicetree/bindings/interrupt-controller/kalray,kv3-1-apic-mailbox.yaml @@ -0,0 +1,75 @@ +# SPDX-License-Identifier: GPL-2.0 +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/interrupt-controller/kalray,kv3-1-apic-mail... +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Kalray kv3-1 APIC-Mailbox + +description: | + Each cluster in the Coolidge SoC includes an Advanced Programmable Interrupt + Controller (APIC) which is split in two part: + - a Generic Interrupt Controller (referred as APIC-GIC) + - a Mailbox Controller (referred as APIC-Mailbox) + The APIC-Mailbox contains 128 mailboxes of 8 bytes (size of a word), + this hardware block is basically a 1 KB of smart memory space. + Each mailbox can be independently configured with a trigger condition + and an input mode function. + + Input mode are: + - write + - bitwise OR + - add + + Interrupts are generated on a write when the mailbox content value + match the configured trigger condition. + Available conditions are: + - doorbell: always raise interruption on write + - match: when the mailbox's value equal the configured trigger value + - barrier: same as match but the mailbox's value is cleared on trigger + - threshold: when the mailbox's value is greater than, or equal to, the + configured trigger value + + Since this hardware block generates IRQs based on writes to some memory + locations, it is both an interrupt controller and an MSI controller. + +allOf: + - $ref: /schemas/interrupt-controller.yaml# + +properties: + compatible: + const: kalray,kv3-1-apic-mailbox + "#interrupt-cells": + const: 1 + description: + The IRQ number. + interrupt-controller: true + interrupt-parent: true + interrupts: + maxItems: 128 + description: | + Specifies the interrupt line(s) in the interrupt-parent controller node; + valid values depend on the type of parent interrupt controller + msi-controller: true + +required: + - compatible + - reg + - "#interrupt-cells" + - interrupt-controller + - interrupt-parent + - interrupts + - msi-controller + +examples: + - | + apic_mailbox: interrupt-controller@a00000 { + compatible = "kalray,kv3-1-apic-gic"; + reg = <0 0xa00000 0 0x0f200>; + #interrupt-cells = <1>; + interrupt-controller; + interrupt-parent = <&apic_gic>; + interrups = <0 1 2 3 4 5 6 7 8 9>; + }; + +... -- 2.37.2

From: Jules Maselbas <jmaselbas(a)kalray.eu&gt; Add documentation for `kalray,coolidge-itgen` binding. Co-developed-by: Jules Maselbas <jmaselbas(a)kalray.eu&gt; Signed-off-by: Jules Maselbas <jmaselbas(a)kalray.eu&gt; Signed-off-by: Yann Sionneau <ysionneau(a)kalray.eu&gt; --- Notes: V1 -> V2: new patch .../kalray,coolidge-itgen.yaml | 48 +++++++++++++++++++ 1 file changed, 48 insertions(+) create mode 100644 Documentation/devicetree/bindings/interrupt-controller/kalray,coolidge-itgen.yaml diff --git a/Documentation/devicetree/bindings/interrupt-controller/kalray,coolidge-itgen.yaml b/Documentation/devicetree/bindings/interrupt-controller/kalray,coolidge-itgen.yaml new file mode 100644 index 000000000000..47b503bff1d9 --- /dev/null +++ b/Documentation/devicetree/bindings/interrupt-controller/kalray,coolidge-itgen.yaml @@ -0,0 +1,48 @@ +# SPDX-License-Identifier: GPL-2.0 +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/interrupt-controller/kalray,coolidge-itgen# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Kalray Coolidge SoC Interrupt Generator (ITGEN) + +description: | + The Interrupt Generator (ITGEN) is an interrupt controller block. + It's purpose is to convert IRQ lines coming from SoC peripherals into writes + on the AXI bus. The ITGEN intended purpose is to write into the APIC mailboxes. + +allOf: + - $ref: /schemas/interrupt-controller.yaml# + +properties: + compatible: + const: kalray,coolidge-itgen + + "#interrupt-cells": + const: 2 + description: | + - 1st cell is for the IRQ number + - 2nd cell is for the trigger type as defined dt-bindings/interrupt-controller/irq.h + + interrupt-controller: true + + msi-parent: true + +required: + - compatible + - reg + - "#interrupt-cells" + - interrupt-controller + - msi-parent + +examples: + - | + itgen: interrupt-controller@27000000 { + compatible = "kalray,coolidge-itgen"; + reg = <0 0x27000000 0 0x1104>; + #interrupt-cells = <2>; + interrupt-controller; + msi-parent = <&apic_mailbox>; + }; + +... -- 2.37.2

From: Jules Maselbas <jmaselbas(a)kalray.eu&gt; Add documentation for `kalray,kv3-1-ipi-ctrl` binding. Co-developed-by: Jules Maselbas <jmaselbas(a)kalray.eu&gt; Signed-off-by: Jules Maselbas <jmaselbas(a)kalray.eu&gt; Signed-off-by: Yann Sionneau <ysionneau(a)kalray.eu&gt; --- Notes: V1 -> V2: new patch .../kalray/kalray,kv3-1-ipi-ctrl.yaml | 44 +++++++++++++++++++ 1 file changed, 44 insertions(+) create mode 100644 Documentation/devicetree/bindings/kalray/kalray,kv3-1-ipi-ctrl.yaml diff --git a/Documentation/devicetree/bindings/kalray/kalray,kv3-1-ipi-ctrl.yaml b/Documentation/devicetree/bindings/kalray/kalray,kv3-1-ipi-ctrl.yaml new file mode 100644 index 000000000000..dc8026b12905 --- /dev/null +++ b/Documentation/devicetree/bindings/kalray/kalray,kv3-1-ipi-ctrl.yaml @@ -0,0 +1,44 @@ +# SPDX-License-Identifier: GPL-2.0 +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/kalray/kalray,kv3-1-ipi-ctrl# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Kalray kv3-1 Inter-Processor Interrupt Controller (IPI) + +description: | + The Inter-Processor Interrupt Controller (IPI) provides a fast synchronization + mechanism to the software. It exposes eight independent set of registers that + can be use to notify each processor in the cluster. + A set of registers contains two 32-bit registers: + - 17-bit interrupt control, one bit per core, raise an interrupt on write + - 17-bit mask, one per core, to enable interrupts + + Bit at offsets 0 to 15 selects cores in the cluster, respectively PE0 to PE15, + while bit at offset 16 is for the cluster Resource Manager (RM) core. + + The eight output interrupts are connected to each processor core interrupt + controller (intc). + +properties: + compatible: + const: kalray,kv3-1-ipi-ctrl + reg: + maxItems: 1 + +required: + - compatible + - reg + - interrupt-parent + - interrupts + +examples: + - | + ipi: inter-processor-interrupt@ad0000 { + compatible = "kalray,kv3-1-ipi-ctrl"; + reg = <0x00 0xad0000 0x00 0x1000>; + interrupt-parent = <&intc>; + interrupts = <24>; + }; + +... -- 2.37.2

Add ELF-related definitions for kvx, including: EM_KVX, AUDIT_ARCH_KVX and NT_KVX_TCA. Co-developed-by: Clement Leger <clement(a)clement-leger.fr&gt; Signed-off-by: Clement Leger <clement(a)clement-leger.fr&gt; Signed-off-by: Yann Sionneau <ysionneau(a)kalray.eu&gt; --- Notes: V1 -> V2: no changes include/uapi/linux/audit.h | 1 + include/uapi/linux/elf-em.h | 1 + include/uapi/linux/elf.h | 1 + 3 files changed, 3 insertions(+) diff --git a/include/uapi/linux/audit.h b/include/uapi/linux/audit.h index d676ed2b246e..4db7aa3f84c7 100644 --- a/include/uapi/linux/audit.h +++ b/include/uapi/linux/audit.h @@ -402,6 +402,7 @@ enum { #define AUDIT_ARCH_HEXAGON (EM_HEXAGON) #define AUDIT_ARCH_I386 (EM_386|__AUDIT_ARCH_LE) #define AUDIT_ARCH_IA64 (EM_IA_64|__AUDIT_ARCH_64BIT|__AUDIT_ARCH_LE) +#define AUDIT_ARCH_KVX (EM_KVX|__AUDIT_ARCH_64BIT|__AUDIT_ARCH_LE) #define AUDIT_ARCH_M32R (EM_M32R) #define AUDIT_ARCH_M68K (EM_68K) #define AUDIT_ARCH_MICROBLAZE (EM_MICROBLAZE) diff --git a/include/uapi/linux/elf-em.h b/include/uapi/linux/elf-em.h index ef38c2bc5ab7..9cc348be7f86 100644 --- a/include/uapi/linux/elf-em.h +++ b/include/uapi/linux/elf-em.h @@ -51,6 +51,7 @@ #define EM_RISCV 243 /* RISC-V */ #define EM_BPF 247 /* Linux BPF - in-kernel virtual machine */ #define EM_CSKY 252 /* C-SKY */ +#define EM_KVX 256 /* Kalray VLIW Architecture */ #define EM_LOONGARCH 258 /* LoongArch */ #define EM_FRV 0x5441 /* Fujitsu FR-V */ diff --git a/include/uapi/linux/elf.h b/include/uapi/linux/elf.h index c7b056af9ef0..49094f3be06c 100644 --- a/include/uapi/linux/elf.h +++ b/include/uapi/linux/elf.h @@ -444,6 +444,7 @@ typedef struct elf64_shdr { #define NT_LOONGARCH_LSX 0xa02 /* LoongArch Loongson SIMD Extension registers */ #define NT_LOONGARCH_LASX 0xa03 /* LoongArch Loongson Advanced SIMD Extension registers */ #define NT_LOONGARCH_LBT 0xa04 /* LoongArch Loongson Binary Translation registers */ +#define NT_KVX_TCA 0x900 /* kvx TCA registers */ /* Note types with note name "GNU" */ #define NT_GNU_PROPERTY_TYPE_0 5 -- 2.37.2

Add common headers for basic kvx support: - CPU definition - SFR (System Function Registers) - Privilege levels owners Co-developed-by: Clement Leger <clement(a)clement-leger.fr&gt; Signed-off-by: Clement Leger <clement(a)clement-leger.fr&gt; Co-developed-by: Guillaume Thouvenin <gthouvenin(a)kalray.eu&gt; Signed-off-by: Guillaume Thouvenin <gthouvenin(a)kalray.eu&gt; Co-developed-by: Julian Vetter <jvetter(a)kalray.eu&gt; Signed-off-by: Julian Vetter <jvetter(a)kalray.eu&gt; Co-developed-by: Marius Gligor <mgligor(a)kalray.eu&gt; Signed-off-by: Marius Gligor <mgligor(a)kalray.eu&gt; Co-developed-by: Yann Sionneau <ysionneau(a)kalray.eu&gt; Signed-off-by: Yann Sionneau <ysionneau(a)kalray.eu&gt; --- Notes: V1 -> V2: removed HW breakpoints arch/kvx/include/asm/privilege.h | 211 ++ arch/kvx/include/asm/processor.h | 172 + arch/kvx/include/asm/sfr.h | 107 + arch/kvx/include/asm/sfr_defs.h | 5028 ++++++++++++++++++++++++++++++ arch/kvx/include/asm/swab.h | 48 + arch/kvx/include/asm/sys_arch.h | 51 + 6 files changed, 5617 insertions(+) create mode 100644 arch/kvx/include/asm/privilege.h create mode 100644 arch/kvx/include/asm/processor.h create mode 100644 arch/kvx/include/asm/sfr.h create mode 100644 arch/kvx/include/asm/sfr_defs.h create mode 100644 arch/kvx/include/asm/swab.h create mode 100644 arch/kvx/include/asm/sys_arch.h diff --git a/arch/kvx/include/asm/privilege.h b/arch/kvx/include/asm/privilege.h new file mode 100644 index 000000000000..e639c4a541c6 --- /dev/null +++ b/arch/kvx/include/asm/privilege.h @@ -0,0 +1,211 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Yann Sionneau + * Marius Gligor + */ + +#ifndef _ASM_KVX_PRIVILEGE_H +#define _ASM_KVX_PRIVILEGE_H + +#include <asm/sys_arch.h> + +/** + * Privilege level stuff + */ + +/* + * When manipulating ring levels, we always use relative values. This means that + * settings a resource owner to value 1 means "Current privilege level + 1. + * Setting it to 0 means "Current privilege level" + */ +#define PL_CUR_PLUS_1 1 +#define PL_CUR 0 + +/** + * Syscall owner configuration + */ +#define SYO_WFXL_OWN(__field, __pl) \ + SFR_SET_VAL_WFXL(SYO, __field, __pl) + +#define SYO_WFXL_VALUE(__pl) (SYO_WFXL_OWN(Q0, __pl) | \ + SYO_WFXL_OWN(Q1, __pl) | \ + SYO_WFXL_OWN(Q2, __pl) | \ + SYO_WFXL_OWN(Q3, __pl)) + +#define SYO_WFXL_VALUE_PL_CUR_PLUS_1 SYO_WFXL_VALUE(PL_CUR_PLUS_1) +#define SYO_WFXL_VALUE_PL_CUR SYO_WFXL_VALUE(PL_CUR) + +/** + * hardware trap owner configuration + */ +#define HTO_WFXL_OWN(__field, __pl) \ + SFR_SET_VAL_WFXL(HTO, __field, __pl) + + +#define HTO_WFXL_VALUE_BASE(__pl) (HTO_WFXL_OWN(OPC, __pl) | \ + HTO_WFXL_OWN(DMIS, __pl) | \ + HTO_WFXL_OWN(PSYS, __pl) | \ + HTO_WFXL_OWN(DSYS, __pl) | \ + HTO_WFXL_OWN(NOMAP, __pl) | \ + HTO_WFXL_OWN(PROT, __pl) | \ + HTO_WFXL_OWN(W2CL, __pl) | \ + HTO_WFXL_OWN(A2CL, __pl) | \ + HTO_WFXL_OWN(VSFR, __pl) | \ + HTO_WFXL_OWN(PLO, __pl)) + +/* + * When alone on the processor, we need to request all traps or the processor + * will die badly without any information at all by jumping to the more + * privilege level even if nobody is there. + */ +#define HTO_WFXL_VALUE_PL_CUR_PLUS_1 (HTO_WFXL_VALUE_BASE(PL_CUR_PLUS_1) | \ + HTO_WFXL_OWN(DECCG, PL_CUR_PLUS_1) | \ + HTO_WFXL_OWN(SECCG, PL_CUR_PLUS_1) | \ + HTO_WFXL_OWN(DE, PL_CUR_PLUS_1)) + +#define HTO_WFXL_VALUE_PL_CUR HTO_WFXL_VALUE_BASE(PL_CUR) + +/** + * Interrupt owner configuration + */ +#define ITO_WFXL_OWN(__field, __pl) \ + SFR_SET_VAL_WFXL(ITO, __field, __pl) + +#define ITO_WFXL_VALUE(__pl) (ITO_WFXL_OWN(IT0, __pl) | \ + ITO_WFXL_OWN(IT1, __pl) | \ + ITO_WFXL_OWN(IT2, __pl) | \ + ITO_WFXL_OWN(IT3, __pl) | \ + ITO_WFXL_OWN(IT4, __pl) | \ + ITO_WFXL_OWN(IT5, __pl) | \ + ITO_WFXL_OWN(IT6, __pl) | \ + ITO_WFXL_OWN(IT7, __pl) | \ + ITO_WFXL_OWN(IT8, __pl) | \ + ITO_WFXL_OWN(IT9, __pl) | \ + ITO_WFXL_OWN(IT10, __pl) | \ + ITO_WFXL_OWN(IT11, __pl) | \ + ITO_WFXL_OWN(IT12, __pl) | \ + ITO_WFXL_OWN(IT13, __pl) | \ + ITO_WFXL_OWN(IT14, __pl) | \ + ITO_WFXL_OWN(IT15, __pl)) + +#define ITO_WFXL_VALUE_PL_CUR_PLUS_1 ITO_WFXL_VALUE(PL_CUR_PLUS_1) +#define ITO_WFXL_VALUE_PL_CUR ITO_WFXL_VALUE(PL_CUR) + +#define ITO_WFXM_OWN(__field, __pl) \ + SFR_SET_VAL_WFXM(ITO, __field, __pl) + +#define ITO_WFXM_VALUE(__pl) (ITO_WFXM_OWN(IT16, __pl) | \ + ITO_WFXM_OWN(IT17, __pl) | \ + ITO_WFXM_OWN(IT18, __pl) | \ + ITO_WFXM_OWN(IT19, __pl) | \ + ITO_WFXM_OWN(IT20, __pl) | \ + ITO_WFXM_OWN(IT21, __pl) | \ + ITO_WFXM_OWN(IT22, __pl) | \ + ITO_WFXM_OWN(IT23, __pl) | \ + ITO_WFXM_OWN(IT24, __pl) | \ + ITO_WFXM_OWN(IT25, __pl) | \ + ITO_WFXM_OWN(IT26, __pl) | \ + ITO_WFXM_OWN(IT27, __pl) | \ + ITO_WFXM_OWN(IT28, __pl) | \ + ITO_WFXM_OWN(IT29, __pl) | \ + ITO_WFXM_OWN(IT30, __pl) | \ + ITO_WFXM_OWN(IT31, __pl)) + +#define ITO_WFXM_VALUE_PL_CUR_PLUS_1 ITO_WFXM_VALUE(PL_CUR_PLUS_1) +#define ITO_WFXM_VALUE_PL_CUR ITO_WFXM_VALUE(PL_CUR) + +/** + * Debug Owner configuration + */ + +#define DO_WFXL_OWN(__field, __pl) \ + SFR_SET_VAL_WFXL(DO, __field, __pl) + +#define DO_WFXL_VALUE(__pl) (DO_WFXL_OWN(B0, __pl) | \ + DO_WFXL_OWN(B1, __pl) | \ + DO_WFXL_OWN(W0, __pl) | \ + DO_WFXL_OWN(W1, __pl)) + +#define DO_WFXL_VALUE_PL_CUR_PLUS_1 DO_WFXL_VALUE(PL_CUR_PLUS_1) +#define DO_WFXL_VALUE_PL_CUR DO_WFXL_VALUE(PL_CUR) + +/** + * Misc owner configuration + */ +#define MO_WFXL_OWN(__field, __pl) \ + SFR_SET_VAL_WFXL(MO, __field, __pl) + +#define MO_WFXL_VALUE(__pl) (MO_WFXL_OWN(MMI, __pl) | \ + MO_WFXL_OWN(RFE, __pl) | \ + MO_WFXL_OWN(STOP, __pl) | \ + MO_WFXL_OWN(SYNC, __pl) | \ + MO_WFXL_OWN(PCR, __pl) | \ + MO_WFXL_OWN(MSG, __pl) | \ + MO_WFXL_OWN(MEN, __pl) | \ + MO_WFXL_OWN(MES, __pl) | \ + MO_WFXL_OWN(CSIT, __pl) | \ + MO_WFXL_OWN(T0, __pl) | \ + MO_WFXL_OWN(T1, __pl) | \ + MO_WFXL_OWN(WD, __pl) | \ + MO_WFXL_OWN(PM0, __pl) | \ + MO_WFXL_OWN(PM1, __pl) | \ + MO_WFXL_OWN(PM2, __pl) | \ + MO_WFXL_OWN(PM3, __pl)) + +#define MO_WFXL_VALUE_PL_CUR_PLUS_1 MO_WFXL_VALUE(PL_CUR_PLUS_1) +#define MO_WFXL_VALUE_PL_CUR MO_WFXL_VALUE(PL_CUR) + +#define MO_WFXM_OWN(__field, __pl) \ + SFR_SET_VAL_WFXM(MO, __field, __pl) + +#define MO_WFXM_VALUE(__pl) (MO_WFXM_OWN(PMIT, __pl)) + +#define MO_WFXM_VALUE_PL_CUR_PLUS_1 MO_WFXM_VALUE(PL_CUR_PLUS_1) +#define MO_WFXM_VALUE_PL_CUR MO_WFXM_VALUE(PL_CUR) + +/** + * $ps owner configuration + */ +#define PSO_WFXL_OWN(__field, __pl) \ + SFR_SET_VAL_WFXL(PSO, __field, __pl) + +#define PSO_WFXL_BASE_VALUE(__pl) (PSO_WFXL_OWN(PL0, __pl) | \ + PSO_WFXL_OWN(PL1, __pl) | \ + PSO_WFXL_OWN(ET, __pl) | \ + PSO_WFXL_OWN(HTD, __pl) | \ + PSO_WFXL_OWN(IE, __pl) | \ + PSO_WFXL_OWN(HLE, __pl) | \ + PSO_WFXL_OWN(SRE, __pl) | \ + PSO_WFXL_OWN(DAUS, __pl) | \ + PSO_WFXL_OWN(ICE, __pl) | \ + PSO_WFXL_OWN(USE, __pl) | \ + PSO_WFXL_OWN(DCE, __pl) | \ + PSO_WFXL_OWN(MME, __pl) | \ + PSO_WFXL_OWN(IL0, __pl) | \ + PSO_WFXL_OWN(IL1, __pl) | \ + PSO_WFXL_OWN(VS0, __pl)) +/* Request additionnal VS1 when alone */ +#define PSO_WFXL_VALUE_PL_CUR_PLUS_1 (PSO_WFXL_BASE_VALUE(PL_CUR_PLUS_1) | \ + PSO_WFXL_OWN(VS1, PL_CUR_PLUS_1)) +#define PSO_WFXL_VALUE_PL_CUR PSO_WFXL_BASE_VALUE(PL_CUR) + +#define PSO_WFXM_OWN(__field, __pl) \ + SFR_SET_VAL_WFXM(PSO, __field, __pl) + +#define PSO_WFXM_VALUE(__pl) (PSO_WFXM_OWN(V64, __pl) | \ + PSO_WFXM_OWN(L2E, __pl) | \ + PSO_WFXM_OWN(SME, __pl) | \ + PSO_WFXM_OWN(SMR, __pl) | \ + PSO_WFXM_OWN(PMJ0, __pl) | \ + PSO_WFXM_OWN(PMJ1, __pl) | \ + PSO_WFXM_OWN(PMJ2, __pl) | \ + PSO_WFXM_OWN(PMJ3, __pl) | \ + PSO_WFXM_OWN(MMUP, __pl)) + +/* Request additionnal VS1 */ +#define PSO_WFXM_VALUE_PL_CUR_PLUS_1 PSO_WFXM_VALUE(PL_CUR_PLUS_1) +#define PSO_WFXM_VALUE_PL_CUR PSO_WFXM_VALUE(PL_CUR) + +#endif /* _ASM_KVX_PRIVILEGE_H */ diff --git a/arch/kvx/include/asm/processor.h b/arch/kvx/include/asm/processor.h new file mode 100644 index 000000000000..76321ebe00db --- /dev/null +++ b/arch/kvx/include/asm/processor.h @@ -0,0 +1,172 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Guillaume Thouvenin + * Marius Gligor + */ + +#ifndef _ASM_KVX_PROCESSOR_H +#define _ASM_KVX_PROCESSOR_H + +#include <asm/mmu.h> +#include <asm/types.h> +#include <asm/ptrace.h> +#include <asm/sfr_defs.h> + +#define ARCH_HAS_PREFETCH +#define ARCH_HAS_PREFETCHW + +static inline void prefetch(const void *x) +{ + __builtin_prefetch(x); +} + +static inline void prefetchw(const void *x) +{ + __builtin_prefetch(x, 1); +} + +#define TASK_SIZE _BITULL(MMU_USR_ADDR_BITS) +#define TASK_SIZE_MAX TASK_SIZE + +/* + * This decides where the kernel will search for a free chunk of vm + * space during mmap's. + */ +#define TASK_UNMAPPED_BASE PAGE_ALIGN(TASK_SIZE >> 1) + +#define STACK_TOP TASK_SIZE +#define STACK_TOP_MAX STACK_TOP + +/* Stack alignment constant */ +#define STACK_ALIGNMENT 32 +#define STACK_ALIGN_MASK (STACK_ALIGNMENT - 1) + +#define cpu_relax() barrier() + +/* Size for register saving area for refill handler (enough for 3 quad regs) */ +#define SAVE_AREA_SIZE 12 + +#define TCA_REG_COUNT 48 + +/* TCA registers are 256 bits wide */ +struct tca_reg { + uint64_t x; + uint64_t y; + uint64_t z; + uint64_t t; +}; + +/** + * According to kvx ABI, the following registers are callee-saved: + * fp (r14) r18 r19 r20 r21 r22 r23 r24 r25 r26 r27 r28 r29 r30 r31. + * In order to switch from a task to another, we only need to save these + * registers + sp (r12) and ra + * + * WARNING WARNING WARNING WARNING WARNING WARNING WARNING WARNING + * + * Do not reorder the following fields ! + * They are used in asm-offset for store octuples so they must be + * all right behind each other + */ +struct ctx_switch_regs { + + uint64_t fp; + + uint64_t ra; /* Return address */ + uint64_t sp; + uint64_t r18; + uint64_t r19; + + uint64_t r20; + uint64_t r21; + uint64_t r22; + uint64_t r23; + + uint64_t r24; + uint64_t r25; + uint64_t r26; + uint64_t r27; + + uint64_t r28; + uint64_t r29; + uint64_t r30; + uint64_t r31; + +#ifdef CONFIG_ENABLE_TCA + struct tca_reg tca_regs[TCA_REG_COUNT]; + uint8_t tca_regs_saved; +#endif +}; + +struct debug_info { +}; + +struct thread_struct { + uint64_t kernel_sp; + uint64_t save_area[SAVE_AREA_SIZE]; /* regs save area */ + +#ifdef CONFIG_KVX_MMU_STATS + uint64_t trap_entry_ts; +#endif + /* Context switch related registers */ + struct ctx_switch_regs ctx_switch; + + /* debugging */ + struct debug_info debug; +} __packed; + +#define INIT_THREAD { \ + .ctx_switch.sp = \ + sizeof(init_stack) + (unsigned long) &init_stack, \ +} + +#define KSTK_ESP(tsk) (task_pt_regs(tsk)->sp) +#define KSTK_EIP(tsk) (task_pt_regs(tsk)->spc) + +#define task_pt_regs(p) \ + ((struct pt_regs *)(task_stack_page(p) + THREAD_SIZE) - 1) + +#define thread_saved_reg(__tsk, __reg) \ + ((unsigned long) ((__tsk)->thread.ctx_switch.__reg)) + +void release_thread(struct task_struct *t); + +void start_thread(struct pt_regs *regs, unsigned long pc, unsigned long sp); + +unsigned long __get_wchan(struct task_struct *p); + +extern void ret_from_kernel_thread(void); + +/* User return function */ +extern void ret_from_fork(void); + +static inline void wait_for_interrupt(void) +{ + __builtin_kvx_await(); + kvx_sfr_set_field(WS, WU0, 0); +} + +static inline void local_cpu_stop(void) +{ + /* Clear Wake-Up 2 to allow stop instruction to work */ + kvx_sfr_set_field(WS, WU2, 0); + __asm__ __volatile__ ( + "1: stop\n" + ";;\n" + "goto 1b\n" + ";;\n" + ); +} + +struct cpuinfo_kvx { + u64 freq; + u8 arch_rev; + u8 uarch_rev; + u8 copro_enable; +}; + +DECLARE_PER_CPU_READ_MOSTLY(struct cpuinfo_kvx, cpu_info); + +#endif /* _ASM_KVX_PROCESSOR_H */ diff --git a/arch/kvx/include/asm/sfr.h b/arch/kvx/include/asm/sfr.h new file mode 100644 index 000000000000..d91aaa335bb9 --- /dev/null +++ b/arch/kvx/include/asm/sfr.h @@ -0,0 +1,107 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Yann Sionneau + * Guillaume Thouvenin + */ + +#ifndef _ASM_KVX_SFR_H +#define _ASM_KVX_SFR_H + +#ifndef __ASSEMBLY__ + +#include <linux/types.h> + +#include <asm/sfr_defs.h> + +#define wfxl(_sfr, _val) __builtin_kvx_wfxl(_sfr, _val) + +#define wfxm(_sfr, _val) __builtin_kvx_wfxm(_sfr, _val) + +static inline void +__kvx_sfr_set_bit(unsigned char sfr, unsigned char bit) +{ + if (bit < 32) + wfxl(sfr, (uint64_t) (1 << bit) << 32); + else + wfxm(sfr, (uint64_t) 1 << bit); +} + +#define kvx_sfr_set_bit(__sfr, __bit) \ + __kvx_sfr_set_bit(KVX_SFR_ ## __sfr, __bit) + +static inline uint64_t make_sfr_val(uint64_t mask, uint64_t value) +{ + return ((value & 0xFFFFFFFF) << 32) | (mask & 0xFFFFFFFF); +} + +static inline void +__kvx_sfr_set_mask(unsigned char sfr, uint64_t mask, uint64_t value) +{ + uint64_t wf_val; + + /* Least significant bits */ + if (mask & 0xFFFFFFFF) { + wf_val = make_sfr_val(mask, value); + wfxl(sfr, wf_val); + } + + /* Most significant bits */ + if (mask & (0xFFFFFFFFULL << 32)) { + value >>= 32; + mask >>= 32; + wf_val = make_sfr_val(mask, value); + wfxm(sfr, wf_val); + } +} + +static inline u64 kvx_sfr_iget(unsigned char sfr) +{ + u64 res = sfr; + + asm volatile ("iget %0" : "+r"(res) :: ); + return res; +} + +#ifdef CONFIG_DEBUG_SFR_SET_MASK +# define kvx_sfr_set_mask(__sfr, __mask, __value) \ + do { \ + BUG_ON(((__value) & (__mask)) != (__value)); \ + __kvx_sfr_set_mask(KVX_SFR_ ## __sfr, __mask, __value); \ + } while (0) + +#else +# define kvx_sfr_set_mask(__sfr, __mask, __value) \ + __kvx_sfr_set_mask(KVX_SFR_ ## __sfr, __mask, __value) +#endif + +#define kvx_sfr_set_field(sfr, field, value) \ + kvx_sfr_set_mask(sfr, KVX_SFR_ ## sfr ## _ ## field ## _MASK, \ + ((uint64_t) (value) << KVX_SFR_ ## sfr ## _ ## field ## _SHIFT)) + +static inline void +__kvx_sfr_clear_bit(unsigned char sfr, unsigned char bit) +{ + if (bit < 32) + wfxl(sfr, (uint64_t) 1 << bit); + else + wfxm(sfr, (uint64_t) 1 << (bit - 32)); +} + +#define kvx_sfr_clear_bit(__sfr, __bit) \ + __kvx_sfr_clear_bit(KVX_SFR_ ## __sfr, __bit) + +#define kvx_sfr_set(_sfr, _val) __builtin_kvx_set(KVX_SFR_ ## _sfr, _val) +#define kvx_sfr_get(_sfr) __builtin_kvx_get(KVX_SFR_ ## _sfr) + +#define kvx_sfr_field_val(_val, _sfr, _field) \ + (((_val) & KVX_SFR_ ## _sfr ## _ ## _field ## _MASK) \ + >> KVX_SFR_ ## _sfr ## _ ## _field ## _SHIFT) + +#define kvx_sfr_bit(_sfr, _field) \ + BIT_ULL(KVX_SFR_ ## _sfr ## _ ## _field ## _SHIFT) + +#endif + +#endif /* _ASM_KVX_SFR_DEFS_H */ diff --git a/arch/kvx/include/asm/sfr_defs.h b/arch/kvx/include/asm/sfr_defs.h new file mode 100644 index 000000000000..7e6318348b83 --- /dev/null +++ b/arch/kvx/include/asm/sfr_defs.h @@ -0,0 +1,5028 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_SFR_DEFS_H +#define _ASM_KVX_SFR_DEFS_H + +#include <linux/const.h> + +/* Register file indices */ +#define KVX_SFR_PC 0 /* Program Counter $pc $s0 */ +#define KVX_SFR_PS 1 /* Processor State $ps $s1 */ +#define KVX_SFR_PCR 2 /* Processing Identification $pcr $s2 */ +#define KVX_SFR_RA 3 /* Return Address $ra $s3 */ +#define KVX_SFR_CS 4 /* Compute Status $cs $s4 */ +#define KVX_SFR_CSIT 5 /* Compute Status arithmetic interrupt $csit $s5 */ +#define KVX_SFR_AESPC 6 /* Arithmetic Exception Saved PC $aespc $s6 */ +#define KVX_SFR_LS 7 /* Loop Start Address $ls $s7 */ +#define KVX_SFR_LE 8 /* Loop Exit Address $le $s8 */ +#define KVX_SFR_LC 9 /* Loop Counter $lc $s9 */ +#define KVX_SFR_IPE 10 /* Inter Processor Event $ipe $s10 */ +#define KVX_SFR_MEN 11 /* Misc External Notifications $men $s11 */ +#define KVX_SFR_PMC 12 /* Performance Monitor Control $pmc $s12 */ +#define KVX_SFR_PM0 13 /* Performance Monitor 0 $pm0 $s13 */ +#define KVX_SFR_PM1 14 /* Performance Monitor 1 $pm1 $s14 */ +#define KVX_SFR_PM2 15 /* Performance Monitor 2 $pm2 $s15 */ +#define KVX_SFR_PM3 16 /* Performance Monitor 3 $pm3 $s16 */ +#define KVX_SFR_PMSA 17 /* Performance Monitor Saved Address $pmsa $s17 */ +#define KVX_SFR_TCR 18 /* Timer Control $tcr $s18 */ +#define KVX_SFR_T0V 19 /* Timer 0 value $t0v $s19 */ +#define KVX_SFR_T1V 20 /* Timer 1 value $t1v $s20 */ +#define KVX_SFR_T0R 21 /* Timer 0 reload value $t0r $s21 */ +#define KVX_SFR_T1R 22 /* Timer 1 reload value $t1r $s22 */ +#define KVX_SFR_WDV 23 /* Watchdog Value $wdv $s23 */ +#define KVX_SFR_WDR 24 /* Watchdog Reload $wdr $s24 */ +#define KVX_SFR_ILE 25 /* Interrupt Line Enable $ile $s25 */ +#define KVX_SFR_ILL 26 /* Interrupt Line Level $ill $s26 */ +#define KVX_SFR_ILR 27 /* Interrupt Line Request $ilr $s27 */ +#define KVX_SFR_MMC 28 /* Memory Management Control $mmc $s28 */ +#define KVX_SFR_TEL 29 /* TLB Entry Low $tel $s29 */ +#define KVX_SFR_TEH 30 /* TLB Entry High $teh $s30 */ +#define KVX_SFR_SYO 32 /* Syscall Owners $syo $s32 */ +#define KVX_SFR_HTO 33 /* Hardware Trap Owners $hto $s33 */ +#define KVX_SFR_ITO 34 /* Interrupt Owners $ito $s34 */ +#define KVX_SFR_DO 35 /* Debug Owners $do $s35 */ +#define KVX_SFR_MO 36 /* Miscellaneous Owners $mo $s36 */ +#define KVX_SFR_PSO 37 /* PS register fields Owners $pso $s37 */ +#define KVX_SFR_DC 40 /* OCE (Debug) Control $dc $s40 */ +#define KVX_SFR_DBA0 41 /* Breakpoint Address 0 $dba0 $s41 */ +#define KVX_SFR_DBA1 42 /* Breakpoint Address 1 $dba1 $s42 */ +#define KVX_SFR_DWA0 43 /* Watchpoint Address 0 $dwa0 $s43 */ +#define KVX_SFR_DWA1 44 /* Watchpoint Address 1 $dwa1 $s44 */ +#define KVX_SFR_MES 45 /* Memory Error Status $mes $s45 */ +#define KVX_SFR_WS 46 /* Wake-Up Status $ws $s46 */ +#define KVX_SFR_SPC_PL0 64 /* Shadow PC PL 0 $spc_pl0 $s64 */ +#define KVX_SFR_SPC_PL1 65 /* Shadow PC PL 1 $spc_pl1 $s65 */ +#define KVX_SFR_SPC_PL2 66 /* Shadow PC PL 2 $spc_pl2 $s66 */ +#define KVX_SFR_SPC_PL3 67 /* Shadow PC PL 3 $spc_pl3 $s67 */ +#define KVX_SFR_SPS_PL0 68 /* Shadow PS PL 0 $sps_pl0 $s68 */ +#define KVX_SFR_SPS_PL1 69 /* Shadow PS PL 1 $sps_pl1 $s69 */ +#define KVX_SFR_SPS_PL2 70 /* Shadow PS PL 2 $sps_pl2 $s70 */ +#define KVX_SFR_SPS_PL3 71 /* Shadow PS PL 3 $sps_pl3 $s71 */ +#define KVX_SFR_EA_PL0 72 /* Effective Address PL0 $ea_pl0 $s72 */ +#define KVX_SFR_EA_PL1 73 /* Effective Address PL1 $ea_pl1 $s73 */ +#define KVX_SFR_EA_PL2 74 /* Effective Address PL2 $ea_pl2 $s74 */ +#define KVX_SFR_EA_PL3 75 /* Effective Address PL3 $ea_pl3 $s75 */ +#define KVX_SFR_EV_PL0 76 /* Exception Vector PL 0 $ev_pl0 $s76 */ +#define KVX_SFR_EV_PL1 77 /* Exception Vector PL 1 $ev_pl1 $s77 */ +#define KVX_SFR_EV_PL2 78 /* Exception Vector PL 2 $ev_pl2 $s78 */ +#define KVX_SFR_EV_PL3 79 /* Exception Vector PL 3 $ev_pl3 $s79 */ +#define KVX_SFR_SR_PL0 80 /* System Register PL 0 $sr_pl0 $s80 */ +#define KVX_SFR_SR_PL1 81 /* System Register PL 1 $sr_pl1 $s81 */ +#define KVX_SFR_SR_PL2 82 /* System Register PL 2 $sr_pl2 $s82 */ +#define KVX_SFR_SR_PL3 83 /* System Register PL 3 $sr_pl3 $s83 */ +#define KVX_SFR_ES_PL0 84 /* Exception Syndrome PL 0 $es_pl0 $s84 */ +#define KVX_SFR_ES_PL1 85 /* Exception Syndrome PL 1 $es_pl1 $s85 */ +#define KVX_SFR_ES_PL2 86 /* Exception Syndrome PL 2 $es_pl2 $s86 */ +#define KVX_SFR_ES_PL3 87 /* Exception Syndrome PL 3 $es_pl3 $s87 */ +#define KVX_SFR_SYOW 96 /* Alias to SYO register $syow $s96 */ +#define KVX_SFR_HTOW 97 /* Alias to HTO register $htow $s97 */ +#define KVX_SFR_ITOW 98 /* Alias to ITO register $itow $s98 */ +#define KVX_SFR_DOW 99 /* Alias to DO register $dow $s99 */ +#define KVX_SFR_MOW 100 /* Alias to MO register $mow $s100 */ +#define KVX_SFR_PSOW 101 /* Alias to PSO register $psow $s101 */ +#define KVX_SFR_SPC 128 /* Shadow PC alias on SPC_PL<i> $spc $s128 */ +#define KVX_SFR_SPS 132 /* Shadow PS alias on PS_PL<i> $sps $s132 */ +#define KVX_SFR_EA 136 /* Effective Address alias on EA_PL<i> $ea $s136 */ +#define KVX_SFR_EV 140 /* Exception Vector alias on EV_PL<i> $ev $s140 */ +#define KVX_SFR_SR 144 /* System Register alias on SR_PL<i> $sr $s144 */ +#define KVX_SFR_ES 148 /* Exception Syndrome alias on ES_PL<i> $es $s148 */ +#define KVX_SFR_VSFR0 256 /* Virtual SFR 0 $vsfr0 $s256 */ +#define KVX_SFR_VSFR1 257 /* Virtual SFR 1 $vsfr1 $s257 */ +#define KVX_SFR_VSFR2 258 /* Virtual SFR 2 $vsfr2 $s258 */ +#define KVX_SFR_VSFR3 259 /* Virtual SFR 3 $vsfr3 $s259 */ +#define KVX_SFR_VSFR4 260 /* Virtual SFR 4 $vsfr4 $s260 */ +#define KVX_SFR_VSFR5 261 /* Virtual SFR 5 $vsfr5 $s261 */ +#define KVX_SFR_VSFR6 262 /* Virtual SFR 6 $vsfr6 $s262 */ +#define KVX_SFR_VSFR7 263 /* Virtual SFR 7 $vsfr7 $s263 */ +#define KVX_SFR_VSFR8 264 /* Virtual SFR 8 $vsfr8 $s264 */ +#define KVX_SFR_VSFR9 265 /* Virtual SFR 9 $vsfr9 $s265 */ +#define KVX_SFR_VSFR10 266 /* Virtual SFR 10 $vsfr10 $s266 */ +#define KVX_SFR_VSFR11 267 /* Virtual SFR 11 $vsfr11 $s267 */ +#define KVX_SFR_VSFR12 268 /* Virtual SFR 12 $vsfr12 $s268 */ +#define KVX_SFR_VSFR13 269 /* Virtual SFR 13 $vsfr13 $s269 */ +#define KVX_SFR_VSFR14 270 /* Virtual SFR 14 $vsfr14 $s270 */ +#define KVX_SFR_VSFR15 271 /* Virtual SFR 15 $vsfr15 $s271 */ +#define KVX_SFR_VSFR16 272 /* Virtual SFR 16 $vsfr16 $s272 */ +#define KVX_SFR_VSFR17 273 /* Virtual SFR 17 $vsfr17 $s273 */ +#define KVX_SFR_VSFR18 274 /* Virtual SFR 18 $vsfr18 $s274 */ +#define KVX_SFR_VSFR19 275 /* Virtual SFR 19 $vsfr19 $s275 */ +#define KVX_SFR_VSFR20 276 /* Virtual SFR 20 $vsfr20 $s276 */ +#define KVX_SFR_VSFR21 277 /* Virtual SFR 21 $vsfr21 $s277 */ +#define KVX_SFR_VSFR22 278 /* Virtual SFR 22 $vsfr22 $s278 */ +#define KVX_SFR_VSFR23 279 /* Virtual SFR 23 $vsfr23 $s279 */ +#define KVX_SFR_VSFR24 280 /* Virtual SFR 24 $vsfr24 $s280 */ +#define KVX_SFR_VSFR25 281 /* Virtual SFR 25 $vsfr25 $s281 */ +#define KVX_SFR_VSFR26 282 /* Virtual SFR 26 $vsfr26 $s282 */ +#define KVX_SFR_VSFR27 283 /* Virtual SFR 27 $vsfr27 $s283 */ +#define KVX_SFR_VSFR28 284 /* Virtual SFR 28 $vsfr28 $s284 */ +#define KVX_SFR_VSFR29 285 /* Virtual SFR 29 $vsfr29 $s285 */ +#define KVX_SFR_VSFR30 286 /* Virtual SFR 30 $vsfr30 $s286 */ +#define KVX_SFR_VSFR31 287 /* Virtual SFR 31 $vsfr31 $s287 */ +#define KVX_SFR_VSFR32 288 /* Virtual SFR 32 $vsfr32 $s288 */ +#define KVX_SFR_VSFR33 289 /* Virtual SFR 33 $vsfr33 $s289 */ +#define KVX_SFR_VSFR34 290 /* Virtual SFR 34 $vsfr34 $s290 */ +#define KVX_SFR_VSFR35 291 /* Virtual SFR 35 $vsfr35 $s291 */ +#define KVX_SFR_VSFR36 292 /* Virtual SFR 36 $vsfr36 $s292 */ +#define KVX_SFR_VSFR37 293 /* Virtual SFR 37 $vsfr37 $s293 */ +#define KVX_SFR_VSFR38 294 /* Virtual SFR 38 $vsfr38 $s294 */ +#define KVX_SFR_VSFR39 295 /* Virtual SFR 39 $vsfr39 $s295 */ +#define KVX_SFR_VSFR40 296 /* Virtual SFR 40 $vsfr40 $s296 */ +#define KVX_SFR_VSFR41 297 /* Virtual SFR 41 $vsfr41 $s297 */ +#define KVX_SFR_VSFR42 298 /* Virtual SFR 42 $vsfr42 $s298 */ +#define KVX_SFR_VSFR43 299 /* Virtual SFR 43 $vsfr43 $s299 */ +#define KVX_SFR_VSFR44 300 /* Virtual SFR 44 $vsfr44 $s300 */ +#define KVX_SFR_VSFR45 301 /* Virtual SFR 45 $vsfr45 $s301 */ +#define KVX_SFR_VSFR46 302 /* Virtual SFR 46 $vsfr46 $s302 */ +#define KVX_SFR_VSFR47 303 /* Virtual SFR 47 $vsfr47 $s303 */ +#define KVX_SFR_VSFR48 304 /* Virtual SFR 48 $vsfr48 $s304 */ +#define KVX_SFR_VSFR49 305 /* Virtual SFR 49 $vsfr49 $s305 */ +#define KVX_SFR_VSFR50 306 /* Virtual SFR 50 $vsfr50 $s306 */ +#define KVX_SFR_VSFR51 307 /* Virtual SFR 51 $vsfr51 $s307 */ +#define KVX_SFR_VSFR52 308 /* Virtual SFR 52 $vsfr52 $s308 */ +#define KVX_SFR_VSFR53 309 /* Virtual SFR 53 $vsfr53 $s309 */ +#define KVX_SFR_VSFR54 310 /* Virtual SFR 54 $vsfr54 $s310 */ +#define KVX_SFR_VSFR55 311 /* Virtual SFR 55 $vsfr55 $s311 */ +#define KVX_SFR_VSFR56 312 /* Virtual SFR 56 $vsfr56 $s312 */ +#define KVX_SFR_VSFR57 313 /* Virtual SFR 57 $vsfr57 $s313 */ +#define KVX_SFR_VSFR58 314 /* Virtual SFR 58 $vsfr58 $s314 */ +#define KVX_SFR_VSFR59 315 /* Virtual SFR 59 $vsfr59 $s315 */ +#define KVX_SFR_VSFR60 316 /* Virtual SFR 60 $vsfr60 $s316 */ +#define KVX_SFR_VSFR61 317 /* Virtual SFR 61 $vsfr61 $s317 */ +#define KVX_SFR_VSFR62 318 /* Virtual SFR 62 $vsfr62 $s318 */ +#define KVX_SFR_VSFR63 319 /* Virtual SFR 63 $vsfr63 $s319 */ +#define KVX_SFR_VSFR64 320 /* Virtual SFR 64 $vsfr64 $s320 */ +#define KVX_SFR_VSFR65 321 /* Virtual SFR 65 $vsfr65 $s321 */ +#define KVX_SFR_VSFR66 322 /* Virtual SFR 66 $vsfr66 $s322 */ +#define KVX_SFR_VSFR67 323 /* Virtual SFR 67 $vsfr67 $s323 */ +#define KVX_SFR_VSFR68 324 /* Virtual SFR 68 $vsfr68 $s324 */ +#define KVX_SFR_VSFR69 325 /* Virtual SFR 69 $vsfr69 $s325 */ +#define KVX_SFR_VSFR70 326 /* Virtual SFR 70 $vsfr70 $s326 */ +#define KVX_SFR_VSFR71 327 /* Virtual SFR 71 $vsfr71 $s327 */ +#define KVX_SFR_VSFR72 328 /* Virtual SFR 72 $vsfr72 $s328 */ +#define KVX_SFR_VSFR73 329 /* Virtual SFR 73 $vsfr73 $s329 */ +#define KVX_SFR_VSFR74 330 /* Virtual SFR 74 $vsfr74 $s330 */ +#define KVX_SFR_VSFR75 331 /* Virtual SFR 75 $vsfr75 $s331 */ +#define KVX_SFR_VSFR76 332 /* Virtual SFR 76 $vsfr76 $s332 */ +#define KVX_SFR_VSFR77 333 /* Virtual SFR 77 $vsfr77 $s333 */ +#define KVX_SFR_VSFR78 334 /* Virtual SFR 78 $vsfr78 $s334 */ +#define KVX_SFR_VSFR79 335 /* Virtual SFR 79 $vsfr79 $s335 */ +#define KVX_SFR_VSFR80 336 /* Virtual SFR 80 $vsfr80 $s336 */ +#define KVX_SFR_VSFR81 337 /* Virtual SFR 81 $vsfr81 $s337 */ +#define KVX_SFR_VSFR82 338 /* Virtual SFR 82 $vsfr82 $s338 */ +#define KVX_SFR_VSFR83 339 /* Virtual SFR 83 $vsfr83 $s339 */ +#define KVX_SFR_VSFR84 340 /* Virtual SFR 84 $vsfr84 $s340 */ +#define KVX_SFR_VSFR85 341 /* Virtual SFR 85 $vsfr85 $s341 */ +#define KVX_SFR_VSFR86 342 /* Virtual SFR 86 $vsfr86 $s342 */ +#define KVX_SFR_VSFR87 343 /* Virtual SFR 87 $vsfr87 $s343 */ +#define KVX_SFR_VSFR88 344 /* Virtual SFR 88 $vsfr88 $s344 */ +#define KVX_SFR_VSFR89 345 /* Virtual SFR 89 $vsfr89 $s345 */ +#define KVX_SFR_VSFR90 346 /* Virtual SFR 90 $vsfr90 $s346 */ +#define KVX_SFR_VSFR91 347 /* Virtual SFR 91 $vsfr91 $s347 */ +#define KVX_SFR_VSFR92 348 /* Virtual SFR 92 $vsfr92 $s348 */ +#define KVX_SFR_VSFR93 349 /* Virtual SFR 93 $vsfr93 $s349 */ +#define KVX_SFR_VSFR94 350 /* Virtual SFR 94 $vsfr94 $s350 */ +#define KVX_SFR_VSFR95 351 /* Virtual SFR 95 $vsfr95 $s351 */ +#define KVX_SFR_VSFR96 352 /* Virtual SFR 96 $vsfr96 $s352 */ +#define KVX_SFR_VSFR97 353 /* Virtual SFR 97 $vsfr97 $s353 */ +#define KVX_SFR_VSFR98 354 /* Virtual SFR 98 $vsfr98 $s354 */ +#define KVX_SFR_VSFR99 355 /* Virtual SFR 99 $vsfr99 $s355 */ +#define KVX_SFR_VSFR100 356 /* Virtual SFR 100 $vsfr100 $s356 */ +#define KVX_SFR_VSFR101 357 /* Virtual SFR 101 $vsfr101 $s357 */ +#define KVX_SFR_VSFR102 358 /* Virtual SFR 102 $vsfr102 $s358 */ +#define KVX_SFR_VSFR103 359 /* Virtual SFR 103 $vsfr103 $s359 */ +#define KVX_SFR_VSFR104 360 /* Virtual SFR 104 $vsfr104 $s360 */ +#define KVX_SFR_VSFR105 361 /* Virtual SFR 105 $vsfr105 $s361 */ +#define KVX_SFR_VSFR106 362 /* Virtual SFR 106 $vsfr106 $s362 */ +#define KVX_SFR_VSFR107 363 /* Virtual SFR 107 $vsfr107 $s363 */ +#define KVX_SFR_VSFR108 364 /* Virtual SFR 108 $vsfr108 $s364 */ +#define KVX_SFR_VSFR109 365 /* Virtual SFR 109 $vsfr109 $s365 */ +#define KVX_SFR_VSFR110 366 /* Virtual SFR 110 $vsfr110 $s366 */ +#define KVX_SFR_VSFR111 367 /* Virtual SFR 111 $vsfr111 $s367 */ +#define KVX_SFR_VSFR112 368 /* Virtual SFR 112 $vsfr112 $s368 */ +#define KVX_SFR_VSFR113 369 /* Virtual SFR 113 $vsfr113 $s369 */ +#define KVX_SFR_VSFR114 370 /* Virtual SFR 114 $vsfr114 $s370 */ +#define KVX_SFR_VSFR115 371 /* Virtual SFR 115 $vsfr115 $s371 */ +#define KVX_SFR_VSFR116 372 /* Virtual SFR 116 $vsfr116 $s372 */ +#define KVX_SFR_VSFR117 373 /* Virtual SFR 117 $vsfr117 $s373 */ +#define KVX_SFR_VSFR118 374 /* Virtual SFR 118 $vsfr118 $s374 */ +#define KVX_SFR_VSFR119 375 /* Virtual SFR 119 $vsfr119 $s375 */ +#define KVX_SFR_VSFR120 376 /* Virtual SFR 120 $vsfr120 $s376 */ +#define KVX_SFR_VSFR121 377 /* Virtual SFR 121 $vsfr121 $s377 */ +#define KVX_SFR_VSFR122 378 /* Virtual SFR 122 $vsfr122 $s378 */ +#define KVX_SFR_VSFR123 379 /* Virtual SFR 123 $vsfr123 $s379 */ +#define KVX_SFR_VSFR124 380 /* Virtual SFR 124 $vsfr124 $s380 */ +#define KVX_SFR_VSFR125 381 /* Virtual SFR 125 $vsfr125 $s381 */ +#define KVX_SFR_VSFR126 382 /* Virtual SFR 126 $vsfr126 $s382 */ +#define KVX_SFR_VSFR127 383 /* Virtual SFR 127 $vsfr127 $s383 */ +#define KVX_SFR_VSFR128 384 /* Virtual SFR 128 $vsfr128 $s384 */ +#define KVX_SFR_VSFR129 385 /* Virtual SFR 129 $vsfr129 $s385 */ +#define KVX_SFR_VSFR130 386 /* Virtual SFR 130 $vsfr130 $s386 */ +#define KVX_SFR_VSFR131 387 /* Virtual SFR 131 $vsfr131 $s387 */ +#define KVX_SFR_VSFR132 388 /* Virtual SFR 132 $vsfr132 $s388 */ +#define KVX_SFR_VSFR133 389 /* Virtual SFR 133 $vsfr133 $s389 */ +#define KVX_SFR_VSFR134 390 /* Virtual SFR 134 $vsfr134 $s390 */ +#define KVX_SFR_VSFR135 391 /* Virtual SFR 135 $vsfr135 $s391 */ +#define KVX_SFR_VSFR136 392 /* Virtual SFR 136 $vsfr136 $s392 */ +#define KVX_SFR_VSFR137 393 /* Virtual SFR 137 $vsfr137 $s393 */ +#define KVX_SFR_VSFR138 394 /* Virtual SFR 138 $vsfr138 $s394 */ +#define KVX_SFR_VSFR139 395 /* Virtual SFR 139 $vsfr139 $s395 */ +#define KVX_SFR_VSFR140 396 /* Virtual SFR 140 $vsfr140 $s396 */ +#define KVX_SFR_VSFR141 397 /* Virtual SFR 141 $vsfr141 $s397 */ +#define KVX_SFR_VSFR142 398 /* Virtual SFR 142 $vsfr142 $s398 */ +#define KVX_SFR_VSFR143 399 /* Virtual SFR 143 $vsfr143 $s399 */ +#define KVX_SFR_VSFR144 400 /* Virtual SFR 144 $vsfr144 $s400 */ +#define KVX_SFR_VSFR145 401 /* Virtual SFR 145 $vsfr145 $s401 */ +#define KVX_SFR_VSFR146 402 /* Virtual SFR 146 $vsfr146 $s402 */ +#define KVX_SFR_VSFR147 403 /* Virtual SFR 147 $vsfr147 $s403 */ +#define KVX_SFR_VSFR148 404 /* Virtual SFR 148 $vsfr148 $s404 */ +#define KVX_SFR_VSFR149 405 /* Virtual SFR 149 $vsfr149 $s405 */ +#define KVX_SFR_VSFR150 406 /* Virtual SFR 150 $vsfr150 $s406 */ +#define KVX_SFR_VSFR151 407 /* Virtual SFR 151 $vsfr151 $s407 */ +#define KVX_SFR_VSFR152 408 /* Virtual SFR 152 $vsfr152 $s408 */ +#define KVX_SFR_VSFR153 409 /* Virtual SFR 153 $vsfr153 $s409 */ +#define KVX_SFR_VSFR154 410 /* Virtual SFR 154 $vsfr154 $s410 */ +#define KVX_SFR_VSFR155 411 /* Virtual SFR 155 $vsfr155 $s411 */ +#define KVX_SFR_VSFR156 412 /* Virtual SFR 156 $vsfr156 $s412 */ +#define KVX_SFR_VSFR157 413 /* Virtual SFR 157 $vsfr157 $s413 */ +#define KVX_SFR_VSFR158 414 /* Virtual SFR 158 $vsfr158 $s414 */ +#define KVX_SFR_VSFR159 415 /* Virtual SFR 159 $vsfr159 $s415 */ +#define KVX_SFR_VSFR160 416 /* Virtual SFR 160 $vsfr160 $s416 */ +#define KVX_SFR_VSFR161 417 /* Virtual SFR 161 $vsfr161 $s417 */ +#define KVX_SFR_VSFR162 418 /* Virtual SFR 162 $vsfr162 $s418 */ +#define KVX_SFR_VSFR163 419 /* Virtual SFR 163 $vsfr163 $s419 */ +#define KVX_SFR_VSFR164 420 /* Virtual SFR 164 $vsfr164 $s420 */ +#define KVX_SFR_VSFR165 421 /* Virtual SFR 165 $vsfr165 $s421 */ +#define KVX_SFR_VSFR166 422 /* Virtual SFR 166 $vsfr166 $s422 */ +#define KVX_SFR_VSFR167 423 /* Virtual SFR 167 $vsfr167 $s423 */ +#define KVX_SFR_VSFR168 424 /* Virtual SFR 168 $vsfr168 $s424 */ +#define KVX_SFR_VSFR169 425 /* Virtual SFR 169 $vsfr169 $s425 */ +#define KVX_SFR_VSFR170 426 /* Virtual SFR 170 $vsfr170 $s426 */ +#define KVX_SFR_VSFR171 427 /* Virtual SFR 171 $vsfr171 $s427 */ +#define KVX_SFR_VSFR172 428 /* Virtual SFR 172 $vsfr172 $s428 */ +#define KVX_SFR_VSFR173 429 /* Virtual SFR 173 $vsfr173 $s429 */ +#define KVX_SFR_VSFR174 430 /* Virtual SFR 174 $vsfr174 $s430 */ +#define KVX_SFR_VSFR175 431 /* Virtual SFR 175 $vsfr175 $s431 */ +#define KVX_SFR_VSFR176 432 /* Virtual SFR 176 $vsfr176 $s432 */ +#define KVX_SFR_VSFR177 433 /* Virtual SFR 177 $vsfr177 $s433 */ +#define KVX_SFR_VSFR178 434 /* Virtual SFR 178 $vsfr178 $s434 */ +#define KVX_SFR_VSFR179 435 /* Virtual SFR 179 $vsfr179 $s435 */ +#define KVX_SFR_VSFR180 436 /* Virtual SFR 180 $vsfr180 $s436 */ +#define KVX_SFR_VSFR181 437 /* Virtual SFR 181 $vsfr181 $s437 */ +#define KVX_SFR_VSFR182 438 /* Virtual SFR 182 $vsfr182 $s438 */ +#define KVX_SFR_VSFR183 439 /* Virtual SFR 183 $vsfr183 $s439 */ +#define KVX_SFR_VSFR184 440 /* Virtual SFR 184 $vsfr184 $s440 */ +#define KVX_SFR_VSFR185 441 /* Virtual SFR 185 $vsfr185 $s441 */ +#define KVX_SFR_VSFR186 442 /* Virtual SFR 186 $vsfr186 $s442 */ +#define KVX_SFR_VSFR187 443 /* Virtual SFR 187 $vsfr187 $s443 */ +#define KVX_SFR_VSFR188 444 /* Virtual SFR 188 $vsfr188 $s444 */ +#define KVX_SFR_VSFR189 445 /* Virtual SFR 189 $vsfr189 $s445 */ +#define KVX_SFR_VSFR190 446 /* Virtual SFR 190 $vsfr190 $s446 */ +#define KVX_SFR_VSFR191 447 /* Virtual SFR 191 $vsfr191 $s447 */ +#define KVX_SFR_VSFR192 448 /* Virtual SFR 192 $vsfr192 $s448 */ +#define KVX_SFR_VSFR193 449 /* Virtual SFR 193 $vsfr193 $s449 */ +#define KVX_SFR_VSFR194 450 /* Virtual SFR 194 $vsfr194 $s450 */ +#define KVX_SFR_VSFR195 451 /* Virtual SFR 195 $vsfr195 $s451 */ +#define KVX_SFR_VSFR196 452 /* Virtual SFR 196 $vsfr196 $s452 */ +#define KVX_SFR_VSFR197 453 /* Virtual SFR 197 $vsfr197 $s453 */ +#define KVX_SFR_VSFR198 454 /* Virtual SFR 198 $vsfr198 $s454 */ +#define KVX_SFR_VSFR199 455 /* Virtual SFR 199 $vsfr199 $s455 */ +#define KVX_SFR_VSFR200 456 /* Virtual SFR 200 $vsfr200 $s456 */ +#define KVX_SFR_VSFR201 457 /* Virtual SFR 201 $vsfr201 $s457 */ +#define KVX_SFR_VSFR202 458 /* Virtual SFR 202 $vsfr202 $s458 */ +#define KVX_SFR_VSFR203 459 /* Virtual SFR 203 $vsfr203 $s459 */ +#define KVX_SFR_VSFR204 460 /* Virtual SFR 204 $vsfr204 $s460 */ +#define KVX_SFR_VSFR205 461 /* Virtual SFR 205 $vsfr205 $s461 */ +#define KVX_SFR_VSFR206 462 /* Virtual SFR 206 $vsfr206 $s462 */ +#define KVX_SFR_VSFR207 463 /* Virtual SFR 207 $vsfr207 $s463 */ +#define KVX_SFR_VSFR208 464 /* Virtual SFR 208 $vsfr208 $s464 */ +#define KVX_SFR_VSFR209 465 /* Virtual SFR 209 $vsfr209 $s465 */ +#define KVX_SFR_VSFR210 466 /* Virtual SFR 210 $vsfr210 $s466 */ +#define KVX_SFR_VSFR211 467 /* Virtual SFR 211 $vsfr211 $s467 */ +#define KVX_SFR_VSFR212 468 /* Virtual SFR 212 $vsfr212 $s468 */ +#define KVX_SFR_VSFR213 469 /* Virtual SFR 213 $vsfr213 $s469 */ +#define KVX_SFR_VSFR214 470 /* Virtual SFR 214 $vsfr214 $s470 */ +#define KVX_SFR_VSFR215 471 /* Virtual SFR 215 $vsfr215 $s471 */ +#define KVX_SFR_VSFR216 472 /* Virtual SFR 216 $vsfr216 $s472 */ +#define KVX_SFR_VSFR217 473 /* Virtual SFR 217 $vsfr217 $s473 */ +#define KVX_SFR_VSFR218 474 /* Virtual SFR 218 $vsfr218 $s474 */ +#define KVX_SFR_VSFR219 475 /* Virtual SFR 219 $vsfr219 $s475 */ +#define KVX_SFR_VSFR220 476 /* Virtual SFR 220 $vsfr220 $s476 */ +#define KVX_SFR_VSFR221 477 /* Virtual SFR 221 $vsfr221 $s477 */ +#define KVX_SFR_VSFR222 478 /* Virtual SFR 222 $vsfr222 $s478 */ +#define KVX_SFR_VSFR223 479 /* Virtual SFR 223 $vsfr223 $s479 */ +#define KVX_SFR_VSFR224 480 /* Virtual SFR 224 $vsfr224 $s480 */ +#define KVX_SFR_VSFR225 481 /* Virtual SFR 225 $vsfr225 $s481 */ +#define KVX_SFR_VSFR226 482 /* Virtual SFR 226 $vsfr226 $s482 */ +#define KVX_SFR_VSFR227 483 /* Virtual SFR 227 $vsfr227 $s483 */ +#define KVX_SFR_VSFR228 484 /* Virtual SFR 228 $vsfr228 $s484 */ +#define KVX_SFR_VSFR229 485 /* Virtual SFR 229 $vsfr229 $s485 */ +#define KVX_SFR_VSFR230 486 /* Virtual SFR 230 $vsfr230 $s486 */ +#define KVX_SFR_VSFR231 487 /* Virtual SFR 231 $vsfr231 $s487 */ +#define KVX_SFR_VSFR232 488 /* Virtual SFR 232 $vsfr232 $s488 */ +#define KVX_SFR_VSFR233 489 /* Virtual SFR 233 $vsfr233 $s489 */ +#define KVX_SFR_VSFR234 490 /* Virtual SFR 234 $vsfr234 $s490 */ +#define KVX_SFR_VSFR235 491 /* Virtual SFR 235 $vsfr235 $s491 */ +#define KVX_SFR_VSFR236 492 /* Virtual SFR 236 $vsfr236 $s492 */ +#define KVX_SFR_VSFR237 493 /* Virtual SFR 237 $vsfr237 $s493 */ +#define KVX_SFR_VSFR238 494 /* Virtual SFR 238 $vsfr238 $s494 */ +#define KVX_SFR_VSFR239 495 /* Virtual SFR 239 $vsfr239 $s495 */ +#define KVX_SFR_VSFR240 496 /* Virtual SFR 240 $vsfr240 $s496 */ +#define KVX_SFR_VSFR241 497 /* Virtual SFR 241 $vsfr241 $s497 */ +#define KVX_SFR_VSFR242 498 /* Virtual SFR 242 $vsfr242 $s498 */ +#define KVX_SFR_VSFR243 499 /* Virtual SFR 243 $vsfr243 $s499 */ +#define KVX_SFR_VSFR244 500 /* Virtual SFR 244 $vsfr244 $s500 */ +#define KVX_SFR_VSFR245 501 /* Virtual SFR 245 $vsfr245 $s501 */ +#define KVX_SFR_VSFR246 502 /* Virtual SFR 246 $vsfr246 $s502 */ +#define KVX_SFR_VSFR247 503 /* Virtual SFR 247 $vsfr247 $s503 */ +#define KVX_SFR_VSFR248 504 /* Virtual SFR 248 $vsfr248 $s504 */ +#define KVX_SFR_VSFR249 505 /* Virtual SFR 249 $vsfr249 $s505 */ +#define KVX_SFR_VSFR250 506 /* Virtual SFR 250 $vsfr250 $s506 */ +#define KVX_SFR_VSFR251 507 /* Virtual SFR 251 $vsfr251 $s507 */ +#define KVX_SFR_VSFR252 508 /* Virtual SFR 252 $vsfr252 $s508 */ +#define KVX_SFR_VSFR253 509 /* Virtual SFR 253 $vsfr253 $s509 */ +#define KVX_SFR_VSFR254 510 /* Virtual SFR 254 $vsfr254 $s510 */ +#define KVX_SFR_VSFR255 511 /* Virtual SFR 255 $vsfr255 $s511 */ + +/* Register field masks */ + +#define KVX_SFR_MEN_MEN_MASK _ULL(0xffff) /* Miscellaneous External Notifications */ +#define KVX_SFR_MEN_MEN_SHIFT 0 +#define KVX_SFR_MEN_MEN_WIDTH 16 +#define KVX_SFR_MEN_MEN_WFXL_MASK _ULL(0xffff) +#define KVX_SFR_MEN_MEN_WFXL_CLEAR _ULL(0xffff) +#define KVX_SFR_MEN_MEN_WFXL_SET _ULL(0xffff00000000) + +#define KVX_SFR_SYO_Q0_MASK _ULL(0x3) /* Quarter 0 syscalls 0 to 1023 owner */ +#define KVX_SFR_SYO_Q0_SHIFT 0 +#define KVX_SFR_SYO_Q0_WIDTH 2 +#define KVX_SFR_SYO_Q0_WFXL_MASK _ULL(0x3) +#define KVX_SFR_SYO_Q0_WFXL_CLEAR _ULL(0x3) +#define KVX_SFR_SYO_Q0_WFXL_SET _ULL(0x300000000) + +#define KVX_SFR_SYO_Q1_MASK _ULL(0xc) /* Quarter 1 syscalls 1024 to 2047 owner */ +#define KVX_SFR_SYO_Q1_SHIFT 2 +#define KVX_SFR_SYO_Q1_WIDTH 2 +#define KVX_SFR_SYO_Q1_WFXL_MASK _ULL(0xc) +#define KVX_SFR_SYO_Q1_WFXL_CLEAR _ULL(0xc) +#define KVX_SFR_SYO_Q1_WFXL_SET _ULL(0xc00000000) + +#define KVX_SFR_SYO_Q2_MASK _ULL(0x30) /* Quarter 2 syscalls 2048 to 3071 owner */ +#define KVX_SFR_SYO_Q2_SHIFT 4 +#define KVX_SFR_SYO_Q2_WIDTH 2 +#define KVX_SFR_SYO_Q2_WFXL_MASK _ULL(0x30) +#define KVX_SFR_SYO_Q2_WFXL_CLEAR _ULL(0x30) +#define KVX_SFR_SYO_Q2_WFXL_SET _ULL(0x3000000000) + +#define KVX_SFR_SYO_Q3_MASK _ULL(0xc0) /* Quarter 3 syscalls 3072 to 4095 owner */ +#define KVX_SFR_SYO_Q3_SHIFT 6 +#define KVX_SFR_SYO_Q3_WIDTH 2 +#define KVX_SFR_SYO_Q3_WFXL_MASK _ULL(0xc0) +#define KVX_SFR_SYO_Q3_WFXL_CLEAR _ULL(0xc0) +#define KVX_SFR_SYO_Q3_WFXL_SET _ULL(0xc000000000) + +#define KVX_SFR_SYOW_Q0_MASK _ULL(0x3) /* Quarter 0 syscalls 0 to 1023 owner */ +#define KVX_SFR_SYOW_Q0_SHIFT 0 +#define KVX_SFR_SYOW_Q0_WIDTH 2 +#define KVX_SFR_SYOW_Q0_WFXL_MASK _ULL(0x3) +#define KVX_SFR_SYOW_Q0_WFXL_CLEAR _ULL(0x3) +#define KVX_SFR_SYOW_Q0_WFXL_SET _ULL(0x300000000) + +#define KVX_SFR_SYOW_Q1_MASK _ULL(0xc) /* Quarter 1 syscalls 1024 to 2047 owner */ +#define KVX_SFR_SYOW_Q1_SHIFT 2 +#define KVX_SFR_SYOW_Q1_WIDTH 2 +#define KVX_SFR_SYOW_Q1_WFXL_MASK _ULL(0xc) +#define KVX_SFR_SYOW_Q1_WFXL_CLEAR _ULL(0xc) +#define KVX_SFR_SYOW_Q1_WFXL_SET _ULL(0xc00000000) + +#define KVX_SFR_SYOW_Q2_MASK _ULL(0x30) /* Quarter 2 syscalls 2048 to 3071 owner */ +#define KVX_SFR_SYOW_Q2_SHIFT 4 +#define KVX_SFR_SYOW_Q2_WIDTH 2 +#define KVX_SFR_SYOW_Q2_WFXL_MASK _ULL(0x30) +#define KVX_SFR_SYOW_Q2_WFXL_CLEAR _ULL(0x30) +#define KVX_SFR_SYOW_Q2_WFXL_SET _ULL(0x3000000000) + +#define KVX_SFR_SYOW_Q3_MASK _ULL(0xc0) /* Quarter 3 syscalls 3072 to 4095 owner */ +#define KVX_SFR_SYOW_Q3_SHIFT 6 +#define KVX_SFR_SYOW_Q3_WIDTH 2 +#define KVX_SFR_SYOW_Q3_WFXL_MASK _ULL(0xc0) +#define KVX_SFR_SYOW_Q3_WFXL_CLEAR _ULL(0xc0) +#define KVX_SFR_SYOW_Q3_WFXL_SET _ULL(0xc000000000) + +#define KVX_SFR_HTO_OPC_MASK _ULL(0x3) /* OPCode trap owner */ +#define KVX_SFR_HTO_OPC_SHIFT 0 +#define KVX_SFR_HTO_OPC_WIDTH 2 +#define KVX_SFR_HTO_OPC_WFXL_MASK _ULL(0x3) +#define KVX_SFR_HTO_OPC_WFXL_CLEAR _ULL(0x3) +#define KVX_SFR_HTO_OPC_WFXL_SET _ULL(0x300000000) + +#define KVX_SFR_HTO_DMIS_MASK _ULL(0xc) /* Data MISalign access trap owner */ +#define KVX_SFR_HTO_DMIS_SHIFT 2 +#define KVX_SFR_HTO_DMIS_WIDTH 2 +#define KVX_SFR_HTO_DMIS_WFXL_MASK _ULL(0xc) +#define KVX_SFR_HTO_DMIS_WFXL_CLEAR _ULL(0xc) +#define KVX_SFR_HTO_DMIS_WFXL_SET _ULL(0xc00000000) + +#define KVX_SFR_HTO_PSYS_MASK _ULL(0x30) /* Program System Error trap owner */ +#define KVX_SFR_HTO_PSYS_SHIFT 4 +#define KVX_SFR_HTO_PSYS_WIDTH 2 +#define KVX_SFR_HTO_PSYS_WFXL_MASK _ULL(0x30) +#define KVX_SFR_HTO_PSYS_WFXL_CLEAR _ULL(0x30) +#define KVX_SFR_HTO_PSYS_WFXL_SET _ULL(0x3000000000) + +#define KVX_SFR_HTO_DSYS_MASK _ULL(0xc0) /* Data System Error trap owner */ +#define KVX_SFR_HTO_DSYS_SHIFT 6 +#define KVX_SFR_HTO_DSYS_WIDTH 2 +#define KVX_SFR_HTO_DSYS_WFXL_MASK _ULL(0xc0) +#define KVX_SFR_HTO_DSYS_WFXL_CLEAR _ULL(0xc0) +#define KVX_SFR_HTO_DSYS_WFXL_SET _ULL(0xc000000000) + +#define KVX_SFR_HTO_DECCG_MASK _ULL(0x300) /* Double ECC traps group owner */ +#define KVX_SFR_HTO_DECCG_SHIFT 8 +#define KVX_SFR_HTO_DECCG_WIDTH 2 +#define KVX_SFR_HTO_DECCG_WFXL_MASK _ULL(0x300) +#define KVX_SFR_HTO_DECCG_WFXL_CLEAR _ULL(0x300) +#define KVX_SFR_HTO_DECCG_WFXL_SET _ULL(0x30000000000) + +#define KVX_SFR_HTO_SECCG_MASK _ULL(0xc00) /* Single ECC traps group owner */ +#define KVX_SFR_HTO_SECCG_SHIFT 10 +#define KVX_SFR_HTO_SECCG_WIDTH 2 +#define KVX_SFR_HTO_SECCG_WFXL_MASK _ULL(0xc00) +#define KVX_SFR_HTO_SECCG_WFXL_CLEAR _ULL(0xc00) +#define KVX_SFR_HTO_SECCG_WFXL_SET _ULL(0xc0000000000) + +#define KVX_SFR_HTO_NOMAP_MASK _ULL(0x3000) /* No mapping trap owner */ +#define KVX_SFR_HTO_NOMAP_SHIFT 12 +#define KVX_SFR_HTO_NOMAP_WIDTH 2 +#define KVX_SFR_HTO_NOMAP_WFXL_MASK _ULL(0x3000) +#define KVX_SFR_HTO_NOMAP_WFXL_CLEAR _ULL(0x3000) +#define KVX_SFR_HTO_NOMAP_WFXL_SET _ULL(0x300000000000) + +#define KVX_SFR_HTO_PROT_MASK _ULL(0xc000) /* PROTection trap owner */ +#define KVX_SFR_HTO_PROT_SHIFT 14 +#define KVX_SFR_HTO_PROT_WIDTH 2 +#define KVX_SFR_HTO_PROT_WFXL_MASK _ULL(0xc000) +#define KVX_SFR_HTO_PROT_WFXL_CLEAR _ULL(0xc000) +#define KVX_SFR_HTO_PROT_WFXL_SET _ULL(0xc00000000000) + +#define KVX_SFR_HTO_W2CL_MASK _ULL(0x30000) /* Write to clean trap owner */ +#define KVX_SFR_HTO_W2CL_SHIFT 16 +#define KVX_SFR_HTO_W2CL_WIDTH 2 +#define KVX_SFR_HTO_W2CL_WFXL_MASK _ULL(0x30000) +#define KVX_SFR_HTO_W2CL_WFXL_CLEAR _ULL(0x30000) +#define KVX_SFR_HTO_W2CL_WFXL_SET _ULL(0x3000000000000) + +#define KVX_SFR_HTO_A2CL_MASK _ULL(0xc0000) /* Atomic to clean trap owner */ +#define KVX_SFR_HTO_A2CL_SHIFT 18 +#define KVX_SFR_HTO_A2CL_WIDTH 2 +#define KVX_SFR_HTO_A2CL_WFXL_MASK _ULL(0xc0000) +#define KVX_SFR_HTO_A2CL_WFXL_CLEAR _ULL(0xc0000) +#define KVX_SFR_HTO_A2CL_WFXL_SET _ULL(0xc000000000000) + +#define KVX_SFR_HTO_DE_MASK _ULL(0x300000) /* Double Exception trap owner */ +#define KVX_SFR_HTO_DE_SHIFT 20 +#define KVX_SFR_HTO_DE_WIDTH 2 +#define KVX_SFR_HTO_DE_WFXL_MASK _ULL(0x300000) +#define KVX_SFR_HTO_DE_WFXL_CLEAR _ULL(0x300000) +#define KVX_SFR_HTO_DE_WFXL_SET _ULL(0x30000000000000) + +#define KVX_SFR_HTO_VSFR_MASK _ULL(0xc00000) /* Virtual SFR trap owner */ +#define KVX_SFR_HTO_VSFR_SHIFT 22 +#define KVX_SFR_HTO_VSFR_WIDTH 2 +#define KVX_SFR_HTO_VSFR_WFXL_MASK _ULL(0xc00000) +#define KVX_SFR_HTO_VSFR_WFXL_CLEAR _ULL(0xc00000) +#define KVX_SFR_HTO_VSFR_WFXL_SET _ULL(0xc0000000000000) + +#define KVX_SFR_HTO_PLO_MASK _ULL(0x3000000) /* Privilege Level Overflow trap owner */ +#define KVX_SFR_HTO_PLO_SHIFT 24 +#define KVX_SFR_HTO_PLO_WIDTH 2 +#define KVX_SFR_HTO_PLO_WFXL_MASK _ULL(0x3000000) +#define KVX_SFR_HTO_PLO_WFXL_CLEAR _ULL(0x3000000) +#define KVX_SFR_HTO_PLO_WFXL_SET _ULL(0x300000000000000) + +#define KVX_SFR_HTOW_OPC_MASK _ULL(0x3) /* OPCode trap owner */ +#define KVX_SFR_HTOW_OPC_SHIFT 0 +#define KVX_SFR_HTOW_OPC_WIDTH 2 +#define KVX_SFR_HTOW_OPC_WFXL_MASK _ULL(0x3) +#define KVX_SFR_HTOW_OPC_WFXL_CLEAR _ULL(0x3) +#define KVX_SFR_HTOW_OPC_WFXL_SET _ULL(0x300000000) + +#define KVX_SFR_HTOW_DMIS_MASK _ULL(0xc) /* Data MISalign access trap owner */ +#define KVX_SFR_HTOW_DMIS_SHIFT 2 +#define KVX_SFR_HTOW_DMIS_WIDTH 2 +#define KVX_SFR_HTOW_DMIS_WFXL_MASK _ULL(0xc) +#define KVX_SFR_HTOW_DMIS_WFXL_CLEAR _ULL(0xc) +#define KVX_SFR_HTOW_DMIS_WFXL_SET _ULL(0xc00000000) + +#define KVX_SFR_HTOW_PSYS_MASK _ULL(0x30) /* Program System Error trap owner */ +#define KVX_SFR_HTOW_PSYS_SHIFT 4 +#define KVX_SFR_HTOW_PSYS_WIDTH 2 +#define KVX_SFR_HTOW_PSYS_WFXL_MASK _ULL(0x30) +#define KVX_SFR_HTOW_PSYS_WFXL_CLEAR _ULL(0x30) +#define KVX_SFR_HTOW_PSYS_WFXL_SET _ULL(0x3000000000) + +#define KVX_SFR_HTOW_DSYS_MASK _ULL(0xc0) /* Data System Error trap owner */ +#define KVX_SFR_HTOW_DSYS_SHIFT 6 +#define KVX_SFR_HTOW_DSYS_WIDTH 2 +#define KVX_SFR_HTOW_DSYS_WFXL_MASK _ULL(0xc0) +#define KVX_SFR_HTOW_DSYS_WFXL_CLEAR _ULL(0xc0) +#define KVX_SFR_HTOW_DSYS_WFXL_SET _ULL(0xc000000000) + +#define KVX_SFR_HTOW_DECCG_MASK _ULL(0x300) /* Double ECC traps group owner */ +#define KVX_SFR_HTOW_DECCG_SHIFT 8 +#define KVX_SFR_HTOW_DECCG_WIDTH 2 +#define KVX_SFR_HTOW_DECCG_WFXL_MASK _ULL(0x300) +#define KVX_SFR_HTOW_DECCG_WFXL_CLEAR _ULL(0x300) +#define KVX_SFR_HTOW_DECCG_WFXL_SET _ULL(0x30000000000) + +#define KVX_SFR_HTOW_SECCG_MASK _ULL(0xc00) /* Single ECC traps group owner */ +#define KVX_SFR_HTOW_SECCG_SHIFT 10 +#define KVX_SFR_HTOW_SECCG_WIDTH 2 +#define KVX_SFR_HTOW_SECCG_WFXL_MASK _ULL(0xc00) +#define KVX_SFR_HTOW_SECCG_WFXL_CLEAR _ULL(0xc00) +#define KVX_SFR_HTOW_SECCG_WFXL_SET _ULL(0xc0000000000) + +#define KVX_SFR_HTOW_NOMAP_MASK _ULL(0x3000) /* No mapping trap owner */ +#define KVX_SFR_HTOW_NOMAP_SHIFT 12 +#define KVX_SFR_HTOW_NOMAP_WIDTH 2 +#define KVX_SFR_HTOW_NOMAP_WFXL_MASK _ULL(0x3000) +#define KVX_SFR_HTOW_NOMAP_WFXL_CLEAR _ULL(0x3000) +#define KVX_SFR_HTOW_NOMAP_WFXL_SET _ULL(0x300000000000) + +#define KVX_SFR_HTOW_PROT_MASK _ULL(0xc000) /* PROTection trap owner */ +#define KVX_SFR_HTOW_PROT_SHIFT 14 +#define KVX_SFR_HTOW_PROT_WIDTH 2 +#define KVX_SFR_HTOW_PROT_WFXL_MASK _ULL(0xc000) +#define KVX_SFR_HTOW_PROT_WFXL_CLEAR _ULL(0xc000) +#define KVX_SFR_HTOW_PROT_WFXL_SET _ULL(0xc00000000000) + +#define KVX_SFR_HTOW_W2CL_MASK _ULL(0x30000) /* Write to clean trap owner */ +#define KVX_SFR_HTOW_W2CL_SHIFT 16 +#define KVX_SFR_HTOW_W2CL_WIDTH 2 +#define KVX_SFR_HTOW_W2CL_WFXL_MASK _ULL(0x30000) +#define KVX_SFR_HTOW_W2CL_WFXL_CLEAR _ULL(0x30000) +#define KVX_SFR_HTOW_W2CL_WFXL_SET _ULL(0x3000000000000) + +#define KVX_SFR_HTOW_A2CL_MASK _ULL(0xc0000) /* Atomic to clean trap owner */ +#define KVX_SFR_HTOW_A2CL_SHIFT 18 +#define KVX_SFR_HTOW_A2CL_WIDTH 2 +#define KVX_SFR_HTOW_A2CL_WFXL_MASK _ULL(0xc0000) +#define KVX_SFR_HTOW_A2CL_WFXL_CLEAR _ULL(0xc0000) +#define KVX_SFR_HTOW_A2CL_WFXL_SET _ULL(0xc000000000000) + +#define KVX_SFR_HTOW_DE_MASK _ULL(0x300000) /* Double Exception trap owner */ +#define KVX_SFR_HTOW_DE_SHIFT 20 +#define KVX_SFR_HTOW_DE_WIDTH 2 +#define KVX_SFR_HTOW_DE_WFXL_MASK _ULL(0x300000) +#define KVX_SFR_HTOW_DE_WFXL_CLEAR _ULL(0x300000) +#define KVX_SFR_HTOW_DE_WFXL_SET _ULL(0x30000000000000) + +#define KVX_SFR_HTOW_VSFR_MASK _ULL(0xc00000) /* Virtual SFR trap owner */ +#define KVX_SFR_HTOW_VSFR_SHIFT 22 +#define KVX_SFR_HTOW_VSFR_WIDTH 2 +#define KVX_SFR_HTOW_VSFR_WFXL_MASK _ULL(0xc00000) +#define KVX_SFR_HTOW_VSFR_WFXL_CLEAR _ULL(0xc00000) +#define KVX_SFR_HTOW_VSFR_WFXL_SET _ULL(0xc0000000000000) + +#define KVX_SFR_HTOW_PLO_MASK _ULL(0x3000000) /* Privilege Level Overflow trap owner */ +#define KVX_SFR_HTOW_PLO_SHIFT 24 +#define KVX_SFR_HTOW_PLO_WIDTH 2 +#define KVX_SFR_HTOW_PLO_WFXL_MASK _ULL(0x3000000) +#define KVX_SFR_HTOW_PLO_WFXL_CLEAR _ULL(0x3000000) +#define KVX_SFR_HTOW_PLO_WFXL_SET _ULL(0x300000000000000) + +#define KVX_SFR_ITO_IT0_MASK _ULL(0x3) /* Interrupt 0 owner */ +#define KVX_SFR_ITO_IT0_SHIFT 0 +#define KVX_SFR_ITO_IT0_WIDTH 2 +#define KVX_SFR_ITO_IT0_WFXL_MASK _ULL(0x3) +#define KVX_SFR_ITO_IT0_WFXL_CLEAR _ULL(0x3) +#define KVX_SFR_ITO_IT0_WFXL_SET _ULL(0x300000000) + +#define KVX_SFR_ITO_IT1_MASK _ULL(0xc) /* Interrupt 1 owner */ +#define KVX_SFR_ITO_IT1_SHIFT 2 +#define KVX_SFR_ITO_IT1_WIDTH 2 +#define KVX_SFR_ITO_IT1_WFXL_MASK _ULL(0xc) +#define KVX_SFR_ITO_IT1_WFXL_CLEAR _ULL(0xc) +#define KVX_SFR_ITO_IT1_WFXL_SET _ULL(0xc00000000) + +#define KVX_SFR_ITO_IT2_MASK _ULL(0x30) /* Interrupt 2 owner */ +#define KVX_SFR_ITO_IT2_SHIFT 4 +#define KVX_SFR_ITO_IT2_WIDTH 2 +#define KVX_SFR_ITO_IT2_WFXL_MASK _ULL(0x30) +#define KVX_SFR_ITO_IT2_WFXL_CLEAR _ULL(0x30) +#define KVX_SFR_ITO_IT2_WFXL_SET _ULL(0x3000000000) + +#define KVX_SFR_ITO_IT3_MASK _ULL(0xc0) /* Interrupt 3 owner */ +#define KVX_SFR_ITO_IT3_SHIFT 6 +#define KVX_SFR_ITO_IT3_WIDTH 2 +#define KVX_SFR_ITO_IT3_WFXL_MASK _ULL(0xc0) +#define KVX_SFR_ITO_IT3_WFXL_CLEAR _ULL(0xc0) +#define KVX_SFR_ITO_IT3_WFXL_SET _ULL(0xc000000000) + +#define KVX_SFR_ITO_IT4_MASK _ULL(0x300) /* Interrupt 4 owner */ +#define KVX_SFR_ITO_IT4_SHIFT 8 +#define KVX_SFR_ITO_IT4_WIDTH 2 +#define KVX_SFR_ITO_IT4_WFXL_MASK _ULL(0x300) +#define KVX_SFR_ITO_IT4_WFXL_CLEAR _ULL(0x300) +#define KVX_SFR_ITO_IT4_WFXL_SET _ULL(0x30000000000) + +#define KVX_SFR_ITO_IT5_MASK _ULL(0xc00) /* Interrupt 5 owner */ +#define KVX_SFR_ITO_IT5_SHIFT 10 +#define KVX_SFR_ITO_IT5_WIDTH 2 +#define KVX_SFR_ITO_IT5_WFXL_MASK _ULL(0xc00) +#define KVX_SFR_ITO_IT5_WFXL_CLEAR _ULL(0xc00) +#define KVX_SFR_ITO_IT5_WFXL_SET _ULL(0xc0000000000) + +#define KVX_SFR_ITO_IT6_MASK _ULL(0x3000) /* Interrupt 6 owner */ +#define KVX_SFR_ITO_IT6_SHIFT 12 +#define KVX_SFR_ITO_IT6_WIDTH 2 +#define KVX_SFR_ITO_IT6_WFXL_MASK _ULL(0x3000) +#define KVX_SFR_ITO_IT6_WFXL_CLEAR _ULL(0x3000) +#define KVX_SFR_ITO_IT6_WFXL_SET _ULL(0x300000000000) + +#define KVX_SFR_ITO_IT7_MASK _ULL(0xc000) /* Interrupt 7 owner */ +#define KVX_SFR_ITO_IT7_SHIFT 14 +#define KVX_SFR_ITO_IT7_WIDTH 2 +#define KVX_SFR_ITO_IT7_WFXL_MASK _ULL(0xc000) +#define KVX_SFR_ITO_IT7_WFXL_CLEAR _ULL(0xc000) +#define KVX_SFR_ITO_IT7_WFXL_SET _ULL(0xc00000000000) + +#define KVX_SFR_ITO_IT8_MASK _ULL(0x30000) /* Interrupt 8 owner */ +#define KVX_SFR_ITO_IT8_SHIFT 16 +#define KVX_SFR_ITO_IT8_WIDTH 2 +#define KVX_SFR_ITO_IT8_WFXL_MASK _ULL(0x30000) +#define KVX_SFR_ITO_IT8_WFXL_CLEAR _ULL(0x30000) +#define KVX_SFR_ITO_IT8_WFXL_SET _ULL(0x3000000000000) + +#define KVX_SFR_ITO_IT9_MASK _ULL(0xc0000) /* Interrupt 9 owner */ +#define KVX_SFR_ITO_IT9_SHIFT 18 +#define KVX_SFR_ITO_IT9_WIDTH 2 +#define KVX_SFR_ITO_IT9_WFXL_MASK _ULL(0xc0000) +#define KVX_SFR_ITO_IT9_WFXL_CLEAR _ULL(0xc0000) +#define KVX_SFR_ITO_IT9_WFXL_SET _ULL(0xc000000000000) + +#define KVX_SFR_ITO_IT10_MASK _ULL(0x300000) /* Interrupt 10 owner */ +#define KVX_SFR_ITO_IT10_SHIFT 20 +#define KVX_SFR_ITO_IT10_WIDTH 2 +#define KVX_SFR_ITO_IT10_WFXL_MASK _ULL(0x300000) +#define KVX_SFR_ITO_IT10_WFXL_CLEAR _ULL(0x300000) +#define KVX_SFR_ITO_IT10_WFXL_SET _ULL(0x30000000000000) + +#define KVX_SFR_ITO_IT11_MASK _ULL(0xc00000) /* Interrupt 11 owner */ +#define KVX_SFR_ITO_IT11_SHIFT 22 +#define KVX_SFR_ITO_IT11_WIDTH 2 +#define KVX_SFR_ITO_IT11_WFXL_MASK _ULL(0xc00000) +#define KVX_SFR_ITO_IT11_WFXL_CLEAR _ULL(0xc00000) +#define KVX_SFR_ITO_IT11_WFXL_SET _ULL(0xc0000000000000) + +#define KVX_SFR_ITO_IT12_MASK _ULL(0x3000000) /* Interrupt 12 owner */ +#define KVX_SFR_ITO_IT12_SHIFT 24 +#define KVX_SFR_ITO_IT12_WIDTH 2 +#define KVX_SFR_ITO_IT12_WFXL_MASK _ULL(0x3000000) +#define KVX_SFR_ITO_IT12_WFXL_CLEAR _ULL(0x3000000) +#define KVX_SFR_ITO_IT12_WFXL_SET _ULL(0x300000000000000) + +#define KVX_SFR_ITO_IT13_MASK _ULL(0xc000000) /* Interrupt 13 owner */ +#define KVX_SFR_ITO_IT13_SHIFT 26 +#define KVX_SFR_ITO_IT13_WIDTH 2 +#define KVX_SFR_ITO_IT13_WFXL_MASK _ULL(0xc000000) +#define KVX_SFR_ITO_IT13_WFXL_CLEAR _ULL(0xc000000) +#define KVX_SFR_ITO_IT13_WFXL_SET _ULL(0xc00000000000000) + +#define KVX_SFR_ITO_IT14_MASK _ULL(0x30000000) /* Interrupt 14 owner */ +#define KVX_SFR_ITO_IT14_SHIFT 28 +#define KVX_SFR_ITO_IT14_WIDTH 2 +#define KVX_SFR_ITO_IT14_WFXL_MASK _ULL(0x30000000) +#define KVX_SFR_ITO_IT14_WFXL_CLEAR _ULL(0x30000000) +#define KVX_SFR_ITO_IT14_WFXL_SET _ULL(0x3000000000000000) + +#define KVX_SFR_ITO_IT15_MASK _ULL(0xc0000000) /* Interrupt 15 owner */ +#define KVX_SFR_ITO_IT15_SHIFT 30 +#define KVX_SFR_ITO_IT15_WIDTH 2 +#define KVX_SFR_ITO_IT15_WFXL_MASK _ULL(0xc0000000) +#define KVX_SFR_ITO_IT15_WFXL_CLEAR _ULL(0xc0000000) +#define KVX_SFR_ITO_IT15_WFXL_SET _ULL(0xc000000000000000) + +#define KVX_SFR_ITO_IT16_MASK _ULL(0x300000000) /* Interrupt 16 owner */ +#define KVX_SFR_ITO_IT16_SHIFT 32 +#define KVX_SFR_ITO_IT16_WIDTH 2 +#define KVX_SFR_ITO_IT16_WFXM_MASK _ULL(0x300000000) +#define KVX_SFR_ITO_IT16_WFXM_CLEAR _ULL(0x3) +#define KVX_SFR_ITO_IT16_WFXM_SET _ULL(0x300000000) + +#define KVX_SFR_ITO_IT17_MASK _ULL(0xc00000000) /* Interrupt 17 owner */ +#define KVX_SFR_ITO_IT17_SHIFT 34 +#define KVX_SFR_ITO_IT17_WIDTH 2 +#define KVX_SFR_ITO_IT17_WFXM_MASK _ULL(0xc00000000) +#define KVX_SFR_ITO_IT17_WFXM_CLEAR _ULL(0xc) +#define KVX_SFR_ITO_IT17_WFXM_SET _ULL(0xc00000000) + +#define KVX_SFR_ITO_IT18_MASK _ULL(0x3000000000) /* Interrupt 18 owner */ +#define KVX_SFR_ITO_IT18_SHIFT 36 +#define KVX_SFR_ITO_IT18_WIDTH 2 +#define KVX_SFR_ITO_IT18_WFXM_MASK _ULL(0x3000000000) +#define KVX_SFR_ITO_IT18_WFXM_CLEAR _ULL(0x30) +#define KVX_SFR_ITO_IT18_WFXM_SET _ULL(0x3000000000) + +#define KVX_SFR_ITO_IT19_MASK _ULL(0xc000000000) /* Interrupt 19 owner */ +#define KVX_SFR_ITO_IT19_SHIFT 38 +#define KVX_SFR_ITO_IT19_WIDTH 2 +#define KVX_SFR_ITO_IT19_WFXM_MASK _ULL(0xc000000000) +#define KVX_SFR_ITO_IT19_WFXM_CLEAR _ULL(0xc0) +#define KVX_SFR_ITO_IT19_WFXM_SET _ULL(0xc000000000) + +#define KVX_SFR_ITO_IT20_MASK _ULL(0x30000000000) /* Interrupt 20 owner */ +#define KVX_SFR_ITO_IT20_SHIFT 40 +#define KVX_SFR_ITO_IT20_WIDTH 2 +#define KVX_SFR_ITO_IT20_WFXM_MASK _ULL(0x30000000000) +#define KVX_SFR_ITO_IT20_WFXM_CLEAR _ULL(0x300) +#define KVX_SFR_ITO_IT20_WFXM_SET _ULL(0x30000000000) + +#define KVX_SFR_ITO_IT21_MASK _ULL(0xc0000000000) /* Interrupt 21 owner */ +#define KVX_SFR_ITO_IT21_SHIFT 42 +#define KVX_SFR_ITO_IT21_WIDTH 2 +#define KVX_SFR_ITO_IT21_WFXM_MASK _ULL(0xc0000000000) +#define KVX_SFR_ITO_IT21_WFXM_CLEAR _ULL(0xc00) +#define KVX_SFR_ITO_IT21_WFXM_SET _ULL(0xc0000000000) + +#define KVX_SFR_ITO_IT22_MASK _ULL(0x300000000000) /* Interrupt 22 owner */ +#define KVX_SFR_ITO_IT22_SHIFT 44 +#define KVX_SFR_ITO_IT22_WIDTH 2 +#define KVX_SFR_ITO_IT22_WFXM_MASK _ULL(0x300000000000) +#define KVX_SFR_ITO_IT22_WFXM_CLEAR _ULL(0x3000) +#define KVX_SFR_ITO_IT22_WFXM_SET _ULL(0x300000000000) + +#define KVX_SFR_ITO_IT23_MASK _ULL(0xc00000000000) /* Interrupt 23 owner */ +#define KVX_SFR_ITO_IT23_SHIFT 46 +#define KVX_SFR_ITO_IT23_WIDTH 2 +#define KVX_SFR_ITO_IT23_WFXM_MASK _ULL(0xc00000000000) +#define KVX_SFR_ITO_IT23_WFXM_CLEAR _ULL(0xc000) +#define KVX_SFR_ITO_IT23_WFXM_SET _ULL(0xc00000000000) + +#define KVX_SFR_ITO_IT24_MASK _ULL(0x3000000000000) /* Interrupt 24 owner */ +#define KVX_SFR_ITO_IT24_SHIFT 48 +#define KVX_SFR_ITO_IT24_WIDTH 2 +#define KVX_SFR_ITO_IT24_WFXM_MASK _ULL(0x3000000000000) +#define KVX_SFR_ITO_IT24_WFXM_CLEAR _ULL(0x30000) +#define KVX_SFR_ITO_IT24_WFXM_SET _ULL(0x3000000000000) + +#define KVX_SFR_ITO_IT25_MASK _ULL(0xc000000000000) /* Interrupt 25 owner */ +#define KVX_SFR_ITO_IT25_SHIFT 50 +#define KVX_SFR_ITO_IT25_WIDTH 2 +#define KVX_SFR_ITO_IT25_WFXM_MASK _ULL(0xc000000000000) +#define KVX_SFR_ITO_IT25_WFXM_CLEAR _ULL(0xc0000) +#define KVX_SFR_ITO_IT25_WFXM_SET _ULL(0xc000000000000) + +#define KVX_SFR_ITO_IT26_MASK _ULL(0x30000000000000) /* Interrupt 26 owner */ +#define KVX_SFR_ITO_IT26_SHIFT 52 +#define KVX_SFR_ITO_IT26_WIDTH 2 +#define KVX_SFR_ITO_IT26_WFXM_MASK _ULL(0x30000000000000) +#define KVX_SFR_ITO_IT26_WFXM_CLEAR _ULL(0x300000) +#define KVX_SFR_ITO_IT26_WFXM_SET _ULL(0x30000000000000) + +#define KVX_SFR_ITO_IT27_MASK _ULL(0xc0000000000000) /* Interrupt 27 owner */ +#define KVX_SFR_ITO_IT27_SHIFT 54 +#define KVX_SFR_ITO_IT27_WIDTH 2 +#define KVX_SFR_ITO_IT27_WFXM_MASK _ULL(0xc0000000000000) +#define KVX_SFR_ITO_IT27_WFXM_CLEAR _ULL(0xc00000) +#define KVX_SFR_ITO_IT27_WFXM_SET _ULL(0xc0000000000000) + +#define KVX_SFR_ITO_IT28_MASK _ULL(0x300000000000000) /* Interrupt 28 owner */ +#define KVX_SFR_ITO_IT28_SHIFT 56 +#define KVX_SFR_ITO_IT28_WIDTH 2 +#define KVX_SFR_ITO_IT28_WFXM_MASK _ULL(0x300000000000000) +#define KVX_SFR_ITO_IT28_WFXM_CLEAR _ULL(0x3000000) +#define KVX_SFR_ITO_IT28_WFXM_SET _ULL(0x300000000000000) + +#define KVX_SFR_ITO_IT29_MASK _ULL(0xc00000000000000) /* Interrupt 29 owner */ +#define KVX_SFR_ITO_IT29_SHIFT 58 +#define KVX_SFR_ITO_IT29_WIDTH 2 +#define KVX_SFR_ITO_IT29_WFXM_MASK _ULL(0xc00000000000000) +#define KVX_SFR_ITO_IT29_WFXM_CLEAR _ULL(0xc000000) +#define KVX_SFR_ITO_IT29_WFXM_SET _ULL(0xc00000000000000) + +#define KVX_SFR_ITO_IT30_MASK _ULL(0x3000000000000000) /* Interrupt 30 owner */ +#define KVX_SFR_ITO_IT30_SHIFT 60 +#define KVX_SFR_ITO_IT30_WIDTH 2 +#define KVX_SFR_ITO_IT30_WFXM_MASK _ULL(0x3000000000000000) +#define KVX_SFR_ITO_IT30_WFXM_CLEAR _ULL(0x30000000) +#define KVX_SFR_ITO_IT30_WFXM_SET _ULL(0x3000000000000000) + +#define KVX_SFR_ITO_IT31_MASK _ULL(0xc000000000000000) /* Interrupt 31 owner */ +#define KVX_SFR_ITO_IT31_SHIFT 62 +#define KVX_SFR_ITO_IT31_WIDTH 2 +#define KVX_SFR_ITO_IT31_WFXM_MASK _ULL(0xc000000000000000) +#define KVX_SFR_ITO_IT31_WFXM_CLEAR _ULL(0xc0000000) +#define KVX_SFR_ITO_IT31_WFXM_SET _ULL(0xc000000000000000) + +#define KVX_SFR_ILE_IT0_MASK _ULL(0x1) /* Interrupt 0 owner */ +#define KVX_SFR_ILE_IT0_SHIFT 0 +#define KVX_SFR_ILE_IT0_WIDTH 1 +#define KVX_SFR_ILE_IT0_WFXL_MASK _ULL(0x1) +#define KVX_SFR_ILE_IT0_WFXL_CLEAR _ULL(0x1) +#define KVX_SFR_ILE_IT0_WFXL_SET _ULL(0x100000000) + +#define KVX_SFR_ILE_IT1_MASK _ULL(0x2) /* Interrupt 1 owner */ +#define KVX_SFR_ILE_IT1_SHIFT 1 +#define KVX_SFR_ILE_IT1_WIDTH 1 +#define KVX_SFR_ILE_IT1_WFXL_MASK _ULL(0x2) +#define KVX_SFR_ILE_IT1_WFXL_CLEAR _ULL(0x2) +#define KVX_SFR_ILE_IT1_WFXL_SET _ULL(0x200000000) + +#define KVX_SFR_ILE_IT2_MASK _ULL(0x4) /* Interrupt 2 owner */ +#define KVX_SFR_ILE_IT2_SHIFT 2 +#define KVX_SFR_ILE_IT2_WIDTH 1 +#define KVX_SFR_ILE_IT2_WFXL_MASK _ULL(0x4) +#define KVX_SFR_ILE_IT2_WFXL_CLEAR _ULL(0x4) +#define KVX_SFR_ILE_IT2_WFXL_SET _ULL(0x400000000) + +#define KVX_SFR_ILE_IT3_MASK _ULL(0x8) /* Interrupt 3 owner */ +#define KVX_SFR_ILE_IT3_SHIFT 3 +#define KVX_SFR_ILE_IT3_WIDTH 1 +#define KVX_SFR_ILE_IT3_WFXL_MASK _ULL(0x8) +#define KVX_SFR_ILE_IT3_WFXL_CLEAR _ULL(0x8) +#define KVX_SFR_ILE_IT3_WFXL_SET _ULL(0x800000000) + +#define KVX_SFR_ILE_IT4_MASK _ULL(0x10) /* Interrupt 4 owner */ +#define KVX_SFR_ILE_IT4_SHIFT 4 +#define KVX_SFR_ILE_IT4_WIDTH 1 +#define KVX_SFR_ILE_IT4_WFXL_MASK _ULL(0x10) +#define KVX_SFR_ILE_IT4_WFXL_CLEAR _ULL(0x10) +#define KVX_SFR_ILE_IT4_WFXL_SET _ULL(0x1000000000) + +#define KVX_SFR_ILE_IT5_MASK _ULL(0x20) /* Interrupt 5 owner */ +#define KVX_SFR_ILE_IT5_SHIFT 5 +#define KVX_SFR_ILE_IT5_WIDTH 1 +#define KVX_SFR_ILE_IT5_WFXL_MASK _ULL(0x20) +#define KVX_SFR_ILE_IT5_WFXL_CLEAR _ULL(0x20) +#define KVX_SFR_ILE_IT5_WFXL_SET _ULL(0x2000000000) + +#define KVX_SFR_ILE_IT6_MASK _ULL(0x40) /* Interrupt 6 owner */ +#define KVX_SFR_ILE_IT6_SHIFT 6 +#define KVX_SFR_ILE_IT6_WIDTH 1 +#define KVX_SFR_ILE_IT6_WFXL_MASK _ULL(0x40) +#define KVX_SFR_ILE_IT6_WFXL_CLEAR _ULL(0x40) +#define KVX_SFR_ILE_IT6_WFXL_SET _ULL(0x4000000000) + +#define KVX_SFR_ILE_IT7_MASK _ULL(0x80) /* Interrupt 7 owner */ +#define KVX_SFR_ILE_IT7_SHIFT 7 +#define KVX_SFR_ILE_IT7_WIDTH 1 +#define KVX_SFR_ILE_IT7_WFXL_MASK _ULL(0x80) +#define KVX_SFR_ILE_IT7_WFXL_CLEAR _ULL(0x80) +#define KVX_SFR_ILE_IT7_WFXL_SET _ULL(0x8000000000) + +#define KVX_SFR_ILE_IT8_MASK _ULL(0x100) /* Interrupt 8 owner */ +#define KVX_SFR_ILE_IT8_SHIFT 8 +#define KVX_SFR_ILE_IT8_WIDTH 1 +#define KVX_SFR_ILE_IT8_WFXL_MASK _ULL(0x100) +#define KVX_SFR_ILE_IT8_WFXL_CLEAR _ULL(0x100) +#define KVX_SFR_ILE_IT8_WFXL_SET _ULL(0x10000000000) + +#define KVX_SFR_ILE_IT9_MASK _ULL(0x200) /* Interrupt 9 owner */ +#define KVX_SFR_ILE_IT9_SHIFT 9 +#define KVX_SFR_ILE_IT9_WIDTH 1 +#define KVX_SFR_ILE_IT9_WFXL_MASK _ULL(0x200) +#define KVX_SFR_ILE_IT9_WFXL_CLEAR _ULL(0x200) +#define KVX_SFR_ILE_IT9_WFXL_SET _ULL(0x20000000000) + +#define KVX_SFR_ILE_IT10_MASK _ULL(0x400) /* Interrupt 10 owner */ +#define KVX_SFR_ILE_IT10_SHIFT 10 +#define KVX_SFR_ILE_IT10_WIDTH 1 +#define KVX_SFR_ILE_IT10_WFXL_MASK _ULL(0x400) +#define KVX_SFR_ILE_IT10_WFXL_CLEAR _ULL(0x400) +#define KVX_SFR_ILE_IT10_WFXL_SET _ULL(0x40000000000) + +#define KVX_SFR_ILE_IT11_MASK _ULL(0x800) /* Interrupt 11 owner */ +#define KVX_SFR_ILE_IT11_SHIFT 11 +#define KVX_SFR_ILE_IT11_WIDTH 1 +#define KVX_SFR_ILE_IT11_WFXL_MASK _ULL(0x800) +#define KVX_SFR_ILE_IT11_WFXL_CLEAR _ULL(0x800) +#define KVX_SFR_ILE_IT11_WFXL_SET _ULL(0x80000000000) + +#define KVX_SFR_ILE_IT12_MASK _ULL(0x1000) /* Interrupt 12 owner */ +#define KVX_SFR_ILE_IT12_SHIFT 12 +#define KVX_SFR_ILE_IT12_WIDTH 1 +#define KVX_SFR_ILE_IT12_WFXL_MASK _ULL(0x1000) +#define KVX_SFR_ILE_IT12_WFXL_CLEAR _ULL(0x1000) +#define KVX_SFR_ILE_IT12_WFXL_SET _ULL(0x100000000000) + +#define KVX_SFR_ILE_IT13_MASK _ULL(0x2000) /* Interrupt 13 owner */ +#define KVX_SFR_ILE_IT13_SHIFT 13 +#define KVX_SFR_ILE_IT13_WIDTH 1 +#define KVX_SFR_ILE_IT13_WFXL_MASK _ULL(0x2000) +#define KVX_SFR_ILE_IT13_WFXL_CLEAR _ULL(0x2000) +#define KVX_SFR_ILE_IT13_WFXL_SET _ULL(0x200000000000) + +#define KVX_SFR_ILE_IT14_MASK _ULL(0x4000) /* Interrupt 14 owner */ +#define KVX_SFR_ILE_IT14_SHIFT 14 +#define KVX_SFR_ILE_IT14_WIDTH 1 +#define KVX_SFR_ILE_IT14_WFXL_MASK _ULL(0x4000) +#define KVX_SFR_ILE_IT14_WFXL_CLEAR _ULL(0x4000) +#define KVX_SFR_ILE_IT14_WFXL_SET _ULL(0x400000000000) + +#define KVX_SFR_ILE_IT15_MASK _ULL(0x8000) /* Interrupt 15 owner */ +#define KVX_SFR_ILE_IT15_SHIFT 15 +#define KVX_SFR_ILE_IT15_WIDTH 1 +#define KVX_SFR_ILE_IT15_WFXL_MASK _ULL(0x8000) +#define KVX_SFR_ILE_IT15_WFXL_CLEAR _ULL(0x8000) +#define KVX_SFR_ILE_IT15_WFXL_SET _ULL(0x800000000000) + +#define KVX_SFR_ILE_IT16_MASK _ULL(0x10000) /* Interrupt 16 owner */ +#define KVX_SFR_ILE_IT16_SHIFT 16 +#define KVX_SFR_ILE_IT16_WIDTH 1 +#define KVX_SFR_ILE_IT16_WFXL_MASK _ULL(0x10000) +#define KVX_SFR_ILE_IT16_WFXL_CLEAR _ULL(0x10000) +#define KVX_SFR_ILE_IT16_WFXL_SET _ULL(0x1000000000000) + +#define KVX_SFR_ILE_IT17_MASK _ULL(0x20000) /* Interrupt 17 owner */ +#define KVX_SFR_ILE_IT17_SHIFT 17 +#define KVX_SFR_ILE_IT17_WIDTH 1 +#define KVX_SFR_ILE_IT17_WFXL_MASK _ULL(0x20000) +#define KVX_SFR_ILE_IT17_WFXL_CLEAR _ULL(0x20000) +#define KVX_SFR_ILE_IT17_WFXL_SET _ULL(0x2000000000000) + +#define KVX_SFR_ILE_IT18_MASK _ULL(0x40000) /* Interrupt 18 owner */ +#define KVX_SFR_ILE_IT18_SHIFT 18 +#define KVX_SFR_ILE_IT18_WIDTH 1 +#define KVX_SFR_ILE_IT18_WFXL_MASK _ULL(0x40000) +#define KVX_SFR_ILE_IT18_WFXL_CLEAR _ULL(0x40000) +#define KVX_SFR_ILE_IT18_WFXL_SET _ULL(0x4000000000000) + +#define KVX_SFR_ILE_IT19_MASK _ULL(0x80000) /* Interrupt 19 owner */ +#define KVX_SFR_ILE_IT19_SHIFT 19 +#define KVX_SFR_ILE_IT19_WIDTH 1 +#define KVX_SFR_ILE_IT19_WFXL_MASK _ULL(0x80000) +#define KVX_SFR_ILE_IT19_WFXL_CLEAR _ULL(0x80000) +#define KVX_SFR_ILE_IT19_WFXL_SET _ULL(0x8000000000000) + +#define KVX_SFR_ILE_IT20_MASK _ULL(0x100000) /* Interrupt 20 owner */ +#define KVX_SFR_ILE_IT20_SHIFT 20 +#define KVX_SFR_ILE_IT20_WIDTH 1 +#define KVX_SFR_ILE_IT20_WFXL_MASK _ULL(0x100000) +#define KVX_SFR_ILE_IT20_WFXL_CLEAR _ULL(0x100000) +#define KVX_SFR_ILE_IT20_WFXL_SET _ULL(0x10000000000000) + +#define KVX_SFR_ILE_IT21_MASK _ULL(0x200000) /* Interrupt 21 owner */ +#define KVX_SFR_ILE_IT21_SHIFT 21 +#define KVX_SFR_ILE_IT21_WIDTH 1 +#define KVX_SFR_ILE_IT21_WFXL_MASK _ULL(0x200000) +#define KVX_SFR_ILE_IT21_WFXL_CLEAR _ULL(0x200000) +#define KVX_SFR_ILE_IT21_WFXL_SET _ULL(0x20000000000000) + +#define KVX_SFR_ILE_IT22_MASK _ULL(0x400000) /* Interrupt 22 owner */ +#define KVX_SFR_ILE_IT22_SHIFT 22 +#define KVX_SFR_ILE_IT22_WIDTH 1 +#define KVX_SFR_ILE_IT22_WFXL_MASK _ULL(0x400000) +#define KVX_SFR_ILE_IT22_WFXL_CLEAR _ULL(0x400000) +#define KVX_SFR_ILE_IT22_WFXL_SET _ULL(0x40000000000000) + +#define KVX_SFR_ILE_IT23_MASK _ULL(0x800000) /* Interrupt 23 owner */ +#define KVX_SFR_ILE_IT23_SHIFT 23 +#define KVX_SFR_ILE_IT23_WIDTH 1 +#define KVX_SFR_ILE_IT23_WFXL_MASK _ULL(0x800000) +#define KVX_SFR_ILE_IT23_WFXL_CLEAR _ULL(0x800000) +#define KVX_SFR_ILE_IT23_WFXL_SET _ULL(0x80000000000000) + +#define KVX_SFR_ILE_IT24_MASK _ULL(0x1000000) /* Interrupt 24 owner */ +#define KVX_SFR_ILE_IT24_SHIFT 24 +#define KVX_SFR_ILE_IT24_WIDTH 1 +#define KVX_SFR_ILE_IT24_WFXL_MASK _ULL(0x1000000) +#define KVX_SFR_ILE_IT24_WFXL_CLEAR _ULL(0x1000000) +#define KVX_SFR_ILE_IT24_WFXL_SET _ULL(0x100000000000000) + +#define KVX_SFR_ILE_IT25_MASK _ULL(0x2000000) /* Interrupt 25 owner */ +#define KVX_SFR_ILE_IT25_SHIFT 25 +#define KVX_SFR_ILE_IT25_WIDTH 1 +#define KVX_SFR_ILE_IT25_WFXL_MASK _ULL(0x2000000) +#define KVX_SFR_ILE_IT25_WFXL_CLEAR _ULL(0x2000000) +#define KVX_SFR_ILE_IT25_WFXL_SET _ULL(0x200000000000000) + +#define KVX_SFR_ILE_IT26_MASK _ULL(0x4000000) /* Interrupt 26 owner */ +#define KVX_SFR_ILE_IT26_SHIFT 26 +#define KVX_SFR_ILE_IT26_WIDTH 1 +#define KVX_SFR_ILE_IT26_WFXL_MASK _ULL(0x4000000) +#define KVX_SFR_ILE_IT26_WFXL_CLEAR _ULL(0x4000000) +#define KVX_SFR_ILE_IT26_WFXL_SET _ULL(0x400000000000000) + +#define KVX_SFR_ILE_IT27_MASK _ULL(0x8000000) /* Interrupt 27 owner */ +#define KVX_SFR_ILE_IT27_SHIFT 27 +#define KVX_SFR_ILE_IT27_WIDTH 1 +#define KVX_SFR_ILE_IT27_WFXL_MASK _ULL(0x8000000) +#define KVX_SFR_ILE_IT27_WFXL_CLEAR _ULL(0x8000000) +#define KVX_SFR_ILE_IT27_WFXL_SET _ULL(0x800000000000000) + +#define KVX_SFR_ILE_IT28_MASK _ULL(0x10000000) /* Interrupt 28 owner */ +#define KVX_SFR_ILE_IT28_SHIFT 28 +#define KVX_SFR_ILE_IT28_WIDTH 1 +#define KVX_SFR_ILE_IT28_WFXL_MASK _ULL(0x10000000) +#define KVX_SFR_ILE_IT28_WFXL_CLEAR _ULL(0x10000000) +#define KVX_SFR_ILE_IT28_WFXL_SET _ULL(0x1000000000000000) + +#define KVX_SFR_ILE_IT29_MASK _ULL(0x20000000) /* Interrupt 29 owner */ +#define KVX_SFR_ILE_IT29_SHIFT 29 +#define KVX_SFR_ILE_IT29_WIDTH 1 +#define KVX_SFR_ILE_IT29_WFXL_MASK _ULL(0x20000000) +#define KVX_SFR_ILE_IT29_WFXL_CLEAR _ULL(0x20000000) +#define KVX_SFR_ILE_IT29_WFXL_SET _ULL(0x2000000000000000) + +#define KVX_SFR_ILE_IT30_MASK _ULL(0x40000000) /* Interrupt 30 owner */ +#define KVX_SFR_ILE_IT30_SHIFT 30 +#define KVX_SFR_ILE_IT30_WIDTH 1 +#define KVX_SFR_ILE_IT30_WFXL_MASK _ULL(0x40000000) +#define KVX_SFR_ILE_IT30_WFXL_CLEAR _ULL(0x40000000) +#define KVX_SFR_ILE_IT30_WFXL_SET _ULL(0x4000000000000000) + +#define KVX_SFR_ILE_IT31_MASK _ULL(0x80000000) /* Interrupt 31 owner */ +#define KVX_SFR_ILE_IT31_SHIFT 31 +#define KVX_SFR_ILE_IT31_WIDTH 1 +#define KVX_SFR_ILE_IT31_WFXL_MASK _ULL(0x80000000) +#define KVX_SFR_ILE_IT31_WFXL_CLEAR _ULL(0x80000000) +#define KVX_SFR_ILE_IT31_WFXL_SET _ULL(0x8000000000000000) + +#define KVX_SFR_ILL_IT0_MASK _ULL(0x3) /* Interrupt 0 owner */ +#define KVX_SFR_ILL_IT0_SHIFT 0 +#define KVX_SFR_ILL_IT0_WIDTH 2 +#define KVX_SFR_ILL_IT0_WFXL_MASK _ULL(0x3) +#define KVX_SFR_ILL_IT0_WFXL_CLEAR _ULL(0x3) +#define KVX_SFR_ILL_IT0_WFXL_SET _ULL(0x300000000) + +#define KVX_SFR_ILL_IT1_MASK _ULL(0xc) /* Interrupt 1 owner */ +#define KVX_SFR_ILL_IT1_SHIFT 2 +#define KVX_SFR_ILL_IT1_WIDTH 2 +#define KVX_SFR_ILL_IT1_WFXL_MASK _ULL(0xc) +#define KVX_SFR_ILL_IT1_WFXL_CLEAR _ULL(0xc) +#define KVX_SFR_ILL_IT1_WFXL_SET _ULL(0xc00000000) + +#define KVX_SFR_ILL_IT2_MASK _ULL(0x30) /* Interrupt 2 owner */ +#define KVX_SFR_ILL_IT2_SHIFT 4 +#define KVX_SFR_ILL_IT2_WIDTH 2 +#define KVX_SFR_ILL_IT2_WFXL_MASK _ULL(0x30) +#define KVX_SFR_ILL_IT2_WFXL_CLEAR _ULL(0x30) +#define KVX_SFR_ILL_IT2_WFXL_SET _ULL(0x3000000000) + +#define KVX_SFR_ILL_IT3_MASK _ULL(0xc0) /* Interrupt 3 owner */ +#define KVX_SFR_ILL_IT3_SHIFT 6 +#define KVX_SFR_ILL_IT3_WIDTH 2 +#define KVX_SFR_ILL_IT3_WFXL_MASK _ULL(0xc0) +#define KVX_SFR_ILL_IT3_WFXL_CLEAR _ULL(0xc0) +#define KVX_SFR_ILL_IT3_WFXL_SET _ULL(0xc000000000) + +#define KVX_SFR_ILL_IT4_MASK _ULL(0x300) /* Interrupt 4 owner */ +#define KVX_SFR_ILL_IT4_SHIFT 8 +#define KVX_SFR_ILL_IT4_WIDTH 2 +#define KVX_SFR_ILL_IT4_WFXL_MASK _ULL(0x300) +#define KVX_SFR_ILL_IT4_WFXL_CLEAR _ULL(0x300) +#define KVX_SFR_ILL_IT4_WFXL_SET _ULL(0x30000000000) + +#define KVX_SFR_ILL_IT5_MASK _ULL(0xc00) /* Interrupt 5 owner */ +#define KVX_SFR_ILL_IT5_SHIFT 10 +#define KVX_SFR_ILL_IT5_WIDTH 2 +#define KVX_SFR_ILL_IT5_WFXL_MASK _ULL(0xc00) +#define KVX_SFR_ILL_IT5_WFXL_CLEAR _ULL(0xc00) +#define KVX_SFR_ILL_IT5_WFXL_SET _ULL(0xc0000000000) + +#define KVX_SFR_ILL_IT6_MASK _ULL(0x3000) /* Interrupt 6 owner */ +#define KVX_SFR_ILL_IT6_SHIFT 12 +#define KVX_SFR_ILL_IT6_WIDTH 2 +#define KVX_SFR_ILL_IT6_WFXL_MASK _ULL(0x3000) +#define KVX_SFR_ILL_IT6_WFXL_CLEAR _ULL(0x3000) +#define KVX_SFR_ILL_IT6_WFXL_SET _ULL(0x300000000000) + +#define KVX_SFR_ILL_IT7_MASK _ULL(0xc000) /* Interrupt 7 owner */ +#define KVX_SFR_ILL_IT7_SHIFT 14 +#define KVX_SFR_ILL_IT7_WIDTH 2 +#define KVX_SFR_ILL_IT7_WFXL_MASK _ULL(0xc000) +#define KVX_SFR_ILL_IT7_WFXL_CLEAR _ULL(0xc000) +#define KVX_SFR_ILL_IT7_WFXL_SET _ULL(0xc00000000000) + +#define KVX_SFR_ILL_IT8_MASK _ULL(0x30000) /* Interrupt 8 owner */ +#define KVX_SFR_ILL_IT8_SHIFT 16 +#define KVX_SFR_ILL_IT8_WIDTH 2 +#define KVX_SFR_ILL_IT8_WFXL_MASK _ULL(0x30000) +#define KVX_SFR_ILL_IT8_WFXL_CLEAR _ULL(0x30000) +#define KVX_SFR_ILL_IT8_WFXL_SET _ULL(0x3000000000000) + +#define KVX_SFR_ILL_IT9_MASK _ULL(0xc0000) /* Interrupt 9 owner */ +#define KVX_SFR_ILL_IT9_SHIFT 18 +#define KVX_SFR_ILL_IT9_WIDTH 2 +#define KVX_SFR_ILL_IT9_WFXL_MASK _ULL(0xc0000) +#define KVX_SFR_ILL_IT9_WFXL_CLEAR _ULL(0xc0000) +#define KVX_SFR_ILL_IT9_WFXL_SET _ULL(0xc000000000000) + +#define KVX_SFR_ILL_IT10_MASK _ULL(0x300000) /* Interrupt 10 owner */ +#define KVX_SFR_ILL_IT10_SHIFT 20 +#define KVX_SFR_ILL_IT10_WIDTH 2 +#define KVX_SFR_ILL_IT10_WFXL_MASK _ULL(0x300000) +#define KVX_SFR_ILL_IT10_WFXL_CLEAR _ULL(0x300000) +#define KVX_SFR_ILL_IT10_WFXL_SET _ULL(0x30000000000000) + +#define KVX_SFR_ILL_IT11_MASK _ULL(0xc00000) /* Interrupt 11 owner */ +#define KVX_SFR_ILL_IT11_SHIFT 22 +#define KVX_SFR_ILL_IT11_WIDTH 2 +#define KVX_SFR_ILL_IT11_WFXL_MASK _ULL(0xc00000) +#define KVX_SFR_ILL_IT11_WFXL_CLEAR _ULL(0xc00000) +#define KVX_SFR_ILL_IT11_WFXL_SET _ULL(0xc0000000000000) + +#define KVX_SFR_ILL_IT12_MASK _ULL(0x3000000) /* Interrupt 12 owner */ +#define KVX_SFR_ILL_IT12_SHIFT 24 +#define KVX_SFR_ILL_IT12_WIDTH 2 +#define KVX_SFR_ILL_IT12_WFXL_MASK _ULL(0x3000000) +#define KVX_SFR_ILL_IT12_WFXL_CLEAR _ULL(0x3000000) +#define KVX_SFR_ILL_IT12_WFXL_SET _ULL(0x300000000000000) + +#define KVX_SFR_ILL_IT13_MASK _ULL(0xc000000) /* Interrupt 13 owner */ +#define KVX_SFR_ILL_IT13_SHIFT 26 +#define KVX_SFR_ILL_IT13_WIDTH 2 +#define KVX_SFR_ILL_IT13_WFXL_MASK _ULL(0xc000000) +#define KVX_SFR_ILL_IT13_WFXL_CLEAR _ULL(0xc000000) +#define KVX_SFR_ILL_IT13_WFXL_SET _ULL(0xc00000000000000) + +#define KVX_SFR_ILL_IT14_MASK _ULL(0x30000000) /* Interrupt 14 owner */ +#define KVX_SFR_ILL_IT14_SHIFT 28 +#define KVX_SFR_ILL_IT14_WIDTH 2 +#define KVX_SFR_ILL_IT14_WFXL_MASK _ULL(0x30000000) +#define KVX_SFR_ILL_IT14_WFXL_CLEAR _ULL(0x30000000) +#define KVX_SFR_ILL_IT14_WFXL_SET _ULL(0x3000000000000000) + +#define KVX_SFR_ILL_IT15_MASK _ULL(0xc0000000) /* Interrupt 15 owner */ +#define KVX_SFR_ILL_IT15_SHIFT 30 +#define KVX_SFR_ILL_IT15_WIDTH 2 +#define KVX_SFR_ILL_IT15_WFXL_MASK _ULL(0xc0000000) +#define KVX_SFR_ILL_IT15_WFXL_CLEAR _ULL(0xc0000000) +#define KVX_SFR_ILL_IT15_WFXL_SET _ULL(0xc000000000000000) + +#define KVX_SFR_ILL_IT16_MASK _ULL(0x300000000) /* Interrupt 16 owner */ +#define KVX_SFR_ILL_IT16_SHIFT 32 +#define KVX_SFR_ILL_IT16_WIDTH 2 +#define KVX_SFR_ILL_IT16_WFXM_MASK _ULL(0x300000000) +#define KVX_SFR_ILL_IT16_WFXM_CLEAR _ULL(0x3) +#define KVX_SFR_ILL_IT16_WFXM_SET _ULL(0x300000000) + +#define KVX_SFR_ILL_IT17_MASK _ULL(0xc00000000) /* Interrupt 17 owner */ +#define KVX_SFR_ILL_IT17_SHIFT 34 +#define KVX_SFR_ILL_IT17_WIDTH 2 +#define KVX_SFR_ILL_IT17_WFXM_MASK _ULL(0xc00000000) +#define KVX_SFR_ILL_IT17_WFXM_CLEAR _ULL(0xc) +#define KVX_SFR_ILL_IT17_WFXM_SET _ULL(0xc00000000) + +#define KVX_SFR_ILL_IT18_MASK _ULL(0x3000000000) /* Interrupt 18 owner */ +#define KVX_SFR_ILL_IT18_SHIFT 36 +#define KVX_SFR_ILL_IT18_WIDTH 2 +#define KVX_SFR_ILL_IT18_WFXM_MASK _ULL(0x3000000000) +#define KVX_SFR_ILL_IT18_WFXM_CLEAR _ULL(0x30) +#define KVX_SFR_ILL_IT18_WFXM_SET _ULL(0x3000000000) + +#define KVX_SFR_ILL_IT19_MASK _ULL(0xc000000000) /* Interrupt 19 owner */ +#define KVX_SFR_ILL_IT19_SHIFT 38 +#define KVX_SFR_ILL_IT19_WIDTH 2 +#define KVX_SFR_ILL_IT19_WFXM_MASK _ULL(0xc000000000) +#define KVX_SFR_ILL_IT19_WFXM_CLEAR _ULL(0xc0) +#define KVX_SFR_ILL_IT19_WFXM_SET _ULL(0xc000000000) + +#define KVX_SFR_ILL_IT20_MASK _ULL(0x30000000000) /* Interrupt 20 owner */ +#define KVX_SFR_ILL_IT20_SHIFT 40 +#define KVX_SFR_ILL_IT20_WIDTH 2 +#define KVX_SFR_ILL_IT20_WFXM_MASK _ULL(0x30000000000) +#define KVX_SFR_ILL_IT20_WFXM_CLEAR _ULL(0x300) +#define KVX_SFR_ILL_IT20_WFXM_SET _ULL(0x30000000000) + +#define KVX_SFR_ILL_IT21_MASK _ULL(0xc0000000000) /* Interrupt 21 owner */ +#define KVX_SFR_ILL_IT21_SHIFT 42 +#define KVX_SFR_ILL_IT21_WIDTH 2 +#define KVX_SFR_ILL_IT21_WFXM_MASK _ULL(0xc0000000000) +#define KVX_SFR_ILL_IT21_WFXM_CLEAR _ULL(0xc00) +#define KVX_SFR_ILL_IT21_WFXM_SET _ULL(0xc0000000000) + +#define KVX_SFR_ILL_IT22_MASK _ULL(0x300000000000) /* Interrupt 22 owner */ +#define KVX_SFR_ILL_IT22_SHIFT 44 +#define KVX_SFR_ILL_IT22_WIDTH 2 +#define KVX_SFR_ILL_IT22_WFXM_MASK _ULL(0x300000000000) +#define KVX_SFR_ILL_IT22_WFXM_CLEAR _ULL(0x3000) +#define KVX_SFR_ILL_IT22_WFXM_SET _ULL(0x300000000000) + +#define KVX_SFR_ILL_IT23_MASK _ULL(0xc00000000000) /* Interrupt 23 owner */ +#define KVX_SFR_ILL_IT23_SHIFT 46 +#define KVX_SFR_ILL_IT23_WIDTH 2 +#define KVX_SFR_ILL_IT23_WFXM_MASK _ULL(0xc00000000000) +#define KVX_SFR_ILL_IT23_WFXM_CLEAR _ULL(0xc000) +#define KVX_SFR_ILL_IT23_WFXM_SET _ULL(0xc00000000000) + +#define KVX_SFR_ILL_IT24_MASK _ULL(0x3000000000000) /* Interrupt 24 owner */ +#define KVX_SFR_ILL_IT24_SHIFT 48 +#define KVX_SFR_ILL_IT24_WIDTH 2 +#define KVX_SFR_ILL_IT24_WFXM_MASK _ULL(0x3000000000000) +#define KVX_SFR_ILL_IT24_WFXM_CLEAR _ULL(0x30000) +#define KVX_SFR_ILL_IT24_WFXM_SET _ULL(0x3000000000000) + +#define KVX_SFR_ILL_IT25_MASK _ULL(0xc000000000000) /* Interrupt 25 owner */ +#define KVX_SFR_ILL_IT25_SHIFT 50 +#define KVX_SFR_ILL_IT25_WIDTH 2 +#define KVX_SFR_ILL_IT25_WFXM_MASK _ULL(0xc000000000000) +#define KVX_SFR_ILL_IT25_WFXM_CLEAR _ULL(0xc0000) +#define KVX_SFR_ILL_IT25_WFXM_SET _ULL(0xc000000000000) + +#define KVX_SFR_ILL_IT26_MASK _ULL(0x30000000000000) /* Interrupt 26 owner */ +#define KVX_SFR_ILL_IT26_SHIFT 52 +#define KVX_SFR_ILL_IT26_WIDTH 2 +#define KVX_SFR_ILL_IT26_WFXM_MASK _ULL(0x30000000000000) +#define KVX_SFR_ILL_IT26_WFXM_CLEAR _ULL(0x300000) +#define KVX_SFR_ILL_IT26_WFXM_SET _ULL(0x30000000000000) + +#define KVX_SFR_ILL_IT27_MASK _ULL(0xc0000000000000) /* Interrupt 27 owner */ +#define KVX_SFR_ILL_IT27_SHIFT 54 +#define KVX_SFR_ILL_IT27_WIDTH 2 +#define KVX_SFR_ILL_IT27_WFXM_MASK _ULL(0xc0000000000000) +#define KVX_SFR_ILL_IT27_WFXM_CLEAR _ULL(0xc00000) +#define KVX_SFR_ILL_IT27_WFXM_SET _ULL(0xc0000000000000) + +#define KVX_SFR_ILL_IT28_MASK _ULL(0x300000000000000) /* Interrupt 28 owner */ +#define KVX_SFR_ILL_IT28_SHIFT 56 +#define KVX_SFR_ILL_IT28_WIDTH 2 +#define KVX_SFR_ILL_IT28_WFXM_MASK _ULL(0x300000000000000) +#define KVX_SFR_ILL_IT28_WFXM_CLEAR _ULL(0x3000000) +#define KVX_SFR_ILL_IT28_WFXM_SET _ULL(0x300000000000000) + +#define KVX_SFR_ILL_IT29_MASK _ULL(0xc00000000000000) /* Interrupt 29 owner */ +#define KVX_SFR_ILL_IT29_SHIFT 58 +#define KVX_SFR_ILL_IT29_WIDTH 2 +#define KVX_SFR_ILL_IT29_WFXM_MASK _ULL(0xc00000000000000) +#define KVX_SFR_ILL_IT29_WFXM_CLEAR _ULL(0xc000000) +#define KVX_SFR_ILL_IT29_WFXM_SET _ULL(0xc00000000000000) + +#define KVX_SFR_ILL_IT30_MASK _ULL(0x3000000000000000) /* Interrupt 30 owner */ +#define KVX_SFR_ILL_IT30_SHIFT 60 +#define KVX_SFR_ILL_IT30_WIDTH 2 +#define KVX_SFR_ILL_IT30_WFXM_MASK _ULL(0x3000000000000000) +#define KVX_SFR_ILL_IT30_WFXM_CLEAR _ULL(0x30000000) +#define KVX_SFR_ILL_IT30_WFXM_SET _ULL(0x3000000000000000) + +#define KVX_SFR_ILL_IT31_MASK _ULL(0xc000000000000000) /* Interrupt 31 owner */ +#define KVX_SFR_ILL_IT31_SHIFT 62 +#define KVX_SFR_ILL_IT31_WIDTH 2 +#define KVX_SFR_ILL_IT31_WFXM_MASK _ULL(0xc000000000000000) +#define KVX_SFR_ILL_IT31_WFXM_CLEAR _ULL(0xc0000000) +#define KVX_SFR_ILL_IT31_WFXM_SET _ULL(0xc000000000000000) + +#define KVX_SFR_ILR_IT0_MASK _ULL(0x1) /* Interrupt 0 owner */ +#define KVX_SFR_ILR_IT0_SHIFT 0 +#define KVX_SFR_ILR_IT0_WIDTH 1 +#define KVX_SFR_ILR_IT0_WFXL_MASK _ULL(0x1) +#define KVX_SFR_ILR_IT0_WFXL_CLEAR _ULL(0x1) +#define KVX_SFR_ILR_IT0_WFXL_SET _ULL(0x100000000) + +#define KVX_SFR_ILR_IT1_MASK _ULL(0x2) /* Interrupt 1 owner */ +#define KVX_SFR_ILR_IT1_SHIFT 1 +#define KVX_SFR_ILR_IT1_WIDTH 1 +#define KVX_SFR_ILR_IT1_WFXL_MASK _ULL(0x2) +#define KVX_SFR_ILR_IT1_WFXL_CLEAR _ULL(0x2) +#define KVX_SFR_ILR_IT1_WFXL_SET _ULL(0x200000000) + +#define KVX_SFR_ILR_IT2_MASK _ULL(0x4) /* Interrupt 2 owner */ +#define KVX_SFR_ILR_IT2_SHIFT 2 +#define KVX_SFR_ILR_IT2_WIDTH 1 +#define KVX_SFR_ILR_IT2_WFXL_MASK _ULL(0x4) +#define KVX_SFR_ILR_IT2_WFXL_CLEAR _ULL(0x4) +#define KVX_SFR_ILR_IT2_WFXL_SET _ULL(0x400000000) + +#define KVX_SFR_ILR_IT3_MASK _ULL(0x8) /* Interrupt 3 owner */ +#define KVX_SFR_ILR_IT3_SHIFT 3 +#define KVX_SFR_ILR_IT3_WIDTH 1 +#define KVX_SFR_ILR_IT3_WFXL_MASK _ULL(0x8) +#define KVX_SFR_ILR_IT3_WFXL_CLEAR _ULL(0x8) +#define KVX_SFR_ILR_IT3_WFXL_SET _ULL(0x800000000) + +#define KVX_SFR_ILR_IT4_MASK _ULL(0x10) /* Interrupt 4 owner */ +#define KVX_SFR_ILR_IT4_SHIFT 4 +#define KVX_SFR_ILR_IT4_WIDTH 1 +#define KVX_SFR_ILR_IT4_WFXL_MASK _ULL(0x10) +#define KVX_SFR_ILR_IT4_WFXL_CLEAR _ULL(0x10) +#define KVX_SFR_ILR_IT4_WFXL_SET _ULL(0x1000000000) + +#define KVX_SFR_ILR_IT5_MASK _ULL(0x20) /* Interrupt 5 owner */ +#define KVX_SFR_ILR_IT5_SHIFT 5 +#define KVX_SFR_ILR_IT5_WIDTH 1 +#define KVX_SFR_ILR_IT5_WFXL_MASK _ULL(0x20) +#define KVX_SFR_ILR_IT5_WFXL_CLEAR _ULL(0x20) +#define KVX_SFR_ILR_IT5_WFXL_SET _ULL(0x2000000000) + +#define KVX_SFR_ILR_IT6_MASK _ULL(0x40) /* Interrupt 6 owner */ +#define KVX_SFR_ILR_IT6_SHIFT 6 +#define KVX_SFR_ILR_IT6_WIDTH 1 +#define KVX_SFR_ILR_IT6_WFXL_MASK _ULL(0x40) +#define KVX_SFR_ILR_IT6_WFXL_CLEAR _ULL(0x40) +#define KVX_SFR_ILR_IT6_WFXL_SET _ULL(0x4000000000) + +#define KVX_SFR_ILR_IT7_MASK _ULL(0x80) /* Interrupt 7 owner */ +#define KVX_SFR_ILR_IT7_SHIFT 7 +#define KVX_SFR_ILR_IT7_WIDTH 1 +#define KVX_SFR_ILR_IT7_WFXL_MASK _ULL(0x80) +#define KVX_SFR_ILR_IT7_WFXL_CLEAR _ULL(0x80) +#define KVX_SFR_ILR_IT7_WFXL_SET _ULL(0x8000000000) + +#define KVX_SFR_ILR_IT8_MASK _ULL(0x100) /* Interrupt 8 owner */ +#define KVX_SFR_ILR_IT8_SHIFT 8 +#define KVX_SFR_ILR_IT8_WIDTH 1 +#define KVX_SFR_ILR_IT8_WFXL_MASK _ULL(0x100) +#define KVX_SFR_ILR_IT8_WFXL_CLEAR _ULL(0x100) +#define KVX_SFR_ILR_IT8_WFXL_SET _ULL(0x10000000000) + +#define KVX_SFR_ILR_IT9_MASK _ULL(0x200) /* Interrupt 9 owner */ +#define KVX_SFR_ILR_IT9_SHIFT 9 +#define KVX_SFR_ILR_IT9_WIDTH 1 +#define KVX_SFR_ILR_IT9_WFXL_MASK _ULL(0x200) +#define KVX_SFR_ILR_IT9_WFXL_CLEAR _ULL(0x200) +#define KVX_SFR_ILR_IT9_WFXL_SET _ULL(0x20000000000) + +#define KVX_SFR_ILR_IT10_MASK _ULL(0x400) /* Interrupt 10 owner */ +#define KVX_SFR_ILR_IT10_SHIFT 10 +#define KVX_SFR_ILR_IT10_WIDTH 1 +#define KVX_SFR_ILR_IT10_WFXL_MASK _ULL(0x400) +#define KVX_SFR_ILR_IT10_WFXL_CLEAR _ULL(0x400) +#define KVX_SFR_ILR_IT10_WFXL_SET _ULL(0x40000000000) + +#define KVX_SFR_ILR_IT11_MASK _ULL(0x800) /* Interrupt 11 owner */ +#define KVX_SFR_ILR_IT11_SHIFT 11 +#define KVX_SFR_ILR_IT11_WIDTH 1 +#define KVX_SFR_ILR_IT11_WFXL_MASK _ULL(0x800) +#define KVX_SFR_ILR_IT11_WFXL_CLEAR _ULL(0x800) +#define KVX_SFR_ILR_IT11_WFXL_SET _ULL(0x80000000000) + +#define KVX_SFR_ILR_IT12_MASK _ULL(0x1000) /* Interrupt 12 owner */ +#define KVX_SFR_ILR_IT12_SHIFT 12 +#define KVX_SFR_ILR_IT12_WIDTH 1 +#define KVX_SFR_ILR_IT12_WFXL_MASK _ULL(0x1000) +#define KVX_SFR_ILR_IT12_WFXL_CLEAR _ULL(0x1000) +#define KVX_SFR_ILR_IT12_WFXL_SET _ULL(0x100000000000) + +#define KVX_SFR_ILR_IT13_MASK _ULL(0x2000) /* Interrupt 13 owner */ +#define KVX_SFR_ILR_IT13_SHIFT 13 +#define KVX_SFR_ILR_IT13_WIDTH 1 +#define KVX_SFR_ILR_IT13_WFXL_MASK _ULL(0x2000) +#define KVX_SFR_ILR_IT13_WFXL_CLEAR _ULL(0x2000) +#define KVX_SFR_ILR_IT13_WFXL_SET _ULL(0x200000000000) + +#define KVX_SFR_ILR_IT14_MASK _ULL(0x4000) /* Interrupt 14 owner */ +#define KVX_SFR_ILR_IT14_SHIFT 14 +#define KVX_SFR_ILR_IT14_WIDTH 1 +#define KVX_SFR_ILR_IT14_WFXL_MASK _ULL(0x4000) +#define KVX_SFR_ILR_IT14_WFXL_CLEAR _ULL(0x4000) +#define KVX_SFR_ILR_IT14_WFXL_SET _ULL(0x400000000000) + +#define KVX_SFR_ILR_IT15_MASK _ULL(0x8000) /* Interrupt 15 owner */ +#define KVX_SFR_ILR_IT15_SHIFT 15 +#define KVX_SFR_ILR_IT15_WIDTH 1 +#define KVX_SFR_ILR_IT15_WFXL_MASK _ULL(0x8000) +#define KVX_SFR_ILR_IT15_WFXL_CLEAR _ULL(0x8000) +#define KVX_SFR_ILR_IT15_WFXL_SET _ULL(0x800000000000) + +#define KVX_SFR_ILR_IT16_MASK _ULL(0x10000) /* Interrupt 16 owner */ +#define KVX_SFR_ILR_IT16_SHIFT 16 +#define KVX_SFR_ILR_IT16_WIDTH 1 +#define KVX_SFR_ILR_IT16_WFXL_MASK _ULL(0x10000) +#define KVX_SFR_ILR_IT16_WFXL_CLEAR _ULL(0x10000) +#define KVX_SFR_ILR_IT16_WFXL_SET _ULL(0x1000000000000) + +#define KVX_SFR_ILR_IT17_MASK _ULL(0x20000) /* Interrupt 17 owner */ +#define KVX_SFR_ILR_IT17_SHIFT 17 +#define KVX_SFR_ILR_IT17_WIDTH 1 +#define KVX_SFR_ILR_IT17_WFXL_MASK _ULL(0x20000) +#define KVX_SFR_ILR_IT17_WFXL_CLEAR _ULL(0x20000) +#define KVX_SFR_ILR_IT17_WFXL_SET _ULL(0x2000000000000) + +#define KVX_SFR_ILR_IT18_MASK _ULL(0x40000) /* Interrupt 18 owner */ +#define KVX_SFR_ILR_IT18_SHIFT 18 +#define KVX_SFR_ILR_IT18_WIDTH 1 +#define KVX_SFR_ILR_IT18_WFXL_MASK _ULL(0x40000) +#define KVX_SFR_ILR_IT18_WFXL_CLEAR _ULL(0x40000) +#define KVX_SFR_ILR_IT18_WFXL_SET _ULL(0x4000000000000) + +#define KVX_SFR_ILR_IT19_MASK _ULL(0x80000) /* Interrupt 19 owner */ +#define KVX_SFR_ILR_IT19_SHIFT 19 +#define KVX_SFR_ILR_IT19_WIDTH 1 +#define KVX_SFR_ILR_IT19_WFXL_MASK _ULL(0x80000) +#define KVX_SFR_ILR_IT19_WFXL_CLEAR _ULL(0x80000) +#define KVX_SFR_ILR_IT19_WFXL_SET _ULL(0x8000000000000) + +#define KVX_SFR_ILR_IT20_MASK _ULL(0x100000) /* Interrupt 20 owner */ +#define KVX_SFR_ILR_IT20_SHIFT 20 +#define KVX_SFR_ILR_IT20_WIDTH 1 +#define KVX_SFR_ILR_IT20_WFXL_MASK _ULL(0x100000) +#define KVX_SFR_ILR_IT20_WFXL_CLEAR _ULL(0x100000) +#define KVX_SFR_ILR_IT20_WFXL_SET _ULL(0x10000000000000) + +#define KVX_SFR_ILR_IT21_MASK _ULL(0x200000) /* Interrupt 21 owner */ +#define KVX_SFR_ILR_IT21_SHIFT 21 +#define KVX_SFR_ILR_IT21_WIDTH 1 +#define KVX_SFR_ILR_IT21_WFXL_MASK _ULL(0x200000) +#define KVX_SFR_ILR_IT21_WFXL_CLEAR _ULL(0x200000) +#define KVX_SFR_ILR_IT21_WFXL_SET _ULL(0x20000000000000) + +#define KVX_SFR_ILR_IT22_MASK _ULL(0x400000) /* Interrupt 22 owner */ +#define KVX_SFR_ILR_IT22_SHIFT 22 +#define KVX_SFR_ILR_IT22_WIDTH 1 +#define KVX_SFR_ILR_IT22_WFXL_MASK _ULL(0x400000) +#define KVX_SFR_ILR_IT22_WFXL_CLEAR _ULL(0x400000) +#define KVX_SFR_ILR_IT22_WFXL_SET _ULL(0x40000000000000) + +#define KVX_SFR_ILR_IT23_MASK _ULL(0x800000) /* Interrupt 23 owner */ +#define KVX_SFR_ILR_IT23_SHIFT 23 +#define KVX_SFR_ILR_IT23_WIDTH 1 +#define KVX_SFR_ILR_IT23_WFXL_MASK _ULL(0x800000) +#define KVX_SFR_ILR_IT23_WFXL_CLEAR _ULL(0x800000) +#define KVX_SFR_ILR_IT23_WFXL_SET _ULL(0x80000000000000) + +#define KVX_SFR_ILR_IT24_MASK _ULL(0x1000000) /* Interrupt 24 owner */ +#define KVX_SFR_ILR_IT24_SHIFT 24 +#define KVX_SFR_ILR_IT24_WIDTH 1 +#define KVX_SFR_ILR_IT24_WFXL_MASK _ULL(0x1000000) +#define KVX_SFR_ILR_IT24_WFXL_CLEAR _ULL(0x1000000) +#define KVX_SFR_ILR_IT24_WFXL_SET _ULL(0x100000000000000) + +#define KVX_SFR_ILR_IT25_MASK _ULL(0x2000000) /* Interrupt 25 owner */ +#define KVX_SFR_ILR_IT25_SHIFT 25 +#define KVX_SFR_ILR_IT25_WIDTH 1 +#define KVX_SFR_ILR_IT25_WFXL_MASK _ULL(0x2000000) +#define KVX_SFR_ILR_IT25_WFXL_CLEAR _ULL(0x2000000) +#define KVX_SFR_ILR_IT25_WFXL_SET _ULL(0x200000000000000) + +#define KVX_SFR_ILR_IT26_MASK _ULL(0x4000000) /* Interrupt 26 owner */ +#define KVX_SFR_ILR_IT26_SHIFT 26 +#define KVX_SFR_ILR_IT26_WIDTH 1 +#define KVX_SFR_ILR_IT26_WFXL_MASK _ULL(0x4000000) +#define KVX_SFR_ILR_IT26_WFXL_CLEAR _ULL(0x4000000) +#define KVX_SFR_ILR_IT26_WFXL_SET _ULL(0x400000000000000) + +#define KVX_SFR_ILR_IT27_MASK _ULL(0x8000000) /* Interrupt 27 owner */ +#define KVX_SFR_ILR_IT27_SHIFT 27 +#define KVX_SFR_ILR_IT27_WIDTH 1 +#define KVX_SFR_ILR_IT27_WFXL_MASK _ULL(0x8000000) +#define KVX_SFR_ILR_IT27_WFXL_CLEAR _ULL(0x8000000) +#define KVX_SFR_ILR_IT27_WFXL_SET _ULL(0x800000000000000) + +#define KVX_SFR_ILR_IT28_MASK _ULL(0x10000000) /* Interrupt 28 owner */ +#define KVX_SFR_ILR_IT28_SHIFT 28 +#define KVX_SFR_ILR_IT28_WIDTH 1 +#define KVX_SFR_ILR_IT28_WFXL_MASK _ULL(0x10000000) +#define KVX_SFR_ILR_IT28_WFXL_CLEAR _ULL(0x10000000) +#define KVX_SFR_ILR_IT28_WFXL_SET _ULL(0x1000000000000000) + +#define KVX_SFR_ILR_IT29_MASK _ULL(0x20000000) /* Interrupt 29 owner */ +#define KVX_SFR_ILR_IT29_SHIFT 29 +#define KVX_SFR_ILR_IT29_WIDTH 1 +#define KVX_SFR_ILR_IT29_WFXL_MASK _ULL(0x20000000) +#define KVX_SFR_ILR_IT29_WFXL_CLEAR _ULL(0x20000000) +#define KVX_SFR_ILR_IT29_WFXL_SET _ULL(0x2000000000000000) + +#define KVX_SFR_ILR_IT30_MASK _ULL(0x40000000) /* Interrupt 30 owner */ +#define KVX_SFR_ILR_IT30_SHIFT 30 +#define KVX_SFR_ILR_IT30_WIDTH 1 +#define KVX_SFR_ILR_IT30_WFXL_MASK _ULL(0x40000000) +#define KVX_SFR_ILR_IT30_WFXL_CLEAR _ULL(0x40000000) +#define KVX_SFR_ILR_IT30_WFXL_SET _ULL(0x4000000000000000) + +#define KVX_SFR_ILR_IT31_MASK _ULL(0x80000000) /* Interrupt 31 owner */ +#define KVX_SFR_ILR_IT31_SHIFT 31 +#define KVX_SFR_ILR_IT31_WIDTH 1 +#define KVX_SFR_ILR_IT31_WFXL_MASK _ULL(0x80000000) +#define KVX_SFR_ILR_IT31_WFXL_CLEAR _ULL(0x80000000) +#define KVX_SFR_ILR_IT31_WFXL_SET _ULL(0x8000000000000000) + +#define KVX_SFR_ITOW_IT0_MASK _ULL(0x3) /* Interrupt 0 owner */ +#define KVX_SFR_ITOW_IT0_SHIFT 0 +#define KVX_SFR_ITOW_IT0_WIDTH 2 +#define KVX_SFR_ITOW_IT0_WFXL_MASK _ULL(0x3) +#define KVX_SFR_ITOW_IT0_WFXL_CLEAR _ULL(0x3) +#define KVX_SFR_ITOW_IT0_WFXL_SET _ULL(0x300000000) + +#define KVX_SFR_ITOW_IT1_MASK _ULL(0xc) /* Interrupt 1 owner */ +#define KVX_SFR_ITOW_IT1_SHIFT 2 +#define KVX_SFR_ITOW_IT1_WIDTH 2 +#define KVX_SFR_ITOW_IT1_WFXL_MASK _ULL(0xc) +#define KVX_SFR_ITOW_IT1_WFXL_CLEAR _ULL(0xc) +#define KVX_SFR_ITOW_IT1_WFXL_SET _ULL(0xc00000000) + +#define KVX_SFR_ITOW_IT2_MASK _ULL(0x30) /* Interrupt 2 owner */ +#define KVX_SFR_ITOW_IT2_SHIFT 4 +#define KVX_SFR_ITOW_IT2_WIDTH 2 +#define KVX_SFR_ITOW_IT2_WFXL_MASK _ULL(0x30) +#define KVX_SFR_ITOW_IT2_WFXL_CLEAR _ULL(0x30) +#define KVX_SFR_ITOW_IT2_WFXL_SET _ULL(0x3000000000) + +#define KVX_SFR_ITOW_IT3_MASK _ULL(0xc0) /* Interrupt 3 owner */ +#define KVX_SFR_ITOW_IT3_SHIFT 6 +#define KVX_SFR_ITOW_IT3_WIDTH 2 +#define KVX_SFR_ITOW_IT3_WFXL_MASK _ULL(0xc0) +#define KVX_SFR_ITOW_IT3_WFXL_CLEAR _ULL(0xc0) +#define KVX_SFR_ITOW_IT3_WFXL_SET _ULL(0xc000000000) + +#define KVX_SFR_ITOW_IT4_MASK _ULL(0x300) /* Interrupt 4 owner */ +#define KVX_SFR_ITOW_IT4_SHIFT 8 +#define KVX_SFR_ITOW_IT4_WIDTH 2 +#define KVX_SFR_ITOW_IT4_WFXL_MASK _ULL(0x300) +#define KVX_SFR_ITOW_IT4_WFXL_CLEAR _ULL(0x300) +#define KVX_SFR_ITOW_IT4_WFXL_SET _ULL(0x30000000000) + +#define KVX_SFR_ITOW_IT5_MASK _ULL(0xc00) /* Interrupt 5 owner */ +#define KVX_SFR_ITOW_IT5_SHIFT 10 +#define KVX_SFR_ITOW_IT5_WIDTH 2 +#define KVX_SFR_ITOW_IT5_WFXL_MASK _ULL(0xc00) +#define KVX_SFR_ITOW_IT5_WFXL_CLEAR _ULL(0xc00) +#define KVX_SFR_ITOW_IT5_WFXL_SET _ULL(0xc0000000000) + +#define KVX_SFR_ITOW_IT6_MASK _ULL(0x3000) /* Interrupt 6 owner */ +#define KVX_SFR_ITOW_IT6_SHIFT 12 +#define KVX_SFR_ITOW_IT6_WIDTH 2 +#define KVX_SFR_ITOW_IT6_WFXL_MASK _ULL(0x3000) +#define KVX_SFR_ITOW_IT6_WFXL_CLEAR _ULL(0x3000) +#define KVX_SFR_ITOW_IT6_WFXL_SET _ULL(0x300000000000) + +#define KVX_SFR_ITOW_IT7_MASK _ULL(0xc000) /* Interrupt 7 owner */ +#define KVX_SFR_ITOW_IT7_SHIFT 14 +#define KVX_SFR_ITOW_IT7_WIDTH 2 +#define KVX_SFR_ITOW_IT7_WFXL_MASK _ULL(0xc000) +#define KVX_SFR_ITOW_IT7_WFXL_CLEAR _ULL(0xc000) +#define KVX_SFR_ITOW_IT7_WFXL_SET _ULL(0xc00000000000) + +#define KVX_SFR_ITOW_IT8_MASK _ULL(0x30000) /* Interrupt 8 owner */ +#define KVX_SFR_ITOW_IT8_SHIFT 16 +#define KVX_SFR_ITOW_IT8_WIDTH 2 +#define KVX_SFR_ITOW_IT8_WFXL_MASK _ULL(0x30000) +#define KVX_SFR_ITOW_IT8_WFXL_CLEAR _ULL(0x30000) +#define KVX_SFR_ITOW_IT8_WFXL_SET _ULL(0x3000000000000) + +#define KVX_SFR_ITOW_IT9_MASK _ULL(0xc0000) /* Interrupt 9 owner */ +#define KVX_SFR_ITOW_IT9_SHIFT 18 +#define KVX_SFR_ITOW_IT9_WIDTH 2 +#define KVX_SFR_ITOW_IT9_WFXL_MASK _ULL(0xc0000) +#define KVX_SFR_ITOW_IT9_WFXL_CLEAR _ULL(0xc0000) +#define KVX_SFR_ITOW_IT9_WFXL_SET _ULL(0xc000000000000) + +#define KVX_SFR_ITOW_IT10_MASK _ULL(0x300000) /* Interrupt 10 owner */ +#define KVX_SFR_ITOW_IT10_SHIFT 20 +#define KVX_SFR_ITOW_IT10_WIDTH 2 +#define KVX_SFR_ITOW_IT10_WFXL_MASK _ULL(0x300000) +#define KVX_SFR_ITOW_IT10_WFXL_CLEAR _ULL(0x300000) +#define KVX_SFR_ITOW_IT10_WFXL_SET _ULL(0x30000000000000) + +#define KVX_SFR_ITOW_IT11_MASK _ULL(0xc00000) /* Interrupt 11 owner */ +#define KVX_SFR_ITOW_IT11_SHIFT 22 +#define KVX_SFR_ITOW_IT11_WIDTH 2 +#define KVX_SFR_ITOW_IT11_WFXL_MASK _ULL(0xc00000) +#define KVX_SFR_ITOW_IT11_WFXL_CLEAR _ULL(0xc00000) +#define KVX_SFR_ITOW_IT11_WFXL_SET _ULL(0xc0000000000000) + +#define KVX_SFR_ITOW_IT12_MASK _ULL(0x3000000) /* Interrupt 12 owner */ +#define KVX_SFR_ITOW_IT12_SHIFT 24 +#define KVX_SFR_ITOW_IT12_WIDTH 2 +#define KVX_SFR_ITOW_IT12_WFXL_MASK _ULL(0x3000000) +#define KVX_SFR_ITOW_IT12_WFXL_CLEAR _ULL(0x3000000) +#define KVX_SFR_ITOW_IT12_WFXL_SET _ULL(0x300000000000000) + +#define KVX_SFR_ITOW_IT13_MASK _ULL(0xc000000) /* Interrupt 13 owner */ +#define KVX_SFR_ITOW_IT13_SHIFT 26 +#define KVX_SFR_ITOW_IT13_WIDTH 2 +#define KVX_SFR_ITOW_IT13_WFXL_MASK _ULL(0xc000000) +#define KVX_SFR_ITOW_IT13_WFXL_CLEAR _ULL(0xc000000) +#define KVX_SFR_ITOW_IT13_WFXL_SET _ULL(0xc00000000000000) + +#define KVX_SFR_ITOW_IT14_MASK _ULL(0x30000000) /* Interrupt 14 owner */ +#define KVX_SFR_ITOW_IT14_SHIFT 28 +#define KVX_SFR_ITOW_IT14_WIDTH 2 +#define KVX_SFR_ITOW_IT14_WFXL_MASK _ULL(0x30000000) +#define KVX_SFR_ITOW_IT14_WFXL_CLEAR _ULL(0x30000000) +#define KVX_SFR_ITOW_IT14_WFXL_SET _ULL(0x3000000000000000) + +#define KVX_SFR_ITOW_IT15_MASK _ULL(0xc0000000) /* Interrupt 15 owner */ +#define KVX_SFR_ITOW_IT15_SHIFT 30 +#define KVX_SFR_ITOW_IT15_WIDTH 2 +#define KVX_SFR_ITOW_IT15_WFXL_MASK _ULL(0xc0000000) +#define KVX_SFR_ITOW_IT15_WFXL_CLEAR _ULL(0xc0000000) +#define KVX_SFR_ITOW_IT15_WFXL_SET _ULL(0xc000000000000000) + +#define KVX_SFR_ITOW_IT16_MASK _ULL(0x300000000) /* Interrupt 16 owner */ +#define KVX_SFR_ITOW_IT16_SHIFT 32 +#define KVX_SFR_ITOW_IT16_WIDTH 2 +#define KVX_SFR_ITOW_IT16_WFXM_MASK _ULL(0x300000000) +#define KVX_SFR_ITOW_IT16_WFXM_CLEAR _ULL(0x3) +#define KVX_SFR_ITOW_IT16_WFXM_SET _ULL(0x300000000) + +#define KVX_SFR_ITOW_IT17_MASK _ULL(0xc00000000) /* Interrupt 17 owner */ +#define KVX_SFR_ITOW_IT17_SHIFT 34 +#define KVX_SFR_ITOW_IT17_WIDTH 2 +#define KVX_SFR_ITOW_IT17_WFXM_MASK _ULL(0xc00000000) +#define KVX_SFR_ITOW_IT17_WFXM_CLEAR _ULL(0xc) +#define KVX_SFR_ITOW_IT17_WFXM_SET _ULL(0xc00000000) + +#define KVX_SFR_ITOW_IT18_MASK _ULL(0x3000000000) /* Interrupt 18 owner */ +#define KVX_SFR_ITOW_IT18_SHIFT 36 +#define KVX_SFR_ITOW_IT18_WIDTH 2 +#define KVX_SFR_ITOW_IT18_WFXM_MASK _ULL(0x3000000000) +#define KVX_SFR_ITOW_IT18_WFXM_CLEAR _ULL(0x30) +#define KVX_SFR_ITOW_IT18_WFXM_SET _ULL(0x3000000000) + +#define KVX_SFR_ITOW_IT19_MASK _ULL(0xc000000000) /* Interrupt 19 owner */ +#define KVX_SFR_ITOW_IT19_SHIFT 38 +#define KVX_SFR_ITOW_IT19_WIDTH 2 +#define KVX_SFR_ITOW_IT19_WFXM_MASK _ULL(0xc000000000) +#define KVX_SFR_ITOW_IT19_WFXM_CLEAR _ULL(0xc0) +#define KVX_SFR_ITOW_IT19_WFXM_SET _ULL(0xc000000000) + +#define KVX_SFR_ITOW_IT20_MASK _ULL(0x30000000000) /* Interrupt 20 owner */ +#define KVX_SFR_ITOW_IT20_SHIFT 40 +#define KVX_SFR_ITOW_IT20_WIDTH 2 +#define KVX_SFR_ITOW_IT20_WFXM_MASK _ULL(0x30000000000) +#define KVX_SFR_ITOW_IT20_WFXM_CLEAR _ULL(0x300) +#define KVX_SFR_ITOW_IT20_WFXM_SET _ULL(0x30000000000) + +#define KVX_SFR_ITOW_IT21_MASK _ULL(0xc0000000000) /* Interrupt 21 owner */ +#define KVX_SFR_ITOW_IT21_SHIFT 42 +#define KVX_SFR_ITOW_IT21_WIDTH 2 +#define KVX_SFR_ITOW_IT21_WFXM_MASK _ULL(0xc0000000000) +#define KVX_SFR_ITOW_IT21_WFXM_CLEAR _ULL(0xc00) +#define KVX_SFR_ITOW_IT21_WFXM_SET _ULL(0xc0000000000) + +#define KVX_SFR_ITOW_IT22_MASK _ULL(0x300000000000) /* Interrupt 22 owner */ +#define KVX_SFR_ITOW_IT22_SHIFT 44 +#define KVX_SFR_ITOW_IT22_WIDTH 2 +#define KVX_SFR_ITOW_IT22_WFXM_MASK _ULL(0x300000000000) +#define KVX_SFR_ITOW_IT22_WFXM_CLEAR _ULL(0x3000) +#define KVX_SFR_ITOW_IT22_WFXM_SET _ULL(0x300000000000) + +#define KVX_SFR_ITOW_IT23_MASK _ULL(0xc00000000000) /* Interrupt 23 owner */ +#define KVX_SFR_ITOW_IT23_SHIFT 46 +#define KVX_SFR_ITOW_IT23_WIDTH 2 +#define KVX_SFR_ITOW_IT23_WFXM_MASK _ULL(0xc00000000000) +#define KVX_SFR_ITOW_IT23_WFXM_CLEAR _ULL(0xc000) +#define KVX_SFR_ITOW_IT23_WFXM_SET _ULL(0xc00000000000) + +#define KVX_SFR_ITOW_IT24_MASK _ULL(0x3000000000000) /* Interrupt 24 owner */ +#define KVX_SFR_ITOW_IT24_SHIFT 48 +#define KVX_SFR_ITOW_IT24_WIDTH 2 +#define KVX_SFR_ITOW_IT24_WFXM_MASK _ULL(0x3000000000000) +#define KVX_SFR_ITOW_IT24_WFXM_CLEAR _ULL(0x30000) +#define KVX_SFR_ITOW_IT24_WFXM_SET _ULL(0x3000000000000) + +#define KVX_SFR_ITOW_IT25_MASK _ULL(0xc000000000000) /* Interrupt 25 owner */ +#define KVX_SFR_ITOW_IT25_SHIFT 50 +#define KVX_SFR_ITOW_IT25_WIDTH 2 +#define KVX_SFR_ITOW_IT25_WFXM_MASK _ULL(0xc000000000000) +#define KVX_SFR_ITOW_IT25_WFXM_CLEAR _ULL(0xc0000) +#define KVX_SFR_ITOW_IT25_WFXM_SET _ULL(0xc000000000000) + +#define KVX_SFR_ITOW_IT26_MASK _ULL(0x30000000000000) /* Interrupt 26 owner */ +#define KVX_SFR_ITOW_IT26_SHIFT 52 +#define KVX_SFR_ITOW_IT26_WIDTH 2 +#define KVX_SFR_ITOW_IT26_WFXM_MASK _ULL(0x30000000000000) +#define KVX_SFR_ITOW_IT26_WFXM_CLEAR _ULL(0x300000) +#define KVX_SFR_ITOW_IT26_WFXM_SET _ULL(0x30000000000000) + +#define KVX_SFR_ITOW_IT27_MASK _ULL(0xc0000000000000) /* Interrupt 27 owner */ +#define KVX_SFR_ITOW_IT27_SHIFT 54 +#define KVX_SFR_ITOW_IT27_WIDTH 2 +#define KVX_SFR_ITOW_IT27_WFXM_MASK _ULL(0xc0000000000000) +#define KVX_SFR_ITOW_IT27_WFXM_CLEAR _ULL(0xc00000) +#define KVX_SFR_ITOW_IT27_WFXM_SET _ULL(0xc0000000000000) + +#define KVX_SFR_ITOW_IT28_MASK _ULL(0x300000000000000) /* Interrupt 28 owner */ +#define KVX_SFR_ITOW_IT28_SHIFT 56 +#define KVX_SFR_ITOW_IT28_WIDTH 2 +#define KVX_SFR_ITOW_IT28_WFXM_MASK _ULL(0x300000000000000) +#define KVX_SFR_ITOW_IT28_WFXM_CLEAR _ULL(0x3000000) +#define KVX_SFR_ITOW_IT28_WFXM_SET _ULL(0x300000000000000) + +#define KVX_SFR_ITOW_IT29_MASK _ULL(0xc00000000000000) /* Interrupt 29 owner */ +#define KVX_SFR_ITOW_IT29_SHIFT 58 +#define KVX_SFR_ITOW_IT29_WIDTH 2 +#define KVX_SFR_ITOW_IT29_WFXM_MASK _ULL(0xc00000000000000) +#define KVX_SFR_ITOW_IT29_WFXM_CLEAR _ULL(0xc000000) +#define KVX_SFR_ITOW_IT29_WFXM_SET _ULL(0xc00000000000000) + +#define KVX_SFR_ITOW_IT30_MASK _ULL(0x3000000000000000) /* Interrupt 30 owner */ +#define KVX_SFR_ITOW_IT30_SHIFT 60 +#define KVX_SFR_ITOW_IT30_WIDTH 2 +#define KVX_SFR_ITOW_IT30_WFXM_MASK _ULL(0x3000000000000000) +#define KVX_SFR_ITOW_IT30_WFXM_CLEAR _ULL(0x30000000) +#define KVX_SFR_ITOW_IT30_WFXM_SET _ULL(0x3000000000000000) + +#define KVX_SFR_ITOW_IT31_MASK _ULL(0xc000000000000000) /* Interrupt 31 owner */ +#define KVX_SFR_ITOW_IT31_SHIFT 62 +#define KVX_SFR_ITOW_IT31_WIDTH 2 +#define KVX_SFR_ITOW_IT31_WFXM_MASK _ULL(0xc000000000000000) +#define KVX_SFR_ITOW_IT31_WFXM_CLEAR _ULL(0xc0000000) +#define KVX_SFR_ITOW_IT31_WFXM_SET _ULL(0xc000000000000000) + +#define KVX_SFR_DO_B0_MASK _ULL(0x3) /* Breakpoint 0 owner. */ +#define KVX_SFR_DO_B0_SHIFT 0 +#define KVX_SFR_DO_B0_WIDTH 2 +#define KVX_SFR_DO_B0_WFXL_MASK _ULL(0x3) +#define KVX_SFR_DO_B0_WFXL_CLEAR _ULL(0x3) +#define KVX_SFR_DO_B0_WFXL_SET _ULL(0x300000000) + +#define KVX_SFR_DO_B1_MASK _ULL(0xc) /* Breakpoint 1 owner. */ +#define KVX_SFR_DO_B1_SHIFT 2 +#define KVX_SFR_DO_B1_WIDTH 2 +#define KVX_SFR_DO_B1_WFXL_MASK _ULL(0xc) +#define KVX_SFR_DO_B1_WFXL_CLEAR _ULL(0xc) +#define KVX_SFR_DO_B1_WFXL_SET _ULL(0xc00000000) + +#define KVX_SFR_DO_W0_MASK _ULL(0x30) /* Watchpoint 0 owner. */ +#define KVX_SFR_DO_W0_SHIFT 4 +#define KVX_SFR_DO_W0_WIDTH 2 +#define KVX_SFR_DO_W0_WFXL_MASK _ULL(0x30) +#define KVX_SFR_DO_W0_WFXL_CLEAR _ULL(0x30) +#define KVX_SFR_DO_W0_WFXL_SET _ULL(0x3000000000) + +#define KVX_SFR_DO_W1_MASK _ULL(0xc0) /* Watchpoint 1 owner. */ +#define KVX_SFR_DO_W1_SHIFT 6 +#define KVX_SFR_DO_W1_WIDTH 2 +#define KVX_SFR_DO_W1_WFXL_MASK _ULL(0xc0) +#define KVX_SFR_DO_W1_WFXL_CLEAR _ULL(0xc0) +#define KVX_SFR_DO_W1_WFXL_SET _ULL(0xc000000000) + +#define KVX_SFR_DBA0_DBA0_MASK _ULL(0xffffffffffffffff) /* Debug Breakpoint Address 0 */ +#define KVX_SFR_DBA0_DBA0_SHIFT 0 +#define KVX_SFR_DBA0_DBA0_WIDTH 64 +#define KVX_SFR_DBA0_DBA0_WFXL_MASK _ULL(0xffffffff) +#define KVX_SFR_DBA0_DBA0_WFXL_CLEAR _ULL(0xffffffff) +#define KVX_SFR_DBA0_DBA0_WFXL_SET _ULL(0xffffffff00000000) +#define KVX_SFR_DBA0_DBA0_WFXM_MASK _ULL(0xffffffff00000000) +#define KVX_SFR_DBA0_DBA0_WFXM_CLEAR _ULL(0xffffffff) +#define KVX_SFR_DBA0_DBA0_WFXM_SET _ULL(0xffffffff00000000) + +#define KVX_SFR_DBA1_DBA1_MASK _ULL(0xffffffffffffffff) /* Debug Breakpoint Address 1 */ +#define KVX_SFR_DBA1_DBA1_SHIFT 0 +#define KVX_SFR_DBA1_DBA1_WIDTH 64 +#define KVX_SFR_DBA1_DBA1_WFXL_MASK _ULL(0xffffffff) +#define KVX_SFR_DBA1_DBA1_WFXL_CLEAR _ULL(0xffffffff) +#define KVX_SFR_DBA1_DBA1_WFXL_SET _ULL(0xffffffff00000000) +#define KVX_SFR_DBA1_DBA1_WFXM_MASK _ULL(0xffffffff00000000) +#define KVX_SFR_DBA1_DBA1_WFXM_CLEAR _ULL(0xffffffff) +#define KVX_SFR_DBA1_DBA1_WFXM_SET _ULL(0xffffffff00000000) + +#define KVX_SFR_DWA0_DWA0_MASK _ULL(0xffffffffffffffff) /* Debug Breakpoint Address 0 */ +#define KVX_SFR_DWA0_DWA0_SHIFT 0 +#define KVX_SFR_DWA0_DWA0_WIDTH 64 +#define KVX_SFR_DWA0_DWA0_WFXL_MASK _ULL(0xffffffff) +#define KVX_SFR_DWA0_DWA0_WFXL_CLEAR _ULL(0xffffffff) +#define KVX_SFR_DWA0_DWA0_WFXL_SET _ULL(0xffffffff00000000) +#define KVX_SFR_DWA0_DWA0_WFXM_MASK _ULL(0xffffffff00000000) +#define KVX_SFR_DWA0_DWA0_WFXM_CLEAR _ULL(0xffffffff) +#define KVX_SFR_DWA0_DWA0_WFXM_SET _ULL(0xffffffff00000000) + +#define KVX_SFR_DWA1_DWA1_MASK _ULL(0xffffffffffffffff) /* Debug Breakpoint Address 1 */ +#define KVX_SFR_DWA1_DWA1_SHIFT 0 +#define KVX_SFR_DWA1_DWA1_WIDTH 64 +#define KVX_SFR_DWA1_DWA1_WFXL_MASK _ULL(0xffffffff) +#define KVX_SFR_DWA1_DWA1_WFXL_CLEAR _ULL(0xffffffff) +#define KVX_SFR_DWA1_DWA1_WFXL_SET _ULL(0xffffffff00000000) +#define KVX_SFR_DWA1_DWA1_WFXM_MASK _ULL(0xffffffff00000000) +#define KVX_SFR_DWA1_DWA1_WFXM_CLEAR _ULL(0xffffffff) +#define KVX_SFR_DWA1_DWA1_WFXM_SET _ULL(0xffffffff00000000) + +#define KVX_SFR_DOW_B0_MASK _ULL(0x3) /* Breakpoint 0 owner. */ +#define KVX_SFR_DOW_B0_SHIFT 0 +#define KVX_SFR_DOW_B0_WIDTH 2 +#define KVX_SFR_DOW_B0_WFXL_MASK _ULL(0x3) +#define KVX_SFR_DOW_B0_WFXL_CLEAR _ULL(0x3) +#define KVX_SFR_DOW_B0_WFXL_SET _ULL(0x300000000) + +#define KVX_SFR_DOW_B1_MASK _ULL(0xc) /* Breakpoint 1 owner. */ +#define KVX_SFR_DOW_B1_SHIFT 2 +#define KVX_SFR_DOW_B1_WIDTH 2 +#define KVX_SFR_DOW_B1_WFXL_MASK _ULL(0xc) +#define KVX_SFR_DOW_B1_WFXL_CLEAR _ULL(0xc) +#define KVX_SFR_DOW_B1_WFXL_SET _ULL(0xc00000000) + +#define KVX_SFR_DOW_W0_MASK _ULL(0x30) /* Watchpoint 0 owner. */ +#define KVX_SFR_DOW_W0_SHIFT 4 +#define KVX_SFR_DOW_W0_WIDTH 2 +#define KVX_SFR_DOW_W0_WFXL_MASK _ULL(0x30) +#define KVX_SFR_DOW_W0_WFXL_CLEAR _ULL(0x30) +#define KVX_SFR_DOW_W0_WFXL_SET _ULL(0x3000000000) + +#define KVX_SFR_DOW_W1_MASK _ULL(0xc0) /* Watchpoint 1 owner. */ +#define KVX_SFR_DOW_W1_SHIFT 6 +#define KVX_SFR_DOW_W1_WIDTH 2 +#define KVX_SFR_DOW_W1_WFXL_MASK _ULL(0xc0) +#define KVX_SFR_DOW_W1_WFXL_CLEAR _ULL(0xc0) +#define KVX_SFR_DOW_W1_WFXL_SET _ULL(0xc000000000) + +#define KVX_SFR_MO_MMI_MASK _ULL(0x3) /* Memory Management Instructions owner. */ +#define KVX_SFR_MO_MMI_SHIFT 0 +#define KVX_SFR_MO_MMI_WIDTH 2 +#define KVX_SFR_MO_MMI_WFXL_MASK _ULL(0x3) +#define KVX_SFR_MO_MMI_WFXL_CLEAR _ULL(0x3) +#define KVX_SFR_MO_MMI_WFXL_SET _ULL(0x300000000) + +#define KVX_SFR_MO_RFE_MASK _ULL(0xc) /* RFE instruction owner. */ +#define KVX_SFR_MO_RFE_SHIFT 2 +#define KVX_SFR_MO_RFE_WIDTH 2 +#define KVX_SFR_MO_RFE_WFXL_MASK _ULL(0xc) +#define KVX_SFR_MO_RFE_WFXL_CLEAR _ULL(0xc) +#define KVX_SFR_MO_RFE_WFXL_SET _ULL(0xc00000000) + +#define KVX_SFR_MO_STOP_MASK _ULL(0x30) /* STOP instruction owner. */ +#define KVX_SFR_MO_STOP_SHIFT 4 +#define KVX_SFR_MO_STOP_WIDTH 2 +#define KVX_SFR_MO_STOP_WFXL_MASK _ULL(0x30) +#define KVX_SFR_MO_STOP_WFXL_CLEAR _ULL(0x30) +#define KVX_SFR_MO_STOP_WFXL_SET _ULL(0x3000000000) + +#define KVX_SFR_MO_SYNC_MASK _ULL(0xc0) /* SYNCGROUP instruction owner. */ +#define KVX_SFR_MO_SYNC_SHIFT 6 +#define KVX_SFR_MO_SYNC_WIDTH 2 +#define KVX_SFR_MO_SYNC_WFXL_MASK _ULL(0xc0) +#define KVX_SFR_MO_SYNC_WFXL_CLEAR _ULL(0xc0) +#define KVX_SFR_MO_SYNC_WFXL_SET _ULL(0xc000000000) + +#define KVX_SFR_MO_PCR_MASK _ULL(0x300) /* PCR register owner. */ +#define KVX_SFR_MO_PCR_SHIFT 8 +#define KVX_SFR_MO_PCR_WIDTH 2 +#define KVX_SFR_MO_PCR_WFXL_MASK _ULL(0x300) +#define KVX_SFR_MO_PCR_WFXL_CLEAR _ULL(0x300) +#define KVX_SFR_MO_PCR_WFXL_SET _ULL(0x30000000000) + +#define KVX_SFR_MO_MSG_MASK _ULL(0xc00) /* MMU SFR GROUP registers owner. */ +#define KVX_SFR_MO_MSG_SHIFT 10 +#define KVX_SFR_MO_MSG_WIDTH 2 +#define KVX_SFR_MO_MSG_WFXL_MASK _ULL(0xc00) +#define KVX_SFR_MO_MSG_WFXL_CLEAR _ULL(0xc00) +#define KVX_SFR_MO_MSG_WFXL_SET _ULL(0xc0000000000) + +#define KVX_SFR_MO_MEN_MASK _ULL(0x3000) /* Miscellaneous External Notifications register owner. */ +#define KVX_SFR_MO_MEN_SHIFT 12 +#define KVX_SFR_MO_MEN_WIDTH 2 +#define KVX_SFR_MO_MEN_WFXL_MASK _ULL(0x3000) +#define KVX_SFR_MO_MEN_WFXL_CLEAR _ULL(0x3000) +#define KVX_SFR_MO_MEN_WFXL_SET _ULL(0x300000000000) + +#define KVX_SFR_MO_MES_MASK _ULL(0xc000) /* Memory Error Status register owner. */ +#define KVX_SFR_MO_MES_SHIFT 14 +#define KVX_SFR_MO_MES_WIDTH 2 +#define KVX_SFR_MO_MES_WFXL_MASK _ULL(0xc000) +#define KVX_SFR_MO_MES_WFXL_CLEAR _ULL(0xc000) +#define KVX_SFR_MO_MES_WFXL_SET _ULL(0xc00000000000) + +#define KVX_SFR_MO_CSIT_MASK _ULL(0x30000) /* Compute Status Artithmetic Interrupt register owner. */ +#define KVX_SFR_MO_CSIT_SHIFT 16 +#define KVX_SFR_MO_CSIT_WIDTH 2 +#define KVX_SFR_MO_CSIT_WFXL_MASK _ULL(0x30000) +#define KVX_SFR_MO_CSIT_WFXL_CLEAR _ULL(0x30000) +#define KVX_SFR_MO_CSIT_WFXL_SET _ULL(0x3000000000000) + +#define KVX_SFR_MO_T0_MASK _ULL(0xc0000) /* Timer 0 register group owner */ +#define KVX_SFR_MO_T0_SHIFT 18 +#define KVX_SFR_MO_T0_WIDTH 2 +#define KVX_SFR_MO_T0_WFXL_MASK _ULL(0xc0000) +#define KVX_SFR_MO_T0_WFXL_CLEAR _ULL(0xc0000) +#define KVX_SFR_MO_T0_WFXL_SET _ULL(0xc000000000000) + +#define KVX_SFR_MO_T1_MASK _ULL(0x300000) /* Timer 1 register group owner */ +#define KVX_SFR_MO_T1_SHIFT 20 +#define KVX_SFR_MO_T1_WIDTH 2 +#define KVX_SFR_MO_T1_WFXL_MASK _ULL(0x300000) +#define KVX_SFR_MO_T1_WFXL_CLEAR _ULL(0x300000) +#define KVX_SFR_MO_T1_WFXL_SET _ULL(0x30000000000000) + +#define KVX_SFR_MO_WD_MASK _ULL(0xc00000) /* Watch Dog register group owner. */ +#define KVX_SFR_MO_WD_SHIFT 22 +#define KVX_SFR_MO_WD_WIDTH 2 +#define KVX_SFR_MO_WD_WFXL_MASK _ULL(0xc00000) +#define KVX_SFR_MO_WD_WFXL_CLEAR _ULL(0xc00000) +#define KVX_SFR_MO_WD_WFXL_SET _ULL(0xc0000000000000) + +#define KVX_SFR_MO_PM0_MASK _ULL(0x3000000) /* Performance Monitor 0 register owner. */ +#define KVX_SFR_MO_PM0_SHIFT 24 +#define KVX_SFR_MO_PM0_WIDTH 2 +#define KVX_SFR_MO_PM0_WFXL_MASK _ULL(0x3000000) +#define KVX_SFR_MO_PM0_WFXL_CLEAR _ULL(0x3000000) +#define KVX_SFR_MO_PM0_WFXL_SET _ULL(0x300000000000000) + +#define KVX_SFR_MO_PM1_MASK _ULL(0xc000000) /* Performance Monitor 1 register owner. */ +#define KVX_SFR_MO_PM1_SHIFT 26 +#define KVX_SFR_MO_PM1_WIDTH 2 +#define KVX_SFR_MO_PM1_WFXL_MASK _ULL(0xc000000) +#define KVX_SFR_MO_PM1_WFXL_CLEAR _ULL(0xc000000) +#define KVX_SFR_MO_PM1_WFXL_SET _ULL(0xc00000000000000) + +#define KVX_SFR_MO_PM2_MASK _ULL(0x30000000) /* Performance Monitor 2 register owner. */ +#define KVX_SFR_MO_PM2_SHIFT 28 +#define KVX_SFR_MO_PM2_WIDTH 2 +#define KVX_SFR_MO_PM2_WFXL_MASK _ULL(0x30000000) +#define KVX_SFR_MO_PM2_WFXL_CLEAR _ULL(0x30000000) +#define KVX_SFR_MO_PM2_WFXL_SET _ULL(0x3000000000000000) + +#define KVX_SFR_MO_PM3_MASK _ULL(0xc0000000) /* Performance Monitor 3 register owner. */ +#define KVX_SFR_MO_PM3_SHIFT 30 +#define KVX_SFR_MO_PM3_WIDTH 2 +#define KVX_SFR_MO_PM3_WFXL_MASK _ULL(0xc0000000) +#define KVX_SFR_MO_PM3_WFXL_CLEAR _ULL(0xc0000000) +#define KVX_SFR_MO_PM3_WFXL_SET _ULL(0xc000000000000000) + +#define KVX_SFR_MO_PMIT_MASK _ULL(0x300000000) /* Performance Monitor Interrupt register group owner. */ +#define KVX_SFR_MO_PMIT_SHIFT 32 +#define KVX_SFR_MO_PMIT_WIDTH 2 +#define KVX_SFR_MO_PMIT_WFXM_MASK _ULL(0x300000000) +#define KVX_SFR_MO_PMIT_WFXM_CLEAR _ULL(0x3) +#define KVX_SFR_MO_PMIT_WFXM_SET _ULL(0x300000000) + +#define KVX_SFR_MOW_MMI_MASK _ULL(0x3) /* Memory Management Instructions owner. */ +#define KVX_SFR_MOW_MMI_SHIFT 0 +#define KVX_SFR_MOW_MMI_WIDTH 2 +#define KVX_SFR_MOW_MMI_WFXL_MASK _ULL(0x3) +#define KVX_SFR_MOW_MMI_WFXL_CLEAR _ULL(0x3) +#define KVX_SFR_MOW_MMI_WFXL_SET _ULL(0x300000000) + +#define KVX_SFR_MOW_RFE_MASK _ULL(0xc) /* RFE instruction owner. */ +#define KVX_SFR_MOW_RFE_SHIFT 2 +#define KVX_SFR_MOW_RFE_WIDTH 2 +#define KVX_SFR_MOW_RFE_WFXL_MASK _ULL(0xc) +#define KVX_SFR_MOW_RFE_WFXL_CLEAR _ULL(0xc) +#define KVX_SFR_MOW_RFE_WFXL_SET _ULL(0xc00000000) + +#define KVX_SFR_MOW_STOP_MASK _ULL(0x30) /* STOP instruction owner. */ +#define KVX_SFR_MOW_STOP_SHIFT 4 +#define KVX_SFR_MOW_STOP_WIDTH 2 +#define KVX_SFR_MOW_STOP_WFXL_MASK _ULL(0x30) +#define KVX_SFR_MOW_STOP_WFXL_CLEAR _ULL(0x30) +#define KVX_SFR_MOW_STOP_WFXL_SET _ULL(0x3000000000) + +#define KVX_SFR_MOW_SYNC_MASK _ULL(0xc0) /* SYNCGROUP instruction owner. */ +#define KVX_SFR_MOW_SYNC_SHIFT 6 +#define KVX_SFR_MOW_SYNC_WIDTH 2 +#define KVX_SFR_MOW_SYNC_WFXL_MASK _ULL(0xc0) +#define KVX_SFR_MOW_SYNC_WFXL_CLEAR _ULL(0xc0) +#define KVX_SFR_MOW_SYNC_WFXL_SET _ULL(0xc000000000) + +#define KVX_SFR_MOW_PCR_MASK _ULL(0x300) /* PCR register owner. */ +#define KVX_SFR_MOW_PCR_SHIFT 8 +#define KVX_SFR_MOW_PCR_WIDTH 2 +#define KVX_SFR_MOW_PCR_WFXL_MASK _ULL(0x300) +#define KVX_SFR_MOW_PCR_WFXL_CLEAR _ULL(0x300) +#define KVX_SFR_MOW_PCR_WFXL_SET _ULL(0x30000000000) + +#define KVX_SFR_MOW_MSG_MASK _ULL(0xc00) /* MMU SFR GROUP registers owner. */ +#define KVX_SFR_MOW_MSG_SHIFT 10 +#define KVX_SFR_MOW_MSG_WIDTH 2 +#define KVX_SFR_MOW_MSG_WFXL_MASK _ULL(0xc00) +#define KVX_SFR_MOW_MSG_WFXL_CLEAR _ULL(0xc00) +#define KVX_SFR_MOW_MSG_WFXL_SET _ULL(0xc0000000000) + +#define KVX_SFR_MOW_MEN_MASK _ULL(0x3000) /* Miscellaneous External Notifications register owner. */ +#define KVX_SFR_MOW_MEN_SHIFT 12 +#define KVX_SFR_MOW_MEN_WIDTH 2 +#define KVX_SFR_MOW_MEN_WFXL_MASK _ULL(0x3000) +#define KVX_SFR_MOW_MEN_WFXL_CLEAR _ULL(0x3000) +#define KVX_SFR_MOW_MEN_WFXL_SET _ULL(0x300000000000) + +#define KVX_SFR_MOW_MES_MASK _ULL(0xc000) /* Memory Error Status register owner. */ +#define KVX_SFR_MOW_MES_SHIFT 14 +#define KVX_SFR_MOW_MES_WIDTH 2 +#define KVX_SFR_MOW_MES_WFXL_MASK _ULL(0xc000) +#define KVX_SFR_MOW_MES_WFXL_CLEAR _ULL(0xc000) +#define KVX_SFR_MOW_MES_WFXL_SET _ULL(0xc00000000000) + +#define KVX_SFR_MOW_CSIT_MASK _ULL(0x30000) /* Compute Status Artithmetic Interrupt register owner. */ +#define KVX_SFR_MOW_CSIT_SHIFT 16 +#define KVX_SFR_MOW_CSIT_WIDTH 2 +#define KVX_SFR_MOW_CSIT_WFXL_MASK _ULL(0x30000) +#define KVX_SFR_MOW_CSIT_WFXL_CLEAR _ULL(0x30000) +#define KVX_SFR_MOW_CSIT_WFXL_SET _ULL(0x3000000000000) + +#define KVX_SFR_MOW_T0_MASK _ULL(0xc0000) /* Timer 0 register group owner */ +#define KVX_SFR_MOW_T0_SHIFT 18 +#define KVX_SFR_MOW_T0_WIDTH 2 +#define KVX_SFR_MOW_T0_WFXL_MASK _ULL(0xc0000) +#define KVX_SFR_MOW_T0_WFXL_CLEAR _ULL(0xc0000) +#define KVX_SFR_MOW_T0_WFXL_SET _ULL(0xc000000000000) + +#define KVX_SFR_MOW_T1_MASK _ULL(0x300000) /* Timer 1 register group owner */ +#define KVX_SFR_MOW_T1_SHIFT 20 +#define KVX_SFR_MOW_T1_WIDTH 2 +#define KVX_SFR_MOW_T1_WFXL_MASK _ULL(0x300000) +#define KVX_SFR_MOW_T1_WFXL_CLEAR _ULL(0x300000) +#define KVX_SFR_MOW_T1_WFXL_SET _ULL(0x30000000000000) + +#define KVX_SFR_MOW_WD_MASK _ULL(0xc00000) /* Watch Dog register group owner. */ +#define KVX_SFR_MOW_WD_SHIFT 22 +#define KVX_SFR_MOW_WD_WIDTH 2 +#define KVX_SFR_MOW_WD_WFXL_MASK _ULL(0xc00000) +#define KVX_SFR_MOW_WD_WFXL_CLEAR _ULL(0xc00000) +#define KVX_SFR_MOW_WD_WFXL_SET _ULL(0xc0000000000000) + +#define KVX_SFR_MOW_PM0_MASK _ULL(0x3000000) /* Performance Monitor 0 register owner. */ +#define KVX_SFR_MOW_PM0_SHIFT 24 +#define KVX_SFR_MOW_PM0_WIDTH 2 +#define KVX_SFR_MOW_PM0_WFXL_MASK _ULL(0x3000000) +#define KVX_SFR_MOW_PM0_WFXL_CLEAR _ULL(0x3000000) +#define KVX_SFR_MOW_PM0_WFXL_SET _ULL(0x300000000000000) + +#define KVX_SFR_MOW_PM1_MASK _ULL(0xc000000) /* Performance Monitor 1 register owner. */ +#define KVX_SFR_MOW_PM1_SHIFT 26 +#define KVX_SFR_MOW_PM1_WIDTH 2 +#define KVX_SFR_MOW_PM1_WFXL_MASK _ULL(0xc000000) +#define KVX_SFR_MOW_PM1_WFXL_CLEAR _ULL(0xc000000) +#define KVX_SFR_MOW_PM1_WFXL_SET _ULL(0xc00000000000000) + +#define KVX_SFR_MOW_PM2_MASK _ULL(0x30000000) /* Performance Monitor 2 register owner. */ +#define KVX_SFR_MOW_PM2_SHIFT 28 +#define KVX_SFR_MOW_PM2_WIDTH 2 +#define KVX_SFR_MOW_PM2_WFXL_MASK _ULL(0x30000000) +#define KVX_SFR_MOW_PM2_WFXL_CLEAR _ULL(0x30000000) +#define KVX_SFR_MOW_PM2_WFXL_SET _ULL(0x3000000000000000) + +#define KVX_SFR_MOW_PM3_MASK _ULL(0xc0000000) /* Performance Monitor 3 register owner. */ +#define KVX_SFR_MOW_PM3_SHIFT 30 +#define KVX_SFR_MOW_PM3_WIDTH 2 +#define KVX_SFR_MOW_PM3_WFXL_MASK _ULL(0xc0000000) +#define KVX_SFR_MOW_PM3_WFXL_CLEAR _ULL(0xc0000000) +#define KVX_SFR_MOW_PM3_WFXL_SET _ULL(0xc000000000000000) + +#define KVX_SFR_MOW_PMIT_MASK _ULL(0x300000000) /* Performance Monitor Interrupt register group owner. */ +#define KVX_SFR_MOW_PMIT_SHIFT 32 +#define KVX_SFR_MOW_PMIT_WIDTH 2 +#define KVX_SFR_MOW_PMIT_WFXM_MASK _ULL(0x300000000) +#define KVX_SFR_MOW_PMIT_WFXM_CLEAR _ULL(0x3) +#define KVX_SFR_MOW_PMIT_WFXM_SET _ULL(0x300000000) + +#define KVX_SFR_PS_PL_MASK _ULL(0x3) /* Current Privilege Level */ +#define KVX_SFR_PS_PL_SHIFT 0 +#define KVX_SFR_PS_PL_WIDTH 2 +#define KVX_SFR_PS_PL_WFXL_MASK _ULL(0x3) +#define KVX_SFR_PS_PL_WFXL_CLEAR _ULL(0x3) +#define KVX_SFR_PS_PL_WFXL_SET _ULL(0x300000000) + +#define KVX_SFR_PS_ET_MASK _ULL(0x4) /* Exception Taken */ +#define KVX_SFR_PS_ET_SHIFT 2 +#define KVX_SFR_PS_ET_WIDTH 1 +#define KVX_SFR_PS_ET_WFXL_MASK _ULL(0x4) +#define KVX_SFR_PS_ET_WFXL_CLEAR _ULL(0x4) +#define KVX_SFR_PS_ET_WFXL_SET _ULL(0x400000000) + +#define KVX_SFR_PS_HTD_MASK _ULL(0x8) /* Hardware Trap Disable */ +#define KVX_SFR_PS_HTD_SHIFT 3 +#define KVX_SFR_PS_HTD_WIDTH 1 +#define KVX_SFR_PS_HTD_WFXL_MASK _ULL(0x8) +#define KVX_SFR_PS_HTD_WFXL_CLEAR _ULL(0x8) +#define KVX_SFR_PS_HTD_WFXL_SET _ULL(0x800000000) + +#define KVX_SFR_PS_IE_MASK _ULL(0x10) /* Interrupt Enable */ +#define KVX_SFR_PS_IE_SHIFT 4 +#define KVX_SFR_PS_IE_WIDTH 1 +#define KVX_SFR_PS_IE_WFXL_MASK _ULL(0x10) +#define KVX_SFR_PS_IE_WFXL_CLEAR _ULL(0x10) +#define KVX_SFR_PS_IE_WFXL_SET _ULL(0x1000000000) + +#define KVX_SFR_PS_HLE_MASK _ULL(0x20) /* Hardware Loop Enable */ +#define KVX_SFR_PS_HLE_SHIFT 5 +#define KVX_SFR_PS_HLE_WIDTH 1 +#define KVX_SFR_PS_HLE_WFXL_MASK _ULL(0x20) +#define KVX_SFR_PS_HLE_WFXL_CLEAR _ULL(0x20) +#define KVX_SFR_PS_HLE_WFXL_SET _ULL(0x2000000000) + +#define KVX_SFR_PS_SRE_MASK _ULL(0x40) /* Software REserved */ +#define KVX_SFR_PS_SRE_SHIFT 6 +#define KVX_SFR_PS_SRE_WIDTH 1 +#define KVX_SFR_PS_SRE_WFXL_MASK _ULL(0x40) +#define KVX_SFR_PS_SRE_WFXL_CLEAR _ULL(0x40) +#define KVX_SFR_PS_SRE_WFXL_SET _ULL(0x4000000000) + +#define KVX_SFR_PS_DAUS_MASK _ULL(0x80) /* Data Accesses Use SPS settings */ +#define KVX_SFR_PS_DAUS_SHIFT 7 +#define KVX_SFR_PS_DAUS_WIDTH 1 +#define KVX_SFR_PS_DAUS_WFXL_MASK _ULL(0x80) +#define KVX_SFR_PS_DAUS_WFXL_CLEAR _ULL(0x80) +#define KVX_SFR_PS_DAUS_WFXL_SET _ULL(0x8000000000) + +#define KVX_SFR_PS_ICE_MASK _ULL(0x100) /* Instruction Cache Enable */ +#define KVX_SFR_PS_ICE_SHIFT 8 +#define KVX_SFR_PS_ICE_WIDTH 1 +#define KVX_SFR_PS_ICE_WFXL_MASK _ULL(0x100) +#define KVX_SFR_PS_ICE_WFXL_CLEAR _ULL(0x100) +#define KVX_SFR_PS_ICE_WFXL_SET _ULL(0x10000000000) + +#define KVX_SFR_PS_USE_MASK _ULL(0x200) /* Uncached Streaming Enable */ +#define KVX_SFR_PS_USE_SHIFT 9 +#define KVX_SFR_PS_USE_WIDTH 1 +#define KVX_SFR_PS_USE_WFXL_MASK _ULL(0x200) +#define KVX_SFR_PS_USE_WFXL_CLEAR _ULL(0x200) +#define KVX_SFR_PS_USE_WFXL_SET _ULL(0x20000000000) + +#define KVX_SFR_PS_DCE_MASK _ULL(0x400) /* Data Cache Enable */ +#define KVX_SFR_PS_DCE_SHIFT 10 +#define KVX_SFR_PS_DCE_WIDTH 1 +#define KVX_SFR_PS_DCE_WFXL_MASK _ULL(0x400) +#define KVX_SFR_PS_DCE_WFXL_CLEAR _ULL(0x400) +#define KVX_SFR_PS_DCE_WFXL_SET _ULL(0x40000000000) + +#define KVX_SFR_PS_MME_MASK _ULL(0x800) /* Memory Management Enable */ +#define KVX_SFR_PS_MME_SHIFT 11 +#define KVX_SFR_PS_MME_WIDTH 1 +#define KVX_SFR_PS_MME_WFXL_MASK _ULL(0x800) +#define KVX_SFR_PS_MME_WFXL_CLEAR _ULL(0x800) +#define KVX_SFR_PS_MME_WFXL_SET _ULL(0x80000000000) + +#define KVX_SFR_PS_IL_MASK _ULL(0x3000) /* Interrupt Level */ +#define KVX_SFR_PS_IL_SHIFT 12 +#define KVX_SFR_PS_IL_WIDTH 2 +#define KVX_SFR_PS_IL_WFXL_MASK _ULL(0x3000) +#define KVX_SFR_PS_IL_WFXL_CLEAR _ULL(0x3000) +#define KVX_SFR_PS_IL_WFXL_SET _ULL(0x300000000000) + +#define KVX_SFR_PS_VS_MASK _ULL(0xc000) /* Virtual Space */ +#define KVX_SFR_PS_VS_SHIFT 14 +#define KVX_SFR_PS_VS_WIDTH 2 +#define KVX_SFR_PS_VS_WFXL_MASK _ULL(0xc000) +#define KVX_SFR_PS_VS_WFXL_CLEAR _ULL(0xc000) +#define KVX_SFR_PS_VS_WFXL_SET _ULL(0xc00000000000) + +#define KVX_SFR_PS_V64_MASK _ULL(0x10000) /* Virtual 64 bits mode. */ +#define KVX_SFR_PS_V64_SHIFT 16 +#define KVX_SFR_PS_V64_WIDTH 1 +#define KVX_SFR_PS_V64_WFXL_MASK _ULL(0x10000) +#define KVX_SFR_PS_V64_WFXL_CLEAR _ULL(0x10000) +#define KVX_SFR_PS_V64_WFXL_SET _ULL(0x1000000000000) + +#define KVX_SFR_PS_L2E_MASK _ULL(0x20000) /* L2 cache Enable. */ +#define KVX_SFR_PS_L2E_SHIFT 17 +#define KVX_SFR_PS_L2E_WIDTH 1 +#define KVX_SFR_PS_L2E_WFXL_MASK _ULL(0x20000) +#define KVX_SFR_PS_L2E_WFXL_CLEAR _ULL(0x20000) +#define KVX_SFR_PS_L2E_WFXL_SET _ULL(0x2000000000000) + +#define KVX_SFR_PS_SME_MASK _ULL(0x40000) /* Step Mode Enabled */ +#define KVX_SFR_PS_SME_SHIFT 18 +#define KVX_SFR_PS_SME_WIDTH 1 +#define KVX_SFR_PS_SME_WFXL_MASK _ULL(0x40000) +#define KVX_SFR_PS_SME_WFXL_CLEAR _ULL(0x40000) +#define KVX_SFR_PS_SME_WFXL_SET _ULL(0x4000000000000) + +#define KVX_SFR_PS_SMR_MASK _ULL(0x80000) /* Step Mode Ready */ +#define KVX_SFR_PS_SMR_SHIFT 19 +#define KVX_SFR_PS_SMR_WIDTH 1 +#define KVX_SFR_PS_SMR_WFXL_MASK _ULL(0x80000) +#define KVX_SFR_PS_SMR_WFXL_CLEAR _ULL(0x80000) +#define KVX_SFR_PS_SMR_WFXL_SET _ULL(0x8000000000000) + +#define KVX_SFR_PS_PMJ_MASK _ULL(0xf00000) /* Page Mask in JTLB. */ +#define KVX_SFR_PS_PMJ_SHIFT 20 +#define KVX_SFR_PS_PMJ_WIDTH 4 +#define KVX_SFR_PS_PMJ_WFXL_MASK _ULL(0xf00000) +#define KVX_SFR_PS_PMJ_WFXL_CLEAR _ULL(0xf00000) +#define KVX_SFR_PS_PMJ_WFXL_SET _ULL(0xf0000000000000) + +#define KVX_SFR_PS_MMUP_MASK _ULL(0x1000000) /* Privileged on MMU. */ +#define KVX_SFR_PS_MMUP_SHIFT 24 +#define KVX_SFR_PS_MMUP_WIDTH 1 +#define KVX_SFR_PS_MMUP_WFXL_MASK _ULL(0x1000000) +#define KVX_SFR_PS_MMUP_WFXL_CLEAR _ULL(0x1000000) +#define KVX_SFR_PS_MMUP_WFXL_SET _ULL(0x100000000000000) + +#define KVX_SFR_SPS_PL_MASK _ULL(0x3) /* Current Privilege Level */ +#define KVX_SFR_SPS_PL_SHIFT 0 +#define KVX_SFR_SPS_PL_WIDTH 2 +#define KVX_SFR_SPS_PL_WFXL_MASK _ULL(0x3) +#define KVX_SFR_SPS_PL_WFXL_CLEAR _ULL(0x3) +#define KVX_SFR_SPS_PL_WFXL_SET _ULL(0x300000000) + +#define KVX_SFR_SPS_ET_MASK _ULL(0x4) /* Exception Taken */ +#define KVX_SFR_SPS_ET_SHIFT 2 +#define KVX_SFR_SPS_ET_WIDTH 1 +#define KVX_SFR_SPS_ET_WFXL_MASK _ULL(0x4) +#define KVX_SFR_SPS_ET_WFXL_CLEAR _ULL(0x4) +#define KVX_SFR_SPS_ET_WFXL_SET _ULL(0x400000000) + +#define KVX_SFR_SPS_HTD_MASK _ULL(0x8) /* Hardware Trap Disable */ +#define KVX_SFR_SPS_HTD_SHIFT 3 +#define KVX_SFR_SPS_HTD_WIDTH 1 +#define KVX_SFR_SPS_HTD_WFXL_MASK _ULL(0x8) +#define KVX_SFR_SPS_HTD_WFXL_CLEAR _ULL(0x8) +#define KVX_SFR_SPS_HTD_WFXL_SET _ULL(0x800000000) + +#define KVX_SFR_SPS_IE_MASK _ULL(0x10) /* Interrupt Enable */ +#define KVX_SFR_SPS_IE_SHIFT 4 +#define KVX_SFR_SPS_IE_WIDTH 1 +#define KVX_SFR_SPS_IE_WFXL_MASK _ULL(0x10) +#define KVX_SFR_SPS_IE_WFXL_CLEAR _ULL(0x10) +#define KVX_SFR_SPS_IE_WFXL_SET _ULL(0x1000000000) + +#define KVX_SFR_SPS_HLE_MASK _ULL(0x20) /* Hardware Loop Enable */ +#define KVX_SFR_SPS_HLE_SHIFT 5 +#define KVX_SFR_SPS_HLE_WIDTH 1 +#define KVX_SFR_SPS_HLE_WFXL_MASK _ULL(0x20) +#define KVX_SFR_SPS_HLE_WFXL_CLEAR _ULL(0x20) +#define KVX_SFR_SPS_HLE_WFXL_SET _ULL(0x2000000000) + +#define KVX_SFR_SPS_SRE_MASK _ULL(0x40) /* Software REserved */ +#define KVX_SFR_SPS_SRE_SHIFT 6 +#define KVX_SFR_SPS_SRE_WIDTH 1 +#define KVX_SFR_SPS_SRE_WFXL_MASK _ULL(0x40) +#define KVX_SFR_SPS_SRE_WFXL_CLEAR _ULL(0x40) +#define KVX_SFR_SPS_SRE_WFXL_SET _ULL(0x4000000000) + +#define KVX_SFR_SPS_DAUS_MASK _ULL(0x80) /* Data Accesses Use SPS settings */ +#define KVX_SFR_SPS_DAUS_SHIFT 7 +#define KVX_SFR_SPS_DAUS_WIDTH 1 +#define KVX_SFR_SPS_DAUS_WFXL_MASK _ULL(0x80) +#define KVX_SFR_SPS_DAUS_WFXL_CLEAR _ULL(0x80) +#define KVX_SFR_SPS_DAUS_WFXL_SET _ULL(0x8000000000) + +#define KVX_SFR_SPS_ICE_MASK _ULL(0x100) /* Instruction Cache Enable */ +#define KVX_SFR_SPS_ICE_SHIFT 8 +#define KVX_SFR_SPS_ICE_WIDTH 1 +#define KVX_SFR_SPS_ICE_WFXL_MASK _ULL(0x100) +#define KVX_SFR_SPS_ICE_WFXL_CLEAR _ULL(0x100) +#define KVX_SFR_SPS_ICE_WFXL_SET _ULL(0x10000000000) + +#define KVX_SFR_SPS_USE_MASK _ULL(0x200) /* Uncached Streaming Enable */ +#define KVX_SFR_SPS_USE_SHIFT 9 +#define KVX_SFR_SPS_USE_WIDTH 1 +#define KVX_SFR_SPS_USE_WFXL_MASK _ULL(0x200) +#define KVX_SFR_SPS_USE_WFXL_CLEAR _ULL(0x200) +#define KVX_SFR_SPS_USE_WFXL_SET _ULL(0x20000000000) + +#define KVX_SFR_SPS_DCE_MASK _ULL(0x400) /* Data Cache Enable */ +#define KVX_SFR_SPS_DCE_SHIFT 10 +#define KVX_SFR_SPS_DCE_WIDTH 1 +#define KVX_SFR_SPS_DCE_WFXL_MASK _ULL(0x400) +#define KVX_SFR_SPS_DCE_WFXL_CLEAR _ULL(0x400) +#define KVX_SFR_SPS_DCE_WFXL_SET _ULL(0x40000000000) + +#define KVX_SFR_SPS_MME_MASK _ULL(0x800) /* Memory Management Enable */ +#define KVX_SFR_SPS_MME_SHIFT 11 +#define KVX_SFR_SPS_MME_WIDTH 1 +#define KVX_SFR_SPS_MME_WFXL_MASK _ULL(0x800) +#define KVX_SFR_SPS_MME_WFXL_CLEAR _ULL(0x800) +#define KVX_SFR_SPS_MME_WFXL_SET _ULL(0x80000000000) + +#define KVX_SFR_SPS_IL_MASK _ULL(0x3000) /* Interrupt Level */ +#define KVX_SFR_SPS_IL_SHIFT 12 +#define KVX_SFR_SPS_IL_WIDTH 2 +#define KVX_SFR_SPS_IL_WFXL_MASK _ULL(0x3000) +#define KVX_SFR_SPS_IL_WFXL_CLEAR _ULL(0x3000) +#define KVX_SFR_SPS_IL_WFXL_SET _ULL(0x300000000000) + +#define KVX_SFR_SPS_VS_MASK _ULL(0xc000) /* Virtual Space */ +#define KVX_SFR_SPS_VS_SHIFT 14 +#define KVX_SFR_SPS_VS_WIDTH 2 +#define KVX_SFR_SPS_VS_WFXL_MASK _ULL(0xc000) +#define KVX_SFR_SPS_VS_WFXL_CLEAR _ULL(0xc000) +#define KVX_SFR_SPS_VS_WFXL_SET _ULL(0xc00000000000) + +#define KVX_SFR_SPS_V64_MASK _ULL(0x10000) /* Virtual 64 bits mode. */ +#define KVX_SFR_SPS_V64_SHIFT 16 +#define KVX_SFR_SPS_V64_WIDTH 1 +#define KVX_SFR_SPS_V64_WFXL_MASK _ULL(0x10000) +#define KVX_SFR_SPS_V64_WFXL_CLEAR _ULL(0x10000) +#define KVX_SFR_SPS_V64_WFXL_SET _ULL(0x1000000000000) + +#define KVX_SFR_SPS_L2E_MASK _ULL(0x20000) /* L2 cache Enable. */ +#define KVX_SFR_SPS_L2E_SHIFT 17 +#define KVX_SFR_SPS_L2E_WIDTH 1 +#define KVX_SFR_SPS_L2E_WFXL_MASK _ULL(0x20000) +#define KVX_SFR_SPS_L2E_WFXL_CLEAR _ULL(0x20000) +#define KVX_SFR_SPS_L2E_WFXL_SET _ULL(0x2000000000000) + +#define KVX_SFR_SPS_SME_MASK _ULL(0x40000) /* Step Mode Enabled */ +#define KVX_SFR_SPS_SME_SHIFT 18 +#define KVX_SFR_SPS_SME_WIDTH 1 +#define KVX_SFR_SPS_SME_WFXL_MASK _ULL(0x40000) +#define KVX_SFR_SPS_SME_WFXL_CLEAR _ULL(0x40000) +#define KVX_SFR_SPS_SME_WFXL_SET _ULL(0x4000000000000) + +#define KVX_SFR_SPS_SMR_MASK _ULL(0x80000) /* Step Mode Ready */ +#define KVX_SFR_SPS_SMR_SHIFT 19 +#define KVX_SFR_SPS_SMR_WIDTH 1 +#define KVX_SFR_SPS_SMR_WFXL_MASK _ULL(0x80000) +#define KVX_SFR_SPS_SMR_WFXL_CLEAR _ULL(0x80000) +#define KVX_SFR_SPS_SMR_WFXL_SET _ULL(0x8000000000000) + +#define KVX_SFR_SPS_PMJ_MASK _ULL(0xf00000) /* Page Mask in JTLB. */ +#define KVX_SFR_SPS_PMJ_SHIFT 20 +#define KVX_SFR_SPS_PMJ_WIDTH 4 +#define KVX_SFR_SPS_PMJ_WFXL_MASK _ULL(0xf00000) +#define KVX_SFR_SPS_PMJ_WFXL_CLEAR _ULL(0xf00000) +#define KVX_SFR_SPS_PMJ_WFXL_SET _ULL(0xf0000000000000) + +#define KVX_SFR_SPS_MMUP_MASK _ULL(0x1000000) /* Privileged on MMU. */ +#define KVX_SFR_SPS_MMUP_SHIFT 24 +#define KVX_SFR_SPS_MMUP_WIDTH 1 +#define KVX_SFR_SPS_MMUP_WFXL_MASK _ULL(0x1000000) +#define KVX_SFR_SPS_MMUP_WFXL_CLEAR _ULL(0x1000000) +#define KVX_SFR_SPS_MMUP_WFXL_SET _ULL(0x100000000000000) + +#define KVX_SFR_SPS_PL0_PL_MASK _ULL(0x3) /* Current Privilege Level */ +#define KVX_SFR_SPS_PL0_PL_SHIFT 0 +#define KVX_SFR_SPS_PL0_PL_WIDTH 2 +#define KVX_SFR_SPS_PL0_PL_WFXL_MASK _ULL(0x3) +#define KVX_SFR_SPS_PL0_PL_WFXL_CLEAR _ULL(0x3) +#define KVX_SFR_SPS_PL0_PL_WFXL_SET _ULL(0x300000000) + +#define KVX_SFR_SPS_PL0_ET_MASK _ULL(0x4) /* Exception Taken */ +#define KVX_SFR_SPS_PL0_ET_SHIFT 2 +#define KVX_SFR_SPS_PL0_ET_WIDTH 1 +#define KVX_SFR_SPS_PL0_ET_WFXL_MASK _ULL(0x4) +#define KVX_SFR_SPS_PL0_ET_WFXL_CLEAR _ULL(0x4) +#define KVX_SFR_SPS_PL0_ET_WFXL_SET _ULL(0x400000000) + +#define KVX_SFR_SPS_PL0_HTD_MASK _ULL(0x8) /* Hardware Trap Disable */ +#define KVX_SFR_SPS_PL0_HTD_SHIFT 3 +#define KVX_SFR_SPS_PL0_HTD_WIDTH 1 +#define KVX_SFR_SPS_PL0_HTD_WFXL_MASK _ULL(0x8) +#define KVX_SFR_SPS_PL0_HTD_WFXL_CLEAR _ULL(0x8) +#define KVX_SFR_SPS_PL0_HTD_WFXL_SET _ULL(0x800000000) + +#define KVX_SFR_SPS_PL0_IE_MASK _ULL(0x10) /* Interrupt Enable */ +#define KVX_SFR_SPS_PL0_IE_SHIFT 4 +#define KVX_SFR_SPS_PL0_IE_WIDTH 1 +#define KVX_SFR_SPS_PL0_IE_WFXL_MASK _ULL(0x10) +#define KVX_SFR_SPS_PL0_IE_WFXL_CLEAR _ULL(0x10) +#define KVX_SFR_SPS_PL0_IE_WFXL_SET _ULL(0x1000000000) + +#define KVX_SFR_SPS_PL0_HLE_MASK _ULL(0x20) /* Hardware Loop Enable */ +#define KVX_SFR_SPS_PL0_HLE_SHIFT 5 +#define KVX_SFR_SPS_PL0_HLE_WIDTH 1 +#define KVX_SFR_SPS_PL0_HLE_WFXL_MASK _ULL(0x20) +#define KVX_SFR_SPS_PL0_HLE_WFXL_CLEAR _ULL(0x20) +#define KVX_SFR_SPS_PL0_HLE_WFXL_SET _ULL(0x2000000000) + +#define KVX_SFR_SPS_PL0_SRE_MASK _ULL(0x40) /* Software REserved */ +#define KVX_SFR_SPS_PL0_SRE_SHIFT 6 +#define KVX_SFR_SPS_PL0_SRE_WIDTH 1 +#define KVX_SFR_SPS_PL0_SRE_WFXL_MASK _ULL(0x40) +#define KVX_SFR_SPS_PL0_SRE_WFXL_CLEAR _ULL(0x40) +#define KVX_SFR_SPS_PL0_SRE_WFXL_SET _ULL(0x4000000000) + +#define KVX_SFR_SPS_PL0_DAUS_MASK _ULL(0x80) /* Data Accesses Use SPS settings */ +#define KVX_SFR_SPS_PL0_DAUS_SHIFT 7 +#define KVX_SFR_SPS_PL0_DAUS_WIDTH 1 +#define KVX_SFR_SPS_PL0_DAUS_WFXL_MASK _ULL(0x80) +#define KVX_SFR_SPS_PL0_DAUS_WFXL_CLEAR _ULL(0x80) +#define KVX_SFR_SPS_PL0_DAUS_WFXL_SET _ULL(0x8000000000) + +#define KVX_SFR_SPS_PL0_ICE_MASK _ULL(0x100) /* Instruction Cache Enable */ +#define KVX_SFR_SPS_PL0_ICE_SHIFT 8 +#define KVX_SFR_SPS_PL0_ICE_WIDTH 1 +#define KVX_SFR_SPS_PL0_ICE_WFXL_MASK _ULL(0x100) +#define KVX_SFR_SPS_PL0_ICE_WFXL_CLEAR _ULL(0x100) +#define KVX_SFR_SPS_PL0_ICE_WFXL_SET _ULL(0x10000000000) + +#define KVX_SFR_SPS_PL0_USE_MASK _ULL(0x200) /* Uncached Streaming Enable */ +#define KVX_SFR_SPS_PL0_USE_SHIFT 9 +#define KVX_SFR_SPS_PL0_USE_WIDTH 1 +#define KVX_SFR_SPS_PL0_USE_WFXL_MASK _ULL(0x200) +#define KVX_SFR_SPS_PL0_USE_WFXL_CLEAR _ULL(0x200) +#define KVX_SFR_SPS_PL0_USE_WFXL_SET _ULL(0x20000000000) + +#define KVX_SFR_SPS_PL0_DCE_MASK _ULL(0x400) /* Data Cache Enable */ +#define KVX_SFR_SPS_PL0_DCE_SHIFT 10 +#define KVX_SFR_SPS_PL0_DCE_WIDTH 1 +#define KVX_SFR_SPS_PL0_DCE_WFXL_MASK _ULL(0x400) +#define KVX_SFR_SPS_PL0_DCE_WFXL_CLEAR _ULL(0x400) +#define KVX_SFR_SPS_PL0_DCE_WFXL_SET _ULL(0x40000000000) + +#define KVX_SFR_SPS_PL0_MME_MASK _ULL(0x800) /* Memory Management Enable */ +#define KVX_SFR_SPS_PL0_MME_SHIFT 11 +#define KVX_SFR_SPS_PL0_MME_WIDTH 1 +#define KVX_SFR_SPS_PL0_MME_WFXL_MASK _ULL(0x800) +#define KVX_SFR_SPS_PL0_MME_WFXL_CLEAR _ULL(0x800) +#define KVX_SFR_SPS_PL0_MME_WFXL_SET _ULL(0x80000000000) + +#define KVX_SFR_SPS_PL0_IL_MASK _ULL(0x3000) /* Interrupt Level */ +#define KVX_SFR_SPS_PL0_IL_SHIFT 12 +#define KVX_SFR_SPS_PL0_IL_WIDTH 2 +#define KVX_SFR_SPS_PL0_IL_WFXL_MASK _ULL(0x3000) +#define KVX_SFR_SPS_PL0_IL_WFXL_CLEAR _ULL(0x3000) +#define KVX_SFR_SPS_PL0_IL_WFXL_SET _ULL(0x300000000000) + +#define KVX_SFR_SPS_PL0_VS_MASK _ULL(0xc000) /* Virtual Space */ +#define KVX_SFR_SPS_PL0_VS_SHIFT 14 +#define KVX_SFR_SPS_PL0_VS_WIDTH 2 +#define KVX_SFR_SPS_PL0_VS_WFXL_MASK _ULL(0xc000) +#define KVX_SFR_SPS_PL0_VS_WFXL_CLEAR _ULL(0xc000) +#define KVX_SFR_SPS_PL0_VS_WFXL_SET _ULL(0xc00000000000) + +#define KVX_SFR_SPS_PL0_V64_MASK _ULL(0x10000) /* Virtual 64 bits mode. */ +#define KVX_SFR_SPS_PL0_V64_SHIFT 16 +#define KVX_SFR_SPS_PL0_V64_WIDTH 1 +#define KVX_SFR_SPS_PL0_V64_WFXL_MASK _ULL(0x10000) +#define KVX_SFR_SPS_PL0_V64_WFXL_CLEAR _ULL(0x10000) +#define KVX_SFR_SPS_PL0_V64_WFXL_SET _ULL(0x1000000000000) + +#define KVX_SFR_SPS_PL0_L2E_MASK _ULL(0x20000) /* L2 cache Enable. */ +#define KVX_SFR_SPS_PL0_L2E_SHIFT 17 +#define KVX_SFR_SPS_PL0_L2E_WIDTH 1 +#define KVX_SFR_SPS_PL0_L2E_WFXL_MASK _ULL(0x20000) +#define KVX_SFR_SPS_PL0_L2E_WFXL_CLEAR _ULL(0x20000) +#define KVX_SFR_SPS_PL0_L2E_WFXL_SET _ULL(0x2000000000000) + +#define KVX_SFR_SPS_PL0_SME_MASK _ULL(0x40000) /* Step Mode Enabled */ +#define KVX_SFR_SPS_PL0_SME_SHIFT 18 +#define KVX_SFR_SPS_PL0_SME_WIDTH 1 +#define KVX_SFR_SPS_PL0_SME_WFXL_MASK _ULL(0x40000) +#define KVX_SFR_SPS_PL0_SME_WFXL_CLEAR _ULL(0x40000) +#define KVX_SFR_SPS_PL0_SME_WFXL_SET _ULL(0x4000000000000) + +#define KVX_SFR_SPS_PL0_SMR_MASK _ULL(0x80000) /* Step Mode Ready */ +#define KVX_SFR_SPS_PL0_SMR_SHIFT 19 +#define KVX_SFR_SPS_PL0_SMR_WIDTH 1 +#define KVX_SFR_SPS_PL0_SMR_WFXL_MASK _ULL(0x80000) +#define KVX_SFR_SPS_PL0_SMR_WFXL_CLEAR _ULL(0x80000) +#define KVX_SFR_SPS_PL0_SMR_WFXL_SET _ULL(0x8000000000000) + +#define KVX_SFR_SPS_PL0_PMJ_MASK _ULL(0xf00000) /* Page Mask in JTLB. */ +#define KVX_SFR_SPS_PL0_PMJ_SHIFT 20 +#define KVX_SFR_SPS_PL0_PMJ_WIDTH 4 +#define KVX_SFR_SPS_PL0_PMJ_WFXL_MASK _ULL(0xf00000) +#define KVX_SFR_SPS_PL0_PMJ_WFXL_CLEAR _ULL(0xf00000) +#define KVX_SFR_SPS_PL0_PMJ_WFXL_SET _ULL(0xf0000000000000) + +#define KVX_SFR_SPS_PL0_MMUP_MASK _ULL(0x1000000) /* Privileged on MMU. */ +#define KVX_SFR_SPS_PL0_MMUP_SHIFT 24 +#define KVX_SFR_SPS_PL0_MMUP_WIDTH 1 +#define KVX_SFR_SPS_PL0_MMUP_WFXL_MASK _ULL(0x1000000) +#define KVX_SFR_SPS_PL0_MMUP_WFXL_CLEAR _ULL(0x1000000) +#define KVX_SFR_SPS_PL0_MMUP_WFXL_SET _ULL(0x100000000000000) + +#define KVX_SFR_SPS_PL1_PL_MASK _ULL(0x3) /* Current Privilege Level */ +#define KVX_SFR_SPS_PL1_PL_SHIFT 0 +#define KVX_SFR_SPS_PL1_PL_WIDTH 2 +#define KVX_SFR_SPS_PL1_PL_WFXL_MASK _ULL(0x3) +#define KVX_SFR_SPS_PL1_PL_WFXL_CLEAR _ULL(0x3) +#define KVX_SFR_SPS_PL1_PL_WFXL_SET _ULL(0x300000000) + +#define KVX_SFR_SPS_PL1_ET_MASK _ULL(0x4) /* Exception Taken */ +#define KVX_SFR_SPS_PL1_ET_SHIFT 2 +#define KVX_SFR_SPS_PL1_ET_WIDTH 1 +#define KVX_SFR_SPS_PL1_ET_WFXL_MASK _ULL(0x4) +#define KVX_SFR_SPS_PL1_ET_WFXL_CLEAR _ULL(0x4) +#define KVX_SFR_SPS_PL1_ET_WFXL_SET _ULL(0x400000000) + +#define KVX_SFR_SPS_PL1_HTD_MASK _ULL(0x8) /* Hardware Trap Disable */ +#define KVX_SFR_SPS_PL1_HTD_SHIFT 3 +#define KVX_SFR_SPS_PL1_HTD_WIDTH 1 +#define KVX_SFR_SPS_PL1_HTD_WFXL_MASK _ULL(0x8) +#define KVX_SFR_SPS_PL1_HTD_WFXL_CLEAR _ULL(0x8) +#define KVX_SFR_SPS_PL1_HTD_WFXL_SET _ULL(0x800000000) + +#define KVX_SFR_SPS_PL1_IE_MASK _ULL(0x10) /* Interrupt Enable */ +#define KVX_SFR_SPS_PL1_IE_SHIFT 4 +#define KVX_SFR_SPS_PL1_IE_WIDTH 1 +#define KVX_SFR_SPS_PL1_IE_WFXL_MASK _ULL(0x10) +#define KVX_SFR_SPS_PL1_IE_WFXL_CLEAR _ULL(0x10) +#define KVX_SFR_SPS_PL1_IE_WFXL_SET _ULL(0x1000000000) + +#define KVX_SFR_SPS_PL1_HLE_MASK _ULL(0x20) /* Hardware Loop Enable */ +#define KVX_SFR_SPS_PL1_HLE_SHIFT 5 +#define KVX_SFR_SPS_PL1_HLE_WIDTH 1 +#define KVX_SFR_SPS_PL1_HLE_WFXL_MASK _ULL(0x20) +#define KVX_SFR_SPS_PL1_HLE_WFXL_CLEAR _ULL(0x20) +#define KVX_SFR_SPS_PL1_HLE_WFXL_SET _ULL(0x2000000000) + +#define KVX_SFR_SPS_PL1_SRE_MASK _ULL(0x40) /* Software REserved */ +#define KVX_SFR_SPS_PL1_SRE_SHIFT 6 +#define KVX_SFR_SPS_PL1_SRE_WIDTH 1 +#define KVX_SFR_SPS_PL1_SRE_WFXL_MASK _ULL(0x40) +#define KVX_SFR_SPS_PL1_SRE_WFXL_CLEAR _ULL(0x40) +#define KVX_SFR_SPS_PL1_SRE_WFXL_SET _ULL(0x4000000000) + +#define KVX_SFR_SPS_PL1_DAUS_MASK _ULL(0x80) /* Data Accesses Use SPS settings */ +#define KVX_SFR_SPS_PL1_DAUS_SHIFT 7 +#define KVX_SFR_SPS_PL1_DAUS_WIDTH 1 +#define KVX_SFR_SPS_PL1_DAUS_WFXL_MASK _ULL(0x80) +#define KVX_SFR_SPS_PL1_DAUS_WFXL_CLEAR _ULL(0x80) +#define KVX_SFR_SPS_PL1_DAUS_WFXL_SET _ULL(0x8000000000) + +#define KVX_SFR_SPS_PL1_ICE_MASK _ULL(0x100) /* Instruction Cache Enable */ +#define KVX_SFR_SPS_PL1_ICE_SHIFT 8 +#define KVX_SFR_SPS_PL1_ICE_WIDTH 1 +#define KVX_SFR_SPS_PL1_ICE_WFXL_MASK _ULL(0x100) +#define KVX_SFR_SPS_PL1_ICE_WFXL_CLEAR _ULL(0x100) +#define KVX_SFR_SPS_PL1_ICE_WFXL_SET _ULL(0x10000000000) + +#define KVX_SFR_SPS_PL1_USE_MASK _ULL(0x200) /* Uncached Streaming Enable */ +#define KVX_SFR_SPS_PL1_USE_SHIFT 9 +#define KVX_SFR_SPS_PL1_USE_WIDTH 1 +#define KVX_SFR_SPS_PL1_USE_WFXL_MASK _ULL(0x200) +#define KVX_SFR_SPS_PL1_USE_WFXL_CLEAR _ULL(0x200) +#define KVX_SFR_SPS_PL1_USE_WFXL_SET _ULL(0x20000000000) + +#define KVX_SFR_SPS_PL1_DCE_MASK _ULL(0x400) /* Data Cache Enable */ +#define KVX_SFR_SPS_PL1_DCE_SHIFT 10 +#define KVX_SFR_SPS_PL1_DCE_WIDTH 1 +#define KVX_SFR_SPS_PL1_DCE_WFXL_MASK _ULL(0x400) +#define KVX_SFR_SPS_PL1_DCE_WFXL_CLEAR _ULL(0x400) +#define KVX_SFR_SPS_PL1_DCE_WFXL_SET _ULL(0x40000000000) + +#define KVX_SFR_SPS_PL1_MME_MASK _ULL(0x800) /* Memory Management Enable */ +#define KVX_SFR_SPS_PL1_MME_SHIFT 11 +#define KVX_SFR_SPS_PL1_MME_WIDTH 1 +#define KVX_SFR_SPS_PL1_MME_WFXL_MASK _ULL(0x800) +#define KVX_SFR_SPS_PL1_MME_WFXL_CLEAR _ULL(0x800) +#define KVX_SFR_SPS_PL1_MME_WFXL_SET _ULL(0x80000000000) + +#define KVX_SFR_SPS_PL1_IL_MASK _ULL(0x3000) /* Interrupt Level */ +#define KVX_SFR_SPS_PL1_IL_SHIFT 12 +#define KVX_SFR_SPS_PL1_IL_WIDTH 2 +#define KVX_SFR_SPS_PL1_IL_WFXL_MASK _ULL(0x3000) +#define KVX_SFR_SPS_PL1_IL_WFXL_CLEAR _ULL(0x3000) +#define KVX_SFR_SPS_PL1_IL_WFXL_SET _ULL(0x300000000000) + +#define KVX_SFR_SPS_PL1_VS_MASK _ULL(0xc000) /* Virtual Space */ +#define KVX_SFR_SPS_PL1_VS_SHIFT 14 +#define KVX_SFR_SPS_PL1_VS_WIDTH 2 +#define KVX_SFR_SPS_PL1_VS_WFXL_MASK _ULL(0xc000) +#define KVX_SFR_SPS_PL1_VS_WFXL_CLEAR _ULL(0xc000) +#define KVX_SFR_SPS_PL1_VS_WFXL_SET _ULL(0xc00000000000) + +#define KVX_SFR_SPS_PL1_V64_MASK _ULL(0x10000) /* Virtual 64 bits mode. */ +#define KVX_SFR_SPS_PL1_V64_SHIFT 16 +#define KVX_SFR_SPS_PL1_V64_WIDTH 1 +#define KVX_SFR_SPS_PL1_V64_WFXL_MASK _ULL(0x10000) +#define KVX_SFR_SPS_PL1_V64_WFXL_CLEAR _ULL(0x10000) +#define KVX_SFR_SPS_PL1_V64_WFXL_SET _ULL(0x1000000000000) + +#define KVX_SFR_SPS_PL1_L2E_MASK _ULL(0x20000) /* L2 cache Enable. */ +#define KVX_SFR_SPS_PL1_L2E_SHIFT 17 +#define KVX_SFR_SPS_PL1_L2E_WIDTH 1 +#define KVX_SFR_SPS_PL1_L2E_WFXL_MASK _ULL(0x20000) +#define KVX_SFR_SPS_PL1_L2E_WFXL_CLEAR _ULL(0x20000) +#define KVX_SFR_SPS_PL1_L2E_WFXL_SET _ULL(0x2000000000000) + +#define KVX_SFR_SPS_PL1_SME_MASK _ULL(0x40000) /* Step Mode Enabled */ +#define KVX_SFR_SPS_PL1_SME_SHIFT 18 +#define KVX_SFR_SPS_PL1_SME_WIDTH 1 +#define KVX_SFR_SPS_PL1_SME_WFXL_MASK _ULL(0x40000) +#define KVX_SFR_SPS_PL1_SME_WFXL_CLEAR _ULL(0x40000) +#define KVX_SFR_SPS_PL1_SME_WFXL_SET _ULL(0x4000000000000) + +#define KVX_SFR_SPS_PL1_SMR_MASK _ULL(0x80000) /* Step Mode Ready */ +#define KVX_SFR_SPS_PL1_SMR_SHIFT 19 +#define KVX_SFR_SPS_PL1_SMR_WIDTH 1 +#define KVX_SFR_SPS_PL1_SMR_WFXL_MASK _ULL(0x80000) +#define KVX_SFR_SPS_PL1_SMR_WFXL_CLEAR _ULL(0x80000) +#define KVX_SFR_SPS_PL1_SMR_WFXL_SET _ULL(0x8000000000000) + +#define KVX_SFR_SPS_PL1_PMJ_MASK _ULL(0xf00000) /* Page Mask in JTLB. */ +#define KVX_SFR_SPS_PL1_PMJ_SHIFT 20 +#define KVX_SFR_SPS_PL1_PMJ_WIDTH 4 +#define KVX_SFR_SPS_PL1_PMJ_WFXL_MASK _ULL(0xf00000) +#define KVX_SFR_SPS_PL1_PMJ_WFXL_CLEAR _ULL(0xf00000) +#define KVX_SFR_SPS_PL1_PMJ_WFXL_SET _ULL(0xf0000000000000) + +#define KVX_SFR_SPS_PL1_MMUP_MASK _ULL(0x1000000) /* Privileged on MMU. */ +#define KVX_SFR_SPS_PL1_MMUP_SHIFT 24 +#define KVX_SFR_SPS_PL1_MMUP_WIDTH 1 +#define KVX_SFR_SPS_PL1_MMUP_WFXL_MASK _ULL(0x1000000) +#define KVX_SFR_SPS_PL1_MMUP_WFXL_CLEAR _ULL(0x1000000) +#define KVX_SFR_SPS_PL1_MMUP_WFXL_SET _ULL(0x100000000000000) + +#define KVX_SFR_SPS_PL2_PL_MASK _ULL(0x3) /* Current Privilege Level */ +#define KVX_SFR_SPS_PL2_PL_SHIFT 0 +#define KVX_SFR_SPS_PL2_PL_WIDTH 2 +#define KVX_SFR_SPS_PL2_PL_WFXL_MASK _ULL(0x3) +#define KVX_SFR_SPS_PL2_PL_WFXL_CLEAR _ULL(0x3) +#define KVX_SFR_SPS_PL2_PL_WFXL_SET _ULL(0x300000000) + +#define KVX_SFR_SPS_PL2_ET_MASK _ULL(0x4) /* Exception Taken */ +#define KVX_SFR_SPS_PL2_ET_SHIFT 2 +#define KVX_SFR_SPS_PL2_ET_WIDTH 1 +#define KVX_SFR_SPS_PL2_ET_WFXL_MASK _ULL(0x4) +#define KVX_SFR_SPS_PL2_ET_WFXL_CLEAR _ULL(0x4) +#define KVX_SFR_SPS_PL2_ET_WFXL_SET _ULL(0x400000000) + +#define KVX_SFR_SPS_PL2_HTD_MASK _ULL(0x8) /* Hardware Trap Disable */ +#define KVX_SFR_SPS_PL2_HTD_SHIFT 3 +#define KVX_SFR_SPS_PL2_HTD_WIDTH 1 +#define KVX_SFR_SPS_PL2_HTD_WFXL_MASK _ULL(0x8) +#define KVX_SFR_SPS_PL2_HTD_WFXL_CLEAR _ULL(0x8) +#define KVX_SFR_SPS_PL2_HTD_WFXL_SET _ULL(0x800000000) + +#define KVX_SFR_SPS_PL2_IE_MASK _ULL(0x10) /* Interrupt Enable */ +#define KVX_SFR_SPS_PL2_IE_SHIFT 4 +#define KVX_SFR_SPS_PL2_IE_WIDTH 1 +#define KVX_SFR_SPS_PL2_IE_WFXL_MASK _ULL(0x10) +#define KVX_SFR_SPS_PL2_IE_WFXL_CLEAR _ULL(0x10) +#define KVX_SFR_SPS_PL2_IE_WFXL_SET _ULL(0x1000000000) + +#define KVX_SFR_SPS_PL2_HLE_MASK _ULL(0x20) /* Hardware Loop Enable */ +#define KVX_SFR_SPS_PL2_HLE_SHIFT 5 +#define KVX_SFR_SPS_PL2_HLE_WIDTH 1 +#define KVX_SFR_SPS_PL2_HLE_WFXL_MASK _ULL(0x20) +#define KVX_SFR_SPS_PL2_HLE_WFXL_CLEAR _ULL(0x20) +#define KVX_SFR_SPS_PL2_HLE_WFXL_SET _ULL(0x2000000000) + +#define KVX_SFR_SPS_PL2_SRE_MASK _ULL(0x40) /* Software REserved */ +#define KVX_SFR_SPS_PL2_SRE_SHIFT 6 +#define KVX_SFR_SPS_PL2_SRE_WIDTH 1 +#define KVX_SFR_SPS_PL2_SRE_WFXL_MASK _ULL(0x40) +#define KVX_SFR_SPS_PL2_SRE_WFXL_CLEAR _ULL(0x40) +#define KVX_SFR_SPS_PL2_SRE_WFXL_SET _ULL(0x4000000000) + +#define KVX_SFR_SPS_PL2_DAUS_MASK _ULL(0x80) /* Data Accesses Use SPS settings */ +#define KVX_SFR_SPS_PL2_DAUS_SHIFT 7 +#define KVX_SFR_SPS_PL2_DAUS_WIDTH 1 +#define KVX_SFR_SPS_PL2_DAUS_WFXL_MASK _ULL(0x80) +#define KVX_SFR_SPS_PL2_DAUS_WFXL_CLEAR _ULL(0x80) +#define KVX_SFR_SPS_PL2_DAUS_WFXL_SET _ULL(0x8000000000) + +#define KVX_SFR_SPS_PL2_ICE_MASK _ULL(0x100) /* Instruction Cache Enable */ +#define KVX_SFR_SPS_PL2_ICE_SHIFT 8 +#define KVX_SFR_SPS_PL2_ICE_WIDTH 1 +#define KVX_SFR_SPS_PL2_ICE_WFXL_MASK _ULL(0x100) +#define KVX_SFR_SPS_PL2_ICE_WFXL_CLEAR _ULL(0x100) +#define KVX_SFR_SPS_PL2_ICE_WFXL_SET _ULL(0x10000000000) + +#define KVX_SFR_SPS_PL2_USE_MASK _ULL(0x200) /* Uncached Streaming Enable */ +#define KVX_SFR_SPS_PL2_USE_SHIFT 9 +#define KVX_SFR_SPS_PL2_USE_WIDTH 1 +#define KVX_SFR_SPS_PL2_USE_WFXL_MASK _ULL(0x200) +#define KVX_SFR_SPS_PL2_USE_WFXL_CLEAR _ULL(0x200) +#define KVX_SFR_SPS_PL2_USE_WFXL_SET _ULL(0x20000000000) + +#define KVX_SFR_SPS_PL2_DCE_MASK _ULL(0x400) /* Data Cache Enable */ +#define KVX_SFR_SPS_PL2_DCE_SHIFT 10 +#define KVX_SFR_SPS_PL2_DCE_WIDTH 1 +#define KVX_SFR_SPS_PL2_DCE_WFXL_MASK _ULL(0x400) +#define KVX_SFR_SPS_PL2_DCE_WFXL_CLEAR _ULL(0x400) +#define KVX_SFR_SPS_PL2_DCE_WFXL_SET _ULL(0x40000000000) + +#define KVX_SFR_SPS_PL2_MME_MASK _ULL(0x800) /* Memory Management Enable */ +#define KVX_SFR_SPS_PL2_MME_SHIFT 11 +#define KVX_SFR_SPS_PL2_MME_WIDTH 1 +#define KVX_SFR_SPS_PL2_MME_WFXL_MASK _ULL(0x800) +#define KVX_SFR_SPS_PL2_MME_WFXL_CLEAR _ULL(0x800) +#define KVX_SFR_SPS_PL2_MME_WFXL_SET _ULL(0x80000000000) + +#define KVX_SFR_SPS_PL2_IL_MASK _ULL(0x3000) /* Interrupt Level */ +#define KVX_SFR_SPS_PL2_IL_SHIFT 12 +#define KVX_SFR_SPS_PL2_IL_WIDTH 2 +#define KVX_SFR_SPS_PL2_IL_WFXL_MASK _ULL(0x3000) +#define KVX_SFR_SPS_PL2_IL_WFXL_CLEAR _ULL(0x3000) +#define KVX_SFR_SPS_PL2_IL_WFXL_SET _ULL(0x300000000000) + +#define KVX_SFR_SPS_PL2_VS_MASK _ULL(0xc000) /* Virtual Space */ +#define KVX_SFR_SPS_PL2_VS_SHIFT 14 +#define KVX_SFR_SPS_PL2_VS_WIDTH 2 +#define KVX_SFR_SPS_PL2_VS_WFXL_MASK _ULL(0xc000) +#define KVX_SFR_SPS_PL2_VS_WFXL_CLEAR _ULL(0xc000) +#define KVX_SFR_SPS_PL2_VS_WFXL_SET _ULL(0xc00000000000) + +#define KVX_SFR_SPS_PL2_V64_MASK _ULL(0x10000) /* Virtual 64 bits mode. */ +#define KVX_SFR_SPS_PL2_V64_SHIFT 16 +#define KVX_SFR_SPS_PL2_V64_WIDTH 1 +#define KVX_SFR_SPS_PL2_V64_WFXL_MASK _ULL(0x10000) +#define KVX_SFR_SPS_PL2_V64_WFXL_CLEAR _ULL(0x10000) +#define KVX_SFR_SPS_PL2_V64_WFXL_SET _ULL(0x1000000000000) + +#define KVX_SFR_SPS_PL2_L2E_MASK _ULL(0x20000) /* L2 cache Enable. */ +#define KVX_SFR_SPS_PL2_L2E_SHIFT 17 +#define KVX_SFR_SPS_PL2_L2E_WIDTH 1 +#define KVX_SFR_SPS_PL2_L2E_WFXL_MASK _ULL(0x20000) +#define KVX_SFR_SPS_PL2_L2E_WFXL_CLEAR _ULL(0x20000) +#define KVX_SFR_SPS_PL2_L2E_WFXL_SET _ULL(0x2000000000000) + +#define KVX_SFR_SPS_PL2_SME_MASK _ULL(0x40000) /* Step Mode Enabled */ +#define KVX_SFR_SPS_PL2_SME_SHIFT 18 +#define KVX_SFR_SPS_PL2_SME_WIDTH 1 +#define KVX_SFR_SPS_PL2_SME_WFXL_MASK _ULL(0x40000) +#define KVX_SFR_SPS_PL2_SME_WFXL_CLEAR _ULL(0x40000) +#define KVX_SFR_SPS_PL2_SME_WFXL_SET _ULL(0x4000000000000) + +#define KVX_SFR_SPS_PL2_SMR_MASK _ULL(0x80000) /* Step Mode Ready */ +#define KVX_SFR_SPS_PL2_SMR_SHIFT 19 +#define KVX_SFR_SPS_PL2_SMR_WIDTH 1 +#define KVX_SFR_SPS_PL2_SMR_WFXL_MASK _ULL(0x80000) +#define KVX_SFR_SPS_PL2_SMR_WFXL_CLEAR _ULL(0x80000) +#define KVX_SFR_SPS_PL2_SMR_WFXL_SET _ULL(0x8000000000000) + +#define KVX_SFR_SPS_PL2_PMJ_MASK _ULL(0xf00000) /* Page Mask in JTLB. */ +#define KVX_SFR_SPS_PL2_PMJ_SHIFT 20 +#define KVX_SFR_SPS_PL2_PMJ_WIDTH 4 +#define KVX_SFR_SPS_PL2_PMJ_WFXL_MASK _ULL(0xf00000) +#define KVX_SFR_SPS_PL2_PMJ_WFXL_CLEAR _ULL(0xf00000) +#define KVX_SFR_SPS_PL2_PMJ_WFXL_SET _ULL(0xf0000000000000) + +#define KVX_SFR_SPS_PL2_MMUP_MASK _ULL(0x1000000) /* Privileged on MMU. */ +#define KVX_SFR_SPS_PL2_MMUP_SHIFT 24 +#define KVX_SFR_SPS_PL2_MMUP_WIDTH 1 +#define KVX_SFR_SPS_PL2_MMUP_WFXL_MASK _ULL(0x1000000) +#define KVX_SFR_SPS_PL2_MMUP_WFXL_CLEAR _ULL(0x1000000) +#define KVX_SFR_SPS_PL2_MMUP_WFXL_SET _ULL(0x100000000000000) + +#define KVX_SFR_SPS_PL3_PL_MASK _ULL(0x3) /* Current Privilege Level */ +#define KVX_SFR_SPS_PL3_PL_SHIFT 0 +#define KVX_SFR_SPS_PL3_PL_WIDTH 2 +#define KVX_SFR_SPS_PL3_PL_WFXL_MASK _ULL(0x3) +#define KVX_SFR_SPS_PL3_PL_WFXL_CLEAR _ULL(0x3) +#define KVX_SFR_SPS_PL3_PL_WFXL_SET _ULL(0x300000000) + +#define KVX_SFR_SPS_PL3_ET_MASK _ULL(0x4) /* Exception Taken */ +#define KVX_SFR_SPS_PL3_ET_SHIFT 2 +#define KVX_SFR_SPS_PL3_ET_WIDTH 1 +#define KVX_SFR_SPS_PL3_ET_WFXL_MASK _ULL(0x4) +#define KVX_SFR_SPS_PL3_ET_WFXL_CLEAR _ULL(0x4) +#define KVX_SFR_SPS_PL3_ET_WFXL_SET _ULL(0x400000000) + +#define KVX_SFR_SPS_PL3_HTD_MASK _ULL(0x8) /* Hardware Trap Disable */ +#define KVX_SFR_SPS_PL3_HTD_SHIFT 3 +#define KVX_SFR_SPS_PL3_HTD_WIDTH 1 +#define KVX_SFR_SPS_PL3_HTD_WFXL_MASK _ULL(0x8) +#define KVX_SFR_SPS_PL3_HTD_WFXL_CLEAR _ULL(0x8) +#define KVX_SFR_SPS_PL3_HTD_WFXL_SET _ULL(0x800000000) + +#define KVX_SFR_SPS_PL3_IE_MASK _ULL(0x10) /* Interrupt Enable */ +#define KVX_SFR_SPS_PL3_IE_SHIFT 4 +#define KVX_SFR_SPS_PL3_IE_WIDTH 1 +#define KVX_SFR_SPS_PL3_IE_WFXL_MASK _ULL(0x10) +#define KVX_SFR_SPS_PL3_IE_WFXL_CLEAR _ULL(0x10) +#define KVX_SFR_SPS_PL3_IE_WFXL_SET _ULL(0x1000000000) + +#define KVX_SFR_SPS_PL3_HLE_MASK _ULL(0x20) /* Hardware Loop Enable */ +#define KVX_SFR_SPS_PL3_HLE_SHIFT 5 +#define KVX_SFR_SPS_PL3_HLE_WIDTH 1 +#define KVX_SFR_SPS_PL3_HLE_WFXL_MASK _ULL(0x20) +#define KVX_SFR_SPS_PL3_HLE_WFXL_CLEAR _ULL(0x20) +#define KVX_SFR_SPS_PL3_HLE_WFXL_SET _ULL(0x2000000000) + +#define KVX_SFR_SPS_PL3_SRE_MASK _ULL(0x40) /* Software REserved */ +#define KVX_SFR_SPS_PL3_SRE_SHIFT 6 +#define KVX_SFR_SPS_PL3_SRE_WIDTH 1 +#define KVX_SFR_SPS_PL3_SRE_WFXL_MASK _ULL(0x40) +#define KVX_SFR_SPS_PL3_SRE_WFXL_CLEAR _ULL(0x40) +#define KVX_SFR_SPS_PL3_SRE_WFXL_SET _ULL(0x4000000000) + +#define KVX_SFR_SPS_PL3_DAUS_MASK _ULL(0x80) /* Data Accesses Use SPS settings */ +#define KVX_SFR_SPS_PL3_DAUS_SHIFT 7 +#define KVX_SFR_SPS_PL3_DAUS_WIDTH 1 +#define KVX_SFR_SPS_PL3_DAUS_WFXL_MASK _ULL(0x80) +#define KVX_SFR_SPS_PL3_DAUS_WFXL_CLEAR _ULL(0x80) +#define KVX_SFR_SPS_PL3_DAUS_WFXL_SET _ULL(0x8000000000) + +#define KVX_SFR_SPS_PL3_ICE_MASK _ULL(0x100) /* Instruction Cache Enable */ +#define KVX_SFR_SPS_PL3_ICE_SHIFT 8 +#define KVX_SFR_SPS_PL3_ICE_WIDTH 1 +#define KVX_SFR_SPS_PL3_ICE_WFXL_MASK _ULL(0x100) +#define KVX_SFR_SPS_PL3_ICE_WFXL_CLEAR _ULL(0x100) +#define KVX_SFR_SPS_PL3_ICE_WFXL_SET _ULL(0x10000000000) + +#define KVX_SFR_SPS_PL3_USE_MASK _ULL(0x200) /* Uncached Streaming Enable */ +#define KVX_SFR_SPS_PL3_USE_SHIFT 9 +#define KVX_SFR_SPS_PL3_USE_WIDTH 1 +#define KVX_SFR_SPS_PL3_USE_WFXL_MASK _ULL(0x200) +#define KVX_SFR_SPS_PL3_USE_WFXL_CLEAR _ULL(0x200) +#define KVX_SFR_SPS_PL3_USE_WFXL_SET _ULL(0x20000000000) + +#define KVX_SFR_SPS_PL3_DCE_MASK _ULL(0x400) /* Data Cache Enable */ +#define KVX_SFR_SPS_PL3_DCE_SHIFT 10 +#define KVX_SFR_SPS_PL3_DCE_WIDTH 1 +#define KVX_SFR_SPS_PL3_DCE_WFXL_MASK _ULL(0x400) +#define KVX_SFR_SPS_PL3_DCE_WFXL_CLEAR _ULL(0x400) +#define KVX_SFR_SPS_PL3_DCE_WFXL_SET _ULL(0x40000000000) + +#define KVX_SFR_SPS_PL3_MME_MASK _ULL(0x800) /* Memory Management Enable */ +#define KVX_SFR_SPS_PL3_MME_SHIFT 11 +#define KVX_SFR_SPS_PL3_MME_WIDTH 1 +#define KVX_SFR_SPS_PL3_MME_WFXL_MASK _ULL(0x800) +#define KVX_SFR_SPS_PL3_MME_WFXL_CLEAR _ULL(0x800) +#define KVX_SFR_SPS_PL3_MME_WFXL_SET _ULL(0x80000000000) + +#define KVX_SFR_SPS_PL3_IL_MASK _ULL(0x3000) /* Interrupt Level */ +#define KVX_SFR_SPS_PL3_IL_SHIFT 12 +#define KVX_SFR_SPS_PL3_IL_WIDTH 2 +#define KVX_SFR_SPS_PL3_IL_WFXL_MASK _ULL(0x3000) +#define KVX_SFR_SPS_PL3_IL_WFXL_CLEAR _ULL(0x3000) +#define KVX_SFR_SPS_PL3_IL_WFXL_SET _ULL(0x300000000000) + +#define KVX_SFR_SPS_PL3_VS_MASK _ULL(0xc000) /* Virtual Space */ +#define KVX_SFR_SPS_PL3_VS_SHIFT 14 +#define KVX_SFR_SPS_PL3_VS_WIDTH 2 +#define KVX_SFR_SPS_PL3_VS_WFXL_MASK _ULL(0xc000) +#define KVX_SFR_SPS_PL3_VS_WFXL_CLEAR _ULL(0xc000) +#define KVX_SFR_SPS_PL3_VS_WFXL_SET _ULL(0xc00000000000) + +#define KVX_SFR_SPS_PL3_V64_MASK _ULL(0x10000) /* Virtual 64 bits mode. */ +#define KVX_SFR_SPS_PL3_V64_SHIFT 16 +#define KVX_SFR_SPS_PL3_V64_WIDTH 1 +#define KVX_SFR_SPS_PL3_V64_WFXL_MASK _ULL(0x10000) +#define KVX_SFR_SPS_PL3_V64_WFXL_CLEAR _ULL(0x10000) +#define KVX_SFR_SPS_PL3_V64_WFXL_SET _ULL(0x1000000000000) + +#define KVX_SFR_SPS_PL3_L2E_MASK _ULL(0x20000) /* L2 cache Enable. */ +#define KVX_SFR_SPS_PL3_L2E_SHIFT 17 +#define KVX_SFR_SPS_PL3_L2E_WIDTH 1 +#define KVX_SFR_SPS_PL3_L2E_WFXL_MASK _ULL(0x20000) +#define KVX_SFR_SPS_PL3_L2E_WFXL_CLEAR _ULL(0x20000) +#define KVX_SFR_SPS_PL3_L2E_WFXL_SET _ULL(0x2000000000000) + +#define KVX_SFR_SPS_PL3_SME_MASK _ULL(0x40000) /* Step Mode Enabled */ +#define KVX_SFR_SPS_PL3_SME_SHIFT 18 +#define KVX_SFR_SPS_PL3_SME_WIDTH 1 +#define KVX_SFR_SPS_PL3_SME_WFXL_MASK _ULL(0x40000) +#define KVX_SFR_SPS_PL3_SME_WFXL_CLEAR _ULL(0x40000) +#define KVX_SFR_SPS_PL3_SME_WFXL_SET _ULL(0x4000000000000) + +#define KVX_SFR_SPS_PL3_SMR_MASK _ULL(0x80000) /* Step Mode Ready */ +#define KVX_SFR_SPS_PL3_SMR_SHIFT 19 +#define KVX_SFR_SPS_PL3_SMR_WIDTH 1 +#define KVX_SFR_SPS_PL3_SMR_WFXL_MASK _ULL(0x80000) +#define KVX_SFR_SPS_PL3_SMR_WFXL_CLEAR _ULL(0x80000) +#define KVX_SFR_SPS_PL3_SMR_WFXL_SET _ULL(0x8000000000000) + +#define KVX_SFR_SPS_PL3_PMJ_MASK _ULL(0xf00000) /* Page Mask in JTLB. */ +#define KVX_SFR_SPS_PL3_PMJ_SHIFT 20 +#define KVX_SFR_SPS_PL3_PMJ_WIDTH 4 +#define KVX_SFR_SPS_PL3_PMJ_WFXL_MASK _ULL(0xf00000) +#define KVX_SFR_SPS_PL3_PMJ_WFXL_CLEAR _ULL(0xf00000) +#define KVX_SFR_SPS_PL3_PMJ_WFXL_SET _ULL(0xf0000000000000) + +#define KVX_SFR_SPS_PL3_MMUP_MASK _ULL(0x1000000) /* Privileged on MMU. */ +#define KVX_SFR_SPS_PL3_MMUP_SHIFT 24 +#define KVX_SFR_SPS_PL3_MMUP_WIDTH 1 +#define KVX_SFR_SPS_PL3_MMUP_WFXL_MASK _ULL(0x1000000) +#define KVX_SFR_SPS_PL3_MMUP_WFXL_CLEAR _ULL(0x1000000) +#define KVX_SFR_SPS_PL3_MMUP_WFXL_SET _ULL(0x100000000000000) + +#define KVX_SFR_PSO_PL0_MASK _ULL(0x3) /* Current Privilege Level bit 0 owner */ +#define KVX_SFR_PSO_PL0_SHIFT 0 +#define KVX_SFR_PSO_PL0_WIDTH 2 +#define KVX_SFR_PSO_PL0_WFXL_MASK _ULL(0x3) +#define KVX_SFR_PSO_PL0_WFXL_CLEAR _ULL(0x3) +#define KVX_SFR_PSO_PL0_WFXL_SET _ULL(0x300000000) + +#define KVX_SFR_PSO_PL1_MASK _ULL(0xc) /* Current Privilege Level bit 1 owner */ +#define KVX_SFR_PSO_PL1_SHIFT 2 +#define KVX_SFR_PSO_PL1_WIDTH 2 +#define KVX_SFR_PSO_PL1_WFXL_MASK _ULL(0xc) +#define KVX_SFR_PSO_PL1_WFXL_CLEAR _ULL(0xc) +#define KVX_SFR_PSO_PL1_WFXL_SET _ULL(0xc00000000) + +#define KVX_SFR_PSO_ET_MASK _ULL(0x30) /* Exception Taken owner */ +#define KVX_SFR_PSO_ET_SHIFT 4 +#define KVX_SFR_PSO_ET_WIDTH 2 +#define KVX_SFR_PSO_ET_WFXL_MASK _ULL(0x30) +#define KVX_SFR_PSO_ET_WFXL_CLEAR _ULL(0x30) +#define KVX_SFR_PSO_ET_WFXL_SET _ULL(0x3000000000) + +#define KVX_SFR_PSO_HTD_MASK _ULL(0xc0) /* Hardware Trap Disable owner */ +#define KVX_SFR_PSO_HTD_SHIFT 6 +#define KVX_SFR_PSO_HTD_WIDTH 2 +#define KVX_SFR_PSO_HTD_WFXL_MASK _ULL(0xc0) +#define KVX_SFR_PSO_HTD_WFXL_CLEAR _ULL(0xc0) +#define KVX_SFR_PSO_HTD_WFXL_SET _ULL(0xc000000000) + +#define KVX_SFR_PSO_IE_MASK _ULL(0x300) /* Interrupt Enable owner */ +#define KVX_SFR_PSO_IE_SHIFT 8 +#define KVX_SFR_PSO_IE_WIDTH 2 +#define KVX_SFR_PSO_IE_WFXL_MASK _ULL(0x300) +#define KVX_SFR_PSO_IE_WFXL_CLEAR _ULL(0x300) +#define KVX_SFR_PSO_IE_WFXL_SET _ULL(0x30000000000) + +#define KVX_SFR_PSO_HLE_MASK _ULL(0xc00) /* Hardware Loop Enable owner */ +#define KVX_SFR_PSO_HLE_SHIFT 10 +#define KVX_SFR_PSO_HLE_WIDTH 2 +#define KVX_SFR_PSO_HLE_WFXL_MASK _ULL(0xc00) +#define KVX_SFR_PSO_HLE_WFXL_CLEAR _ULL(0xc00) +#define KVX_SFR_PSO_HLE_WFXL_SET _ULL(0xc0000000000) + +#define KVX_SFR_PSO_SRE_MASK _ULL(0x3000) /* Software REserved owner */ +#define KVX_SFR_PSO_SRE_SHIFT 12 +#define KVX_SFR_PSO_SRE_WIDTH 2 +#define KVX_SFR_PSO_SRE_WFXL_MASK _ULL(0x3000) +#define KVX_SFR_PSO_SRE_WFXL_CLEAR _ULL(0x3000) +#define KVX_SFR_PSO_SRE_WFXL_SET _ULL(0x300000000000) + +#define KVX_SFR_PSO_DAUS_MASK _ULL(0xc000) /* Data Accesses Use SPS settings owner */ +#define KVX_SFR_PSO_DAUS_SHIFT 14 +#define KVX_SFR_PSO_DAUS_WIDTH 2 +#define KVX_SFR_PSO_DAUS_WFXL_MASK _ULL(0xc000) +#define KVX_SFR_PSO_DAUS_WFXL_CLEAR _ULL(0xc000) +#define KVX_SFR_PSO_DAUS_WFXL_SET _ULL(0xc00000000000) + +#define KVX_SFR_PSO_ICE_MASK _ULL(0x30000) /* Instruction Cache Enable owner */ +#define KVX_SFR_PSO_ICE_SHIFT 16 +#define KVX_SFR_PSO_ICE_WIDTH 2 +#define KVX_SFR_PSO_ICE_WFXL_MASK _ULL(0x30000) +#define KVX_SFR_PSO_ICE_WFXL_CLEAR _ULL(0x30000) +#define KVX_SFR_PSO_ICE_WFXL_SET _ULL(0x3000000000000) + +#define KVX_SFR_PSO_USE_MASK _ULL(0xc0000) /* Uncached Streaming Enable owner */ +#define KVX_SFR_PSO_USE_SHIFT 18 +#define KVX_SFR_PSO_USE_WIDTH 2 +#define KVX_SFR_PSO_USE_WFXL_MASK _ULL(0xc0000) +#define KVX_SFR_PSO_USE_WFXL_CLEAR _ULL(0xc0000) +#define KVX_SFR_PSO_USE_WFXL_SET _ULL(0xc000000000000) + +#define KVX_SFR_PSO_DCE_MASK _ULL(0x300000) /* Data Cache Enable owner */ +#define KVX_SFR_PSO_DCE_SHIFT 20 +#define KVX_SFR_PSO_DCE_WIDTH 2 +#define KVX_SFR_PSO_DCE_WFXL_MASK _ULL(0x300000) +#define KVX_SFR_PSO_DCE_WFXL_CLEAR _ULL(0x300000) +#define KVX_SFR_PSO_DCE_WFXL_SET _ULL(0x30000000000000) + +#define KVX_SFR_PSO_MME_MASK _ULL(0xc00000) /* Memory Management Enable owner */ +#define KVX_SFR_PSO_MME_SHIFT 22 +#define KVX_SFR_PSO_MME_WIDTH 2 +#define KVX_SFR_PSO_MME_WFXL_MASK _ULL(0xc00000) +#define KVX_SFR_PSO_MME_WFXL_CLEAR _ULL(0xc00000) +#define KVX_SFR_PSO_MME_WFXL_SET _ULL(0xc0000000000000) + +#define KVX_SFR_PSO_IL0_MASK _ULL(0x3000000) /* Interrupt Level bit 0 owner */ +#define KVX_SFR_PSO_IL0_SHIFT 24 +#define KVX_SFR_PSO_IL0_WIDTH 2 +#define KVX_SFR_PSO_IL0_WFXL_MASK _ULL(0x3000000) +#define KVX_SFR_PSO_IL0_WFXL_CLEAR _ULL(0x3000000) +#define KVX_SFR_PSO_IL0_WFXL_SET _ULL(0x300000000000000) + +#define KVX_SFR_PSO_IL1_MASK _ULL(0xc000000) /* Interrupt Level bit 1 owner */ +#define KVX_SFR_PSO_IL1_SHIFT 26 +#define KVX_SFR_PSO_IL1_WIDTH 2 +#define KVX_SFR_PSO_IL1_WFXL_MASK _ULL(0xc000000) +#define KVX_SFR_PSO_IL1_WFXL_CLEAR _ULL(0xc000000) +#define KVX_SFR_PSO_IL1_WFXL_SET _ULL(0xc00000000000000) + +#define KVX_SFR_PSO_VS0_MASK _ULL(0x30000000) /* Virtual Space bit 0 owner */ +#define KVX_SFR_PSO_VS0_SHIFT 28 +#define KVX_SFR_PSO_VS0_WIDTH 2 +#define KVX_SFR_PSO_VS0_WFXL_MASK _ULL(0x30000000) +#define KVX_SFR_PSO_VS0_WFXL_CLEAR _ULL(0x30000000) +#define KVX_SFR_PSO_VS0_WFXL_SET _ULL(0x3000000000000000) + +#define KVX_SFR_PSO_VS1_MASK _ULL(0xc0000000) /* Virtual Space bit 1 owner */ +#define KVX_SFR_PSO_VS1_SHIFT 30 +#define KVX_SFR_PSO_VS1_WIDTH 2 +#define KVX_SFR_PSO_VS1_WFXL_MASK _ULL(0xc0000000) +#define KVX_SFR_PSO_VS1_WFXL_CLEAR _ULL(0xc0000000) +#define KVX_SFR_PSO_VS1_WFXL_SET _ULL(0xc000000000000000) + +#define KVX_SFR_PSO_V64_MASK _ULL(0x300000000) /* Virtual 64 bits mode owner */ +#define KVX_SFR_PSO_V64_SHIFT 32 +#define KVX_SFR_PSO_V64_WIDTH 2 +#define KVX_SFR_PSO_V64_WFXM_MASK _ULL(0x300000000) +#define KVX_SFR_PSO_V64_WFXM_CLEAR _ULL(0x3) +#define KVX_SFR_PSO_V64_WFXM_SET _ULL(0x300000000) + +#define KVX_SFR_PSO_L2E_MASK _ULL(0xc00000000) /* L2 cache Enable owner */ +#define KVX_SFR_PSO_L2E_SHIFT 34 +#define KVX_SFR_PSO_L2E_WIDTH 2 +#define KVX_SFR_PSO_L2E_WFXM_MASK _ULL(0xc00000000) +#define KVX_SFR_PSO_L2E_WFXM_CLEAR _ULL(0xc) +#define KVX_SFR_PSO_L2E_WFXM_SET _ULL(0xc00000000) + +#define KVX_SFR_PSO_SME_MASK _ULL(0x3000000000) /* Step Mode Enabled owner */ +#define KVX_SFR_PSO_SME_SHIFT 36 +#define KVX_SFR_PSO_SME_WIDTH 2 +#define KVX_SFR_PSO_SME_WFXM_MASK _ULL(0x3000000000) +#define KVX_SFR_PSO_SME_WFXM_CLEAR _ULL(0x30) +#define KVX_SFR_PSO_SME_WFXM_SET _ULL(0x3000000000) + +#define KVX_SFR_PSO_SMR_MASK _ULL(0xc000000000) /* Step Mode Ready owner */ +#define KVX_SFR_PSO_SMR_SHIFT 38 +#define KVX_SFR_PSO_SMR_WIDTH 2 +#define KVX_SFR_PSO_SMR_WFXM_MASK _ULL(0xc000000000) +#define KVX_SFR_PSO_SMR_WFXM_CLEAR _ULL(0xc0) +#define KVX_SFR_PSO_SMR_WFXM_SET _ULL(0xc000000000) + +#define KVX_SFR_PSO_PMJ0_MASK _ULL(0x30000000000) /* Page Mask in JTLB bit 0 owner */ +#define KVX_SFR_PSO_PMJ0_SHIFT 40 +#define KVX_SFR_PSO_PMJ0_WIDTH 2 +#define KVX_SFR_PSO_PMJ0_WFXM_MASK _ULL(0x30000000000) +#define KVX_SFR_PSO_PMJ0_WFXM_CLEAR _ULL(0x300) +#define KVX_SFR_PSO_PMJ0_WFXM_SET _ULL(0x30000000000) + +#define KVX_SFR_PSO_PMJ1_MASK _ULL(0xc0000000000) /* Page Mask in JTLB bit 1 owner */ +#define KVX_SFR_PSO_PMJ1_SHIFT 42 +#define KVX_SFR_PSO_PMJ1_WIDTH 2 +#define KVX_SFR_PSO_PMJ1_WFXM_MASK _ULL(0xc0000000000) +#define KVX_SFR_PSO_PMJ1_WFXM_CLEAR _ULL(0xc00) +#define KVX_SFR_PSO_PMJ1_WFXM_SET _ULL(0xc0000000000) + +#define KVX_SFR_PSO_PMJ2_MASK _ULL(0x300000000000) /* Page Mask in JTLB bit 2 owner */ +#define KVX_SFR_PSO_PMJ2_SHIFT 44 +#define KVX_SFR_PSO_PMJ2_WIDTH 2 +#define KVX_SFR_PSO_PMJ2_WFXM_MASK _ULL(0x300000000000) +#define KVX_SFR_PSO_PMJ2_WFXM_CLEAR _ULL(0x3000) +#define KVX_SFR_PSO_PMJ2_WFXM_SET _ULL(0x300000000000) + +#define KVX_SFR_PSO_PMJ3_MASK _ULL(0xc00000000000) /* Page Mask in JTLB bit 3 owner */ +#define KVX_SFR_PSO_PMJ3_SHIFT 46 +#define KVX_SFR_PSO_PMJ3_WIDTH 2 +#define KVX_SFR_PSO_PMJ3_WFXM_MASK _ULL(0xc00000000000) +#define KVX_SFR_PSO_PMJ3_WFXM_CLEAR _ULL(0xc000) +#define KVX_SFR_PSO_PMJ3_WFXM_SET _ULL(0xc00000000000) + +#define KVX_SFR_PSO_MMUP_MASK _ULL(0x3000000000000) /* Privileged on MMU owner. */ +#define KVX_SFR_PSO_MMUP_SHIFT 48 +#define KVX_SFR_PSO_MMUP_WIDTH 2 +#define KVX_SFR_PSO_MMUP_WFXM_MASK _ULL(0x3000000000000) +#define KVX_SFR_PSO_MMUP_WFXM_CLEAR _ULL(0x30000) +#define KVX_SFR_PSO_MMUP_WFXM_SET _ULL(0x3000000000000) + +#define KVX_SFR_PSOW_PL0_MASK _ULL(0x3) /* Current Privilege Level bit 0 owner */ +#define KVX_SFR_PSOW_PL0_SHIFT 0 +#define KVX_SFR_PSOW_PL0_WIDTH 2 +#define KVX_SFR_PSOW_PL0_WFXL_MASK _ULL(0x3) +#define KVX_SFR_PSOW_PL0_WFXL_CLEAR _ULL(0x3) +#define KVX_SFR_PSOW_PL0_WFXL_SET _ULL(0x300000000) + +#define KVX_SFR_PSOW_PL1_MASK _ULL(0xc) /* Current Privilege Level bit 1 owner */ +#define KVX_SFR_PSOW_PL1_SHIFT 2 +#define KVX_SFR_PSOW_PL1_WIDTH 2 +#define KVX_SFR_PSOW_PL1_WFXL_MASK _ULL(0xc) +#define KVX_SFR_PSOW_PL1_WFXL_CLEAR _ULL(0xc) +#define KVX_SFR_PSOW_PL1_WFXL_SET _ULL(0xc00000000) + +#define KVX_SFR_PSOW_ET_MASK _ULL(0x30) /* Exception Taken owner */ +#define KVX_SFR_PSOW_ET_SHIFT 4 +#define KVX_SFR_PSOW_ET_WIDTH 2 +#define KVX_SFR_PSOW_ET_WFXL_MASK _ULL(0x30) +#define KVX_SFR_PSOW_ET_WFXL_CLEAR _ULL(0x30) +#define KVX_SFR_PSOW_ET_WFXL_SET _ULL(0x3000000000) + +#define KVX_SFR_PSOW_HTD_MASK _ULL(0xc0) /* Hardware Trap Disable owner */ +#define KVX_SFR_PSOW_HTD_SHIFT 6 +#define KVX_SFR_PSOW_HTD_WIDTH 2 +#define KVX_SFR_PSOW_HTD_WFXL_MASK _ULL(0xc0) +#define KVX_SFR_PSOW_HTD_WFXL_CLEAR _ULL(0xc0) +#define KVX_SFR_PSOW_HTD_WFXL_SET _ULL(0xc000000000) + +#define KVX_SFR_PSOW_IE_MASK _ULL(0x300) /* Interrupt Enable owner */ +#define KVX_SFR_PSOW_IE_SHIFT 8 +#define KVX_SFR_PSOW_IE_WIDTH 2 +#define KVX_SFR_PSOW_IE_WFXL_MASK _ULL(0x300) +#define KVX_SFR_PSOW_IE_WFXL_CLEAR _ULL(0x300) +#define KVX_SFR_PSOW_IE_WFXL_SET _ULL(0x30000000000) + +#define KVX_SFR_PSOW_HLE_MASK _ULL(0xc00) /* Hardware Loop Enable owner */ +#define KVX_SFR_PSOW_HLE_SHIFT 10 +#define KVX_SFR_PSOW_HLE_WIDTH 2 +#define KVX_SFR_PSOW_HLE_WFXL_MASK _ULL(0xc00) +#define KVX_SFR_PSOW_HLE_WFXL_CLEAR _ULL(0xc00) +#define KVX_SFR_PSOW_HLE_WFXL_SET _ULL(0xc0000000000) + +#define KVX_SFR_PSOW_SRE_MASK _ULL(0x3000) /* Software REserved owner */ +#define KVX_SFR_PSOW_SRE_SHIFT 12 +#define KVX_SFR_PSOW_SRE_WIDTH 2 +#define KVX_SFR_PSOW_SRE_WFXL_MASK _ULL(0x3000) +#define KVX_SFR_PSOW_SRE_WFXL_CLEAR _ULL(0x3000) +#define KVX_SFR_PSOW_SRE_WFXL_SET _ULL(0x300000000000) + +#define KVX_SFR_PSOW_DAUS_MASK _ULL(0xc000) /* Data Accesses Use SPS settings owner */ +#define KVX_SFR_PSOW_DAUS_SHIFT 14 +#define KVX_SFR_PSOW_DAUS_WIDTH 2 +#define KVX_SFR_PSOW_DAUS_WFXL_MASK _ULL(0xc000) +#define KVX_SFR_PSOW_DAUS_WFXL_CLEAR _ULL(0xc000) +#define KVX_SFR_PSOW_DAUS_WFXL_SET _ULL(0xc00000000000) + +#define KVX_SFR_PSOW_ICE_MASK _ULL(0x30000) /* Instruction Cache Enable owner */ +#define KVX_SFR_PSOW_ICE_SHIFT 16 +#define KVX_SFR_PSOW_ICE_WIDTH 2 +#define KVX_SFR_PSOW_ICE_WFXL_MASK _ULL(0x30000) +#define KVX_SFR_PSOW_ICE_WFXL_CLEAR _ULL(0x30000) +#define KVX_SFR_PSOW_ICE_WFXL_SET _ULL(0x3000000000000) + +#define KVX_SFR_PSOW_USE_MASK _ULL(0xc0000) /* Uncached Streaming Enable owner */ +#define KVX_SFR_PSOW_USE_SHIFT 18 +#define KVX_SFR_PSOW_USE_WIDTH 2 +#define KVX_SFR_PSOW_USE_WFXL_MASK _ULL(0xc0000) +#define KVX_SFR_PSOW_USE_WFXL_CLEAR _ULL(0xc0000) +#define KVX_SFR_PSOW_USE_WFXL_SET _ULL(0xc000000000000) + +#define KVX_SFR_PSOW_DCE_MASK _ULL(0x300000) /* Data Cache Enable owner */ +#define KVX_SFR_PSOW_DCE_SHIFT 20 +#define KVX_SFR_PSOW_DCE_WIDTH 2 +#define KVX_SFR_PSOW_DCE_WFXL_MASK _ULL(0x300000) +#define KVX_SFR_PSOW_DCE_WFXL_CLEAR _ULL(0x300000) +#define KVX_SFR_PSOW_DCE_WFXL_SET _ULL(0x30000000000000) + +#define KVX_SFR_PSOW_MME_MASK _ULL(0xc00000) /* Memory Management Enable owner */ +#define KVX_SFR_PSOW_MME_SHIFT 22 +#define KVX_SFR_PSOW_MME_WIDTH 2 +#define KVX_SFR_PSOW_MME_WFXL_MASK _ULL(0xc00000) +#define KVX_SFR_PSOW_MME_WFXL_CLEAR _ULL(0xc00000) +#define KVX_SFR_PSOW_MME_WFXL_SET _ULL(0xc0000000000000) + +#define KVX_SFR_PSOW_IL0_MASK _ULL(0x3000000) /* Interrupt Level bit 0 owner */ +#define KVX_SFR_PSOW_IL0_SHIFT 24 +#define KVX_SFR_PSOW_IL0_WIDTH 2 +#define KVX_SFR_PSOW_IL0_WFXL_MASK _ULL(0x3000000) +#define KVX_SFR_PSOW_IL0_WFXL_CLEAR _ULL(0x3000000) +#define KVX_SFR_PSOW_IL0_WFXL_SET _ULL(0x300000000000000) + +#define KVX_SFR_PSOW_IL1_MASK _ULL(0xc000000) /* Interrupt Level bit 1 owner */ +#define KVX_SFR_PSOW_IL1_SHIFT 26 +#define KVX_SFR_PSOW_IL1_WIDTH 2 +#define KVX_SFR_PSOW_IL1_WFXL_MASK _ULL(0xc000000) +#define KVX_SFR_PSOW_IL1_WFXL_CLEAR _ULL(0xc000000) +#define KVX_SFR_PSOW_IL1_WFXL_SET _ULL(0xc00000000000000) + +#define KVX_SFR_PSOW_VS0_MASK _ULL(0x30000000) /* Virtual Space bit 0 owner */ +#define KVX_SFR_PSOW_VS0_SHIFT 28 +#define KVX_SFR_PSOW_VS0_WIDTH 2 +#define KVX_SFR_PSOW_VS0_WFXL_MASK _ULL(0x30000000) +#define KVX_SFR_PSOW_VS0_WFXL_CLEAR _ULL(0x30000000) +#define KVX_SFR_PSOW_VS0_WFXL_SET _ULL(0x3000000000000000) + +#define KVX_SFR_PSOW_VS1_MASK _ULL(0xc0000000) /* Virtual Space bit 1 owner */ +#define KVX_SFR_PSOW_VS1_SHIFT 30 +#define KVX_SFR_PSOW_VS1_WIDTH 2 +#define KVX_SFR_PSOW_VS1_WFXL_MASK _ULL(0xc0000000) +#define KVX_SFR_PSOW_VS1_WFXL_CLEAR _ULL(0xc0000000) +#define KVX_SFR_PSOW_VS1_WFXL_SET _ULL(0xc000000000000000) + +#define KVX_SFR_PSOW_V64_MASK _ULL(0x300000000) /* Virtual 64 bits mode owner */ +#define KVX_SFR_PSOW_V64_SHIFT 32 +#define KVX_SFR_PSOW_V64_WIDTH 2 +#define KVX_SFR_PSOW_V64_WFXM_MASK _ULL(0x300000000) +#define KVX_SFR_PSOW_V64_WFXM_CLEAR _ULL(0x3) +#define KVX_SFR_PSOW_V64_WFXM_SET _ULL(0x300000000) + +#define KVX_SFR_PSOW_L2E_MASK _ULL(0xc00000000) /* L2 cache Enable owner */ +#define KVX_SFR_PSOW_L2E_SHIFT 34 +#define KVX_SFR_PSOW_L2E_WIDTH 2 +#define KVX_SFR_PSOW_L2E_WFXM_MASK _ULL(0xc00000000) +#define KVX_SFR_PSOW_L2E_WFXM_CLEAR _ULL(0xc) +#define KVX_SFR_PSOW_L2E_WFXM_SET _ULL(0xc00000000) + +#define KVX_SFR_PSOW_SME_MASK _ULL(0x3000000000) /* Step Mode Enabled owner */ +#define KVX_SFR_PSOW_SME_SHIFT 36 +#define KVX_SFR_PSOW_SME_WIDTH 2 +#define KVX_SFR_PSOW_SME_WFXM_MASK _ULL(0x3000000000) +#define KVX_SFR_PSOW_SME_WFXM_CLEAR _ULL(0x30) +#define KVX_SFR_PSOW_SME_WFXM_SET _ULL(0x3000000000) + +#define KVX_SFR_PSOW_SMR_MASK _ULL(0xc000000000) /* Step Mode Ready owner */ +#define KVX_SFR_PSOW_SMR_SHIFT 38 +#define KVX_SFR_PSOW_SMR_WIDTH 2 +#define KVX_SFR_PSOW_SMR_WFXM_MASK _ULL(0xc000000000) +#define KVX_SFR_PSOW_SMR_WFXM_CLEAR _ULL(0xc0) +#define KVX_SFR_PSOW_SMR_WFXM_SET _ULL(0xc000000000) + +#define KVX_SFR_PSOW_PMJ0_MASK _ULL(0x30000000000) /* Page Mask in JTLB bit 0 owner */ +#define KVX_SFR_PSOW_PMJ0_SHIFT 40 +#define KVX_SFR_PSOW_PMJ0_WIDTH 2 +#define KVX_SFR_PSOW_PMJ0_WFXM_MASK _ULL(0x30000000000) +#define KVX_SFR_PSOW_PMJ0_WFXM_CLEAR _ULL(0x300) +#define KVX_SFR_PSOW_PMJ0_WFXM_SET _ULL(0x30000000000) + +#define KVX_SFR_PSOW_PMJ1_MASK _ULL(0xc0000000000) /* Page Mask in JTLB bit 1 owner */ +#define KVX_SFR_PSOW_PMJ1_SHIFT 42 +#define KVX_SFR_PSOW_PMJ1_WIDTH 2 +#define KVX_SFR_PSOW_PMJ1_WFXM_MASK _ULL(0xc0000000000) +#define KVX_SFR_PSOW_PMJ1_WFXM_CLEAR _ULL(0xc00) +#define KVX_SFR_PSOW_PMJ1_WFXM_SET _ULL(0xc0000000000) + +#define KVX_SFR_PSOW_PMJ2_MASK _ULL(0x300000000000) /* Page Mask in JTLB bit 2 owner */ +#define KVX_SFR_PSOW_PMJ2_SHIFT 44 +#define KVX_SFR_PSOW_PMJ2_WIDTH 2 +#define KVX_SFR_PSOW_PMJ2_WFXM_MASK _ULL(0x300000000000) +#define KVX_SFR_PSOW_PMJ2_WFXM_CLEAR _ULL(0x3000) +#define KVX_SFR_PSOW_PMJ2_WFXM_SET _ULL(0x300000000000) + +#define KVX_SFR_PSOW_PMJ3_MASK _ULL(0xc00000000000) /* Page Mask in JTLB bit 3 owner */ +#define KVX_SFR_PSOW_PMJ3_SHIFT 46 +#define KVX_SFR_PSOW_PMJ3_WIDTH 2 +#define KVX_SFR_PSOW_PMJ3_WFXM_MASK _ULL(0xc00000000000) +#define KVX_SFR_PSOW_PMJ3_WFXM_CLEAR _ULL(0xc000) +#define KVX_SFR_PSOW_PMJ3_WFXM_SET _ULL(0xc00000000000) + +#define KVX_SFR_PSOW_MMUP_MASK _ULL(0x3000000000000) /* Privileged on MMU owner. */ +#define KVX_SFR_PSOW_MMUP_SHIFT 48 +#define KVX_SFR_PSOW_MMUP_WIDTH 2 +#define KVX_SFR_PSOW_MMUP_WFXM_MASK _ULL(0x3000000000000) +#define KVX_SFR_PSOW_MMUP_WFXM_CLEAR _ULL(0x30000) +#define KVX_SFR_PSOW_MMUP_WFXM_SET _ULL(0x3000000000000) + +#define KVX_SFR_CS_IC_MASK _ULL(0x1) /* Integer Carry */ +#define KVX_SFR_CS_IC_SHIFT 0 +#define KVX_SFR_CS_IC_WIDTH 1 +#define KVX_SFR_CS_IC_WFXL_MASK _ULL(0x1) +#define KVX_SFR_CS_IC_WFXL_CLEAR _ULL(0x1) +#define KVX_SFR_CS_IC_WFXL_SET _ULL(0x100000000) + +#define KVX_SFR_CS_IO_MASK _ULL(0x2) /* IEEE 754 Invalid Operation */ +#define KVX_SFR_CS_IO_SHIFT 1 +#define KVX_SFR_CS_IO_WIDTH 1 +#define KVX_SFR_CS_IO_WFXL_MASK _ULL(0x2) +#define KVX_SFR_CS_IO_WFXL_CLEAR _ULL(0x2) +#define KVX_SFR_CS_IO_WFXL_SET _ULL(0x200000000) + +#define KVX_SFR_CS_DZ_MASK _ULL(0x4) /* IEEE 754 Divide by Zero */ +#define KVX_SFR_CS_DZ_SHIFT 2 +#define KVX_SFR_CS_DZ_WIDTH 1 +#define KVX_SFR_CS_DZ_WFXL_MASK _ULL(0x4) +#define KVX_SFR_CS_DZ_WFXL_CLEAR _ULL(0x4) +#define KVX_SFR_CS_DZ_WFXL_SET _ULL(0x400000000) + +#define KVX_SFR_CS_OV_MASK _ULL(0x8) /* IEEE 754 Overflow */ +#define KVX_SFR_CS_OV_SHIFT 3 +#define KVX_SFR_CS_OV_WIDTH 1 +#define KVX_SFR_CS_OV_WFXL_MASK _ULL(0x8) +#define KVX_SFR_CS_OV_WFXL_CLEAR _ULL(0x8) +#define KVX_SFR_CS_OV_WFXL_SET _ULL(0x800000000) + +#define KVX_SFR_CS_UN_MASK _ULL(0x10) /* IEEE 754 Underflow */ +#define KVX_SFR_CS_UN_SHIFT 4 +#define KVX_SFR_CS_UN_WIDTH 1 +#define KVX_SFR_CS_UN_WFXL_MASK _ULL(0x10) +#define KVX_SFR_CS_UN_WFXL_CLEAR _ULL(0x10) +#define KVX_SFR_CS_UN_WFXL_SET _ULL(0x1000000000) + +#define KVX_SFR_CS_IN_MASK _ULL(0x20) /* IEEE 754 Inexact */ +#define KVX_SFR_CS_IN_SHIFT 5 +#define KVX_SFR_CS_IN_WIDTH 1 +#define KVX_SFR_CS_IN_WFXL_MASK _ULL(0x20) +#define KVX_SFR_CS_IN_WFXL_CLEAR _ULL(0x20) +#define KVX_SFR_CS_IN_WFXL_SET _ULL(0x2000000000) + +#define KVX_SFR_CS_XIO_MASK _ULL(0x200) /* Extension IEEE 754 Invalid Operation */ +#define KVX_SFR_CS_XIO_SHIFT 9 +#define KVX_SFR_CS_XIO_WIDTH 1 +#define KVX_SFR_CS_XIO_WFXL_MASK _ULL(0x200) +#define KVX_SFR_CS_XIO_WFXL_CLEAR _ULL(0x200) +#define KVX_SFR_CS_XIO_WFXL_SET _ULL(0x20000000000) + +#define KVX_SFR_CS_XDZ_MASK _ULL(0x400) /* Extension IEEE 754 Divide by Zero */ +#define KVX_SFR_CS_XDZ_SHIFT 10 +#define KVX_SFR_CS_XDZ_WIDTH 1 +#define KVX_SFR_CS_XDZ_WFXL_MASK _ULL(0x400) +#define KVX_SFR_CS_XDZ_WFXL_CLEAR _ULL(0x400) +#define KVX_SFR_CS_XDZ_WFXL_SET _ULL(0x40000000000) + +#define KVX_SFR_CS_XOV_MASK _ULL(0x800) /* Extension IEEE 754 Overflow */ +#define KVX_SFR_CS_XOV_SHIFT 11 +#define KVX_SFR_CS_XOV_WIDTH 1 +#define KVX_SFR_CS_XOV_WFXL_MASK _ULL(0x800) +#define KVX_SFR_CS_XOV_WFXL_CLEAR _ULL(0x800) +#define KVX_SFR_CS_XOV_WFXL_SET _ULL(0x80000000000) + +#define KVX_SFR_CS_XUN_MASK _ULL(0x1000) /* Extension IEEE 754 Underflow */ +#define KVX_SFR_CS_XUN_SHIFT 12 +#define KVX_SFR_CS_XUN_WIDTH 1 +#define KVX_SFR_CS_XUN_WFXL_MASK _ULL(0x1000) +#define KVX_SFR_CS_XUN_WFXL_CLEAR _ULL(0x1000) +#define KVX_SFR_CS_XUN_WFXL_SET _ULL(0x100000000000) + +#define KVX_SFR_CS_XIN_MASK _ULL(0x2000) /* Extension IEEE 754 Inexact */ +#define KVX_SFR_CS_XIN_SHIFT 13 +#define KVX_SFR_CS_XIN_WIDTH 1 +#define KVX_SFR_CS_XIN_WFXL_MASK _ULL(0x2000) +#define KVX_SFR_CS_XIN_WFXL_CLEAR _ULL(0x2000) +#define KVX_SFR_CS_XIN_WFXL_SET _ULL(0x200000000000) + +#define KVX_SFR_CS_RM_MASK _ULL(0x30000) /* IEEE 754 Rounding Mode */ +#define KVX_SFR_CS_RM_SHIFT 16 +#define KVX_SFR_CS_RM_WIDTH 2 +#define KVX_SFR_CS_RM_WFXL_MASK _ULL(0x30000) +#define KVX_SFR_CS_RM_WFXL_CLEAR _ULL(0x30000) +#define KVX_SFR_CS_RM_WFXL_SET _ULL(0x3000000000000) + +#define KVX_SFR_CS_XRM_MASK _ULL(0x300000) /* Extension IEEE 754 Rounding Mode */ +#define KVX_SFR_CS_XRM_SHIFT 20 +#define KVX_SFR_CS_XRM_WIDTH 2 +#define KVX_SFR_CS_XRM_WFXL_MASK _ULL(0x300000) +#define KVX_SFR_CS_XRM_WFXL_CLEAR _ULL(0x300000) +#define KVX_SFR_CS_XRM_WFXL_SET _ULL(0x30000000000000) + +#define KVX_SFR_CS_XMF_MASK _ULL(0x1000000) /* eXtension ModiFied */ +#define KVX_SFR_CS_XMF_SHIFT 24 +#define KVX_SFR_CS_XMF_WIDTH 1 +#define KVX_SFR_CS_XMF_WFXL_MASK _ULL(0x1000000) +#define KVX_SFR_CS_XMF_WFXL_CLEAR _ULL(0x1000000) +#define KVX_SFR_CS_XMF_WFXL_SET _ULL(0x100000000000000) + +#define KVX_SFR_CS_CC_MASK _ULL(0xffff00000000) /* Carry Counter */ +#define KVX_SFR_CS_CC_SHIFT 32 +#define KVX_SFR_CS_CC_WIDTH 16 +#define KVX_SFR_CS_CC_WFXM_MASK _ULL(0xffff00000000) +#define KVX_SFR_CS_CC_WFXM_CLEAR _ULL(0xffff) +#define KVX_SFR_CS_CC_WFXM_SET _ULL(0xffff00000000) + +#define KVX_SFR_CS_XDROP_MASK _ULL(0x3f000000000000) /* Extension Conversion Drop Bits */ +#define KVX_SFR_CS_XDROP_SHIFT 48 +#define KVX_SFR_CS_XDROP_WIDTH 6 +#define KVX_SFR_CS_XDROP_WFXM_MASK _ULL(0x3f000000000000) +#define KVX_SFR_CS_XDROP_WFXM_CLEAR _ULL(0x3f0000) +#define KVX_SFR_CS_XDROP_WFXM_SET _ULL(0x3f000000000000) + +#define KVX_SFR_CS_XPOW2_MASK _ULL(0xfc0000000000000) /* Extension FScale Power of Two */ +#define KVX_SFR_CS_XPOW2_SHIFT 54 +#define KVX_SFR_CS_XPOW2_WIDTH 6 +#define KVX_SFR_CS_XPOW2_WFXM_MASK _ULL(0xfc0000000000000) +#define KVX_SFR_CS_XPOW2_WFXM_CLEAR _ULL(0xfc00000) +#define KVX_SFR_CS_XPOW2_WFXM_SET _ULL(0xfc0000000000000) + +#define KVX_SFR_AESPC_AESPC_MASK _ULL(0xffffffffffffffff) /* Arithmetic Exception Saved PC */ +#define KVX_SFR_AESPC_AESPC_SHIFT 0 +#define KVX_SFR_AESPC_AESPC_WIDTH 64 +#define KVX_SFR_AESPC_AESPC_WFXL_MASK _ULL(0xffffffff) +#define KVX_SFR_AESPC_AESPC_WFXL_CLEAR _ULL(0xffffffff) +#define KVX_SFR_AESPC_AESPC_WFXL_SET _ULL(0xffffffff00000000) +#define KVX_SFR_AESPC_AESPC_WFXM_MASK _ULL(0xffffffff00000000) +#define KVX_SFR_AESPC_AESPC_WFXM_CLEAR _ULL(0xffffffff) +#define KVX_SFR_AESPC_AESPC_WFXM_SET _ULL(0xffffffff00000000) + +#define KVX_SFR_CSIT_ICIE_MASK _ULL(0x1) /* Integer Carry Interrupt Enable */ +#define KVX_SFR_CSIT_ICIE_SHIFT 0 +#define KVX_SFR_CSIT_ICIE_WIDTH 1 +#define KVX_SFR_CSIT_ICIE_WFXL_MASK _ULL(0x1) +#define KVX_SFR_CSIT_ICIE_WFXL_CLEAR _ULL(0x1) +#define KVX_SFR_CSIT_ICIE_WFXL_SET _ULL(0x100000000) + +#define KVX_SFR_CSIT_IOIE_MASK _ULL(0x2) /* IEEE 754 Invalid Operation Interrupt Enable */ +#define KVX_SFR_CSIT_IOIE_SHIFT 1 +#define KVX_SFR_CSIT_IOIE_WIDTH 1 +#define KVX_SFR_CSIT_IOIE_WFXL_MASK _ULL(0x2) +#define KVX_SFR_CSIT_IOIE_WFXL_CLEAR _ULL(0x2) +#define KVX_SFR_CSIT_IOIE_WFXL_SET _ULL(0x200000000) + +#define KVX_SFR_CSIT_DZIE_MASK _ULL(0x4) /* IEEE 754 Divide by Zero Interrupt Enable */ +#define KVX_SFR_CSIT_DZIE_SHIFT 2 +#define KVX_SFR_CSIT_DZIE_WIDTH 1 +#define KVX_SFR_CSIT_DZIE_WFXL_MASK _ULL(0x4) +#define KVX_SFR_CSIT_DZIE_WFXL_CLEAR _ULL(0x4) +#define KVX_SFR_CSIT_DZIE_WFXL_SET _ULL(0x400000000) + +#define KVX_SFR_CSIT_OVIE_MASK _ULL(0x8) /* IEEE 754 Overflow Interrupt Enable */ +#define KVX_SFR_CSIT_OVIE_SHIFT 3 +#define KVX_SFR_CSIT_OVIE_WIDTH 1 +#define KVX_SFR_CSIT_OVIE_WFXL_MASK _ULL(0x8) +#define KVX_SFR_CSIT_OVIE_WFXL_CLEAR _ULL(0x8) +#define KVX_SFR_CSIT_OVIE_WFXL_SET _ULL(0x800000000) + +#define KVX_SFR_CSIT_UNIE_MASK _ULL(0x10) /* IEEE 754 Underflow Interrupt Enable */ +#define KVX_SFR_CSIT_UNIE_SHIFT 4 +#define KVX_SFR_CSIT_UNIE_WIDTH 1 +#define KVX_SFR_CSIT_UNIE_WFXL_MASK _ULL(0x10) +#define KVX_SFR_CSIT_UNIE_WFXL_CLEAR _ULL(0x10) +#define KVX_SFR_CSIT_UNIE_WFXL_SET _ULL(0x1000000000) + +#define KVX_SFR_CSIT_INIE_MASK _ULL(0x20) /* IEEE 754 Inexact Interrupt Enable */ +#define KVX_SFR_CSIT_INIE_SHIFT 5 +#define KVX_SFR_CSIT_INIE_WIDTH 1 +#define KVX_SFR_CSIT_INIE_WFXL_MASK _ULL(0x20) +#define KVX_SFR_CSIT_INIE_WFXL_CLEAR _ULL(0x20) +#define KVX_SFR_CSIT_INIE_WFXL_SET _ULL(0x2000000000) + +#define KVX_SFR_CSIT_XIOIE_MASK _ULL(0x200) /* Extension IEEE 754 Invalid Operation Interrupt Enable */ +#define KVX_SFR_CSIT_XIOIE_SHIFT 9 +#define KVX_SFR_CSIT_XIOIE_WIDTH 1 +#define KVX_SFR_CSIT_XIOIE_WFXL_MASK _ULL(0x200) +#define KVX_SFR_CSIT_XIOIE_WFXL_CLEAR _ULL(0x200) +#define KVX_SFR_CSIT_XIOIE_WFXL_SET _ULL(0x20000000000) + +#define KVX_SFR_CSIT_XDZIE_MASK _ULL(0x400) /* Extension IEEE 754 Divide by Zero Interrupt Enable */ +#define KVX_SFR_CSIT_XDZIE_SHIFT 10 +#define KVX_SFR_CSIT_XDZIE_WIDTH 1 +#define KVX_SFR_CSIT_XDZIE_WFXL_MASK _ULL(0x400) +#define KVX_SFR_CSIT_XDZIE_WFXL_CLEAR _ULL(0x400) +#define KVX_SFR_CSIT_XDZIE_WFXL_SET _ULL(0x40000000000) + +#define KVX_SFR_CSIT_XOVIE_MASK _ULL(0x800) /* Extension IEEE 754 Overflow Interrupt Enable */ +#define KVX_SFR_CSIT_XOVIE_SHIFT 11 +#define KVX_SFR_CSIT_XOVIE_WIDTH 1 +#define KVX_SFR_CSIT_XOVIE_WFXL_MASK _ULL(0x800) +#define KVX_SFR_CSIT_XOVIE_WFXL_CLEAR _ULL(0x800) +#define KVX_SFR_CSIT_XOVIE_WFXL_SET _ULL(0x80000000000) + +#define KVX_SFR_CSIT_XUNIE_MASK _ULL(0x1000) /* Extension IEEE 754 Underflow Interrupt Enable */ +#define KVX_SFR_CSIT_XUNIE_SHIFT 12 +#define KVX_SFR_CSIT_XUNIE_WIDTH 1 +#define KVX_SFR_CSIT_XUNIE_WFXL_MASK _ULL(0x1000) +#define KVX_SFR_CSIT_XUNIE_WFXL_CLEAR _ULL(0x1000) +#define KVX_SFR_CSIT_XUNIE_WFXL_SET _ULL(0x100000000000) + +#define KVX_SFR_CSIT_XINIE_MASK _ULL(0x2000) /* Extension IEEE 754 Inexact Interrupt Enable */ +#define KVX_SFR_CSIT_XINIE_SHIFT 13 +#define KVX_SFR_CSIT_XINIE_WIDTH 1 +#define KVX_SFR_CSIT_XINIE_WFXL_MASK _ULL(0x2000) +#define KVX_SFR_CSIT_XINIE_WFXL_CLEAR _ULL(0x2000) +#define KVX_SFR_CSIT_XINIE_WFXL_SET _ULL(0x200000000000) + +#define KVX_SFR_CSIT_AEIR_MASK _ULL(0x10000) /* Arithmetic Exception Interrupt Raised */ +#define KVX_SFR_CSIT_AEIR_SHIFT 16 +#define KVX_SFR_CSIT_AEIR_WIDTH 1 +#define KVX_SFR_CSIT_AEIR_WFXL_MASK _ULL(0x10000) +#define KVX_SFR_CSIT_AEIR_WFXL_CLEAR _ULL(0x10000) +#define KVX_SFR_CSIT_AEIR_WFXL_SET _ULL(0x1000000000000) + +#define KVX_SFR_CSIT_AEC_MASK _ULL(0xe0000) /* Arithmetic Exception Code */ +#define KVX_SFR_CSIT_AEC_SHIFT 17 +#define KVX_SFR_CSIT_AEC_WIDTH 3 +#define KVX_SFR_CSIT_AEC_WFXL_MASK _ULL(0xe0000) +#define KVX_SFR_CSIT_AEC_WFXL_CLEAR _ULL(0xe0000) +#define KVX_SFR_CSIT_AEC_WFXL_SET _ULL(0xe000000000000) + +#define KVX_SFR_CSIT_SPCV_MASK _ULL(0x100000) /* SPC Valid */ +#define KVX_SFR_CSIT_SPCV_SHIFT 20 +#define KVX_SFR_CSIT_SPCV_WIDTH 1 +#define KVX_SFR_CSIT_SPCV_WFXL_MASK _ULL(0x100000) +#define KVX_SFR_CSIT_SPCV_WFXL_CLEAR _ULL(0x100000) +#define KVX_SFR_CSIT_SPCV_WFXL_SET _ULL(0x10000000000000) + +#define KVX_SFR_ES_EC_MASK _ULL(0xf) /* Exception Class */ +#define KVX_SFR_ES_EC_SHIFT 0 +#define KVX_SFR_ES_EC_WIDTH 4 +#define KVX_SFR_ES_EC_WFXL_MASK _ULL(0xf) +#define KVX_SFR_ES_EC_WFXL_CLEAR _ULL(0xf) +#define KVX_SFR_ES_EC_WFXL_SET _ULL(0xf00000000) + +#define KVX_SFR_ES_ED_MASK _ULL(0xfffffffffffffff0) /* Exception Details */ +#define KVX_SFR_ES_ED_SHIFT 4 +#define KVX_SFR_ES_ED_WIDTH 60 +#define KVX_SFR_ES_ED_WFXL_MASK _ULL(0xfffffff0) +#define KVX_SFR_ES_ED_WFXL_CLEAR _ULL(0xfffffff0) +#define KVX_SFR_ES_ED_WFXL_SET _ULL(0xfffffff000000000) +#define KVX_SFR_ES_ED_WFXM_MASK _ULL(0xffffffff00000000) +#define KVX_SFR_ES_ED_WFXM_CLEAR _ULL(0xffffffff) +#define KVX_SFR_ES_ED_WFXM_SET _ULL(0xffffffff00000000) + +#define KVX_SFR_ES_OAPL_MASK _ULL(0x30) /* Origin Absolute PL */ +#define KVX_SFR_ES_OAPL_SHIFT 4 +#define KVX_SFR_ES_OAPL_WIDTH 2 +#define KVX_SFR_ES_OAPL_WFXL_MASK _ULL(0x30) +#define KVX_SFR_ES_OAPL_WFXL_CLEAR _ULL(0x30) +#define KVX_SFR_ES_OAPL_WFXL_SET _ULL(0x3000000000) + +#define KVX_SFR_ES_ORPL_MASK _ULL(0xc0) /* Origin Relative PL */ +#define KVX_SFR_ES_ORPL_SHIFT 6 +#define KVX_SFR_ES_ORPL_WIDTH 2 +#define KVX_SFR_ES_ORPL_WFXL_MASK _ULL(0xc0) +#define KVX_SFR_ES_ORPL_WFXL_CLEAR _ULL(0xc0) +#define KVX_SFR_ES_ORPL_WFXL_SET _ULL(0xc000000000) + +#define KVX_SFR_ES_PTAPL_MASK _ULL(0x300) /* Primary Target Absolute PL */ +#define KVX_SFR_ES_PTAPL_SHIFT 8 +#define KVX_SFR_ES_PTAPL_WIDTH 2 +#define KVX_SFR_ES_PTAPL_WFXL_MASK _ULL(0x300) +#define KVX_SFR_ES_PTAPL_WFXL_CLEAR _ULL(0x300) +#define KVX_SFR_ES_PTAPL_WFXL_SET _ULL(0x30000000000) + +#define KVX_SFR_ES_PTRPL_MASK _ULL(0xc00) /* Primary Target Relative PL */ +#define KVX_SFR_ES_PTRPL_SHIFT 10 +#define KVX_SFR_ES_PTRPL_WIDTH 2 +#define KVX_SFR_ES_PTRPL_WFXL_MASK _ULL(0xc00) +#define KVX_SFR_ES_PTRPL_WFXL_CLEAR _ULL(0xc00) +#define KVX_SFR_ES_PTRPL_WFXL_SET _ULL(0xc0000000000) + +#define KVX_SFR_ES_ITN_MASK _ULL(0x1f000) /* InTerrupt Number */ +#define KVX_SFR_ES_ITN_SHIFT 12 +#define KVX_SFR_ES_ITN_WIDTH 5 +#define KVX_SFR_ES_ITN_WFXL_MASK _ULL(0x1f000) +#define KVX_SFR_ES_ITN_WFXL_CLEAR _ULL(0x1f000) +#define KVX_SFR_ES_ITN_WFXL_SET _ULL(0x1f00000000000) + +#define KVX_SFR_ES_ITL_MASK _ULL(0x60000) /* InTerrupt Level */ +#define KVX_SFR_ES_ITL_SHIFT 17 +#define KVX_SFR_ES_ITL_WIDTH 2 +#define KVX_SFR_ES_ITL_WFXL_MASK _ULL(0x60000) +#define KVX_SFR_ES_ITL_WFXL_CLEAR _ULL(0x60000) +#define KVX_SFR_ES_ITL_WFXL_SET _ULL(0x6000000000000) + +#define KVX_SFR_ES_ITI_MASK _ULL(0x1ff80000) /* InTerrupt Info */ +#define KVX_SFR_ES_ITI_SHIFT 19 +#define KVX_SFR_ES_ITI_WIDTH 10 +#define KVX_SFR_ES_ITI_WFXL_MASK _ULL(0x1ff80000) +#define KVX_SFR_ES_ITI_WFXL_CLEAR _ULL(0x1ff80000) +#define KVX_SFR_ES_ITI_WFXL_SET _ULL(0x1ff8000000000000) + +#define KVX_SFR_ES_SN_MASK _ULL(0xfff000) /* Syscall Number */ +#define KVX_SFR_ES_SN_SHIFT 12 +#define KVX_SFR_ES_SN_WIDTH 12 +#define KVX_SFR_ES_SN_WFXL_MASK _ULL(0xfff000) +#define KVX_SFR_ES_SN_WFXL_CLEAR _ULL(0xfff000) +#define KVX_SFR_ES_SN_WFXL_SET _ULL(0xfff00000000000) + +#define KVX_SFR_ES_HTC_MASK _ULL(0x1f000) /* Hardware Trap Cause */ +#define KVX_SFR_ES_HTC_SHIFT 12 +#define KVX_SFR_ES_HTC_WIDTH 5 +#define KVX_SFR_ES_HTC_WFXL_MASK _ULL(0x1f000) +#define KVX_SFR_ES_HTC_WFXL_CLEAR _ULL(0x1f000) +#define KVX_SFR_ES_HTC_WFXL_SET _ULL(0x1f00000000000) + +#define KVX_SFR_ES_SFRT_MASK _ULL(0x20000) /* SFR Trap */ +#define KVX_SFR_ES_SFRT_SHIFT 17 +#define KVX_SFR_ES_SFRT_WIDTH 1 +#define KVX_SFR_ES_SFRT_WFXL_MASK _ULL(0x20000) +#define KVX_SFR_ES_SFRT_WFXL_CLEAR _ULL(0x20000) +#define KVX_SFR_ES_SFRT_WFXL_SET _ULL(0x2000000000000) + +#define KVX_SFR_ES_SFRI_MASK _ULL(0x1c0000) /* SFR Instruction */ +#define KVX_SFR_ES_SFRI_SHIFT 18 +#define KVX_SFR_ES_SFRI_WIDTH 3 +#define KVX_SFR_ES_SFRI_WFXL_MASK _ULL(0x1c0000) +#define KVX_SFR_ES_SFRI_WFXL_CLEAR _ULL(0x1c0000) +#define KVX_SFR_ES_SFRI_WFXL_SET _ULL(0x1c000000000000) + +#define KVX_SFR_ES_GPRP_MASK _ULL(0x7e00000) /* GPR Pointer */ +#define KVX_SFR_ES_GPRP_SHIFT 21 +#define KVX_SFR_ES_GPRP_WIDTH 6 +#define KVX_SFR_ES_GPRP_WFXL_MASK _ULL(0x7e00000) +#define KVX_SFR_ES_GPRP_WFXL_CLEAR _ULL(0x7e00000) +#define KVX_SFR_ES_GPRP_WFXL_SET _ULL(0x7e0000000000000) + +#define KVX_SFR_ES_SFRP_MASK _ULL(0xff8000000) /* SFR Pointer */ +#define KVX_SFR_ES_SFRP_SHIFT 27 +#define KVX_SFR_ES_SFRP_WIDTH 9 +#define KVX_SFR_ES_SFRP_WFXL_MASK _ULL(0xf8000000) +#define KVX_SFR_ES_SFRP_WFXL_CLEAR _ULL(0xf8000000) +#define KVX_SFR_ES_SFRP_WFXL_SET _ULL(0xf800000000000000) +#define KVX_SFR_ES_SFRP_WFXM_MASK _ULL(0xf00000000) +#define KVX_SFR_ES_SFRP_WFXM_CLEAR _ULL(0xf) +#define KVX_SFR_ES_SFRP_WFXM_SET _ULL(0xf00000000) + +#define KVX_SFR_ES_DHT_MASK _ULL(0x1000000000) /* Disabled Hardware Trap */ +#define KVX_SFR_ES_DHT_SHIFT 36 +#define KVX_SFR_ES_DHT_WIDTH 1 +#define KVX_SFR_ES_DHT_WFXM_MASK _ULL(0x1000000000) +#define KVX_SFR_ES_DHT_WFXM_CLEAR _ULL(0x10) +#define KVX_SFR_ES_DHT_WFXM_SET _ULL(0x1000000000) + +#define KVX_SFR_ES_RWX_MASK _ULL(0x38000000000) /* Read Write Execute */ +#define KVX_SFR_ES_RWX_SHIFT 39 +#define KVX_SFR_ES_RWX_WIDTH 3 +#define KVX_SFR_ES_RWX_WFXM_MASK _ULL(0x38000000000) +#define KVX_SFR_ES_RWX_WFXM_CLEAR _ULL(0x380) +#define KVX_SFR_ES_RWX_WFXM_SET _ULL(0x38000000000) + +#define KVX_SFR_ES_NTA_MASK _ULL(0x40000000000) /* Non-Trapping Access */ +#define KVX_SFR_ES_NTA_SHIFT 42 +#define KVX_SFR_ES_NTA_WIDTH 1 +#define KVX_SFR_ES_NTA_WFXM_MASK _ULL(0x40000000000) +#define KVX_SFR_ES_NTA_WFXM_CLEAR _ULL(0x400) +#define KVX_SFR_ES_NTA_WFXM_SET _ULL(0x40000000000) + +#define KVX_SFR_ES_UCA_MASK _ULL(0x80000000000) /* Un-Cached Access */ +#define KVX_SFR_ES_UCA_SHIFT 43 +#define KVX_SFR_ES_UCA_WIDTH 1 +#define KVX_SFR_ES_UCA_WFXM_MASK _ULL(0x80000000000) +#define KVX_SFR_ES_UCA_WFXM_CLEAR _ULL(0x800) +#define KVX_SFR_ES_UCA_WFXM_SET _ULL(0x80000000000) + +#define KVX_SFR_ES_AS_MASK _ULL(0x3f00000000000) /* Access Size */ +#define KVX_SFR_ES_AS_SHIFT 44 +#define KVX_SFR_ES_AS_WIDTH 6 +#define KVX_SFR_ES_AS_WFXM_MASK _ULL(0x3f00000000000) +#define KVX_SFR_ES_AS_WFXM_CLEAR _ULL(0x3f000) +#define KVX_SFR_ES_AS_WFXM_SET _ULL(0x3f00000000000) + +#define KVX_SFR_ES_BS_MASK _ULL(0x3c000000000000) /* Bundle Size */ +#define KVX_SFR_ES_BS_SHIFT 50 +#define KVX_SFR_ES_BS_WIDTH 4 +#define KVX_SFR_ES_BS_WFXM_MASK _ULL(0x3c000000000000) +#define KVX_SFR_ES_BS_WFXM_CLEAR _ULL(0x3c0000) +#define KVX_SFR_ES_BS_WFXM_SET _ULL(0x3c000000000000) + +#define KVX_SFR_ES_DRI_MASK _ULL(0xfc0000000000000) /* Data Register Index */ +#define KVX_SFR_ES_DRI_SHIFT 54 +#define KVX_SFR_ES_DRI_WIDTH 6 +#define KVX_SFR_ES_DRI_WFXM_MASK _ULL(0xfc0000000000000) +#define KVX_SFR_ES_DRI_WFXM_CLEAR _ULL(0xfc00000) +#define KVX_SFR_ES_DRI_WFXM_SET _ULL(0xfc0000000000000) + +#define KVX_SFR_ES_PIC_MASK _ULL(0xf000000000000000) /* Privileged Instruction Code */ +#define KVX_SFR_ES_PIC_SHIFT 60 +#define KVX_SFR_ES_PIC_WIDTH 4 +#define KVX_SFR_ES_PIC_WFXM_MASK _ULL(0xf000000000000000) +#define KVX_SFR_ES_PIC_WFXM_CLEAR _ULL(0xf0000000) +#define KVX_SFR_ES_PIC_WFXM_SET _ULL(0xf000000000000000) + +#define KVX_SFR_ES_DC_MASK _ULL(0x3000) /* Debug Cause */ +#define KVX_SFR_ES_DC_SHIFT 12 +#define KVX_SFR_ES_DC_WIDTH 2 +#define KVX_SFR_ES_DC_WFXL_MASK _ULL(0x3000) +#define KVX_SFR_ES_DC_WFXL_CLEAR _ULL(0x3000) +#define KVX_SFR_ES_DC_WFXL_SET _ULL(0x300000000000) + +#define KVX_SFR_ES_BN_MASK _ULL(0x4000) /* Breakpoint Number */ +#define KVX_SFR_ES_BN_SHIFT 14 +#define KVX_SFR_ES_BN_WIDTH 1 +#define KVX_SFR_ES_BN_WFXL_MASK _ULL(0x4000) +#define KVX_SFR_ES_BN_WFXL_CLEAR _ULL(0x4000) +#define KVX_SFR_ES_BN_WFXL_SET _ULL(0x400000000000) + +#define KVX_SFR_ES_WN_MASK _ULL(0x8000) /* Watchpoint Number */ +#define KVX_SFR_ES_WN_SHIFT 15 +#define KVX_SFR_ES_WN_WIDTH 1 +#define KVX_SFR_ES_WN_WFXL_MASK _ULL(0x8000) +#define KVX_SFR_ES_WN_WFXL_CLEAR _ULL(0x8000) +#define KVX_SFR_ES_WN_WFXL_SET _ULL(0x800000000000) + +#define KVX_SFR_ES_PL0_EC_MASK _ULL(0xf) /* Exception Class */ +#define KVX_SFR_ES_PL0_EC_SHIFT 0 +#define KVX_SFR_ES_PL0_EC_WIDTH 4 +#define KVX_SFR_ES_PL0_EC_WFXL_MASK _ULL(0xf) +#define KVX_SFR_ES_PL0_EC_WFXL_CLEAR _ULL(0xf) +#define KVX_SFR_ES_PL0_EC_WFXL_SET _ULL(0xf00000000) + +#define KVX_SFR_ES_PL0_ED_MASK _ULL(0xfffffffffffffff0) /* Exception Details */ +#define KVX_SFR_ES_PL0_ED_SHIFT 4 +#define KVX_SFR_ES_PL0_ED_WIDTH 60 +#define KVX_SFR_ES_PL0_ED_WFXL_MASK _ULL(0xfffffff0) +#define KVX_SFR_ES_PL0_ED_WFXL_CLEAR _ULL(0xfffffff0) +#define KVX_SFR_ES_PL0_ED_WFXL_SET _ULL(0xfffffff000000000) +#define KVX_SFR_ES_PL0_ED_WFXM_MASK _ULL(0xffffffff00000000) +#define KVX_SFR_ES_PL0_ED_WFXM_CLEAR _ULL(0xffffffff) +#define KVX_SFR_ES_PL0_ED_WFXM_SET _ULL(0xffffffff00000000) + +#define KVX_SFR_ES_PL0_OAPL_MASK _ULL(0x30) /* Origin Absolute PL */ +#define KVX_SFR_ES_PL0_OAPL_SHIFT 4 +#define KVX_SFR_ES_PL0_OAPL_WIDTH 2 +#define KVX_SFR_ES_PL0_OAPL_WFXL_MASK _ULL(0x30) +#define KVX_SFR_ES_PL0_OAPL_WFXL_CLEAR _ULL(0x30) +#define KVX_SFR_ES_PL0_OAPL_WFXL_SET _ULL(0x3000000000) + +#define KVX_SFR_ES_PL0_ORPL_MASK _ULL(0xc0) /* Origin Relative PL */ +#define KVX_SFR_ES_PL0_ORPL_SHIFT 6 +#define KVX_SFR_ES_PL0_ORPL_WIDTH 2 +#define KVX_SFR_ES_PL0_ORPL_WFXL_MASK _ULL(0xc0) +#define KVX_SFR_ES_PL0_ORPL_WFXL_CLEAR _ULL(0xc0) +#define KVX_SFR_ES_PL0_ORPL_WFXL_SET _ULL(0xc000000000) + +#define KVX_SFR_ES_PL0_PTAPL_MASK _ULL(0x300) /* Target Absolute PL */ +#define KVX_SFR_ES_PL0_PTAPL_SHIFT 8 +#define KVX_SFR_ES_PL0_PTAPL_WIDTH 2 +#define KVX_SFR_ES_PL0_PTAPL_WFXL_MASK _ULL(0x300) +#define KVX_SFR_ES_PL0_PTAPL_WFXL_CLEAR _ULL(0x300) +#define KVX_SFR_ES_PL0_PTAPL_WFXL_SET _ULL(0x30000000000) + +#define KVX_SFR_ES_PL0_PTRPL_MASK _ULL(0xc00) /* Target Relative PL */ +#define KVX_SFR_ES_PL0_PTRPL_SHIFT 10 +#define KVX_SFR_ES_PL0_PTRPL_WIDTH 2 +#define KVX_SFR_ES_PL0_PTRPL_WFXL_MASK _ULL(0xc00) +#define KVX_SFR_ES_PL0_PTRPL_WFXL_CLEAR _ULL(0xc00) +#define KVX_SFR_ES_PL0_PTRPL_WFXL_SET _ULL(0xc0000000000) + +#define KVX_SFR_ES_PL0_ITN_MASK _ULL(0x1f000) /* InTerrupt Number */ +#define KVX_SFR_ES_PL0_ITN_SHIFT 12 +#define KVX_SFR_ES_PL0_ITN_WIDTH 5 +#define KVX_SFR_ES_PL0_ITN_WFXL_MASK _ULL(0x1f000) +#define KVX_SFR_ES_PL0_ITN_WFXL_CLEAR _ULL(0x1f000) +#define KVX_SFR_ES_PL0_ITN_WFXL_SET _ULL(0x1f00000000000) + +#define KVX_SFR_ES_PL0_ITL_MASK _ULL(0x60000) /* InTerrupt Level */ +#define KVX_SFR_ES_PL0_ITL_SHIFT 17 +#define KVX_SFR_ES_PL0_ITL_WIDTH 2 +#define KVX_SFR_ES_PL0_ITL_WFXL_MASK _ULL(0x60000) +#define KVX_SFR_ES_PL0_ITL_WFXL_CLEAR _ULL(0x60000) +#define KVX_SFR_ES_PL0_ITL_WFXL_SET _ULL(0x6000000000000) + +#define KVX_SFR_ES_PL0_ITI_MASK _ULL(0x1ff80000) /* InTerrupt Info */ +#define KVX_SFR_ES_PL0_ITI_SHIFT 19 +#define KVX_SFR_ES_PL0_ITI_WIDTH 10 +#define KVX_SFR_ES_PL0_ITI_WFXL_MASK _ULL(0x1ff80000) +#define KVX_SFR_ES_PL0_ITI_WFXL_CLEAR _ULL(0x1ff80000) +#define KVX_SFR_ES_PL0_ITI_WFXL_SET _ULL(0x1ff8000000000000) + +#define KVX_SFR_ES_PL0_SN_MASK _ULL(0xfff000) /* Syscall Number */ +#define KVX_SFR_ES_PL0_SN_SHIFT 12 +#define KVX_SFR_ES_PL0_SN_WIDTH 12 +#define KVX_SFR_ES_PL0_SN_WFXL_MASK _ULL(0xfff000) +#define KVX_SFR_ES_PL0_SN_WFXL_CLEAR _ULL(0xfff000) +#define KVX_SFR_ES_PL0_SN_WFXL_SET _ULL(0xfff00000000000) + +#define KVX_SFR_ES_PL0_HTC_MASK _ULL(0x1f000) /* Hardware Trap Cause */ +#define KVX_SFR_ES_PL0_HTC_SHIFT 12 +#define KVX_SFR_ES_PL0_HTC_WIDTH 5 +#define KVX_SFR_ES_PL0_HTC_WFXL_MASK _ULL(0x1f000) +#define KVX_SFR_ES_PL0_HTC_WFXL_CLEAR _ULL(0x1f000) +#define KVX_SFR_ES_PL0_HTC_WFXL_SET _ULL(0x1f00000000000) + +#define KVX_SFR_ES_PL0_SFRT_MASK _ULL(0x20000) /* SFR Trap */ +#define KVX_SFR_ES_PL0_SFRT_SHIFT 17 +#define KVX_SFR_ES_PL0_SFRT_WIDTH 1 +#define KVX_SFR_ES_PL0_SFRT_WFXL_MASK _ULL(0x20000) +#define KVX_SFR_ES_PL0_SFRT_WFXL_CLEAR _ULL(0x20000) +#define KVX_SFR_ES_PL0_SFRT_WFXL_SET _ULL(0x2000000000000) + +#define KVX_SFR_ES_PL0_SFRI_MASK _ULL(0x1c0000) /* SFR Instruction */ +#define KVX_SFR_ES_PL0_SFRI_SHIFT 18 +#define KVX_SFR_ES_PL0_SFRI_WIDTH 3 +#define KVX_SFR_ES_PL0_SFRI_WFXL_MASK _ULL(0x1c0000) +#define KVX_SFR_ES_PL0_SFRI_WFXL_CLEAR _ULL(0x1c0000) +#define KVX_SFR_ES_PL0_SFRI_WFXL_SET _ULL(0x1c000000000000) + +#define KVX_SFR_ES_PL0_GPRP_MASK _ULL(0x7e00000) /* GPR Pointer */ +#define KVX_SFR_ES_PL0_GPRP_SHIFT 21 +#define KVX_SFR_ES_PL0_GPRP_WIDTH 6 +#define KVX_SFR_ES_PL0_GPRP_WFXL_MASK _ULL(0x7e00000) +#define KVX_SFR_ES_PL0_GPRP_WFXL_CLEAR _ULL(0x7e00000) +#define KVX_SFR_ES_PL0_GPRP_WFXL_SET _ULL(0x7e0000000000000) + +#define KVX_SFR_ES_PL0_SFRP_MASK _ULL(0xff8000000) /* SFR Pointer */ +#define KVX_SFR_ES_PL0_SFRP_SHIFT 27 +#define KVX_SFR_ES_PL0_SFRP_WIDTH 9 +#define KVX_SFR_ES_PL0_SFRP_WFXL_MASK _ULL(0xf8000000) +#define KVX_SFR_ES_PL0_SFRP_WFXL_CLEAR _ULL(0xf8000000) +#define KVX_SFR_ES_PL0_SFRP_WFXL_SET _ULL(0xf800000000000000) +#define KVX_SFR_ES_PL0_SFRP_WFXM_MASK _ULL(0xf00000000) +#define KVX_SFR_ES_PL0_SFRP_WFXM_CLEAR _ULL(0xf) +#define KVX_SFR_ES_PL0_SFRP_WFXM_SET _ULL(0xf00000000) + +#define KVX_SFR_ES_PL0_DHT_MASK _ULL(0x1000000000) /* Disabled Hardware Trap */ +#define KVX_SFR_ES_PL0_DHT_SHIFT 36 +#define KVX_SFR_ES_PL0_DHT_WIDTH 1 +#define KVX_SFR_ES_PL0_DHT_WFXM_MASK _ULL(0x1000000000) +#define KVX_SFR_ES_PL0_DHT_WFXM_CLEAR _ULL(0x10) +#define KVX_SFR_ES_PL0_DHT_WFXM_SET _ULL(0x1000000000) + +#define KVX_SFR_ES_PL0_RWX_MASK _ULL(0x38000000000) /* Read Write Execute */ +#define KVX_SFR_ES_PL0_RWX_SHIFT 39 +#define KVX_SFR_ES_PL0_RWX_WIDTH 3 +#define KVX_SFR_ES_PL0_RWX_WFXM_MASK _ULL(0x38000000000) +#define KVX_SFR_ES_PL0_RWX_WFXM_CLEAR _ULL(0x380) +#define KVX_SFR_ES_PL0_RWX_WFXM_SET _ULL(0x38000000000) + +#define KVX_SFR_ES_PL0_NTA_MASK _ULL(0x40000000000) /* Non-Trapping Access */ +#define KVX_SFR_ES_PL0_NTA_SHIFT 42 +#define KVX_SFR_ES_PL0_NTA_WIDTH 1 +#define KVX_SFR_ES_PL0_NTA_WFXM_MASK _ULL(0x40000000000) +#define KVX_SFR_ES_PL0_NTA_WFXM_CLEAR _ULL(0x400) +#define KVX_SFR_ES_PL0_NTA_WFXM_SET _ULL(0x40000000000) + +#define KVX_SFR_ES_PL0_UCA_MASK _ULL(0x80000000000) /* Un-Cached Access */ +#define KVX_SFR_ES_PL0_UCA_SHIFT 43 +#define KVX_SFR_ES_PL0_UCA_WIDTH 1 +#define KVX_SFR_ES_PL0_UCA_WFXM_MASK _ULL(0x80000000000) +#define KVX_SFR_ES_PL0_UCA_WFXM_CLEAR _ULL(0x800) +#define KVX_SFR_ES_PL0_UCA_WFXM_SET _ULL(0x80000000000) + +#define KVX_SFR_ES_PL0_AS_MASK _ULL(0x3f00000000000) /* Access Size */ +#define KVX_SFR_ES_PL0_AS_SHIFT 44 +#define KVX_SFR_ES_PL0_AS_WIDTH 6 +#define KVX_SFR_ES_PL0_AS_WFXM_MASK _ULL(0x3f00000000000) +#define KVX_SFR_ES_PL0_AS_WFXM_CLEAR _ULL(0x3f000) +#define KVX_SFR_ES_PL0_AS_WFXM_SET _ULL(0x3f00000000000) + +#define KVX_SFR_ES_PL0_BS_MASK _ULL(0x3c000000000000) /* Bundle Size */ +#define KVX_SFR_ES_PL0_BS_SHIFT 50 +#define KVX_SFR_ES_PL0_BS_WIDTH 4 +#define KVX_SFR_ES_PL0_BS_WFXM_MASK _ULL(0x3c000000000000) +#define KVX_SFR_ES_PL0_BS_WFXM_CLEAR _ULL(0x3c0000) +#define KVX_SFR_ES_PL0_BS_WFXM_SET _ULL(0x3c000000000000) + +#define KVX_SFR_ES_PL0_DRI_MASK _ULL(0xfc0000000000000) /* Data Register Index */ +#define KVX_SFR_ES_PL0_DRI_SHIFT 54 +#define KVX_SFR_ES_PL0_DRI_WIDTH 6 +#define KVX_SFR_ES_PL0_DRI_WFXM_MASK _ULL(0xfc0000000000000) +#define KVX_SFR_ES_PL0_DRI_WFXM_CLEAR _ULL(0xfc00000) +#define KVX_SFR_ES_PL0_DRI_WFXM_SET _ULL(0xfc0000000000000) + +#define KVX_SFR_ES_PL0_PIC_MASK _ULL(0xf000000000000000) /* Privileged Instruction Code */ +#define KVX_SFR_ES_PL0_PIC_SHIFT 60 +#define KVX_SFR_ES_PL0_PIC_WIDTH 4 +#define KVX_SFR_ES_PL0_PIC_WFXM_MASK _ULL(0xf000000000000000) +#define KVX_SFR_ES_PL0_PIC_WFXM_CLEAR _ULL(0xf0000000) +#define KVX_SFR_ES_PL0_PIC_WFXM_SET _ULL(0xf000000000000000) + +#define KVX_SFR_ES_PL0_DC_MASK _ULL(0x3000) /* Debug Cause */ +#define KVX_SFR_ES_PL0_DC_SHIFT 12 +#define KVX_SFR_ES_PL0_DC_WIDTH 2 +#define KVX_SFR_ES_PL0_DC_WFXL_MASK _ULL(0x3000) +#define KVX_SFR_ES_PL0_DC_WFXL_CLEAR _ULL(0x3000) +#define KVX_SFR_ES_PL0_DC_WFXL_SET _ULL(0x300000000000) + +#define KVX_SFR_ES_PL0_BN_MASK _ULL(0x4000) /* Breakpoint Number */ +#define KVX_SFR_ES_PL0_BN_SHIFT 14 +#define KVX_SFR_ES_PL0_BN_WIDTH 1 +#define KVX_SFR_ES_PL0_BN_WFXL_MASK _ULL(0x4000) +#define KVX_SFR_ES_PL0_BN_WFXL_CLEAR _ULL(0x4000) +#define KVX_SFR_ES_PL0_BN_WFXL_SET _ULL(0x400000000000) + +#define KVX_SFR_ES_PL0_WN_MASK _ULL(0x8000) /* Watchpoint Number */ +#define KVX_SFR_ES_PL0_WN_SHIFT 15 +#define KVX_SFR_ES_PL0_WN_WIDTH 1 +#define KVX_SFR_ES_PL0_WN_WFXL_MASK _ULL(0x8000) +#define KVX_SFR_ES_PL0_WN_WFXL_CLEAR _ULL(0x8000) +#define KVX_SFR_ES_PL0_WN_WFXL_SET _ULL(0x800000000000) + +#define KVX_SFR_ES_PL1_EC_MASK _ULL(0xf) /* Exception Class */ +#define KVX_SFR_ES_PL1_EC_SHIFT 0 +#define KVX_SFR_ES_PL1_EC_WIDTH 4 +#define KVX_SFR_ES_PL1_EC_WFXL_MASK _ULL(0xf) +#define KVX_SFR_ES_PL1_EC_WFXL_CLEAR _ULL(0xf) +#define KVX_SFR_ES_PL1_EC_WFXL_SET _ULL(0xf00000000) + +#define KVX_SFR_ES_PL1_ED_MASK _ULL(0xfffffffffffffff0) /* Exception Details */ +#define KVX_SFR_ES_PL1_ED_SHIFT 4 +#define KVX_SFR_ES_PL1_ED_WIDTH 60 +#define KVX_SFR_ES_PL1_ED_WFXL_MASK _ULL(0xfffffff0) +#define KVX_SFR_ES_PL1_ED_WFXL_CLEAR _ULL(0xfffffff0) +#define KVX_SFR_ES_PL1_ED_WFXL_SET _ULL(0xfffffff000000000) +#define KVX_SFR_ES_PL1_ED_WFXM_MASK _ULL(0xffffffff00000000) +#define KVX_SFR_ES_PL1_ED_WFXM_CLEAR _ULL(0xffffffff) +#define KVX_SFR_ES_PL1_ED_WFXM_SET _ULL(0xffffffff00000000) + +#define KVX_SFR_ES_PL1_OAPL_MASK _ULL(0x30) /* Origin Absolute PL */ +#define KVX_SFR_ES_PL1_OAPL_SHIFT 4 +#define KVX_SFR_ES_PL1_OAPL_WIDTH 2 +#define KVX_SFR_ES_PL1_OAPL_WFXL_MASK _ULL(0x30) +#define KVX_SFR_ES_PL1_OAPL_WFXL_CLEAR _ULL(0x30) +#define KVX_SFR_ES_PL1_OAPL_WFXL_SET _ULL(0x3000000000) + +#define KVX_SFR_ES_PL1_ORPL_MASK _ULL(0xc0) /* Origin Relative PL */ +#define KVX_SFR_ES_PL1_ORPL_SHIFT 6 +#define KVX_SFR_ES_PL1_ORPL_WIDTH 2 +#define KVX_SFR_ES_PL1_ORPL_WFXL_MASK _ULL(0xc0) +#define KVX_SFR_ES_PL1_ORPL_WFXL_CLEAR _ULL(0xc0) +#define KVX_SFR_ES_PL1_ORPL_WFXL_SET _ULL(0xc000000000) + +#define KVX_SFR_ES_PL1_PTAPL_MASK _ULL(0x300) /* Target Absolute PL */ +#define KVX_SFR_ES_PL1_PTAPL_SHIFT 8 +#define KVX_SFR_ES_PL1_PTAPL_WIDTH 2 +#define KVX_SFR_ES_PL1_PTAPL_WFXL_MASK _ULL(0x300) +#define KVX_SFR_ES_PL1_PTAPL_WFXL_CLEAR _ULL(0x300) +#define KVX_SFR_ES_PL1_PTAPL_WFXL_SET _ULL(0x30000000000) + +#define KVX_SFR_ES_PL1_PTRPL_MASK _ULL(0xc00) /* Target Relative PL */ +#define KVX_SFR_ES_PL1_PTRPL_SHIFT 10 +#define KVX_SFR_ES_PL1_PTRPL_WIDTH 2 +#define KVX_SFR_ES_PL1_PTRPL_WFXL_MASK _ULL(0xc00) +#define KVX_SFR_ES_PL1_PTRPL_WFXL_CLEAR _ULL(0xc00) +#define KVX_SFR_ES_PL1_PTRPL_WFXL_SET _ULL(0xc0000000000) + +#define KVX_SFR_ES_PL1_ITN_MASK _ULL(0x1f000) /* InTerrupt Number */ +#define KVX_SFR_ES_PL1_ITN_SHIFT 12 +#define KVX_SFR_ES_PL1_ITN_WIDTH 5 +#define KVX_SFR_ES_PL1_ITN_WFXL_MASK _ULL(0x1f000) +#define KVX_SFR_ES_PL1_ITN_WFXL_CLEAR _ULL(0x1f000) +#define KVX_SFR_ES_PL1_ITN_WFXL_SET _ULL(0x1f00000000000) + +#define KVX_SFR_ES_PL1_ITL_MASK _ULL(0x60000) /* InTerrupt Level */ +#define KVX_SFR_ES_PL1_ITL_SHIFT 17 +#define KVX_SFR_ES_PL1_ITL_WIDTH 2 +#define KVX_SFR_ES_PL1_ITL_WFXL_MASK _ULL(0x60000) +#define KVX_SFR_ES_PL1_ITL_WFXL_CLEAR _ULL(0x60000) +#define KVX_SFR_ES_PL1_ITL_WFXL_SET _ULL(0x6000000000000) + +#define KVX_SFR_ES_PL1_ITI_MASK _ULL(0x1ff80000) /* InTerrupt Info */ +#define KVX_SFR_ES_PL1_ITI_SHIFT 19 +#define KVX_SFR_ES_PL1_ITI_WIDTH 10 +#define KVX_SFR_ES_PL1_ITI_WFXL_MASK _ULL(0x1ff80000) +#define KVX_SFR_ES_PL1_ITI_WFXL_CLEAR _ULL(0x1ff80000) +#define KVX_SFR_ES_PL1_ITI_WFXL_SET _ULL(0x1ff8000000000000) + +#define KVX_SFR_ES_PL1_SN_MASK _ULL(0xfff000) /* Syscall Number */ +#define KVX_SFR_ES_PL1_SN_SHIFT 12 +#define KVX_SFR_ES_PL1_SN_WIDTH 12 +#define KVX_SFR_ES_PL1_SN_WFXL_MASK _ULL(0xfff000) +#define KVX_SFR_ES_PL1_SN_WFXL_CLEAR _ULL(0xfff000) +#define KVX_SFR_ES_PL1_SN_WFXL_SET _ULL(0xfff00000000000) + +#define KVX_SFR_ES_PL1_HTC_MASK _ULL(0x1f000) /* Hardware Trap Cause */ +#define KVX_SFR_ES_PL1_HTC_SHIFT 12 +#define KVX_SFR_ES_PL1_HTC_WIDTH 5 +#define KVX_SFR_ES_PL1_HTC_WFXL_MASK _ULL(0x1f000) +#define KVX_SFR_ES_PL1_HTC_WFXL_CLEAR _ULL(0x1f000) +#define KVX_SFR_ES_PL1_HTC_WFXL_SET _ULL(0x1f00000000000) + +#define KVX_SFR_ES_PL1_SFRT_MASK _ULL(0x20000) /* SFR Trap */ +#define KVX_SFR_ES_PL1_SFRT_SHIFT 17 +#define KVX_SFR_ES_PL1_SFRT_WIDTH 1 +#define KVX_SFR_ES_PL1_SFRT_WFXL_MASK _ULL(0x20000) +#define KVX_SFR_ES_PL1_SFRT_WFXL_CLEAR _ULL(0x20000) +#define KVX_SFR_ES_PL1_SFRT_WFXL_SET _ULL(0x2000000000000) + +#define KVX_SFR_ES_PL1_SFRI_MASK _ULL(0x1c0000) /* SFR Instruction */ +#define KVX_SFR_ES_PL1_SFRI_SHIFT 18 +#define KVX_SFR_ES_PL1_SFRI_WIDTH 3 +#define KVX_SFR_ES_PL1_SFRI_WFXL_MASK _ULL(0x1c0000) +#define KVX_SFR_ES_PL1_SFRI_WFXL_CLEAR _ULL(0x1c0000) +#define KVX_SFR_ES_PL1_SFRI_WFXL_SET _ULL(0x1c000000000000) + +#define KVX_SFR_ES_PL1_GPRP_MASK _ULL(0x7e00000) /* GPR Pointer */ +#define KVX_SFR_ES_PL1_GPRP_SHIFT 21 +#define KVX_SFR_ES_PL1_GPRP_WIDTH 6 +#define KVX_SFR_ES_PL1_GPRP_WFXL_MASK _ULL(0x7e00000) +#define KVX_SFR_ES_PL1_GPRP_WFXL_CLEAR _ULL(0x7e00000) +#define KVX_SFR_ES_PL1_GPRP_WFXL_SET _ULL(0x7e0000000000000) + +#define KVX_SFR_ES_PL1_SFRP_MASK _ULL(0xff8000000) /* SFR Pointer */ +#define KVX_SFR_ES_PL1_SFRP_SHIFT 27 +#define KVX_SFR_ES_PL1_SFRP_WIDTH 9 +#define KVX_SFR_ES_PL1_SFRP_WFXL_MASK _ULL(0xf8000000) +#define KVX_SFR_ES_PL1_SFRP_WFXL_CLEAR _ULL(0xf8000000) +#define KVX_SFR_ES_PL1_SFRP_WFXL_SET _ULL(0xf800000000000000) +#define KVX_SFR_ES_PL1_SFRP_WFXM_MASK _ULL(0xf00000000) +#define KVX_SFR_ES_PL1_SFRP_WFXM_CLEAR _ULL(0xf) +#define KVX_SFR_ES_PL1_SFRP_WFXM_SET _ULL(0xf00000000) + +#define KVX_SFR_ES_PL1_DHT_MASK _ULL(0x1000000000) /* Disabled Hardware Trap */ +#define KVX_SFR_ES_PL1_DHT_SHIFT 36 +#define KVX_SFR_ES_PL1_DHT_WIDTH 1 +#define KVX_SFR_ES_PL1_DHT_WFXM_MASK _ULL(0x1000000000) +#define KVX_SFR_ES_PL1_DHT_WFXM_CLEAR _ULL(0x10) +#define KVX_SFR_ES_PL1_DHT_WFXM_SET _ULL(0x1000000000) + +#define KVX_SFR_ES_PL1_RWX_MASK _ULL(0x38000000000) /* Read Write Execute */ +#define KVX_SFR_ES_PL1_RWX_SHIFT 39 +#define KVX_SFR_ES_PL1_RWX_WIDTH 3 +#define KVX_SFR_ES_PL1_RWX_WFXM_MASK _ULL(0x38000000000) +#define KVX_SFR_ES_PL1_RWX_WFXM_CLEAR _ULL(0x380) +#define KVX_SFR_ES_PL1_RWX_WFXM_SET _ULL(0x38000000000) + +#define KVX_SFR_ES_PL1_NTA_MASK _ULL(0x40000000000) /* Non-Trapping Access */ +#define KVX_SFR_ES_PL1_NTA_SHIFT 42 +#define KVX_SFR_ES_PL1_NTA_WIDTH 1 +#define KVX_SFR_ES_PL1_NTA_WFXM_MASK _ULL(0x40000000000) +#define KVX_SFR_ES_PL1_NTA_WFXM_CLEAR _ULL(0x400) +#define KVX_SFR_ES_PL1_NTA_WFXM_SET _ULL(0x40000000000) + +#define KVX_SFR_ES_PL1_UCA_MASK _ULL(0x80000000000) /* Un-Cached Access */ +#define KVX_SFR_ES_PL1_UCA_SHIFT 43 +#define KVX_SFR_ES_PL1_UCA_WIDTH 1 +#define KVX_SFR_ES_PL1_UCA_WFXM_MASK _ULL(0x80000000000) +#define KVX_SFR_ES_PL1_UCA_WFXM_CLEAR _ULL(0x800) +#define KVX_SFR_ES_PL1_UCA_WFXM_SET _ULL(0x80000000000) + +#define KVX_SFR_ES_PL1_AS_MASK _ULL(0x3f00000000000) /* Access Size */ +#define KVX_SFR_ES_PL1_AS_SHIFT 44 +#define KVX_SFR_ES_PL1_AS_WIDTH 6 +#define KVX_SFR_ES_PL1_AS_WFXM_MASK _ULL(0x3f00000000000) +#define KVX_SFR_ES_PL1_AS_WFXM_CLEAR _ULL(0x3f000) +#define KVX_SFR_ES_PL1_AS_WFXM_SET _ULL(0x3f00000000000) + +#define KVX_SFR_ES_PL1_BS_MASK _ULL(0x3c000000000000) /* Bundle Size */ +#define KVX_SFR_ES_PL1_BS_SHIFT 50 +#define KVX_SFR_ES_PL1_BS_WIDTH 4 +#define KVX_SFR_ES_PL1_BS_WFXM_MASK _ULL(0x3c000000000000) +#define KVX_SFR_ES_PL1_BS_WFXM_CLEAR _ULL(0x3c0000) +#define KVX_SFR_ES_PL1_BS_WFXM_SET _ULL(0x3c000000000000) + +#define KVX_SFR_ES_PL1_DRI_MASK _ULL(0xfc0000000000000) /* Data Register Index */ +#define KVX_SFR_ES_PL1_DRI_SHIFT 54 +#define KVX_SFR_ES_PL1_DRI_WIDTH 6 +#define KVX_SFR_ES_PL1_DRI_WFXM_MASK _ULL(0xfc0000000000000) +#define KVX_SFR_ES_PL1_DRI_WFXM_CLEAR _ULL(0xfc00000) +#define KVX_SFR_ES_PL1_DRI_WFXM_SET _ULL(0xfc0000000000000) + +#define KVX_SFR_ES_PL1_PIC_MASK _ULL(0xf000000000000000) /* Privileged Instruction Code */ +#define KVX_SFR_ES_PL1_PIC_SHIFT 60 +#define KVX_SFR_ES_PL1_PIC_WIDTH 4 +#define KVX_SFR_ES_PL1_PIC_WFXM_MASK _ULL(0xf000000000000000) +#define KVX_SFR_ES_PL1_PIC_WFXM_CLEAR _ULL(0xf0000000) +#define KVX_SFR_ES_PL1_PIC_WFXM_SET _ULL(0xf000000000000000) + +#define KVX_SFR_ES_PL1_DC_MASK _ULL(0x3000) /* Debug Cause */ +#define KVX_SFR_ES_PL1_DC_SHIFT 12 +#define KVX_SFR_ES_PL1_DC_WIDTH 2 +#define KVX_SFR_ES_PL1_DC_WFXL_MASK _ULL(0x3000) +#define KVX_SFR_ES_PL1_DC_WFXL_CLEAR _ULL(0x3000) +#define KVX_SFR_ES_PL1_DC_WFXL_SET _ULL(0x300000000000) + +#define KVX_SFR_ES_PL1_BN_MASK _ULL(0x4000) /* Breakpoint Number */ +#define KVX_SFR_ES_PL1_BN_SHIFT 14 +#define KVX_SFR_ES_PL1_BN_WIDTH 1 +#define KVX_SFR_ES_PL1_BN_WFXL_MASK _ULL(0x4000) +#define KVX_SFR_ES_PL1_BN_WFXL_CLEAR _ULL(0x4000) +#define KVX_SFR_ES_PL1_BN_WFXL_SET _ULL(0x400000000000) + +#define KVX_SFR_ES_PL1_WN_MASK _ULL(0x8000) /* Watchpoint Number */ +#define KVX_SFR_ES_PL1_WN_SHIFT 15 +#define KVX_SFR_ES_PL1_WN_WIDTH 1 +#define KVX_SFR_ES_PL1_WN_WFXL_MASK _ULL(0x8000) +#define KVX_SFR_ES_PL1_WN_WFXL_CLEAR _ULL(0x8000) +#define KVX_SFR_ES_PL1_WN_WFXL_SET _ULL(0x800000000000) + +#define KVX_SFR_ES_PL2_EC_MASK _ULL(0xf) /* Exception Class */ +#define KVX_SFR_ES_PL2_EC_SHIFT 0 +#define KVX_SFR_ES_PL2_EC_WIDTH 4 +#define KVX_SFR_ES_PL2_EC_WFXL_MASK _ULL(0xf) +#define KVX_SFR_ES_PL2_EC_WFXL_CLEAR _ULL(0xf) +#define KVX_SFR_ES_PL2_EC_WFXL_SET _ULL(0xf00000000) + +#define KVX_SFR_ES_PL2_ED_MASK _ULL(0xfffffffffffffff0) /* Exception Details */ +#define KVX_SFR_ES_PL2_ED_SHIFT 4 +#define KVX_SFR_ES_PL2_ED_WIDTH 60 +#define KVX_SFR_ES_PL2_ED_WFXL_MASK _ULL(0xfffffff0) +#define KVX_SFR_ES_PL2_ED_WFXL_CLEAR _ULL(0xfffffff0) +#define KVX_SFR_ES_PL2_ED_WFXL_SET _ULL(0xfffffff000000000) +#define KVX_SFR_ES_PL2_ED_WFXM_MASK _ULL(0xffffffff00000000) +#define KVX_SFR_ES_PL2_ED_WFXM_CLEAR _ULL(0xffffffff) +#define KVX_SFR_ES_PL2_ED_WFXM_SET _ULL(0xffffffff00000000) + +#define KVX_SFR_ES_PL2_OAPL_MASK _ULL(0x30) /* Origin Absolute PL */ +#define KVX_SFR_ES_PL2_OAPL_SHIFT 4 +#define KVX_SFR_ES_PL2_OAPL_WIDTH 2 +#define KVX_SFR_ES_PL2_OAPL_WFXL_MASK _ULL(0x30) +#define KVX_SFR_ES_PL2_OAPL_WFXL_CLEAR _ULL(0x30) +#define KVX_SFR_ES_PL2_OAPL_WFXL_SET _ULL(0x3000000000) + +#define KVX_SFR_ES_PL2_ORPL_MASK _ULL(0xc0) /* Origin Relative PL */ +#define KVX_SFR_ES_PL2_ORPL_SHIFT 6 +#define KVX_SFR_ES_PL2_ORPL_WIDTH 2 +#define KVX_SFR_ES_PL2_ORPL_WFXL_MASK _ULL(0xc0) +#define KVX_SFR_ES_PL2_ORPL_WFXL_CLEAR _ULL(0xc0) +#define KVX_SFR_ES_PL2_ORPL_WFXL_SET _ULL(0xc000000000) + +#define KVX_SFR_ES_PL2_PTAPL_MASK _ULL(0x300) /* Target Absolute PL */ +#define KVX_SFR_ES_PL2_PTAPL_SHIFT 8 +#define KVX_SFR_ES_PL2_PTAPL_WIDTH 2 +#define KVX_SFR_ES_PL2_PTAPL_WFXL_MASK _ULL(0x300) +#define KVX_SFR_ES_PL2_PTAPL_WFXL_CLEAR _ULL(0x300) +#define KVX_SFR_ES_PL2_PTAPL_WFXL_SET _ULL(0x30000000000) + +#define KVX_SFR_ES_PL2_PTRPL_MASK _ULL(0xc00) /* Target Relative PL */ +#define KVX_SFR_ES_PL2_PTRPL_SHIFT 10 +#define KVX_SFR_ES_PL2_PTRPL_WIDTH 2 +#define KVX_SFR_ES_PL2_PTRPL_WFXL_MASK _ULL(0xc00) +#define KVX_SFR_ES_PL2_PTRPL_WFXL_CLEAR _ULL(0xc00) +#define KVX_SFR_ES_PL2_PTRPL_WFXL_SET _ULL(0xc0000000000) + +#define KVX_SFR_ES_PL2_ITN_MASK _ULL(0x1f000) /* InTerrupt Number */ +#define KVX_SFR_ES_PL2_ITN_SHIFT 12 +#define KVX_SFR_ES_PL2_ITN_WIDTH 5 +#define KVX_SFR_ES_PL2_ITN_WFXL_MASK _ULL(0x1f000) +#define KVX_SFR_ES_PL2_ITN_WFXL_CLEAR _ULL(0x1f000) +#define KVX_SFR_ES_PL2_ITN_WFXL_SET _ULL(0x1f00000000000) + +#define KVX_SFR_ES_PL2_ITL_MASK _ULL(0x60000) /* InTerrupt Level */ +#define KVX_SFR_ES_PL2_ITL_SHIFT 17 +#define KVX_SFR_ES_PL2_ITL_WIDTH 2 +#define KVX_SFR_ES_PL2_ITL_WFXL_MASK _ULL(0x60000) +#define KVX_SFR_ES_PL2_ITL_WFXL_CLEAR _ULL(0x60000) +#define KVX_SFR_ES_PL2_ITL_WFXL_SET _ULL(0x6000000000000) + +#define KVX_SFR_ES_PL2_ITI_MASK _ULL(0x1ff80000) /* InTerrupt Info */ +#define KVX_SFR_ES_PL2_ITI_SHIFT 19 +#define KVX_SFR_ES_PL2_ITI_WIDTH 10 +#define KVX_SFR_ES_PL2_ITI_WFXL_MASK _ULL(0x1ff80000) +#define KVX_SFR_ES_PL2_ITI_WFXL_CLEAR _ULL(0x1ff80000) +#define KVX_SFR_ES_PL2_ITI_WFXL_SET _ULL(0x1ff8000000000000) + +#define KVX_SFR_ES_PL2_SN_MASK _ULL(0xfff000) /* Syscall Number */ +#define KVX_SFR_ES_PL2_SN_SHIFT 12 +#define KVX_SFR_ES_PL2_SN_WIDTH 12 +#define KVX_SFR_ES_PL2_SN_WFXL_MASK _ULL(0xfff000) +#define KVX_SFR_ES_PL2_SN_WFXL_CLEAR _ULL(0xfff000) +#define KVX_SFR_ES_PL2_SN_WFXL_SET _ULL(0xfff00000000000) + +#define KVX_SFR_ES_PL2_HTC_MASK _ULL(0x1f000) /* Hardware Trap Cause */ +#define KVX_SFR_ES_PL2_HTC_SHIFT 12 +#define KVX_SFR_ES_PL2_HTC_WIDTH 5 +#define KVX_SFR_ES_PL2_HTC_WFXL_MASK _ULL(0x1f000) +#define KVX_SFR_ES_PL2_HTC_WFXL_CLEAR _ULL(0x1f000) +#define KVX_SFR_ES_PL2_HTC_WFXL_SET _ULL(0x1f00000000000) + +#define KVX_SFR_ES_PL2_SFRT_MASK _ULL(0x20000) /* SFR Trap */ +#define KVX_SFR_ES_PL2_SFRT_SHIFT 17 +#define KVX_SFR_ES_PL2_SFRT_WIDTH 1 +#define KVX_SFR_ES_PL2_SFRT_WFXL_MASK _ULL(0x20000) +#define KVX_SFR_ES_PL2_SFRT_WFXL_CLEAR _ULL(0x20000) +#define KVX_SFR_ES_PL2_SFRT_WFXL_SET _ULL(0x2000000000000) + +#define KVX_SFR_ES_PL2_SFRI_MASK _ULL(0x1c0000) /* SFR Instruction */ +#define KVX_SFR_ES_PL2_SFRI_SHIFT 18 +#define KVX_SFR_ES_PL2_SFRI_WIDTH 3 +#define KVX_SFR_ES_PL2_SFRI_WFXL_MASK _ULL(0x1c0000) +#define KVX_SFR_ES_PL2_SFRI_WFXL_CLEAR _ULL(0x1c0000) +#define KVX_SFR_ES_PL2_SFRI_WFXL_SET _ULL(0x1c000000000000) + +#define KVX_SFR_ES_PL2_GPRP_MASK _ULL(0x7e00000) /* GPR Pointer */ +#define KVX_SFR_ES_PL2_GPRP_SHIFT 21 +#define KVX_SFR_ES_PL2_GPRP_WIDTH 6 +#define KVX_SFR_ES_PL2_GPRP_WFXL_MASK _ULL(0x7e00000) +#define KVX_SFR_ES_PL2_GPRP_WFXL_CLEAR _ULL(0x7e00000) +#define KVX_SFR_ES_PL2_GPRP_WFXL_SET _ULL(0x7e0000000000000) + +#define KVX_SFR_ES_PL2_SFRP_MASK _ULL(0xff8000000) /* SFR Pointer */ +#define KVX_SFR_ES_PL2_SFRP_SHIFT 27 +#define KVX_SFR_ES_PL2_SFRP_WIDTH 9 +#define KVX_SFR_ES_PL2_SFRP_WFXL_MASK _ULL(0xf8000000) +#define KVX_SFR_ES_PL2_SFRP_WFXL_CLEAR _ULL(0xf8000000) +#define KVX_SFR_ES_PL2_SFRP_WFXL_SET _ULL(0xf800000000000000) +#define KVX_SFR_ES_PL2_SFRP_WFXM_MASK _ULL(0xf00000000) +#define KVX_SFR_ES_PL2_SFRP_WFXM_CLEAR _ULL(0xf) +#define KVX_SFR_ES_PL2_SFRP_WFXM_SET _ULL(0xf00000000) + +#define KVX_SFR_ES_PL2_DHT_MASK _ULL(0x1000000000) /* Disabled Hardware Trap */ +#define KVX_SFR_ES_PL2_DHT_SHIFT 36 +#define KVX_SFR_ES_PL2_DHT_WIDTH 1 +#define KVX_SFR_ES_PL2_DHT_WFXM_MASK _ULL(0x1000000000) +#define KVX_SFR_ES_PL2_DHT_WFXM_CLEAR _ULL(0x10) +#define KVX_SFR_ES_PL2_DHT_WFXM_SET _ULL(0x1000000000) + +#define KVX_SFR_ES_PL2_RWX_MASK _ULL(0x38000000000) /* Read Write Execute */ +#define KVX_SFR_ES_PL2_RWX_SHIFT 39 +#define KVX_SFR_ES_PL2_RWX_WIDTH 3 +#define KVX_SFR_ES_PL2_RWX_WFXM_MASK _ULL(0x38000000000) +#define KVX_SFR_ES_PL2_RWX_WFXM_CLEAR _ULL(0x380) +#define KVX_SFR_ES_PL2_RWX_WFXM_SET _ULL(0x38000000000) + +#define KVX_SFR_ES_PL2_NTA_MASK _ULL(0x40000000000) /* Non-Trapping Access */ +#define KVX_SFR_ES_PL2_NTA_SHIFT 42 +#define KVX_SFR_ES_PL2_NTA_WIDTH 1 +#define KVX_SFR_ES_PL2_NTA_WFXM_MASK _ULL(0x40000000000) +#define KVX_SFR_ES_PL2_NTA_WFXM_CLEAR _ULL(0x400) +#define KVX_SFR_ES_PL2_NTA_WFXM_SET _ULL(0x40000000000) + +#define KVX_SFR_ES_PL2_UCA_MASK _ULL(0x80000000000) /* Un-Cached Access */ +#define KVX_SFR_ES_PL2_UCA_SHIFT 43 +#define KVX_SFR_ES_PL2_UCA_WIDTH 1 +#define KVX_SFR_ES_PL2_UCA_WFXM_MASK _ULL(0x80000000000) +#define KVX_SFR_ES_PL2_UCA_WFXM_CLEAR _ULL(0x800) +#define KVX_SFR_ES_PL2_UCA_WFXM_SET _ULL(0x80000000000) + +#define KVX_SFR_ES_PL2_AS_MASK _ULL(0x3f00000000000) /* Access Size */ +#define KVX_SFR_ES_PL2_AS_SHIFT 44 +#define KVX_SFR_ES_PL2_AS_WIDTH 6 +#define KVX_SFR_ES_PL2_AS_WFXM_MASK _ULL(0x3f00000000000) +#define KVX_SFR_ES_PL2_AS_WFXM_CLEAR _ULL(0x3f000) +#define KVX_SFR_ES_PL2_AS_WFXM_SET _ULL(0x3f00000000000) + +#define KVX_SFR_ES_PL2_BS_MASK _ULL(0x3c000000000000) /* Bundle Size */ +#define KVX_SFR_ES_PL2_BS_SHIFT 50 +#define KVX_SFR_ES_PL2_BS_WIDTH 4 +#define KVX_SFR_ES_PL2_BS_WFXM_MASK _ULL(0x3c000000000000) +#define KVX_SFR_ES_PL2_BS_WFXM_CLEAR _ULL(0x3c0000) +#define KVX_SFR_ES_PL2_BS_WFXM_SET _ULL(0x3c000000000000) + +#define KVX_SFR_ES_PL2_DRI_MASK _ULL(0xfc0000000000000) /* Data Register Index */ +#define KVX_SFR_ES_PL2_DRI_SHIFT 54 +#define KVX_SFR_ES_PL2_DRI_WIDTH 6 +#define KVX_SFR_ES_PL2_DRI_WFXM_MASK _ULL(0xfc0000000000000) +#define KVX_SFR_ES_PL2_DRI_WFXM_CLEAR _ULL(0xfc00000) +#define KVX_SFR_ES_PL2_DRI_WFXM_SET _ULL(0xfc0000000000000) + +#define KVX_SFR_ES_PL2_PIC_MASK _ULL(0xf000000000000000) /* Privileged Instruction Code */ +#define KVX_SFR_ES_PL2_PIC_SHIFT 60 +#define KVX_SFR_ES_PL2_PIC_WIDTH 4 +#define KVX_SFR_ES_PL2_PIC_WFXM_MASK _ULL(0xf000000000000000) +#define KVX_SFR_ES_PL2_PIC_WFXM_CLEAR _ULL(0xf0000000) +#define KVX_SFR_ES_PL2_PIC_WFXM_SET _ULL(0xf000000000000000) + +#define KVX_SFR_ES_PL2_DC_MASK _ULL(0x3000) /* Debug Cause */ +#define KVX_SFR_ES_PL2_DC_SHIFT 12 +#define KVX_SFR_ES_PL2_DC_WIDTH 2 +#define KVX_SFR_ES_PL2_DC_WFXL_MASK _ULL(0x3000) +#define KVX_SFR_ES_PL2_DC_WFXL_CLEAR _ULL(0x3000) +#define KVX_SFR_ES_PL2_DC_WFXL_SET _ULL(0x300000000000) + +#define KVX_SFR_ES_PL2_BN_MASK _ULL(0x4000) /* Breakpoint Number */ +#define KVX_SFR_ES_PL2_BN_SHIFT 14 +#define KVX_SFR_ES_PL2_BN_WIDTH 1 +#define KVX_SFR_ES_PL2_BN_WFXL_MASK _ULL(0x4000) +#define KVX_SFR_ES_PL2_BN_WFXL_CLEAR _ULL(0x4000) +#define KVX_SFR_ES_PL2_BN_WFXL_SET _ULL(0x400000000000) + +#define KVX_SFR_ES_PL2_WN_MASK _ULL(0x8000) /* Watchpoint Number */ +#define KVX_SFR_ES_PL2_WN_SHIFT 15 +#define KVX_SFR_ES_PL2_WN_WIDTH 1 +#define KVX_SFR_ES_PL2_WN_WFXL_MASK _ULL(0x8000) +#define KVX_SFR_ES_PL2_WN_WFXL_CLEAR _ULL(0x8000) +#define KVX_SFR_ES_PL2_WN_WFXL_SET _ULL(0x800000000000) + +#define KVX_SFR_ES_PL3_EC_MASK _ULL(0xf) /* Exception Class */ +#define KVX_SFR_ES_PL3_EC_SHIFT 0 +#define KVX_SFR_ES_PL3_EC_WIDTH 4 +#define KVX_SFR_ES_PL3_EC_WFXL_MASK _ULL(0xf) +#define KVX_SFR_ES_PL3_EC_WFXL_CLEAR _ULL(0xf) +#define KVX_SFR_ES_PL3_EC_WFXL_SET _ULL(0xf00000000) + +#define KVX_SFR_ES_PL3_ED_MASK _ULL(0xfffffffffffffff0) /* Exception Details */ +#define KVX_SFR_ES_PL3_ED_SHIFT 4 +#define KVX_SFR_ES_PL3_ED_WIDTH 60 +#define KVX_SFR_ES_PL3_ED_WFXL_MASK _ULL(0xfffffff0) +#define KVX_SFR_ES_PL3_ED_WFXL_CLEAR _ULL(0xfffffff0) +#define KVX_SFR_ES_PL3_ED_WFXL_SET _ULL(0xfffffff000000000) +#define KVX_SFR_ES_PL3_ED_WFXM_MASK _ULL(0xffffffff00000000) +#define KVX_SFR_ES_PL3_ED_WFXM_CLEAR _ULL(0xffffffff) +#define KVX_SFR_ES_PL3_ED_WFXM_SET _ULL(0xffffffff00000000) + +#define KVX_SFR_ES_PL3_OAPL_MASK _ULL(0x30) /* Origin Absolute PL */ +#define KVX_SFR_ES_PL3_OAPL_SHIFT 4 +#define KVX_SFR_ES_PL3_OAPL_WIDTH 2 +#define KVX_SFR_ES_PL3_OAPL_WFXL_MASK _ULL(0x30) +#define KVX_SFR_ES_PL3_OAPL_WFXL_CLEAR _ULL(0x30) +#define KVX_SFR_ES_PL3_OAPL_WFXL_SET _ULL(0x3000000000) + +#define KVX_SFR_ES_PL3_ORPL_MASK _ULL(0xc0) /* Origin Relative PL */ +#define KVX_SFR_ES_PL3_ORPL_SHIFT 6 +#define KVX_SFR_ES_PL3_ORPL_WIDTH 2 +#define KVX_SFR_ES_PL3_ORPL_WFXL_MASK _ULL(0xc0) +#define KVX_SFR_ES_PL3_ORPL_WFXL_CLEAR _ULL(0xc0) +#define KVX_SFR_ES_PL3_ORPL_WFXL_SET _ULL(0xc000000000) + +#define KVX_SFR_ES_PL3_PTAPL_MASK _ULL(0x300) /* Target Absolute PL */ +#define KVX_SFR_ES_PL3_PTAPL_SHIFT 8 +#define KVX_SFR_ES_PL3_PTAPL_WIDTH 2 +#define KVX_SFR_ES_PL3_PTAPL_WFXL_MASK _ULL(0x300) +#define KVX_SFR_ES_PL3_PTAPL_WFXL_CLEAR _ULL(0x300) +#define KVX_SFR_ES_PL3_PTAPL_WFXL_SET _ULL(0x30000000000) + +#define KVX_SFR_ES_PL3_PTRPL_MASK _ULL(0xc00) /* Target Relative PL */ +#define KVX_SFR_ES_PL3_PTRPL_SHIFT 10 +#define KVX_SFR_ES_PL3_PTRPL_WIDTH 2 +#define KVX_SFR_ES_PL3_PTRPL_WFXL_MASK _ULL(0xc00) +#define KVX_SFR_ES_PL3_PTRPL_WFXL_CLEAR _ULL(0xc00) +#define KVX_SFR_ES_PL3_PTRPL_WFXL_SET _ULL(0xc0000000000) + +#define KVX_SFR_ES_PL3_ITN_MASK _ULL(0x1f000) /* InTerrupt Number */ +#define KVX_SFR_ES_PL3_ITN_SHIFT 12 +#define KVX_SFR_ES_PL3_ITN_WIDTH 5 +#define KVX_SFR_ES_PL3_ITN_WFXL_MASK _ULL(0x1f000) +#define KVX_SFR_ES_PL3_ITN_WFXL_CLEAR _ULL(0x1f000) +#define KVX_SFR_ES_PL3_ITN_WFXL_SET _ULL(0x1f00000000000) + +#define KVX_SFR_ES_PL3_ITL_MASK _ULL(0x60000) /* InTerrupt Level */ +#define KVX_SFR_ES_PL3_ITL_SHIFT 17 +#define KVX_SFR_ES_PL3_ITL_WIDTH 2 +#define KVX_SFR_ES_PL3_ITL_WFXL_MASK _ULL(0x60000) +#define KVX_SFR_ES_PL3_ITL_WFXL_CLEAR _ULL(0x60000) +#define KVX_SFR_ES_PL3_ITL_WFXL_SET _ULL(0x6000000000000) + +#define KVX_SFR_ES_PL3_ITI_MASK _ULL(0x1ff80000) /* InTerrupt Info */ +#define KVX_SFR_ES_PL3_ITI_SHIFT 19 +#define KVX_SFR_ES_PL3_ITI_WIDTH 10 +#define KVX_SFR_ES_PL3_ITI_WFXL_MASK _ULL(0x1ff80000) +#define KVX_SFR_ES_PL3_ITI_WFXL_CLEAR _ULL(0x1ff80000) +#define KVX_SFR_ES_PL3_ITI_WFXL_SET _ULL(0x1ff8000000000000) + +#define KVX_SFR_ES_PL3_SN_MASK _ULL(0xfff000) /* Syscall Number */ +#define KVX_SFR_ES_PL3_SN_SHIFT 12 +#define KVX_SFR_ES_PL3_SN_WIDTH 12 +#define KVX_SFR_ES_PL3_SN_WFXL_MASK _ULL(0xfff000) +#define KVX_SFR_ES_PL3_SN_WFXL_CLEAR _ULL(0xfff000) +#define KVX_SFR_ES_PL3_SN_WFXL_SET _ULL(0xfff00000000000) + +#define KVX_SFR_ES_PL3_HTC_MASK _ULL(0x1f000) /* Hardware Trap Cause */ +#define KVX_SFR_ES_PL3_HTC_SHIFT 12 +#define KVX_SFR_ES_PL3_HTC_WIDTH 5 +#define KVX_SFR_ES_PL3_HTC_WFXL_MASK _ULL(0x1f000) +#define KVX_SFR_ES_PL3_HTC_WFXL_CLEAR _ULL(0x1f000) +#define KVX_SFR_ES_PL3_HTC_WFXL_SET _ULL(0x1f00000000000) + +#define KVX_SFR_ES_PL3_SFRT_MASK _ULL(0x20000) /* SFR Trap */ +#define KVX_SFR_ES_PL3_SFRT_SHIFT 17 +#define KVX_SFR_ES_PL3_SFRT_WIDTH 1 +#define KVX_SFR_ES_PL3_SFRT_WFXL_MASK _ULL(0x20000) +#define KVX_SFR_ES_PL3_SFRT_WFXL_CLEAR _ULL(0x20000) +#define KVX_SFR_ES_PL3_SFRT_WFXL_SET _ULL(0x2000000000000) + +#define KVX_SFR_ES_PL3_SFRI_MASK _ULL(0x1c0000) /* SFR Instruction */ +#define KVX_SFR_ES_PL3_SFRI_SHIFT 18 +#define KVX_SFR_ES_PL3_SFRI_WIDTH 3 +#define KVX_SFR_ES_PL3_SFRI_WFXL_MASK _ULL(0x1c0000) +#define KVX_SFR_ES_PL3_SFRI_WFXL_CLEAR _ULL(0x1c0000) +#define KVX_SFR_ES_PL3_SFRI_WFXL_SET _ULL(0x1c000000000000) + +#define KVX_SFR_ES_PL3_GPRP_MASK _ULL(0x7e00000) /* GPR Pointer */ +#define KVX_SFR_ES_PL3_GPRP_SHIFT 21 +#define KVX_SFR_ES_PL3_GPRP_WIDTH 6 +#define KVX_SFR_ES_PL3_GPRP_WFXL_MASK _ULL(0x7e00000) +#define KVX_SFR_ES_PL3_GPRP_WFXL_CLEAR _ULL(0x7e00000) +#define KVX_SFR_ES_PL3_GPRP_WFXL_SET _ULL(0x7e0000000000000) + +#define KVX_SFR_ES_PL3_SFRP_MASK _ULL(0xff8000000) /* SFR Pointer */ +#define KVX_SFR_ES_PL3_SFRP_SHIFT 27 +#define KVX_SFR_ES_PL3_SFRP_WIDTH 9 +#define KVX_SFR_ES_PL3_SFRP_WFXL_MASK _ULL(0xf8000000) +#define KVX_SFR_ES_PL3_SFRP_WFXL_CLEAR _ULL(0xf8000000) +#define KVX_SFR_ES_PL3_SFRP_WFXL_SET _ULL(0xf800000000000000) +#define KVX_SFR_ES_PL3_SFRP_WFXM_MASK _ULL(0xf00000000) +#define KVX_SFR_ES_PL3_SFRP_WFXM_CLEAR _ULL(0xf) +#define KVX_SFR_ES_PL3_SFRP_WFXM_SET _ULL(0xf00000000) + +#define KVX_SFR_ES_PL3_DHT_MASK _ULL(0x1000000000) /* Disabled Hardware Trap */ +#define KVX_SFR_ES_PL3_DHT_SHIFT 36 +#define KVX_SFR_ES_PL3_DHT_WIDTH 1 +#define KVX_SFR_ES_PL3_DHT_WFXM_MASK _ULL(0x1000000000) +#define KVX_SFR_ES_PL3_DHT_WFXM_CLEAR _ULL(0x10) +#define KVX_SFR_ES_PL3_DHT_WFXM_SET _ULL(0x1000000000) + +#define KVX_SFR_ES_PL3_RWX_MASK _ULL(0x38000000000) /* Read Write Execute */ +#define KVX_SFR_ES_PL3_RWX_SHIFT 39 +#define KVX_SFR_ES_PL3_RWX_WIDTH 3 +#define KVX_SFR_ES_PL3_RWX_WFXM_MASK _ULL(0x38000000000) +#define KVX_SFR_ES_PL3_RWX_WFXM_CLEAR _ULL(0x380) +#define KVX_SFR_ES_PL3_RWX_WFXM_SET _ULL(0x38000000000) + +#define KVX_SFR_ES_PL3_NTA_MASK _ULL(0x40000000000) /* Non-Trapping Access */ +#define KVX_SFR_ES_PL3_NTA_SHIFT 42 +#define KVX_SFR_ES_PL3_NTA_WIDTH 1 +#define KVX_SFR_ES_PL3_NTA_WFXM_MASK _ULL(0x40000000000) +#define KVX_SFR_ES_PL3_NTA_WFXM_CLEAR _ULL(0x400) +#define KVX_SFR_ES_PL3_NTA_WFXM_SET _ULL(0x40000000000) + +#define KVX_SFR_ES_PL3_UCA_MASK _ULL(0x80000000000) /* Un-Cached Access */ +#define KVX_SFR_ES_PL3_UCA_SHIFT 43 +#define KVX_SFR_ES_PL3_UCA_WIDTH 1 +#define KVX_SFR_ES_PL3_UCA_WFXM_MASK _ULL(0x80000000000) +#define KVX_SFR_ES_PL3_UCA_WFXM_CLEAR _ULL(0x800) +#define KVX_SFR_ES_PL3_UCA_WFXM_SET _ULL(0x80000000000) + +#define KVX_SFR_ES_PL3_AS_MASK _ULL(0x3f00000000000) /* Access Size */ +#define KVX_SFR_ES_PL3_AS_SHIFT 44 +#define KVX_SFR_ES_PL3_AS_WIDTH 6 +#define KVX_SFR_ES_PL3_AS_WFXM_MASK _ULL(0x3f00000000000) +#define KVX_SFR_ES_PL3_AS_WFXM_CLEAR _ULL(0x3f000) +#define KVX_SFR_ES_PL3_AS_WFXM_SET _ULL(0x3f00000000000) + +#define KVX_SFR_ES_PL3_BS_MASK _ULL(0x3c000000000000) /* Bundle Size */ +#define KVX_SFR_ES_PL3_BS_SHIFT 50 +#define KVX_SFR_ES_PL3_BS_WIDTH 4 +#define KVX_SFR_ES_PL3_BS_WFXM_MASK _ULL(0x3c000000000000) +#define KVX_SFR_ES_PL3_BS_WFXM_CLEAR _ULL(0x3c0000) +#define KVX_SFR_ES_PL3_BS_WFXM_SET _ULL(0x3c000000000000) + +#define KVX_SFR_ES_PL3_DRI_MASK _ULL(0xfc0000000000000) /* Data Register Index */ +#define KVX_SFR_ES_PL3_DRI_SHIFT 54 +#define KVX_SFR_ES_PL3_DRI_WIDTH 6 +#define KVX_SFR_ES_PL3_DRI_WFXM_MASK _ULL(0xfc0000000000000) +#define KVX_SFR_ES_PL3_DRI_WFXM_CLEAR _ULL(0xfc00000) +#define KVX_SFR_ES_PL3_DRI_WFXM_SET _ULL(0xfc0000000000000) + +#define KVX_SFR_ES_PL3_PIC_MASK _ULL(0xf000000000000000) /* Privileged Instruction Code */ +#define KVX_SFR_ES_PL3_PIC_SHIFT 60 +#define KVX_SFR_ES_PL3_PIC_WIDTH 4 +#define KVX_SFR_ES_PL3_PIC_WFXM_MASK _ULL(0xf000000000000000) +#define KVX_SFR_ES_PL3_PIC_WFXM_CLEAR _ULL(0xf0000000) +#define KVX_SFR_ES_PL3_PIC_WFXM_SET _ULL(0xf000000000000000) + +#define KVX_SFR_ES_PL3_DC_MASK _ULL(0x3000) /* Debug Cause */ +#define KVX_SFR_ES_PL3_DC_SHIFT 12 +#define KVX_SFR_ES_PL3_DC_WIDTH 2 +#define KVX_SFR_ES_PL3_DC_WFXL_MASK _ULL(0x3000) +#define KVX_SFR_ES_PL3_DC_WFXL_CLEAR _ULL(0x3000) +#define KVX_SFR_ES_PL3_DC_WFXL_SET _ULL(0x300000000000) + +#define KVX_SFR_ES_PL3_BN_MASK _ULL(0x4000) /* Breakpoint Number */ +#define KVX_SFR_ES_PL3_BN_SHIFT 14 +#define KVX_SFR_ES_PL3_BN_WIDTH 1 +#define KVX_SFR_ES_PL3_BN_WFXL_MASK _ULL(0x4000) +#define KVX_SFR_ES_PL3_BN_WFXL_CLEAR _ULL(0x4000) +#define KVX_SFR_ES_PL3_BN_WFXL_SET _ULL(0x400000000000) + +#define KVX_SFR_ES_PL3_WN_MASK _ULL(0x8000) /* Watchpoint Number */ +#define KVX_SFR_ES_PL3_WN_SHIFT 15 +#define KVX_SFR_ES_PL3_WN_WIDTH 1 +#define KVX_SFR_ES_PL3_WN_WFXL_MASK _ULL(0x8000) +#define KVX_SFR_ES_PL3_WN_WFXL_CLEAR _ULL(0x8000) +#define KVX_SFR_ES_PL3_WN_WFXL_SET _ULL(0x800000000000) + +#define KVX_SFR_TCR_T0CE_MASK _ULL(0x10000) /* Timer 0 Count Enable */ +#define KVX_SFR_TCR_T0CE_SHIFT 16 +#define KVX_SFR_TCR_T0CE_WIDTH 1 +#define KVX_SFR_TCR_T0CE_WFXL_MASK _ULL(0x10000) +#define KVX_SFR_TCR_T0CE_WFXL_CLEAR _ULL(0x10000) +#define KVX_SFR_TCR_T0CE_WFXL_SET _ULL(0x1000000000000) + +#define KVX_SFR_TCR_T1CE_MASK _ULL(0x20000) /* Timer 1 Count Enable */ +#define KVX_SFR_TCR_T1CE_SHIFT 17 +#define KVX_SFR_TCR_T1CE_WIDTH 1 +#define KVX_SFR_TCR_T1CE_WFXL_MASK _ULL(0x20000) +#define KVX_SFR_TCR_T1CE_WFXL_CLEAR _ULL(0x20000) +#define KVX_SFR_TCR_T1CE_WFXL_SET _ULL(0x2000000000000) + +#define KVX_SFR_TCR_T0IE_MASK _ULL(0x40000) /* Timer 0 Interrupt Enable */ +#define KVX_SFR_TCR_T0IE_SHIFT 18 +#define KVX_SFR_TCR_T0IE_WIDTH 1 +#define KVX_SFR_TCR_T0IE_WFXL_MASK _ULL(0x40000) +#define KVX_SFR_TCR_T0IE_WFXL_CLEAR _ULL(0x40000) +#define KVX_SFR_TCR_T0IE_WFXL_SET _ULL(0x4000000000000) + +#define KVX_SFR_TCR_T1IE_MASK _ULL(0x80000) /* Timer 1 Interrupt Enable */ +#define KVX_SFR_TCR_T1IE_SHIFT 19 +#define KVX_SFR_TCR_T1IE_WIDTH 1 +#define KVX_SFR_TCR_T1IE_WFXL_MASK _ULL(0x80000) +#define KVX_SFR_TCR_T1IE_WFXL_CLEAR _ULL(0x80000) +#define KVX_SFR_TCR_T1IE_WFXL_SET _ULL(0x8000000000000) + +#define KVX_SFR_TCR_T0ST_MASK _ULL(0x100000) /* Timer 0 Status */ +#define KVX_SFR_TCR_T0ST_SHIFT 20 +#define KVX_SFR_TCR_T0ST_WIDTH 1 +#define KVX_SFR_TCR_T0ST_WFXL_MASK _ULL(0x100000) +#define KVX_SFR_TCR_T0ST_WFXL_CLEAR _ULL(0x100000) +#define KVX_SFR_TCR_T0ST_WFXL_SET _ULL(0x10000000000000) + +#define KVX_SFR_TCR_T1ST_MASK _ULL(0x200000) /* Timer 1 Status */ +#define KVX_SFR_TCR_T1ST_SHIFT 21 +#define KVX_SFR_TCR_T1ST_WIDTH 1 +#define KVX_SFR_TCR_T1ST_WFXL_MASK _ULL(0x200000) +#define KVX_SFR_TCR_T1ST_WFXL_CLEAR _ULL(0x200000) +#define KVX_SFR_TCR_T1ST_WFXL_SET _ULL(0x20000000000000) + +#define KVX_SFR_TCR_T0SI_MASK _ULL(0x400000) /* Stop Timer 0 in Idle */ +#define KVX_SFR_TCR_T0SI_SHIFT 22 +#define KVX_SFR_TCR_T0SI_WIDTH 1 +#define KVX_SFR_TCR_T0SI_WFXL_MASK _ULL(0x400000) +#define KVX_SFR_TCR_T0SI_WFXL_CLEAR _ULL(0x400000) +#define KVX_SFR_TCR_T0SI_WFXL_SET _ULL(0x40000000000000) + +#define KVX_SFR_TCR_T1SI_MASK _ULL(0x800000) /* Stop Timer 1 in Idle */ +#define KVX_SFR_TCR_T1SI_SHIFT 23 +#define KVX_SFR_TCR_T1SI_WIDTH 1 +#define KVX_SFR_TCR_T1SI_WFXL_MASK _ULL(0x800000) +#define KVX_SFR_TCR_T1SI_WFXL_CLEAR _ULL(0x800000) +#define KVX_SFR_TCR_T1SI_WFXL_SET _ULL(0x80000000000000) + +#define KVX_SFR_TCR_WCE_MASK _ULL(0x1000000) /* Watchdog Counting Enable */ +#define KVX_SFR_TCR_WCE_SHIFT 24 +#define KVX_SFR_TCR_WCE_WIDTH 1 +#define KVX_SFR_TCR_WCE_WFXL_MASK _ULL(0x1000000) +#define KVX_SFR_TCR_WCE_WFXL_CLEAR _ULL(0x1000000) +#define KVX_SFR_TCR_WCE_WFXL_SET _ULL(0x100000000000000) + +#define KVX_SFR_TCR_WIE_MASK _ULL(0x2000000) /* Watchdog Interrupt Enable */ +#define KVX_SFR_TCR_WIE_SHIFT 25 +#define KVX_SFR_TCR_WIE_WIDTH 1 +#define KVX_SFR_TCR_WIE_WFXL_MASK _ULL(0x2000000) +#define KVX_SFR_TCR_WIE_WFXL_CLEAR _ULL(0x2000000) +#define KVX_SFR_TCR_WIE_WFXL_SET _ULL(0x200000000000000) + +#define KVX_SFR_TCR_WUI_MASK _ULL(0x4000000) /* Watchdog Underflow Inform */ +#define KVX_SFR_TCR_WUI_SHIFT 26 +#define KVX_SFR_TCR_WUI_WIDTH 1 +#define KVX_SFR_TCR_WUI_WFXL_MASK _ULL(0x4000000) +#define KVX_SFR_TCR_WUI_WFXL_CLEAR _ULL(0x4000000) +#define KVX_SFR_TCR_WUI_WFXL_SET _ULL(0x400000000000000) + +#define KVX_SFR_TCR_WUS_MASK _ULL(0x8000000) /* Watchdog Underflow Status */ +#define KVX_SFR_TCR_WUS_SHIFT 27 +#define KVX_SFR_TCR_WUS_WIDTH 1 +#define KVX_SFR_TCR_WUS_WFXL_MASK _ULL(0x8000000) +#define KVX_SFR_TCR_WUS_WFXL_CLEAR _ULL(0x8000000) +#define KVX_SFR_TCR_WUS_WFXL_SET _ULL(0x800000000000000) + +#define KVX_SFR_TCR_WSI_MASK _ULL(0x10000000) /* Watchdog Stop in Idle */ +#define KVX_SFR_TCR_WSI_SHIFT 28 +#define KVX_SFR_TCR_WSI_WIDTH 1 +#define KVX_SFR_TCR_WSI_WFXL_MASK _ULL(0x10000000) +#define KVX_SFR_TCR_WSI_WFXL_CLEAR _ULL(0x10000000) +#define KVX_SFR_TCR_WSI_WFXL_SET _ULL(0x1000000000000000) + +#define KVX_SFR_PM0_PM0_MASK _ULL(0xffffffffffffffff) /* Performance Monitor 0 */ +#define KVX_SFR_PM0_PM0_SHIFT 0 +#define KVX_SFR_PM0_PM0_WIDTH 64 +#define KVX_SFR_PM0_PM0_WFXL_MASK _ULL(0xffffffff) +#define KVX_SFR_PM0_PM0_WFXL_CLEAR _ULL(0xffffffff) +#define KVX_SFR_PM0_PM0_WFXL_SET _ULL(0xffffffff00000000) +#define KVX_SFR_PM0_PM0_WFXM_MASK _ULL(0xffffffff00000000) +#define KVX_SFR_PM0_PM0_WFXM_CLEAR _ULL(0xffffffff) +#define KVX_SFR_PM0_PM0_WFXM_SET _ULL(0xffffffff00000000) + +#define KVX_SFR_PM1_PM1_MASK _ULL(0xffffffffffffffff) /* Performance Monitor 1 */ +#define KVX_SFR_PM1_PM1_SHIFT 0 +#define KVX_SFR_PM1_PM1_WIDTH 64 +#define KVX_SFR_PM1_PM1_WFXL_MASK _ULL(0xffffffff) +#define KVX_SFR_PM1_PM1_WFXL_CLEAR _ULL(0xffffffff) +#define KVX_SFR_PM1_PM1_WFXL_SET _ULL(0xffffffff00000000) +#define KVX_SFR_PM1_PM1_WFXM_MASK _ULL(0xffffffff00000000) +#define KVX_SFR_PM1_PM1_WFXM_CLEAR _ULL(0xffffffff) +#define KVX_SFR_PM1_PM1_WFXM_SET _ULL(0xffffffff00000000) + +#define KVX_SFR_PM2_PM2_MASK _ULL(0xffffffffffffffff) /* Performance Monitor 2 */ +#define KVX_SFR_PM2_PM2_SHIFT 0 +#define KVX_SFR_PM2_PM2_WIDTH 64 +#define KVX_SFR_PM2_PM2_WFXL_MASK _ULL(0xffffffff) +#define KVX_SFR_PM2_PM2_WFXL_CLEAR _ULL(0xffffffff) +#define KVX_SFR_PM2_PM2_WFXL_SET _ULL(0xffffffff00000000) +#define KVX_SFR_PM2_PM2_WFXM_MASK _ULL(0xffffffff00000000) +#define KVX_SFR_PM2_PM2_WFXM_CLEAR _ULL(0xffffffff) +#define KVX_SFR_PM2_PM2_WFXM_SET _ULL(0xffffffff00000000) + +#define KVX_SFR_PM3_PM3_MASK _ULL(0xffffffffffffffff) /* Performance Monitor 3 */ +#define KVX_SFR_PM3_PM3_SHIFT 0 +#define KVX_SFR_PM3_PM3_WIDTH 64 +#define KVX_SFR_PM3_PM3_WFXL_MASK _ULL(0xffffffff) +#define KVX_SFR_PM3_PM3_WFXL_CLEAR _ULL(0xffffffff) +#define KVX_SFR_PM3_PM3_WFXL_SET _ULL(0xffffffff00000000) +#define KVX_SFR_PM3_PM3_WFXM_MASK _ULL(0xffffffff00000000) +#define KVX_SFR_PM3_PM3_WFXM_CLEAR _ULL(0xffffffff) +#define KVX_SFR_PM3_PM3_WFXM_SET _ULL(0xffffffff00000000) + +#define KVX_SFR_PMSA_PMSA_MASK _ULL(0xffffffffffffffff) /* Performance Monitor Saved Address */ +#define KVX_SFR_PMSA_PMSA_SHIFT 0 +#define KVX_SFR_PMSA_PMSA_WIDTH 64 +#define KVX_SFR_PMSA_PMSA_WFXL_MASK _ULL(0xffffffff) +#define KVX_SFR_PMSA_PMSA_WFXL_CLEAR _ULL(0xffffffff) +#define KVX_SFR_PMSA_PMSA_WFXL_SET _ULL(0xffffffff00000000) +#define KVX_SFR_PMSA_PMSA_WFXM_MASK _ULL(0xffffffff00000000) +#define KVX_SFR_PMSA_PMSA_WFXM_CLEAR _ULL(0xffffffff) +#define KVX_SFR_PMSA_PMSA_WFXM_SET _ULL(0xffffffff00000000) + +#define KVX_SFR_T0V_T0V_MASK _ULL(0xffffffffffffffff) /* Timer 0 value */ +#define KVX_SFR_T0V_T0V_SHIFT 0 +#define KVX_SFR_T0V_T0V_WIDTH 64 +#define KVX_SFR_T0V_T0V_WFXL_MASK _ULL(0xffffffff) +#define KVX_SFR_T0V_T0V_WFXL_CLEAR _ULL(0xffffffff) +#define KVX_SFR_T0V_T0V_WFXL_SET _ULL(0xffffffff00000000) +#define KVX_SFR_T0V_T0V_WFXM_MASK _ULL(0xffffffff00000000) +#define KVX_SFR_T0V_T0V_WFXM_CLEAR _ULL(0xffffffff) +#define KVX_SFR_T0V_T0V_WFXM_SET _ULL(0xffffffff00000000) + +#define KVX_SFR_T1V_T1V_MASK _ULL(0xffffffffffffffff) /* Timer 1 value */ +#define KVX_SFR_T1V_T1V_SHIFT 0 +#define KVX_SFR_T1V_T1V_WIDTH 64 +#define KVX_SFR_T1V_T1V_WFXL_MASK _ULL(0xffffffff) +#define KVX_SFR_T1V_T1V_WFXL_CLEAR _ULL(0xffffffff) +#define KVX_SFR_T1V_T1V_WFXL_SET _ULL(0xffffffff00000000) +#define KVX_SFR_T1V_T1V_WFXM_MASK _ULL(0xffffffff00000000) +#define KVX_SFR_T1V_T1V_WFXM_CLEAR _ULL(0xffffffff) +#define KVX_SFR_T1V_T1V_WFXM_SET _ULL(0xffffffff00000000) + +#define KVX_SFR_T0R_T0R_MASK _ULL(0xffffffffffffffff) /* Timer 0 reload value */ +#define KVX_SFR_T0R_T0R_SHIFT 0 +#define KVX_SFR_T0R_T0R_WIDTH 64 +#define KVX_SFR_T0R_T0R_WFXL_MASK _ULL(0xffffffff) +#define KVX_SFR_T0R_T0R_WFXL_CLEAR _ULL(0xffffffff) +#define KVX_SFR_T0R_T0R_WFXL_SET _ULL(0xffffffff00000000) +#define KVX_SFR_T0R_T0R_WFXM_MASK _ULL(0xffffffff00000000) +#define KVX_SFR_T0R_T0R_WFXM_CLEAR _ULL(0xffffffff) +#define KVX_SFR_T0R_T0R_WFXM_SET _ULL(0xffffffff00000000) + +#define KVX_SFR_T1R_T1R_MASK _ULL(0xffffffffffffffff) /* Timer 1 reload value */ +#define KVX_SFR_T1R_T1R_SHIFT 0 +#define KVX_SFR_T1R_T1R_WIDTH 64 +#define KVX_SFR_T1R_T1R_WFXL_MASK _ULL(0xffffffff) +#define KVX_SFR_T1R_T1R_WFXL_CLEAR _ULL(0xffffffff) +#define KVX_SFR_T1R_T1R_WFXL_SET _ULL(0xffffffff00000000) +#define KVX_SFR_T1R_T1R_WFXM_MASK _ULL(0xffffffff00000000) +#define KVX_SFR_T1R_T1R_WFXM_CLEAR _ULL(0xffffffff) +#define KVX_SFR_T1R_T1R_WFXM_SET _ULL(0xffffffff00000000) + +#define KVX_SFR_WDV_WDV_MASK _ULL(0xffffffffffffffff) /* Watchdog Value */ +#define KVX_SFR_WDV_WDV_SHIFT 0 +#define KVX_SFR_WDV_WDV_WIDTH 64 +#define KVX_SFR_WDV_WDV_WFXL_MASK _ULL(0xffffffff) +#define KVX_SFR_WDV_WDV_WFXL_CLEAR _ULL(0xffffffff) +#define KVX_SFR_WDV_WDV_WFXL_SET _ULL(0xffffffff00000000) +#define KVX_SFR_WDV_WDV_WFXM_MASK _ULL(0xffffffff00000000) +#define KVX_SFR_WDV_WDV_WFXM_CLEAR _ULL(0xffffffff) +#define KVX_SFR_WDV_WDV_WFXM_SET _ULL(0xffffffff00000000) + +#define KVX_SFR_WDR_WDR_MASK _ULL(0xffffffffffffffff) /* Watchdog Reload Value */ +#define KVX_SFR_WDR_WDR_SHIFT 0 +#define KVX_SFR_WDR_WDR_WIDTH 64 +#define KVX_SFR_WDR_WDR_WFXL_MASK _ULL(0xffffffff) +#define KVX_SFR_WDR_WDR_WFXL_CLEAR _ULL(0xffffffff) +#define KVX_SFR_WDR_WDR_WFXL_SET _ULL(0xffffffff00000000) +#define KVX_SFR_WDR_WDR_WFXM_MASK _ULL(0xffffffff00000000) +#define KVX_SFR_WDR_WDR_WFXM_CLEAR _ULL(0xffffffff) +#define KVX_SFR_WDR_WDR_WFXM_SET _ULL(0xffffffff00000000) + +#define KVX_SFR_PMC_PM0C_MASK _ULL(0x3f) /* PM0 Configuration */ +#define KVX_SFR_PMC_PM0C_SHIFT 0 +#define KVX_SFR_PMC_PM0C_WIDTH 6 +#define KVX_SFR_PMC_PM0C_WFXL_MASK _ULL(0x3f) +#define KVX_SFR_PMC_PM0C_WFXL_CLEAR _ULL(0x3f) +#define KVX_SFR_PMC_PM0C_WFXL_SET _ULL(0x3f00000000) + +#define KVX_SFR_PMC_PM1C_MASK _ULL(0x1f80) /* PM1 Configuration */ +#define KVX_SFR_PMC_PM1C_SHIFT 7 +#define KVX_SFR_PMC_PM1C_WIDTH 6 +#define KVX_SFR_PMC_PM1C_WFXL_MASK _ULL(0x1f80) +#define KVX_SFR_PMC_PM1C_WFXL_CLEAR _ULL(0x1f80) +#define KVX_SFR_PMC_PM1C_WFXL_SET _ULL(0x1f8000000000) + +#define KVX_SFR_PMC_PM2C_MASK _ULL(0xfc000) /* PM2 Configuration */ +#define KVX_SFR_PMC_PM2C_SHIFT 14 +#define KVX_SFR_PMC_PM2C_WIDTH 6 +#define KVX_SFR_PMC_PM2C_WFXL_MASK _ULL(0xfc000) +#define KVX_SFR_PMC_PM2C_WFXL_CLEAR _ULL(0xfc000) +#define KVX_SFR_PMC_PM2C_WFXL_SET _ULL(0xfc00000000000) + +#define KVX_SFR_PMC_PM3C_MASK _ULL(0x7e00000) /* PM3 Configuration */ +#define KVX_SFR_PMC_PM3C_SHIFT 21 +#define KVX_SFR_PMC_PM3C_WIDTH 6 +#define KVX_SFR_PMC_PM3C_WFXL_MASK _ULL(0x7e00000) +#define KVX_SFR_PMC_PM3C_WFXL_CLEAR _ULL(0x7e00000) +#define KVX_SFR_PMC_PM3C_WFXL_SET _ULL(0x7e0000000000000) + +#define KVX_SFR_PMC_SAV_MASK _ULL(0x40000000) /* Saved Address Valid */ +#define KVX_SFR_PMC_SAV_SHIFT 30 +#define KVX_SFR_PMC_SAV_WIDTH 1 +#define KVX_SFR_PMC_SAV_WFXL_MASK _ULL(0x40000000) +#define KVX_SFR_PMC_SAV_WFXL_CLEAR _ULL(0x40000000) +#define KVX_SFR_PMC_SAV_WFXL_SET _ULL(0x4000000000000000) + +#define KVX_SFR_PMC_PM0IE_MASK _ULL(0x100000000) /* PM0 Interrupt Enable */ +#define KVX_SFR_PMC_PM0IE_SHIFT 32 +#define KVX_SFR_PMC_PM0IE_WIDTH 1 +#define KVX_SFR_PMC_PM0IE_WFXM_MASK _ULL(0x100000000) +#define KVX_SFR_PMC_PM0IE_WFXM_CLEAR _ULL(0x1) +#define KVX_SFR_PMC_PM0IE_WFXM_SET _ULL(0x100000000) + +#define KVX_SFR_PMC_PM1IE_MASK _ULL(0x200000000) /* PM1 Interrupt Enable */ +#define KVX_SFR_PMC_PM1IE_SHIFT 33 +#define KVX_SFR_PMC_PM1IE_WIDTH 1 +#define KVX_SFR_PMC_PM1IE_WFXM_MASK _ULL(0x200000000) +#define KVX_SFR_PMC_PM1IE_WFXM_CLEAR _ULL(0x2) +#define KVX_SFR_PMC_PM1IE_WFXM_SET _ULL(0x200000000) + +#define KVX_SFR_PMC_PM2IE_MASK _ULL(0x400000000) /* PM2 Interrupt Enable */ +#define KVX_SFR_PMC_PM2IE_SHIFT 34 +#define KVX_SFR_PMC_PM2IE_WIDTH 1 +#define KVX_SFR_PMC_PM2IE_WFXM_MASK _ULL(0x400000000) +#define KVX_SFR_PMC_PM2IE_WFXM_CLEAR _ULL(0x4) +#define KVX_SFR_PMC_PM2IE_WFXM_SET _ULL(0x400000000) + +#define KVX_SFR_PMC_PM3IE_MASK _ULL(0x800000000) /* PM3 Interrupt Enable */ +#define KVX_SFR_PMC_PM3IE_SHIFT 35 +#define KVX_SFR_PMC_PM3IE_WIDTH 1 +#define KVX_SFR_PMC_PM3IE_WFXM_MASK _ULL(0x800000000) +#define KVX_SFR_PMC_PM3IE_WFXM_CLEAR _ULL(0x8) +#define KVX_SFR_PMC_PM3IE_WFXM_SET _ULL(0x800000000) + +#define KVX_SFR_PMC_SAT_MASK _ULL(0x3000000000) /* Saved Address Type */ +#define KVX_SFR_PMC_SAT_SHIFT 36 +#define KVX_SFR_PMC_SAT_WIDTH 2 +#define KVX_SFR_PMC_SAT_WFXM_MASK _ULL(0x3000000000) +#define KVX_SFR_PMC_SAT_WFXM_CLEAR _ULL(0x30) +#define KVX_SFR_PMC_SAT_WFXM_SET _ULL(0x3000000000) + +#define KVX_SFR_PCR_PID_MASK _ULL(0xff) /* Processing Identifier in cluster */ +#define KVX_SFR_PCR_PID_SHIFT 0 +#define KVX_SFR_PCR_PID_WIDTH 8 +#define KVX_SFR_PCR_PID_WFXL_MASK _ULL(0xff) +#define KVX_SFR_PCR_PID_WFXL_CLEAR _ULL(0xff) +#define KVX_SFR_PCR_PID_WFXL_SET _ULL(0xff00000000) + +#define KVX_SFR_PCR_CID_MASK _ULL(0xff00) /* Cluster Identifier in system */ +#define KVX_SFR_PCR_CID_SHIFT 8 +#define KVX_SFR_PCR_CID_WIDTH 8 +#define KVX_SFR_PCR_CID_WFXL_MASK _ULL(0xff00) +#define KVX_SFR_PCR_CID_WFXL_CLEAR _ULL(0xff00) +#define KVX_SFR_PCR_CID_WFXL_SET _ULL(0xff0000000000) + +#define KVX_SFR_PCR_MID_MASK _ULL(0xff0000) /* MPPA Identifier */ +#define KVX_SFR_PCR_MID_SHIFT 16 +#define KVX_SFR_PCR_MID_WIDTH 8 +#define KVX_SFR_PCR_MID_WFXL_MASK _ULL(0xff0000) +#define KVX_SFR_PCR_MID_WFXL_CLEAR _ULL(0xff0000) +#define KVX_SFR_PCR_MID_WFXL_SET _ULL(0xff000000000000) + +#define KVX_SFR_PCR_CAR_MASK _ULL(0xf000000) /* Core Architecture Revision ID */ +#define KVX_SFR_PCR_CAR_SHIFT 24 +#define KVX_SFR_PCR_CAR_WIDTH 4 +#define KVX_SFR_PCR_CAR_WFXL_MASK _ULL(0xf000000) +#define KVX_SFR_PCR_CAR_WFXL_CLEAR _ULL(0xf000000) +#define KVX_SFR_PCR_CAR_WFXL_SET _ULL(0xf00000000000000) + +#define KVX_SFR_PCR_CMA_MASK _ULL(0xf0000000) /* Core Micro-Architecture Revision ID */ +#define KVX_SFR_PCR_CMA_SHIFT 28 +#define KVX_SFR_PCR_CMA_WIDTH 4 +#define KVX_SFR_PCR_CMA_WFXL_MASK _ULL(0xf0000000) +#define KVX_SFR_PCR_CMA_WFXL_CLEAR _ULL(0xf0000000) +#define KVX_SFR_PCR_CMA_WFXL_SET _ULL(0xf000000000000000) + +#define KVX_SFR_PCR_SV_MASK _ULL(0xff00000000) /* System-On-Chip Version */ +#define KVX_SFR_PCR_SV_SHIFT 32 +#define KVX_SFR_PCR_SV_WIDTH 8 +#define KVX_SFR_PCR_SV_WFXM_MASK _ULL(0xff00000000) +#define KVX_SFR_PCR_SV_WFXM_CLEAR _ULL(0xff) +#define KVX_SFR_PCR_SV_WFXM_SET _ULL(0xff00000000) + +#define KVX_SFR_PCR_ST_MASK _ULL(0xf0000000000) /* System-On-Chip Type */ +#define KVX_SFR_PCR_ST_SHIFT 40 +#define KVX_SFR_PCR_ST_WIDTH 4 +#define KVX_SFR_PCR_ST_WFXM_MASK _ULL(0xf0000000000) +#define KVX_SFR_PCR_ST_WFXM_CLEAR _ULL(0xf00) +#define KVX_SFR_PCR_ST_WFXM_SET _ULL(0xf0000000000) + +#define KVX_SFR_PCR_BM_MASK _ULL(0xff00000000000) /* Boot Mode */ +#define KVX_SFR_PCR_BM_SHIFT 44 +#define KVX_SFR_PCR_BM_WIDTH 8 +#define KVX_SFR_PCR_BM_WFXM_MASK _ULL(0xff00000000000) +#define KVX_SFR_PCR_BM_WFXM_CLEAR _ULL(0xff000) +#define KVX_SFR_PCR_BM_WFXM_SET _ULL(0xff00000000000) + +#define KVX_SFR_PCR_COE_MASK _ULL(0x10000000000000) /* COprocessor Enable */ +#define KVX_SFR_PCR_COE_SHIFT 52 +#define KVX_SFR_PCR_COE_WIDTH 1 +#define KVX_SFR_PCR_COE_WFXM_MASK _ULL(0x10000000000000) +#define KVX_SFR_PCR_COE_WFXM_CLEAR _ULL(0x100000) +#define KVX_SFR_PCR_COE_WFXM_SET _ULL(0x10000000000000) + +#define KVX_SFR_PCR_L1CE_MASK _ULL(0x20000000000000) /* L1 cache Coherency Enable */ +#define KVX_SFR_PCR_L1CE_SHIFT 53 +#define KVX_SFR_PCR_L1CE_WIDTH 1 +#define KVX_SFR_PCR_L1CE_WFXM_MASK _ULL(0x20000000000000) +#define KVX_SFR_PCR_L1CE_WFXM_CLEAR _ULL(0x200000) +#define KVX_SFR_PCR_L1CE_WFXM_SET _ULL(0x20000000000000) + +#define KVX_SFR_PCR_DSEM_MASK _ULL(0x40000000000000) /* Data Simple Ecc exception Mode */ +#define KVX_SFR_PCR_DSEM_SHIFT 54 +#define KVX_SFR_PCR_DSEM_WIDTH 1 +#define KVX_SFR_PCR_DSEM_WFXM_MASK _ULL(0x40000000000000) +#define KVX_SFR_PCR_DSEM_WFXM_CLEAR _ULL(0x400000) +#define KVX_SFR_PCR_DSEM_WFXM_SET _ULL(0x40000000000000) + +#define KVX_SFR_MMC_ASN_MASK _ULL(0x1ff) /* Address Space Number */ +#define KVX_SFR_MMC_ASN_SHIFT 0 +#define KVX_SFR_MMC_ASN_WIDTH 9 +#define KVX_SFR_MMC_ASN_WFXL_MASK _ULL(0x1ff) +#define KVX_SFR_MMC_ASN_WFXL_CLEAR _ULL(0x1ff) +#define KVX_SFR_MMC_ASN_WFXL_SET _ULL(0x1ff00000000) + +#define KVX_SFR_MMC_S_MASK _ULL(0x200) /* Speculative */ +#define KVX_SFR_MMC_S_SHIFT 9 +#define KVX_SFR_MMC_S_WIDTH 1 +#define KVX_SFR_MMC_S_WFXL_MASK _ULL(0x200) +#define KVX_SFR_MMC_S_WFXL_CLEAR _ULL(0x200) +#define KVX_SFR_MMC_S_WFXL_SET _ULL(0x20000000000) + +#define KVX_SFR_MMC_SNE_MASK _ULL(0x4000) /* Speculative NOMAPPING Enable */ +#define KVX_SFR_MMC_SNE_SHIFT 14 +#define KVX_SFR_MMC_SNE_WIDTH 1 +#define KVX_SFR_MMC_SNE_WFXL_MASK _ULL(0x4000) +#define KVX_SFR_MMC_SNE_WFXL_CLEAR _ULL(0x4000) +#define KVX_SFR_MMC_SNE_WFXL_SET _ULL(0x400000000000) + +#define KVX_SFR_MMC_SPE_MASK _ULL(0x8000) /* Speculative PROTECTION Enable */ +#define KVX_SFR_MMC_SPE_SHIFT 15 +#define KVX_SFR_MMC_SPE_WIDTH 1 +#define KVX_SFR_MMC_SPE_WFXL_MASK _ULL(0x8000) +#define KVX_SFR_MMC_SPE_WFXL_CLEAR _ULL(0x8000) +#define KVX_SFR_MMC_SPE_WFXL_SET _ULL(0x800000000000) + +#define KVX_SFR_MMC_PTC_MASK _ULL(0x30000) /* Protection Trap Cause */ +#define KVX_SFR_MMC_PTC_SHIFT 16 +#define KVX_SFR_MMC_PTC_WIDTH 2 +#define KVX_SFR_MMC_PTC_WFXL_MASK _ULL(0x30000) +#define KVX_SFR_MMC_PTC_WFXL_CLEAR _ULL(0x30000) +#define KVX_SFR_MMC_PTC_WFXL_SET _ULL(0x3000000000000) + +#define KVX_SFR_MMC_SW_MASK _ULL(0x3c0000) /* Select Way */ +#define KVX_SFR_MMC_SW_SHIFT 18 +#define KVX_SFR_MMC_SW_WIDTH 4 +#define KVX_SFR_MMC_SW_WFXL_MASK _ULL(0x3c0000) +#define KVX_SFR_MMC_SW_WFXL_CLEAR _ULL(0x3c0000) +#define KVX_SFR_MMC_SW_WFXL_SET _ULL(0x3c000000000000) + +#define KVX_SFR_MMC_SS_MASK _ULL(0xfc00000) /* Select Set */ +#define KVX_SFR_MMC_SS_SHIFT 22 +#define KVX_SFR_MMC_SS_WIDTH 6 +#define KVX_SFR_MMC_SS_WFXL_MASK _ULL(0xfc00000) +#define KVX_SFR_MMC_SS_WFXL_CLEAR _ULL(0xfc00000) +#define KVX_SFR_MMC_SS_WFXL_SET _ULL(0xfc0000000000000) + +#define KVX_SFR_MMC_SB_MASK _ULL(0x10000000) /* Select Buffer */ +#define KVX_SFR_MMC_SB_SHIFT 28 +#define KVX_SFR_MMC_SB_WIDTH 1 +#define KVX_SFR_MMC_SB_WFXL_MASK _ULL(0x10000000) +#define KVX_SFR_MMC_SB_WFXL_CLEAR _ULL(0x10000000) +#define KVX_SFR_MMC_SB_WFXL_SET _ULL(0x1000000000000000) + +#define KVX_SFR_MMC_PAR_MASK _ULL(0x40000000) /* PARity error flag */ +#define KVX_SFR_MMC_PAR_SHIFT 30 +#define KVX_SFR_MMC_PAR_WIDTH 1 +#define KVX_SFR_MMC_PAR_WFXL_MASK _ULL(0x40000000) +#define KVX_SFR_MMC_PAR_WFXL_CLEAR _ULL(0x40000000) +#define KVX_SFR_MMC_PAR_WFXL_SET _ULL(0x4000000000000000) + +#define KVX_SFR_MMC_E_MASK _ULL(0x80000000) /* Error Flag */ +#define KVX_SFR_MMC_E_SHIFT 31 +#define KVX_SFR_MMC_E_WIDTH 1 +#define KVX_SFR_MMC_E_WFXL_MASK _ULL(0x80000000) +#define KVX_SFR_MMC_E_WFXL_CLEAR _ULL(0x80000000) +#define KVX_SFR_MMC_E_WFXL_SET _ULL(0x8000000000000000) + +#define KVX_SFR_TEL_ES_MASK _ULL(0x3) /* Entry Status */ +#define KVX_SFR_TEL_ES_SHIFT 0 +#define KVX_SFR_TEL_ES_WIDTH 2 +#define KVX_SFR_TEL_ES_WFXL_MASK _ULL(0x3) +#define KVX_SFR_TEL_ES_WFXL_CLEAR _ULL(0x3) +#define KVX_SFR_TEL_ES_WFXL_SET _ULL(0x300000000) + +#define KVX_SFR_TEL_CP_MASK _ULL(0xc) /* Cache Policy */ +#define KVX_SFR_TEL_CP_SHIFT 2 +#define KVX_SFR_TEL_CP_WIDTH 2 +#define KVX_SFR_TEL_CP_WFXL_MASK _ULL(0xc) +#define KVX_SFR_TEL_CP_WFXL_CLEAR _ULL(0xc) +#define KVX_SFR_TEL_CP_WFXL_SET _ULL(0xc00000000) + +#define KVX_SFR_TEL_PA_MASK _ULL(0xf0) /* Protection Attributes */ +#define KVX_SFR_TEL_PA_SHIFT 4 +#define KVX_SFR_TEL_PA_WIDTH 4 +#define KVX_SFR_TEL_PA_WFXL_MASK _ULL(0xf0) +#define KVX_SFR_TEL_PA_WFXL_CLEAR _ULL(0xf0) +#define KVX_SFR_TEL_PA_WFXL_SET _ULL(0xf000000000) + +#define KVX_SFR_TEL_PS_MASK _ULL(0xc00) /* Page Size */ +#define KVX_SFR_TEL_PS_SHIFT 10 +#define KVX_SFR_TEL_PS_WIDTH 2 +#define KVX_SFR_TEL_PS_WFXL_MASK _ULL(0xc00) +#define KVX_SFR_TEL_PS_WFXL_CLEAR _ULL(0xc00) +#define KVX_SFR_TEL_PS_WFXL_SET _ULL(0xc0000000000) + +#define KVX_SFR_TEL_FN_MASK _ULL(0xfffffff000) /* Frame Number */ +#define KVX_SFR_TEL_FN_SHIFT 12 +#define KVX_SFR_TEL_FN_WIDTH 28 +#define KVX_SFR_TEL_FN_WFXL_MASK _ULL(0xfffff000) +#define KVX_SFR_TEL_FN_WFXL_CLEAR _ULL(0xfffff000) +#define KVX_SFR_TEL_FN_WFXL_SET _ULL(0xfffff00000000000) +#define KVX_SFR_TEL_FN_WFXM_MASK _ULL(0xff00000000) +#define KVX_SFR_TEL_FN_WFXM_CLEAR _ULL(0xff) +#define KVX_SFR_TEL_FN_WFXM_SET _ULL(0xff00000000) + +#define KVX_SFR_TEH_ASN_MASK _ULL(0x1ff) /* Address Space Number */ +#define KVX_SFR_TEH_ASN_SHIFT 0 +#define KVX_SFR_TEH_ASN_WIDTH 9 +#define KVX_SFR_TEH_ASN_WFXL_MASK _ULL(0x1ff) +#define KVX_SFR_TEH_ASN_WFXL_CLEAR _ULL(0x1ff) +#define KVX_SFR_TEH_ASN_WFXL_SET _ULL(0x1ff00000000) + +#define KVX_SFR_TEH_G_MASK _ULL(0x200) /* Global page indicator */ +#define KVX_SFR_TEH_G_SHIFT 9 +#define KVX_SFR_TEH_G_WIDTH 1 +#define KVX_SFR_TEH_G_WFXL_MASK _ULL(0x200) +#define KVX_SFR_TEH_G_WFXL_CLEAR _ULL(0x200) +#define KVX_SFR_TEH_G_WFXL_SET _ULL(0x20000000000) + +#define KVX_SFR_TEH_VS_MASK _ULL(0xc00) /* Virtual Space */ +#define KVX_SFR_TEH_VS_SHIFT 10 +#define KVX_SFR_TEH_VS_WIDTH 2 +#define KVX_SFR_TEH_VS_WFXL_MASK _ULL(0xc00) +#define KVX_SFR_TEH_VS_WFXL_CLEAR _ULL(0xc00) +#define KVX_SFR_TEH_VS_WFXL_SET _ULL(0xc0000000000) + +#define KVX_SFR_TEH_PN_MASK _ULL(0x1fffffff000) /* Page Number */ +#define KVX_SFR_TEH_PN_SHIFT 12 +#define KVX_SFR_TEH_PN_WIDTH 29 +#define KVX_SFR_TEH_PN_WFXL_MASK _ULL(0xfffff000) +#define KVX_SFR_TEH_PN_WFXL_CLEAR _ULL(0xfffff000) +#define KVX_SFR_TEH_PN_WFXL_SET _ULL(0xfffff00000000000) +#define KVX_SFR_TEH_PN_WFXM_MASK _ULL(0x1ff00000000) +#define KVX_SFR_TEH_PN_WFXM_CLEAR _ULL(0x1ff) +#define KVX_SFR_TEH_PN_WFXM_SET _ULL(0x1ff00000000) + +#define KVX_SFR_DC_BE0_MASK _ULL(0x1) /* Breakpoint 0 Enable */ +#define KVX_SFR_DC_BE0_SHIFT 0 +#define KVX_SFR_DC_BE0_WIDTH 1 +#define KVX_SFR_DC_BE0_WFXL_MASK _ULL(0x1) +#define KVX_SFR_DC_BE0_WFXL_CLEAR _ULL(0x1) +#define KVX_SFR_DC_BE0_WFXL_SET _ULL(0x100000000) + +#define KVX_SFR_DC_BR0_MASK _ULL(0x7e) /* Breakpoint 0 Range */ +#define KVX_SFR_DC_BR0_SHIFT 1 +#define KVX_SFR_DC_BR0_WIDTH 6 +#define KVX_SFR_DC_BR0_WFXL_MASK _ULL(0x7e) +#define KVX_SFR_DC_BR0_WFXL_CLEAR _ULL(0x7e) +#define KVX_SFR_DC_BR0_WFXL_SET _ULL(0x7e00000000) + +#define KVX_SFR_DC_BE1_MASK _ULL(0x80) /* Breakpoint 1 Enable */ +#define KVX_SFR_DC_BE1_SHIFT 7 +#define KVX_SFR_DC_BE1_WIDTH 1 +#define KVX_SFR_DC_BE1_WFXL_MASK _ULL(0x80) +#define KVX_SFR_DC_BE1_WFXL_CLEAR _ULL(0x80) +#define KVX_SFR_DC_BE1_WFXL_SET _ULL(0x8000000000) + +#define KVX_SFR_DC_BR1_MASK _ULL(0x3f00) /* Breakpoint 1 Range */ +#define KVX_SFR_DC_BR1_SHIFT 8 +#define KVX_SFR_DC_BR1_WIDTH 6 +#define KVX_SFR_DC_BR1_WFXL_MASK _ULL(0x3f00) +#define KVX_SFR_DC_BR1_WFXL_CLEAR _ULL(0x3f00) +#define KVX_SFR_DC_BR1_WFXL_SET _ULL(0x3f0000000000) + +#define KVX_SFR_DC_WE0_MASK _ULL(0x4000) /* Watchpoint 0 Enable */ +#define KVX_SFR_DC_WE0_SHIFT 14 +#define KVX_SFR_DC_WE0_WIDTH 1 +#define KVX_SFR_DC_WE0_WFXL_MASK _ULL(0x4000) +#define KVX_SFR_DC_WE0_WFXL_CLEAR _ULL(0x4000) +#define KVX_SFR_DC_WE0_WFXL_SET _ULL(0x400000000000) + +#define KVX_SFR_DC_WR0_MASK _ULL(0x1f8000) /* Watchpoint 0 Range */ +#define KVX_SFR_DC_WR0_SHIFT 15 +#define KVX_SFR_DC_WR0_WIDTH 6 +#define KVX_SFR_DC_WR0_WFXL_MASK _ULL(0x1f8000) +#define KVX_SFR_DC_WR0_WFXL_CLEAR _ULL(0x1f8000) +#define KVX_SFR_DC_WR0_WFXL_SET _ULL(0x1f800000000000) + +#define KVX_SFR_DC_WE1_MASK _ULL(0x200000) /* Watchpoint 1 Enable */ +#define KVX_SFR_DC_WE1_SHIFT 21 +#define KVX_SFR_DC_WE1_WIDTH 1 +#define KVX_SFR_DC_WE1_WFXL_MASK _ULL(0x200000) +#define KVX_SFR_DC_WE1_WFXL_CLEAR _ULL(0x200000) +#define KVX_SFR_DC_WE1_WFXL_SET _ULL(0x20000000000000) + +#define KVX_SFR_DC_WR1_MASK _ULL(0xfc00000) /* Watchpoint 1 Range */ +#define KVX_SFR_DC_WR1_SHIFT 22 +#define KVX_SFR_DC_WR1_WIDTH 6 +#define KVX_SFR_DC_WR1_WFXL_MASK _ULL(0xfc00000) +#define KVX_SFR_DC_WR1_WFXL_CLEAR _ULL(0xfc00000) +#define KVX_SFR_DC_WR1_WFXL_SET _ULL(0xfc0000000000000) + +#define KVX_SFR_MES_PSE_MASK _ULL(0x1) /* Program Simple Ecc */ +#define KVX_SFR_MES_PSE_SHIFT 0 +#define KVX_SFR_MES_PSE_WIDTH 1 +#define KVX_SFR_MES_PSE_WFXL_MASK _ULL(0x1) +#define KVX_SFR_MES_PSE_WFXL_CLEAR _ULL(0x1) +#define KVX_SFR_MES_PSE_WFXL_SET _ULL(0x100000000) + +#define KVX_SFR_MES_PILSY_MASK _ULL(0x2) /* Program cache Invalidated Line following pSYs error. */ +#define KVX_SFR_MES_PILSY_SHIFT 1 +#define KVX_SFR_MES_PILSY_WIDTH 1 +#define KVX_SFR_MES_PILSY_WFXL_MASK _ULL(0x2) +#define KVX_SFR_MES_PILSY_WFXL_CLEAR _ULL(0x2) +#define KVX_SFR_MES_PILSY_WFXL_SET _ULL(0x200000000) + +#define KVX_SFR_MES_PILDE_MASK _ULL(0x4) /* Program cache Invalidated Line following pDEcc error. */ +#define KVX_SFR_MES_PILDE_SHIFT 2 +#define KVX_SFR_MES_PILDE_WIDTH 1 +#define KVX_SFR_MES_PILDE_WFXL_MASK _ULL(0x4) +#define KVX_SFR_MES_PILDE_WFXL_CLEAR _ULL(0x4) +#define KVX_SFR_MES_PILDE_WFXL_SET _ULL(0x400000000) + +#define KVX_SFR_MES_PILPA_MASK _ULL(0x8) /* Program cache Invalidated Line following pPArity error. */ +#define KVX_SFR_MES_PILPA_SHIFT 3 +#define KVX_SFR_MES_PILPA_WIDTH 1 +#define KVX_SFR_MES_PILPA_WFXL_MASK _ULL(0x8) +#define KVX_SFR_MES_PILPA_WFXL_CLEAR _ULL(0x8) +#define KVX_SFR_MES_PILPA_WFXL_SET _ULL(0x800000000) + +#define KVX_SFR_MES_DSE_MASK _ULL(0x10) /* Data Simple Ecc */ +#define KVX_SFR_MES_DSE_SHIFT 4 +#define KVX_SFR_MES_DSE_WIDTH 1 +#define KVX_SFR_MES_DSE_WFXL_MASK _ULL(0x10) +#define KVX_SFR_MES_DSE_WFXL_CLEAR _ULL(0x10) +#define KVX_SFR_MES_DSE_WFXL_SET _ULL(0x1000000000) + +#define KVX_SFR_MES_DILSY_MASK _ULL(0x20) /* Data cache Invalidated Line following dSYs error. */ +#define KVX_SFR_MES_DILSY_SHIFT 5 +#define KVX_SFR_MES_DILSY_WIDTH 1 +#define KVX_SFR_MES_DILSY_WFXL_MASK _ULL(0x20) +#define KVX_SFR_MES_DILSY_WFXL_CLEAR _ULL(0x20) +#define KVX_SFR_MES_DILSY_WFXL_SET _ULL(0x2000000000) + +#define KVX_SFR_MES_DILDE_MASK _ULL(0x40) /* Data cache Invalidated Line following dDEcc error. */ +#define KVX_SFR_MES_DILDE_SHIFT 6 +#define KVX_SFR_MES_DILDE_WIDTH 1 +#define KVX_SFR_MES_DILDE_WFXL_MASK _ULL(0x40) +#define KVX_SFR_MES_DILDE_WFXL_CLEAR _ULL(0x40) +#define KVX_SFR_MES_DILDE_WFXL_SET _ULL(0x4000000000) + +#define KVX_SFR_MES_DILPA_MASK _ULL(0x80) /* Data cache Invalidated Line following dPArity error. */ +#define KVX_SFR_MES_DILPA_SHIFT 7 +#define KVX_SFR_MES_DILPA_WIDTH 1 +#define KVX_SFR_MES_DILPA_WFXL_MASK _ULL(0x80) +#define KVX_SFR_MES_DILPA_WFXL_CLEAR _ULL(0x80) +#define KVX_SFR_MES_DILPA_WFXL_SET _ULL(0x8000000000) + +#define KVX_SFR_MES_DDEE_MASK _ULL(0x100) /* Data DEcc Error. */ +#define KVX_SFR_MES_DDEE_SHIFT 8 +#define KVX_SFR_MES_DDEE_WIDTH 1 +#define KVX_SFR_MES_DDEE_WFXL_MASK _ULL(0x100) +#define KVX_SFR_MES_DDEE_WFXL_CLEAR _ULL(0x100) +#define KVX_SFR_MES_DDEE_WFXL_SET _ULL(0x10000000000) + +#define KVX_SFR_MES_DSYE_MASK _ULL(0x200) /* Data dSYs Error. */ +#define KVX_SFR_MES_DSYE_SHIFT 9 +#define KVX_SFR_MES_DSYE_WIDTH 1 +#define KVX_SFR_MES_DSYE_WFXL_MASK _ULL(0x200) +#define KVX_SFR_MES_DSYE_WFXL_CLEAR _ULL(0x200) +#define KVX_SFR_MES_DSYE_WFXL_SET _ULL(0x20000000000) + +#define KVX_SFR_WS_WU0_MASK _ULL(0x1) /* Wake-Up 0 */ +#define KVX_SFR_WS_WU0_SHIFT 0 +#define KVX_SFR_WS_WU0_WIDTH 1 +#define KVX_SFR_WS_WU0_WFXL_MASK _ULL(0x1) +#define KVX_SFR_WS_WU0_WFXL_CLEAR _ULL(0x1) +#define KVX_SFR_WS_WU0_WFXL_SET _ULL(0x100000000) + +#define KVX_SFR_WS_WU1_MASK _ULL(0x2) /* Wake-Up 1 */ +#define KVX_SFR_WS_WU1_SHIFT 1 +#define KVX_SFR_WS_WU1_WIDTH 1 +#define KVX_SFR_WS_WU1_WFXL_MASK _ULL(0x2) +#define KVX_SFR_WS_WU1_WFXL_CLEAR _ULL(0x2) +#define KVX_SFR_WS_WU1_WFXL_SET _ULL(0x200000000) + +#define KVX_SFR_WS_WU2_MASK _ULL(0x4) /* Wake-Up 2 */ +#define KVX_SFR_WS_WU2_SHIFT 2 +#define KVX_SFR_WS_WU2_WIDTH 1 +#define KVX_SFR_WS_WU2_WFXL_MASK _ULL(0x4) +#define KVX_SFR_WS_WU2_WFXL_CLEAR _ULL(0x4) +#define KVX_SFR_WS_WU2_WFXL_SET _ULL(0x400000000) + +#define KVX_SFR_IPE_FE_MASK _ULL(0xffff) /* Forward Events */ +#define KVX_SFR_IPE_FE_SHIFT 0 +#define KVX_SFR_IPE_FE_WIDTH 16 +#define KVX_SFR_IPE_FE_WFXL_MASK _ULL(0xffff) +#define KVX_SFR_IPE_FE_WFXL_CLEAR _ULL(0xffff) +#define KVX_SFR_IPE_FE_WFXL_SET _ULL(0xffff00000000) + +#define KVX_SFR_IPE_BE_MASK _ULL(0xffff0000) /* Backward Events */ +#define KVX_SFR_IPE_BE_SHIFT 16 +#define KVX_SFR_IPE_BE_WIDTH 16 +#define KVX_SFR_IPE_BE_WFXL_MASK _ULL(0xffff0000) +#define KVX_SFR_IPE_BE_WFXL_CLEAR _ULL(0xffff0000) +#define KVX_SFR_IPE_BE_WFXL_SET _ULL(0xffff000000000000) + +#define KVX_SFR_IPE_FM_MASK _ULL(0xffff00000000) /* Forward Mode */ +#define KVX_SFR_IPE_FM_SHIFT 32 +#define KVX_SFR_IPE_FM_WIDTH 16 +#define KVX_SFR_IPE_FM_WFXM_MASK _ULL(0xffff00000000) +#define KVX_SFR_IPE_FM_WFXM_CLEAR _ULL(0xffff) +#define KVX_SFR_IPE_FM_WFXM_SET _ULL(0xffff00000000) + +#define KVX_SFR_IPE_BM_MASK _ULL(0xffff000000000000) /* Backward Modes */ +#define KVX_SFR_IPE_BM_SHIFT 48 +#define KVX_SFR_IPE_BM_WIDTH 16 +#define KVX_SFR_IPE_BM_WFXM_MASK _ULL(0xffff000000000000) +#define KVX_SFR_IPE_BM_WFXM_CLEAR _ULL(0xffff0000) +#define KVX_SFR_IPE_BM_WFXM_SET _ULL(0xffff000000000000) + +#endif /*_ASM_KVX_SFR_DEFS_H */ diff --git a/arch/kvx/include/asm/swab.h b/arch/kvx/include/asm/swab.h new file mode 100644 index 000000000000..83a48871c8fb --- /dev/null +++ b/arch/kvx/include/asm/swab.h @@ -0,0 +1,48 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_SWAB_H +#define _ASM_KVX_SWAB_H + +#include <linux/types.h> + +#define U64_BYTE_SWAP_MATRIX 0x0102040810204080ULL +#define U32_BYTE_SWAP_MATRIX 0x0000000001020408ULL +#define U16_BYTE_SWAP_MATRIX 0x0000000000000102ULL +#define U32_WORD_SWAP_MATRIX 0x0000000002010804ULL +#define U32_HL_BYTE_SWAP_MATRIX 0x0000000004080102ULL + +static inline __attribute_const__ __u64 __arch_swab64(__u64 val) +{ + return __builtin_kvx_sbmm8(val, U64_BYTE_SWAP_MATRIX); +} + +static inline __attribute_const__ __u32 __arch_swab32(__u32 val) +{ + return __builtin_kvx_sbmm8(val, U32_BYTE_SWAP_MATRIX); +} + +static inline __attribute_const__ __u16 __arch_swab16(__u16 val) +{ + return __builtin_kvx_sbmm8(val, U16_BYTE_SWAP_MATRIX); +} + +static inline __attribute_const__ __u32 __arch_swahw32(__u32 val) +{ + return __builtin_kvx_sbmm8(val, U32_WORD_SWAP_MATRIX); +} + +static inline __attribute_const__ __u32 __arch_swahb32(__u32 val) +{ + return __builtin_kvx_sbmm8(val, U32_HL_BYTE_SWAP_MATRIX); +} + +#define __arch_swab64 __arch_swab64 +#define __arch_swab32 __arch_swab32 +#define __arch_swab16 __arch_swab16 +#define __arch_swahw32 __arch_swahw32 +#define __arch_swahb32 __arch_swahb32 +#endif diff --git a/arch/kvx/include/asm/sys_arch.h b/arch/kvx/include/asm/sys_arch.h new file mode 100644 index 000000000000..ae518106a0a8 --- /dev/null +++ b/arch/kvx/include/asm/sys_arch.h @@ -0,0 +1,51 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_SYS_ARCH_H +#define _ASM_KVX_SYS_ARCH_H + +#include <asm/sfr_defs.h> + +#define EXCEPTION_STRIDE 0x40 +#define EXCEPTION_ALIGNMENT 0x100 + +#define kvx_cluster_id() ((int) \ + ((kvx_sfr_get(PCR) & KVX_SFR_PCR_CID_MASK) \ + >> KVX_SFR_PCR_CID_SHIFT)) + +#define KVX_SFR_START(__sfr_reg) \ + (KVX_SFR_## __sfr_reg ## _SHIFT) + +#define KVX_SFR_END(__sfr_reg) \ + (KVX_SFR_## __sfr_reg ## _SHIFT + KVX_SFR_## __sfr_reg ## _WIDTH - 1) + +/** + * Get the value to clear a sfr + */ +#define SFR_CLEAR(__sfr, __field, __lm) \ + KVX_SFR_## __sfr ## _ ## __field ## _ ## __lm ## _CLEAR + +#define SFR_CLEAR_WFXL(__sfr, __field) SFR_CLEAR(__sfr, __field, WFXL) +#define SFR_CLEAR_WFXM(__sfr, __field) SFR_CLEAR(__sfr, __field, WFXM) + +/** + * Get the value to set a sfr. + */ +#define SFR_SET_WFXL(__sfr, __field, __val) \ + (__val << (KVX_SFR_ ## __sfr ## _ ## __field ## _SHIFT + 32)) + +#define SFR_SET_WFXM(__sfr, __field, __val) \ + (__val << (KVX_SFR_ ## __sfr ## _ ## __field ## _SHIFT)) + +/** + * Generate the mask to clear and set a value using wfx{m|l}. + */ +#define SFR_SET_VAL_WFXL(__sfr, __field, __val) \ + (SFR_SET_WFXL(__sfr, __field, __val) | SFR_CLEAR_WFXL(__sfr, __field)) +#define SFR_SET_VAL_WFXM(__sfr, __field, __val) \ + (SFR_SET_WFXM(__sfr, __field, __val) | SFR_CLEAR_WFXM(__sfr, __field)) + +#endif /* _ASM_KVX_SYS_ARCH_H */ -- 2.37.2

On Fri, Jan 20, 2023 at 10:13 PM Yann Sionneau <ysionneau(a)kalray.eu&gt; wrote: What's the acswapw/d machine code? Seems all RMW-atomic operations are based on it. -- Best Regards Guo Ren

Add basic boot, setup and reset routines for kvx. Co-developed-by: Alex Michon <amichon(a)kalray.eu&gt; Signed-off-by: Alex Michon <amichon(a)kalray.eu&gt; Co-developed-by: Clement Leger <clement(a)clement-leger.fr&gt; Signed-off-by: Clement Leger <clement(a)clement-leger.fr&gt; Co-developed-by: Guillaume Missonnier <gmissonnier(a)kalray.eu&gt; Signed-off-by: Guillaume Missonnier <gmissonnier(a)kalray.eu&gt; Co-developed-by: Guillaume Thouvenin <gthouvenin(a)kalray.eu&gt; Signed-off-by: Guillaume Thouvenin <gthouvenin(a)kalray.eu&gt; Co-developed-by: Jules Maselbas <jmaselbas(a)kalray.eu&gt; Signed-off-by: Jules Maselbas <jmaselbas(a)kalray.eu&gt; Co-developed-by: Julian Vetter <jvetter(a)kalray.eu&gt; Signed-off-by: Julian Vetter <jvetter(a)kalray.eu&gt; Co-developed-by: Julien Hascoet <jhascoet(a)kalray.eu&gt; Signed-off-by: Julien Hascoet <jhascoet(a)kalray.eu&gt; Co-developed-by: Julien Villette <jvillette(a)kalray.eu&gt; Signed-off-by: Julien Villette <jvillette(a)kalray.eu&gt; Co-developed-by: Marc Poulhiès <dkm(a)kataplop.net&gt; Signed-off-by: Marc Poulhiès <dkm(a)kataplop.net&gt; Co-developed-by: Luc Michel <lmichel(a)kalray.eu&gt; Signed-off-by: Luc Michel <lmichel(a)kalray.eu&gt; Co-developed-by: Marius Gligor <mgligor(a)kalray.eu&gt; Signed-off-by: Marius Gligor <mgligor(a)kalray.eu&gt; Co-developed-by: Yann Sionneau <ysionneau(a)kalray.eu&gt; Signed-off-by: Yann Sionneau <ysionneau(a)kalray.eu&gt; --- Notes: V1 -> V2: removed L2 cache firmware starting code arch/kvx/include/asm/setup.h | 29 ++ arch/kvx/kernel/common.c | 11 + arch/kvx/kernel/head.S | 568 +++++++++++++++++++++++++++++++++++ arch/kvx/kernel/prom.c | 24 ++ arch/kvx/kernel/reset.c | 37 +++ arch/kvx/kernel/setup.c | 177 +++++++++++ arch/kvx/kernel/time.c | 242 +++++++++++++++ 7 files changed, 1088 insertions(+) create mode 100644 arch/kvx/include/asm/setup.h create mode 100644 arch/kvx/kernel/common.c create mode 100644 arch/kvx/kernel/head.S create mode 100644 arch/kvx/kernel/prom.c create mode 100644 arch/kvx/kernel/reset.c create mode 100644 arch/kvx/kernel/setup.c create mode 100644 arch/kvx/kernel/time.c diff --git a/arch/kvx/include/asm/setup.h b/arch/kvx/include/asm/setup.h new file mode 100644 index 000000000000..9c27d5981442 --- /dev/null +++ b/arch/kvx/include/asm/setup.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_SETUP_H +#define _ASM_KVX_SETUP_H + +#include <linux/const.h> + +#include <asm-generic/setup.h> + +/* Magic is found in r0 when some parameters are given to kernel */ +#define LINUX_BOOT_PARAM_MAGIC ULL(0x31564752414E494C) + +#ifndef __ASSEMBLY__ + +void early_fixmap_init(void); + +void setup_device_tree(void); + +void setup_arch_memory(void); + +void kvx_init_mmu(void); + +#endif + +#endif /* _ASM_KVX_SETUP_H */ diff --git a/arch/kvx/kernel/common.c b/arch/kvx/kernel/common.c new file mode 100644 index 000000000000..322498f034fd --- /dev/null +++ b/arch/kvx/kernel/common.c @@ -0,0 +1,11 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#include <linux/percpu-defs.h> +#include <linux/sched/task.h> + +DEFINE_PER_CPU(int, __preempt_count) = INIT_PREEMPT_COUNT; +EXPORT_PER_CPU_SYMBOL(__preempt_count); diff --git a/arch/kvx/kernel/head.S b/arch/kvx/kernel/head.S new file mode 100644 index 000000000000..6badd2f6a2a6 --- /dev/null +++ b/arch/kvx/kernel/head.S @@ -0,0 +1,568 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Guillaume Thouvenin + * Marius Gligor + * Julian Vetter + * Julien Hascoet + * Yann Sionneau + * Marc Poulhiès + */ +#include <asm/thread_info.h> +#include <asm/page_size.h> +#include <asm/pwr_ctrl.h> +#include <asm/sfr_defs.h> +#include <asm/sys_arch.h> +#include <asm/privilege.h> +#include <asm/tlb_defs.h> +#include <asm/mem_map.h> +#include <asm/rm_fw.h> +#include <asm/setup.h> +#include <asm/page.h> + +#include <linux/linkage.h> +#include <linux/init.h> + +#ifdef CONFIG_SMP +#define SECONDARY_START_ADDR smp_secondary_start +#else +#define SECONDARY_START_ADDR proc_power_off +#endif + +#define PS_VAL_WFXL(__field, __val) \ + SFR_SET_VAL_WFXL(PS, __field, __val) + +#define PS_WFXL_VALUE PS_VAL_WFXL(HLE, 1) | \ + PS_VAL_WFXL(USE, 1) | \ + PS_VAL_WFXL(DCE, 1) | \ + PS_VAL_WFXL(ICE, 1) | \ + PS_VAL_WFXL(MME, 1) | \ + PS_VAL_WFXL(MMUP, 1) | \ + PS_VAL_WFXL(ET, 0) | \ + PS_VAL_WFXL(HTD, 0) | \ + PS_VAL_WFXL(PMJ, KVX_SUPPORTED_PSIZE) + +#define PCR_VAL_WFXM(__field, __val) \ + SFR_SET_VAL_WFXM(PCR, __field, __val) + +#define PCR_WFXM_VALUE PCR_VAL_WFXM(L1CE, 1) + +/* 30 sec for primary watchdog timeout */ +#define PRIMARY_WATCHDOG_VALUE (30000000000UL) + +#define TCR_WFXL_VALUE SFR_SET_VAL_WFXL(TCR, WUI, 1) | \ + SFR_SET_VAL_WFXL(TCR, WCE, 1) + +/* Enable STOP in WS */ +#define WS_ENABLE_WU2 (KVX_SFR_WS_WU2_MASK) +/* We only want to clear bits in ws */ +#define WS_WFXL_VALUE (WS_ENABLE_WU2) + +/* SMP stuff */ +#define RM_PID_MASK ((KVX_RM_ID) << KVX_SFR_PCR_PID_SHIFT) + +#define PWR_CTRL_ADDR 0xA40000 + +#define PWR_CTRL_GLOBAL_CONFIG_VALUE \ + (1 << KVX_PWR_CTRL_GLOBAL_SET_PE_EN_SHIFT) + +/* Clean error and selected buffer */ +#define MMC_CLEAR_ERROR (KVX_SFR_MMC_E_MASK) + +#define TEH_VIRTUAL_MEMORY \ + TLB_MK_TEH_ENTRY(PAGE_OFFSET, 0, TLB_G_GLOBAL, 0) + +#define TEL_VIRTUAL_MEMORY \ + TLB_MK_TEL_ENTRY(PHYS_OFFSET, TLB_PS_512M, TLB_ES_A_MODIFIED,\ + TLB_CP_W_C, TLB_PA_NA_RWX) + +/* (TEH|TEL)_SHARED_MEMORY are mapping 0x0 to 0x0 */ +#define TEH_SHARED_MEMORY \ + TLB_MK_TEH_ENTRY(0, 0, TLB_G_GLOBAL, 0) + +#define TEL_SHARED_MEMORY \ + TLB_MK_TEL_ENTRY(0, TLB_PS_2M, TLB_ES_A_MODIFIED,\ + TLB_CP_W_C, TLB_PA_NA_RWX) + +#define TEH_GDB_PAGE_MEMORY \ + TLB_MK_TEH_ENTRY(0, 0, TLB_G_GLOBAL, 0) + +#define TEL_GDB_PAGE_MEMORY \ + TLB_MK_TEL_ENTRY(0, TLB_PS_4K, TLB_ES_A_MODIFIED,\ + TLB_CP_U_U, TLB_PA_RWX_RWX) + +/** + * Macros + */ +.altmacro + +/* To select the JTLB we clear SB from MMC */ +.macro select_jtlb scratch_reg + make \scratch_reg, KVX_SFR_MMC_SB_MASK + ;; + wfxl $mmc, \scratch_reg +.endm + +/* To select the LTLB we set SB from MMC */ +.macro select_ltlb scratch_reg + make \scratch_reg, KVX_SFR_MMC_SB_MASK << 32 + ;; + wfxl $mmc, \scratch_reg +.endm + +/* Set SW of the MMC with number found in the reg register */ +.macro select_way_from_register reg scratch1 scratch2 + slld \scratch1 = \reg, KVX_SFR_MMC_SW_SHIFT + make \scratch2 = KVX_SFR_MMC_SW_MASK + ;; + slld \scratch1 = \scratch1, 32 + ;; + ord \scratch1 = \scratch1, \scratch2 + ;; + wfxl $mmc = \scratch1 +.endm + +/* Set SW of the MMC with the immediate */ +.macro select_way_from_immediate imm scratch1 scratch2 + make \scratch1 = (\imm << KVX_SFR_MMC_SW_SHIFT) << 32 + make \scratch2 = KVX_SFR_MMC_SW_MASK + ;; + ord \scratch1 = \scratch1, \scratch2 + ;; + wfxl $mmc = \scratch1 +.endm + +/* write tlb after setting teh and tel registers */ +.macro write_tlb_entry teh tel + set $teh = \teh + ;; + set $tel = \tel + ;; + tlbwrite +.endm + +/* Boot args */ +#define BOOT_ARGS_COUNT 2 +.align 16 +.section .boot.data, "aw", @progbits +rm_boot_args: +.skip BOOT_ARGS_COUNT * 8 + +/* + * This is our entry point. When entering from bootloader, + * the following registers are set: + * $r0 is a magic (LINUX_BOOT_PARAM_MAGIC) + * $r1 device tree pointer + * + * WARNING WARNING WARNING + * ! DO NOT CLOBBER THEM ! + * WARNING WARNING WARNING + * + * Try to use register above $r20 to ease parameter adding in future + */ + +__HEAD + +.align 8 +.section .boot.startup, "ax", @progbits + +ENTRY(kvx_start) + /* Setup 64 bit really early to avoid bugs */ + make $r21 = PS_VAL_WFXL(V64, 1) + ;; + wfxl $ps = $r21 + ;; + call asm_init_pl + ;; + get $r20 = $pcr + ;; + andd $r21 = $r20, RM_PID_MASK + ;; + cb.dnez $r21 ? asm_rm_cfg_pwr_ctrl + ;; +init_core: + /** + * Setup watchdog early to catch potential + * crash before watchdog driver probe + */ + make $r25 = PRIMARY_WATCHDOG_VALUE + make $r26 = TCR_WFXL_VALUE + ;; + set $wdv = $r25 + ;; + wfxl $tcr, $r26 + ;; + call asm_init_mmu + ;; + /* Setup default processor status */ + make $r25 = PS_WFXL_VALUE + make $r26 = PCR_WFXM_VALUE + ;; + /** + * There is nothing much we can do if we take a early trap since the + * kernel is not yet ready to handle them. + * Register this as the early exception handler to at least avoid + * going in a black hole. + */ + make $r27 = __early_exception_start + ;; + set $ev = $r27 + ;; + wfxm $pcr = $r26 + ;; + wfxl $ps = $r25 + ;; + /* Use as break point for debugging purpose. + See Documentation/kvx/kvx.txt for more details. */ +gdb_mmu_enabled: + /* Extract processor identifier */ + get $r24 = $pcr + ;; + extfz $r24 = $r24, KVX_SFR_END(PCR_PID), KVX_SFR_START(PCR_PID) + ;; + /* If proc 0, then go to clear bss and do normal boot */ + cb.deqz $r24? clear_bss + make $r25 = SECONDARY_START_ADDR + ;; + icall $r25 + ;; +clear_bss: + /* Copy bootloader arguments before cloberring them */ + copyd $r20 = $r0 + copyd $r21 = $r1 + ;; + /* Clear BSS */ + make $r0 = __bss_start + make $r1 = __bss_stop + call asm_memzero + ;; + /* Setup stack */ + make $r40 = init_thread_union + make $r41 = init_task + ;; + set $sr = $r41 + copyd $r0 = $r20 + copyd $r1 = $r21 + ;; + addd $sp = $r40, THREAD_SIZE + /* Clear frame pointer */ + make $fp = 0x0 + /* Setup the exception handler */ + make $r27 = __exception_start + ;; + set $ev = $r27 + /* Here we go ! start the C stuff */ + make $r20 = arch_low_level_start + ;; + icall $r20 + ;; + make $r20 = proc_power_off + ;; + igoto $r20 + ;; +ENDPROC(kvx_start) + +/** + * When PE 0 is started from the RM, arguments from the bootloaders are copied + * into rm_boot_args. It allows to give parameters from RM to PE. + * Note that the 4K alignment is required by the reset pc register... + */ +.align (4 * 1024) +ENTRY(pe_start_wrapper) + make $r0 = rm_boot_args + make $r27 = PWR_CTRL_ADDR + make $r28 = kvx_start + ;; + lq $r0r1 = 0[$r0] + ;; + /* Set reset PC back to original value for SMP start */ + sd KVX_PWR_CTRL_RESET_PC_OFFSET[$r27] = $r28 + ;; + fence + goto kvx_start + ;; +ENDPROC(pe_start_wrapper) + +/** + * asm_memzero - Clear a memory zone with zeroes + * $r0 is the start of memory zone (must be align on 32 bytes boundary) + * $r1 is the end of memory zone (must be align on 32 bytes boundary) + */ +ENTRY(asm_memzero) + sbfd $r32 = $r0, $r1 + make $r36 = 0 + make $r37 = 0 + ;; + make $r38 = 0 + make $r39 = 0 + /* Divide by 32 for hardware loop */ + srld $r32, $r32, 5 + ;; + /* Clear memory with hardware loop */ + loopdo $r32, clear_mem_done + ;; + so 0[$r0] = $r36r37r38r39 + addd $r0 = $r0, 32 + ;; + clear_mem_done: + ret + ;; +ENDPROC(asm_memzero) + +/** + * Configure the power controller to be accessible by PEs + */ +ENTRY(asm_rm_cfg_pwr_ctrl) + /* Enable hwloop for memzero */ + make $r32 = PS_VAL_WFXL(HLE, 1) + ;; + wfxl $ps = $r32 + ;; + make $r26 = PWR_CTRL_ADDR + make $r27 = PWR_CTRL_GLOBAL_CONFIG_VALUE + ;; + /* Set PE enable in power controller */ + sd PWR_CTRL_GLOBAL_CONFIG_OFFSET[$r26] = $r27 + make $r28 = rm_boot_args + ;; + /* Store parameters for PE0 */ + sq 0[$r28] = $r0r1 + make $r29 = pe_start_wrapper + ;; + /* Set PE reset PC to arguments wrapper */ + sd KVX_PWR_CTRL_RESET_PC_OFFSET[$r26] = $r29 + ;; + /* Fence to make sure parameters will be visible by PE 0 */ + fence + ;; + /* Start PE 0 (1 << cpu) */ + make $r27 = 1 + make $r26 = PWR_CTRL_ADDR + ;; + /* Wake up PE0 */ + sd PWR_CTRL_WUP_SET_OFFSET[$r26] = $r27 + ;; + /* And clear wakeup to allow PE0 to sleep */ + sd PWR_CTRL_WUP_CLEAR_OFFSET[$r26] = $r27 + ;; + make $r20 = proc_power_off + ;; + igoto $r20 + ;; +ENDPROC(asm_rm_cfg_pwr_ctrl) + +#define request_ownership(__pl) ;\ + make $r21 = SYO_WFXL_VALUE_##__pl ;\ + ;; ;\ + wfxl $syow = $r21 ;\ + ;; ;\ + make $r21 = HTO_WFXL_VALUE_##__pl ;\ + ;; ;\ + wfxl $htow = $r21 ;\ + ;; ;\ + make $r21 = MO_WFXL_VALUE_##__pl ;\ + make $r22 = MO_WFXM_VALUE_##__pl ;\ + ;; ;\ + wfxl $mow = $r21 ;\ + ;; ;\ + wfxm $mow = $r22 ;\ + ;; ;\ + make $r21 = ITO_WFXL_VALUE_##__pl ;\ + make $r22 = ITO_WFXM_VALUE_##__pl ;\ + ;; ;\ + wfxl $itow = $r21 ;\ + ;; ;\ + wfxm $itow = $r22 ;\ + ;; ;\ + make $r21 = PSO_WFXL_VALUE_##__pl ;\ + make $r22 = PSO_WFXM_VALUE_##__pl ;\ + ;; ;\ + wfxl $psow = $r21 ;\ + ;; ;\ + wfxm $psow = $r22 ;\ + ;; ;\ + make $r21 = DO_WFXL_VALUE_##__pl ;\ + ;; ;\ + wfxl $dow = $r21 ;\ + ;; + +/** + * Initialize privilege level for Kernel + */ +ENTRY(asm_init_pl) + get $r21 = $ps + ;; + /* Extract privilege level from $ps to check if we need to + * lower our privilege level + */ + extfz $r20 = $r21, KVX_SFR_END(PS_PL), KVX_SFR_START(PS_PL) + ;; + /* If our privilege level is 0, then we need to lower in execution level + * to ring 1 in order to let the debug routines be inserted at runtime + * by the JTAG. In both case, we will request the resources we need for + * linux to run. + */ + cb.deqz $r20? delegate_pl + ;; + /* + * When someone is already above us, request the resources we need to + * run the kernel. No need to request double exception or ECC traps for + * instance. When doing so, the more privileged level will trap for + * permission and delegate us the required resources. + */ + request_ownership(PL_CUR) + ;; + ret + ;; +delegate_pl: + request_ownership(PL_CUR_PLUS_1) + ;; + /* Copy our $ps into $sps for 1:1 restoration */ + get $r22 = $ps + ;; + /* We will return to $ra after rfe */ + get $r21 = $ra + /* Set privilege level to +1 is $sps */ + addd $r22 = $r22, PL_CUR_PLUS_1 + ;; + set $spc = $r21 + ;; + set $sps = $r22 + ;; + rfe + ;; +ENDPROC(asm_init_pl) + +/** + * Reset and initialize minimal tlb entries + */ +ENTRY(asm_init_mmu) + make $r20 = MMC_CLEAR_ERROR + ;; + wfxl $mmc = $r20 + ;; + /* Reset the JTLB */ + select_jtlb $r20 + ;; + make $r20 = (MMU_JTLB_SETS - 1) /* Used to select the set */ + make $r21 = 0 /* Used for shifting and as scratch register */ + ;; + set $tel = $r21 /* tel is always equal to 0 */ + ;; + clear_jtlb: + slld $r21 = $r20, KVX_SFR_TEH_PN_SHIFT + addd $r20 = $r20, -1 + ;; + set $teh = $r21 + ;; + make $r22 = (MMU_JTLB_WAYS - 1) /* Used to select the way */ + ;; + loop_jtlb_way: + select_way_from_register $r22 $r23 $r24 + ;; + tlbwrite + ;; + addd $r22 = $r22, -1 + ;; + cb.dgez $r22? loop_jtlb_way + ;; + /* loop_jtlb_way done */ + cb.dgez $r20? clear_jtlb + ;; + clear_jtlb_done: + /* Reset the LTLB */ + select_ltlb $r20 + ;; + clear_ltlb: + /* There is only one set that is 0 so we can reuse the same + values for TEH and TEL. */ + make $r20 = (MMU_LTLB_WAYS - 1) + ;; + loop_ltlb_way: + select_way_from_register $r20, $r21, $r22 + ;; + tlbwrite + ;; + addd $r20 = $r20, -1 + ;; + cb.dgez $r20? loop_ltlb_way + ;; + clear_ltlb_done: + + /* See Documentation/kvx/kvx.txt for details about the settings of + the LTLB */ + select_way_from_immediate LTLB_ENTRY_KERNEL_TEXT, $r20, $r21 + ;; + make $r20 = TEH_VIRTUAL_MEMORY + make $r21 = TEL_VIRTUAL_MEMORY + ;; + write_tlb_entry $r20, $r21 + ;; + select_way_from_immediate LTLB_ENTRY_EARLY_SMEM, $r20, $r21 + ;; + make $r20 = TEH_SHARED_MEMORY + make $r21 = TEL_SHARED_MEMORY + ;; + write_tlb_entry $r20, $r21 + ;; + select_way_from_immediate LTLB_ENTRY_GDB_PAGE, $r20, $r21 + ;; + make $r20 = _debug_start_lma + make $r21 = _debug_start + ;; + andd $r20 = $r20, KVX_SFR_TEH_PN_MASK + andd $r21 = $r21, KVX_SFR_TEL_FN_MASK + ;; + addd $r20 = $r20, TEH_GDB_PAGE_MEMORY + addd $r21 = $r21, TEL_GDB_PAGE_MEMORY + ;; + write_tlb_entry $r20, $r21 + ;; + ret + ;; +ENDPROC(asm_init_mmu) + +/** + * Entry point for secondary processors + * $r24 has been set in caller and is the proc id + */ +ENTRY(smp_secondary_start) +#ifdef CONFIG_SMP + dinval + ;; + iinval + ;; + barrier + ;; + make $r25 = __cpu_up_task_pointer + make $r26 = __cpu_up_stack_pointer + ;; + ld.xs $sp = $r24[$r26] + /* Clear frame pointer */ + make $fp = 0x0 + ;; + ld.xs $r25 = $r24[$r25] + ;; + set $sr = $r25 + make $r27 = __exception_start + ;; + set $ev = $r27 + make $r26 = start_kernel_secondary + ;; + icall $r26 + ;; +#endif +ENDPROC(smp_secondary_start) + +ENTRY(proc_power_off) + make $r1 = WS_WFXL_VALUE + ;; + /* Enable STOP */ + wfxl $ws, $r1 + ;; +1: stop + ;; + goto 1b + ;; +ENDPROC(proc_power_off) diff --git a/arch/kvx/kernel/prom.c b/arch/kvx/kernel/prom.c new file mode 100644 index 000000000000..a5241aa66903 --- /dev/null +++ b/arch/kvx/kernel/prom.c @@ -0,0 +1,24 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#include <linux/of_platform.h> +#include <linux/of_fdt.h> +#include <linux/printk.h> +#include <linux/init.h> + +void __init setup_device_tree(void) +{ + const char *name; + + name = of_flat_dt_get_machine_name(); + if (!name) + return; + + pr_info("Machine model: %s\n", name); + dump_stack_set_arch_desc("%s (DT)", name); + + unflatten_device_tree(); +} diff --git a/arch/kvx/kernel/reset.c b/arch/kvx/kernel/reset.c new file mode 100644 index 000000000000..afa0ceb9d7e9 --- /dev/null +++ b/arch/kvx/kernel/reset.c @@ -0,0 +1,37 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#include <linux/pm.h> +#include <linux/reboot.h> + +#include <asm/processor.h> + +static void kvx_default_power_off(void) +{ + smp_send_stop(); + local_cpu_stop(); +} + +void (*pm_power_off)(void) = kvx_default_power_off; +EXPORT_SYMBOL(pm_power_off); + +void machine_restart(char *cmd) +{ + smp_send_stop(); + do_kernel_restart(cmd); + pr_err("Reboot failed -- System halted\n"); + local_cpu_stop(); +} + +void machine_halt(void) +{ + pm_power_off(); +} + +void machine_power_off(void) +{ + pm_power_off(); +} diff --git a/arch/kvx/kernel/setup.c b/arch/kvx/kernel/setup.c new file mode 100644 index 000000000000..e155341a37fd --- /dev/null +++ b/arch/kvx/kernel/setup.c @@ -0,0 +1,177 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include <linux/start_kernel.h> +#include <linux/screen_info.h> +#include <linux/console.h> +#include <linux/linkage.h> +#include <linux/export.h> +#include <linux/of_fdt.h> +#include <linux/kernel.h> +#include <linux/string.h> +#include <linux/sched.h> +#include <linux/init.h> +#include <linux/smp.h> + +#include <asm/processor.h> +#include <asm/sections.h> +#include <asm/hw_irq.h> +#include <asm/setup.h> +#include <asm/rm_fw.h> +#include <asm/page.h> +#include <asm/sfr.h> +#include <asm/mmu.h> +#include <asm/smp.h> + +struct screen_info screen_info; + +unsigned long memory_start; +EXPORT_SYMBOL(memory_start); +unsigned long memory_end; +EXPORT_SYMBOL(memory_end); + +DEFINE_PER_CPU_READ_MOSTLY(struct cpuinfo_kvx, cpu_info); +EXPORT_PER_CPU_SYMBOL(cpu_info); + +static bool use_streaming = true; +static int __init parse_kvx_streaming(char *arg) +{ + strtobool(arg, &use_streaming); + + if (!use_streaming) { + pr_info("disabling streaming\n"); + kvx_sfr_set_field(PS, USE, 0); + } + + return 0; +} +early_param("kvx.streaming", parse_kvx_streaming); + +static void __init setup_user_privilege(void) +{ + /* + * We want to let the user control various fields of ps: + * - hardware loop + * - instruction cache enable + * - streaming enable + */ + uint64_t mask = KVX_SFR_PSOW_HLE_MASK | + KVX_SFR_PSOW_ICE_MASK | + KVX_SFR_PSOW_USE_MASK; + + uint64_t value = (1 << KVX_SFR_PSOW_HLE_SHIFT) | + (1 << KVX_SFR_PSOW_ICE_SHIFT) | + (1 << KVX_SFR_PSOW_USE_SHIFT); + + kvx_sfr_set_mask(PSOW, mask, value); +} + +void __init setup_cpuinfo(void) +{ + struct cpuinfo_kvx *n = this_cpu_ptr(&cpu_info); + u64 pcr = kvx_sfr_get(PCR); + + n->copro_enable = kvx_sfr_field_val(pcr, PCR, COE); + n->arch_rev = kvx_sfr_field_val(pcr, PCR, CAR); + n->uarch_rev = kvx_sfr_field_val(pcr, PCR, CMA); +} + +/* + * Everything that needs to be setup PER cpu should be put here. + * This function will be called by per-cpu setup routine. + */ +void __init setup_processor(void) +{ + /* Clear performance monitor 0 */ + kvx_sfr_set_field(PMC, PM0C, 0); + +#ifdef CONFIG_ENABLE_TCA + /* Enable TCA (COE = Coprocessor Enable) */ + kvx_sfr_set_field(PCR, COE, 1); +#else + kvx_sfr_set_field(PCR, COE, 0); +#endif + + /* + * On kvx, we have speculative accesses which differ from normal + * accesses by the fact their trapping policy is directed by mmc.sne + * (speculative no-mapping enable) and mmc.spe (speculative protection + * enabled). + * To handle these accesses properly, we disable all traps on + * speculative accesses while in kernel and user (sne & spe) + * in order to silently discard data if fetched. + * This allows to do an effective prefetch. + */ + kvx_sfr_set_field(MMC, SNE, 0); + kvx_sfr_set_field(MMC, SPE, 0); + + if (!use_streaming) + kvx_sfr_set_field(PS, USE, 0); + + kvx_init_core_irq(); + + setup_user_privilege(); + + setup_cpuinfo(); +} + +static char builtin_cmdline[COMMAND_LINE_SIZE] __initdata = CONFIG_CMDLINE; + +void __init setup_arch(char **cmdline_p) +{ + if (builtin_cmdline[0]) { + /* append boot loader cmdline to builtin */ + strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE); + strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE); + strscpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE); + } + + *cmdline_p = boot_command_line; + + setup_processor(); + + /* Jump labels needs fixmap to be setup for text modifications */ + early_fixmap_init(); + + /* Parameters might set static keys */ + jump_label_init(); + /* + * Parse early param after setting up arch memory since + * we need fixmap for earlycon and fixedmap need to do + * memory allocation (fixed_range_init). + */ + parse_early_param(); + + setup_arch_memory(); + + paging_init(); + + setup_device_tree(); + + smp_init_cpus(); + +#ifdef CONFIG_VT + conswitchp = &dummy_con; +#endif +} + +asmlinkage __visible void __init arch_low_level_start(unsigned long r0, + void *dtb_ptr) +{ + void *dt = __dtb_start; + + kvx_mmu_early_setup(); + + if (r0 == LINUX_BOOT_PARAM_MAGIC) + dt = __va(dtb_ptr); + + if (!early_init_dt_scan(dt)) + panic("Missing device tree\n"); + + start_kernel(); +} diff --git a/arch/kvx/kernel/time.c b/arch/kvx/kernel/time.c new file mode 100644 index 000000000000..f27103a1a6f4 --- /dev/null +++ b/arch/kvx/kernel/time.c @@ -0,0 +1,242 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Yann Sionneau + * Guillaume Thouvenin + * Luc Michel + * Julian Vetter + */ + +#include <linux/of.h> +#include <linux/clk.h> +#include <linux/init.h> +#include <linux/of_irq.h> +#include <linux/interrupt.h> +#include <linux/cpuhotplug.h> +#include <linux/clockchips.h> +#include <linux/clocksource.h> +#include <linux/clk-provider.h> +#include <linux/of_address.h> +#include <linux/sched_clock.h> + +#include <asm/sfr_defs.h> + +#define KVX_TIMER_MIN_DELTA 1 +#define KVX_TIMER_MAX_DELTA 0xFFFFFFFFFFFFFFFFULL +#define KVX_TIMER_MAX_VALUE 0xFFFFFFFFFFFFFFFFULL + +/* + * Clockevent + */ +static unsigned int kvx_timer_frequency; +static unsigned int kvx_periodic_timer_value; +static unsigned int kvx_timer_irq; + +static void kvx_timer_set_value(unsigned long value, unsigned long reload_value) +{ + kvx_sfr_set(T0R, reload_value); + kvx_sfr_set(T0V, value); + /* Enable timer */ + kvx_sfr_set_field(TCR, T0CE, 1); +} + +static int kvx_clkevent_set_next_event(unsigned long cycles, + struct clock_event_device *dev) +{ + /* + * Hardware does not support oneshot mode. + * In order to support it, set a really high reload value. + * Then, during the interrupt handler, disable the timer if + * in oneshot mode + */ + kvx_timer_set_value(cycles - 1, KVX_TIMER_MAX_VALUE); + + return 0; +} + +/* + * Configure the rtc to periodically tick HZ times per second + */ +static int kvx_clkevent_set_state_periodic(struct clock_event_device *dev) +{ + kvx_timer_set_value(kvx_periodic_timer_value, + kvx_periodic_timer_value); + + return 0; +} + +static int kvx_clkevent_set_state_oneshot(struct clock_event_device *dev) +{ + /* Same as for kvx_clkevent_set_next_event */ + kvx_clkevent_set_next_event(kvx_periodic_timer_value, dev); + + return 0; +} + +static int kvx_clkevent_set_state_shutdown(struct clock_event_device *dev) +{ + kvx_sfr_set_field(TCR, T0CE, 0); + + return 0; +} + +static DEFINE_PER_CPU(struct clock_event_device, kvx_clockevent_device) = { + .name = "kvx-timer-0", + .features = CLOCK_EVT_FEAT_ONESHOT | CLOCK_EVT_FEAT_PERIODIC, + /* arbitrary rating for this clockevent */ + .rating = 300, + .set_next_event = kvx_clkevent_set_next_event, + .set_state_periodic = kvx_clkevent_set_state_periodic, + .set_state_oneshot = kvx_clkevent_set_state_oneshot, + .set_state_shutdown = kvx_clkevent_set_state_shutdown, +}; + +irqreturn_t kvx_timer_irq_handler(int irq, void *dev_id) +{ + struct clock_event_device *evt = this_cpu_ptr(&kvx_clockevent_device); + + /* Disable timer if in oneshot mode before reloading */ + if (likely(clockevent_state_oneshot(evt))) + kvx_sfr_set_field(TCR, T0CE, 0); + + evt->event_handler(evt); + + return IRQ_HANDLED; +} + +static int kvx_timer_starting_cpu(unsigned int cpu) +{ + struct clock_event_device *evt = this_cpu_ptr(&kvx_clockevent_device); + + evt->cpumask = cpumask_of(cpu); + evt->irq = kvx_timer_irq; + + clockevents_config_and_register(evt, kvx_timer_frequency, + KVX_TIMER_MIN_DELTA, + KVX_TIMER_MAX_DELTA); + + /* Enable timer interrupt */ + kvx_sfr_set_field(TCR, T0IE, 1); + + enable_percpu_irq(kvx_timer_irq, IRQ_TYPE_NONE); + + return 0; +} + +static int kvx_timer_dying_cpu(unsigned int cpu) +{ + disable_percpu_irq(kvx_timer_irq); + + return 0; +} + +static int __init kvx_setup_core_timer(struct device_node *np) +{ + struct clock_event_device *evt = this_cpu_ptr(&kvx_clockevent_device); + struct clk *clk; + int err; + + clk = of_clk_get(np, 0); + if (IS_ERR(clk)) { + pr_err("kvx_core_timer: Failed to get CPU clock: %ld\n", + PTR_ERR(clk)); + return 1; + } + + kvx_timer_frequency = clk_get_rate(clk); + clk_put(clk); + kvx_periodic_timer_value = kvx_timer_frequency / HZ; + + kvx_timer_irq = irq_of_parse_and_map(np, 0); + if (!kvx_timer_irq) { + pr_err("kvx_core_timer: Failed to parse irq: %d\n", + kvx_timer_irq); + return -EINVAL; + } + + err = request_percpu_irq(kvx_timer_irq, kvx_timer_irq_handler, + "kvx_core_timer", evt); + if (err) { + pr_err("kvx_core_timer: can't register interrupt %d (%d)\n", + kvx_timer_irq, err); + return err; + } + + err = cpuhp_setup_state(CPUHP_AP_KVX_TIMER_STARTING, + "kvx/time:online", + kvx_timer_starting_cpu, + kvx_timer_dying_cpu); + if (err < 0) { + pr_err("kvx_core_timer: Failed to setup hotplug state"); + return err; + } + + return 0; +} + +TIMER_OF_DECLARE(kvx_core_timer, "kalray,kvx-core-timer", + kvx_setup_core_timer); + +/* + * Clocksource + */ +static u64 kvx_dsu_clocksource_read(struct clocksource *cs) +{ + return readq(cs->archdata.regs); +} + +static struct clocksource kvx_dsu_clocksource = { + .name = "kvx-dsu-clock", + .rating = 400, + .read = kvx_dsu_clocksource_read, + .mask = CLOCKSOURCE_MASK(64), + .flags = CLOCK_SOURCE_IS_CONTINUOUS, +}; + +static u64 notrace kvx_dsu_sched_read(void) +{ + return readq_relaxed(kvx_dsu_clocksource.archdata.regs); +} + +static int __init kvx_setup_dsu_clock(struct device_node *np) +{ + int ret; + struct clk *clk; + unsigned long kvx_dsu_frequency; + + kvx_dsu_clocksource.archdata.regs = of_iomap(np, 0); + + WARN_ON(!kvx_dsu_clocksource.archdata.regs); + if (!kvx_dsu_clocksource.archdata.regs) + return -ENXIO; + + clk = of_clk_get(np, 0); + if (IS_ERR(clk)) { + pr_err("Failed to get CPU clock: %ld\n", PTR_ERR(clk)); + return PTR_ERR(clk); + } + + kvx_dsu_frequency = clk_get_rate(clk); + clk_put(clk); + + ret = clocksource_register_hz(&kvx_dsu_clocksource, + kvx_dsu_frequency); + if (ret) { + pr_err("failed to register dsu clocksource"); + return ret; + } + + sched_clock_register(kvx_dsu_sched_read, 64, kvx_dsu_frequency); + return 0; +} + +TIMER_OF_DECLARE(kvx_dsu_clock, "kalray,kvx-dsu-clock", + kvx_setup_dsu_clock); + +void __init time_init(void) +{ + of_clk_init(NULL); + + timer_probe(); +} -- 2.37.2

Each Cluster of the Coolidge SoC includes an Advanced Programmable Interrupt Controller (APIC) and Generic Interrupt Controller (GIC). The APIC GIC acts as an intermediary interrupt controller, muxing/routing incoming interrupts to cores in the cluster. The 139 possible input interrupt lines are organized as follow: - 128 from the mailbox controller (one it per mailboxes) - 1 from the NoC router - 5 from IOMMUs - 1 from L2 cache DMA job FIFO - 1 from cluster watchdog - 2 for SECC, DECC - 1 from Data NoC The 72 possible output interrupt line: - 68 : 4 interrupts per cores (17 cores) - 1 for L2 cache controller - 3 extra that are for padding Co-developed-by: Clement Leger <clement(a)clement-leger.fr&gt; Signed-off-by: Clement Leger <clement(a)clement-leger.fr&gt; Co-developed-by: Julian Vetter <jvetter(a)kalray.eu&gt; Signed-off-by: Julian Vetter <jvetter(a)kalray.eu&gt; Co-developed-by: Vincent Chardon <vincent.chardon(a)elsys-design.com&gt; Signed-off-by: Vincent Chardon <vincent.chardon(a)elsys-design.com&gt; Signed-off-by: Jules Maselbas <jmaselbas(a)kalray.eu&gt; Signed-off-by: Yann Sionneau <ysionneau(a)kalray.eu&gt; --- Notes: V1 -> V2: - removed irq-kvx-itgen driver (moved in its own patch) - removed irq-kvx-apic-mailbox driver (moved in its own patch) - removed irq-kvx-core-intc driver (moved in its own patch) - removed print on probe success drivers/irqchip/Kconfig | 6 + drivers/irqchip/Makefile | 1 + drivers/irqchip/irq-kvx-apic-gic.c | 356 +++++++++++++++++++++++++++++ 3 files changed, 363 insertions(+) create mode 100644 drivers/irqchip/irq-kvx-apic-gic.c diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig index 7ef9f5e696d3..2433e4ba0759 100644 --- a/drivers/irqchip/Kconfig +++ b/drivers/irqchip/Kconfig @@ -334,6 +334,12 @@ config MIPS_GIC select IRQ_DOMAIN_HIERARCHY select MIPS_CM +config KVX_APIC_GIC + bool + depends on KVX + select IRQ_DOMAIN + select IRQ_DOMAIN_HIERARCHY + config INGENIC_IRQ bool depends on MACH_INGENIC diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile index 87b49a10962c..8ac1dd880420 100644 --- a/drivers/irqchip/Makefile +++ b/drivers/irqchip/Makefile @@ -69,6 +69,7 @@ obj-$(CONFIG_BCM7120_L2_IRQ) += irq-bcm7120-l2.o obj-$(CONFIG_BRCMSTB_L2_IRQ) += irq-brcmstb-l2.o obj-$(CONFIG_KEYSTONE_IRQ) += irq-keystone.o obj-$(CONFIG_MIPS_GIC) += irq-mips-gic.o +obj-$(CONFIG_KVX_APIC_GIC) += irq-kvx-apic-gic.o obj-$(CONFIG_ARCH_MEDIATEK) += irq-mtk-sysirq.o irq-mtk-cirq.o obj-$(CONFIG_ARCH_DIGICOLOR) += irq-digicolor.o obj-$(CONFIG_ARCH_SA1100) += irq-sa11x0.o diff --git a/drivers/irqchip/irq-kvx-apic-gic.c b/drivers/irqchip/irq-kvx-apic-gic.c new file mode 100644 index 000000000000..cc234a075473 --- /dev/null +++ b/drivers/irqchip/irq-kvx-apic-gic.c @@ -0,0 +1,356 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2017 - 2022 Kalray Inc. + * Author(s): Clement Leger + * Julian Vetter + */ + +#define pr_fmt(fmt) "kvx_apic_gic: " fmt + +#include <linux/of_address.h> +#include <linux/cpuhotplug.h> +#include <linux/interrupt.h> +#include <linux/irqdomain.h> +#include <linux/spinlock.h> +#include <linux/irqchip.h> +#include <linux/of_irq.h> +#include <linux/module.h> +#include <linux/init.h> +#include <linux/of.h> + +/* APIC is organized in 18 groups of 4 output lines + * However, the two upper lines are for Secure RM and DMA engine + * Thus, we do not have to use them + */ +#define KVX_GIC_PER_CPU_IT_COUNT 4 +#define KVX_GIC_INPUT_IT_COUNT 0x9D +#define KVX_GIC_OUTPUT_IT_COUNT 0x10 + +/* GIC enable register definitions */ +#define KVX_GIC_ENABLE_OFFSET 0x0 +#define KVX_GIC_ENABLE_ELEM_SIZE 0x1 +#define KVX_GIC_ELEM_SIZE 0x400 + +/* GIC status lac register definitions */ +#define KVX_GIC_STATUS_LAC_OFFSET 0x120 +#define KVX_GIC_STATUS_LAC_ELEM_SIZE 0x8 +#define KVX_GIC_STATUS_LAC_ARRAY_SIZE 0x3 + +/** + * For each CPU, there is 4 output lines coming from the apic GIC. + * We only use 1 line and this structure represent this line. + * @base Output line base address + * @cpu CPU associated to this line + */ +struct gic_out_irq_line { + void __iomem *base; + unsigned int cpu; +}; + +/** + * Input irq line. + * This structure is used to store the status of the input line and the + * associated output line. + * @enabled Boolean for line status + * @cpu CPU currently receiving this interrupt + * @it_num Interrupt number + */ +struct gic_in_irq_line { + bool enabled; + struct gic_out_irq_line *out_line; + unsigned int it_num; +}; + +/** + * struct kvx_apic_gic - kvx apic gic + * @base: Base address of the controller + * @domain Domain for this controller + * @input_nr_irqs: maximum number of supported input interrupts + * @cpus: Per cpu interrupt configuration + * @output_irq: Array of output irq lines + * @input_irq: Array of input irq lines + */ +struct kvx_apic_gic { + raw_spinlock_t lock; + void __iomem *base; + struct irq_domain *domain; + uint32_t input_nr_irqs; + /* For each cpu, there is an output IT line */ + struct gic_out_irq_line output_irq[KVX_GIC_OUTPUT_IT_COUNT]; + /* Input interrupt status */ + struct gic_in_irq_line input_irq[KVX_GIC_INPUT_IT_COUNT]; +}; + +static int gic_parent_irq; + +/** + * Enable/Disable an output irq line + * This function is used by both mask/unmask to disable/enable the line. + */ +static void irq_line_set_enable(struct gic_out_irq_line *irq_line, + struct gic_in_irq_line *in_irq_line, + int enable) +{ + void __iomem *enable_line_addr = irq_line->base + + KVX_GIC_ENABLE_OFFSET + + in_irq_line->it_num * KVX_GIC_ENABLE_ELEM_SIZE; + + writeb((uint8_t) enable ? 1 : 0, enable_line_addr); + in_irq_line->enabled = enable; +} + +static void kvx_apic_gic_set_line(struct irq_data *data, int enable) +{ + struct kvx_apic_gic *gic = irq_data_get_irq_chip_data(data); + unsigned int in_irq = irqd_to_hwirq(data); + struct gic_in_irq_line *in_line = &gic->input_irq[in_irq]; + struct gic_out_irq_line *out_line = in_line->out_line; + + raw_spin_lock(&gic->lock); + /* Set line enable on currently assigned cpu */ + irq_line_set_enable(out_line, in_line, enable); + raw_spin_unlock(&gic->lock); +} + +static void kvx_apic_gic_mask(struct irq_data *data) +{ + kvx_apic_gic_set_line(data, 0); +} + +static void kvx_apic_gic_unmask(struct irq_data *data) +{ + kvx_apic_gic_set_line(data, 1); +} + +#ifdef CONFIG_SMP + +static int kvx_apic_gic_set_affinity(struct irq_data *d, + const struct cpumask *cpumask, + bool force) +{ + struct kvx_apic_gic *gic = irq_data_get_irq_chip_data(d); + unsigned int new_cpu; + unsigned int hw_irq = irqd_to_hwirq(d); + struct gic_in_irq_line *input_line = &gic->input_irq[hw_irq]; + struct gic_out_irq_line *new_out_line; + + /* We assume there is only one cpu in the mask */ + new_cpu = cpumask_first(cpumask); + new_out_line = &gic->output_irq[new_cpu]; + + raw_spin_lock(&gic->lock); + + /* Nothing to do, line is the same */ + if (new_out_line == input_line->out_line) + goto out; + + /* If old line was enabled, enable the new one before disabling + * the old one + */ + if (input_line->enabled) + irq_line_set_enable(new_out_line, input_line, 1); + + /* Disable it on old line */ + irq_line_set_enable(input_line->out_line, input_line, 0); + + /* Assign new output line to input IRQ */ + input_line->out_line = new_out_line; + +out: + raw_spin_unlock(&gic->lock); + + irq_data_update_effective_affinity(d, cpumask_of(new_cpu)); + + return IRQ_SET_MASK_OK; +} +#endif + +static struct irq_chip kvx_apic_gic_chip = { + .name = "kvx apic gic", + .irq_mask = kvx_apic_gic_mask, + .irq_unmask = kvx_apic_gic_unmask, +#ifdef CONFIG_SMP + .irq_set_affinity = kvx_apic_gic_set_affinity, +#endif +}; + +static int kvx_apic_gic_alloc(struct irq_domain *domain, unsigned int virq, + unsigned int nr_irqs, void *args) +{ + int i; + struct irq_fwspec *fwspec = args; + int hwirq = fwspec->param[0]; + + for (i = 0; i < nr_irqs; i++) { + irq_domain_set_info(domain, virq + i, hwirq + i, + &kvx_apic_gic_chip, + domain->host_data, handle_simple_irq, + NULL, NULL); + } + + return 0; +} + +static const struct irq_domain_ops kvx_apic_gic_domain_ops = { + .alloc = kvx_apic_gic_alloc, + .free = irq_domain_free_irqs_common, +}; + +static void irq_line_get_status_lac(struct gic_out_irq_line *out_irq_line, + uint64_t status[KVX_GIC_STATUS_LAC_ARRAY_SIZE]) +{ + int i; + + for (i = 0; i < KVX_GIC_STATUS_LAC_ARRAY_SIZE; i++) { + status[i] = readq(out_irq_line->base + + KVX_GIC_STATUS_LAC_OFFSET + + i * KVX_GIC_STATUS_LAC_ELEM_SIZE); + } +} + +static void kvx_apic_gic_handle_irq(struct irq_desc *desc) +{ + struct kvx_apic_gic *gic_data = irq_desc_get_handler_data(desc); + struct gic_out_irq_line *out_line; + uint64_t status[KVX_GIC_STATUS_LAC_ARRAY_SIZE]; + unsigned long irqn, cascade_irq; + unsigned long cpu = smp_processor_id(); + + out_line = &gic_data->output_irq[cpu]; + + irq_line_get_status_lac(out_line, status); + + for_each_set_bit(irqn, (unsigned long *) status, + KVX_GIC_STATUS_LAC_ARRAY_SIZE * BITS_PER_LONG) { + + cascade_irq = irq_find_mapping(gic_data->domain, irqn); + + generic_handle_irq(cascade_irq); + } +} + +static void __init apic_gic_init(struct kvx_apic_gic *gic) +{ + unsigned int cpu, line; + struct gic_in_irq_line *input_irq_line; + struct gic_out_irq_line *output_irq_line; + uint64_t status[KVX_GIC_STATUS_LAC_ARRAY_SIZE]; + + /* Initialize all input lines (device -> )*/ + for (line = 0; line < KVX_GIC_INPUT_IT_COUNT; line++) { + input_irq_line = &gic->input_irq[line]; + input_irq_line->enabled = false; + /* All input lines map on output 0 */ + input_irq_line->out_line = &gic->output_irq[0]; + input_irq_line->it_num = line; + } + + /* Clear all output lines (-> cpus) */ + for (cpu = 0; cpu < KVX_GIC_OUTPUT_IT_COUNT; cpu++) { + output_irq_line = &gic->output_irq[cpu]; + output_irq_line->cpu = cpu; + output_irq_line->base = gic->base + + cpu * (KVX_GIC_ELEM_SIZE * KVX_GIC_PER_CPU_IT_COUNT); + + /* Disable all external lines on this core */ + for (line = 0; line < KVX_GIC_INPUT_IT_COUNT; line++) + irq_line_set_enable(output_irq_line, + &gic->input_irq[line], 0x0); + + irq_line_get_status_lac(output_irq_line, status); + } +} + +static int kvx_gic_starting_cpu(unsigned int cpu) +{ + enable_percpu_irq(gic_parent_irq, IRQ_TYPE_NONE); + + return 0; +} + +static int kvx_gic_dying_cpu(unsigned int cpu) +{ + disable_percpu_irq(gic_parent_irq); + + return 0; +} + +static int __init kvx_init_apic_gic(struct device_node *node, + struct device_node *parent) +{ + struct kvx_apic_gic *gic; + int ret; + unsigned int irq; + + if (!parent) { + pr_err("kvx apic gic does not have parent\n"); + return -EINVAL; + } + + gic = kzalloc(sizeof(*gic), GFP_KERNEL); + if (!gic) + return -ENOMEM; + + if (of_property_read_u32(node, "kalray,intc-nr-irqs", + &gic->input_nr_irqs)) + gic->input_nr_irqs = KVX_GIC_INPUT_IT_COUNT; + + if (WARN_ON(gic->input_nr_irqs > KVX_GIC_INPUT_IT_COUNT)) { + ret = -EINVAL; + goto err_kfree; + } + + gic->base = of_io_request_and_map(node, 0, node->name); + if (!gic->base) { + ret = -EINVAL; + goto err_kfree; + } + + raw_spin_lock_init(&gic->lock); + apic_gic_init(gic); + + gic->domain = irq_domain_add_linear(node, + gic->input_nr_irqs, + &kvx_apic_gic_domain_ops, + gic); + if (!gic->domain) { + pr_err("Failed to add IRQ domain\n"); + ret = -EINVAL; + goto err_iounmap; + } + + irq = irq_of_parse_and_map(node, 0); + if (irq <= 0) { + pr_err("unable to parse irq\n"); + ret = -EINVAL; + goto err_irq_domain_remove; + } + + irq_set_chained_handler_and_data(irq, kvx_apic_gic_handle_irq, + gic); + + gic_parent_irq = irq; + ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, + "kvx/gic:online", + kvx_gic_starting_cpu, + kvx_gic_dying_cpu); + if (ret < 0) { + pr_err("Failed to setup hotplug state"); + goto err_irq_unmap; + } + + return 0; + +err_irq_unmap: + irq_dispose_mapping(irq); +err_irq_domain_remove: + irq_domain_remove(gic->domain); +err_iounmap: + iounmap(gic->base); +err_kfree: + kfree(gic); + + return ret; +} + +IRQCHIP_DECLARE(kvx_apic_gic, "kalray,kvx-apic-gic", kvx_init_apic_gic); -- 2.37.2

From: Jules Maselbas <jmaselbas(a)kalray.eu&gt; The APIC includes a mailbox controller, containing 128 mailboxes. Each mailbox is a word of 8 bytes in the controller internal memory. Each mailbox can be independently configured with a trigger condition and an input function. After a write to a mailbox if the mailbox's trigger condition is met then an interrupt will be generated. Since this hardware block generates IRQs based on writes at some memory locations, it is both an interrupt controller and an MSI controller. Co-developed-by: Clement Leger <clement(a)clement-leger.fr&gt; Signed-off-by: Clement Leger <clement(a)clement-leger.fr&gt; Co-developed-by: Jules Maselbas <jmaselbas(a)kalray.eu&gt; Signed-off-by: Jules Maselbas <jmaselbas(a)kalray.eu&gt; Co-developed-by: Julian Vetter <jvetter(a)kalray.eu&gt; Signed-off-by: Julian Vetter <jvetter(a)kalray.eu&gt; Co-developed-by: Luc Michel <lmichel(a)kalray.eu&gt; Signed-off-by: Luc Michel <lmichel(a)kalray.eu&gt; Co-developed-by: Yann Sionneau <ysionneau(a)kalray.eu&gt; Signed-off-by: Yann Sionneau <ysionneau(a)kalray.eu&gt; --- Notes: V1 -> V2: new patch - removed print on probe success drivers/irqchip/Kconfig | 8 + drivers/irqchip/Makefile | 1 + drivers/irqchip/irq-kvx-apic-mailbox.c | 480 +++++++++++++++++++++++++ 3 files changed, 489 insertions(+) create mode 100644 drivers/irqchip/irq-kvx-apic-mailbox.c diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig index 546bc611f3f3..806adbc7b2a4 100644 --- a/drivers/irqchip/Kconfig +++ b/drivers/irqchip/Kconfig @@ -348,6 +348,14 @@ config KVX_ITGEN select IRQ_DOMAIN select IRQ_DOMAIN_HIERARCHY +config KVX_APIC_MAILBOX + bool + depends on KVX + select GENERIC_IRQ_IPI if SMP + select GENERIC_MSI_IRQ_DOMAIN + select IRQ_DOMAIN + select IRQ_DOMAIN_HIERARCHY + config INGENIC_IRQ bool depends on MACH_INGENIC diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile index 6b8f459d8a21..7eaea87ca9ab 100644 --- a/drivers/irqchip/Makefile +++ b/drivers/irqchip/Makefile @@ -71,6 +71,7 @@ obj-$(CONFIG_KEYSTONE_IRQ) += irq-keystone.o obj-$(CONFIG_MIPS_GIC) += irq-mips-gic.o obj-$(CONFIG_KVX_APIC_GIC) += irq-kvx-apic-gic.o obj-$(CONFIG_KVX_ITGEN) += irq-kvx-itgen.o +obj-$(CONFIG_KVX_APIC_MAILBOX) += irq-kvx-apic-mailbox.o obj-$(CONFIG_ARCH_MEDIATEK) += irq-mtk-sysirq.o irq-mtk-cirq.o obj-$(CONFIG_ARCH_DIGICOLOR) += irq-digicolor.o obj-$(CONFIG_ARCH_SA1100) += irq-sa11x0.o diff --git a/drivers/irqchip/irq-kvx-apic-mailbox.c b/drivers/irqchip/irq-kvx-apic-mailbox.c new file mode 100644 index 000000000000..33249df047b6 --- /dev/null +++ b/drivers/irqchip/irq-kvx-apic-mailbox.c @@ -0,0 +1,480 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Jules Maselbas + */ + +#define pr_fmt(fmt) "kvx_apic_mailbox: " fmt + +#include <linux/irqchip/chained_irq.h> +#include <linux/of_address.h> +#include <linux/interrupt.h> +#include <linux/irqdomain.h> +#include <linux/iommu.h> +#include <linux/irqchip.h> +#include <linux/module.h> +#include <linux/of_irq.h> +#include <linux/init.h> +#include <linux/msi.h> +#include <linux/of.h> + +#define KVX_MAILBOX_MODE_WRITE 0x0 +#define KVX_MAILBOX_MODE_OR 0x1 +#define KVX_MAILBOX_MODE_ADD 0x2 + +#define KVX_MAILBOX_TRIG_NO_TRIG 0x0 +#define KVX_MAILBOX_TRIG_DOORBELL 0x1 +#define KVX_MAILBOX_TRIG_MATCH 0x2 +#define KVX_MAILBOX_TRIG_BARRIER 0x3 +#define KVX_MAILBOX_TRIG_THRESHOLD 0x4 + +#define KVX_MAILBOX_OFFSET 0x0 +#define KVX_MAILBOX_ELEM_SIZE 0x200 +#define KVX_MAILBOX_MASK_OFFSET 0x10 +#define KVX_MAILBOX_FUNCT_OFFSET 0x18 +#define KVX_MAILBOX_LAC_OFFSET 0x8 +#define KVX_MAILBOX_VALUE_OFFSET 0x0 +#define KVX_MAILBOX_FUNCT_MODE_SHIFT 0x0 +#define KVX_MAILBOX_FUNCT_TRIG_SHIFT 0x8 + +#define MAILBOXES_MAX_COUNT 128 + +/* Mailboxes are 64 bits wide */ +#define MAILBOXES_BIT_SIZE 64 + +/* Maximum number of mailboxes available */ +#define MAILBOXES_MAX_BIT_COUNT (MAILBOXES_MAX_COUNT * MAILBOXES_BIT_SIZE) + +/* Mailboxes are grouped by 8 in a single page */ +#define MAILBOXES_BITS_PER_PAGE (8 * MAILBOXES_BIT_SIZE) + +/** + * struct mb_data - per mailbox data + * @cpu: CPU on which the mailbox is routed + * @parent_irq: Parent IRQ on the GIC + */ +struct mb_data { + unsigned int cpu; + unsigned int parent_irq; +}; + +/** + * struct kvx_apic_mailbox - kvx apic mailbox + * @base: base address of the controller + * @device_domain: IRQ device domain for mailboxes + * @msi_domain: platform MSI domain for MSI interface + * @domain_info: Domain information needed for the MSI domain + * @mb_count: Count of mailboxes we are handling + * @available: bitmap of availables bits in mailboxes + * @mailboxes_lock: lock for irq migration + * @mask_lock: lock for irq masking + * @mb_data: data associated to each mailbox + */ +struct kvx_apic_mailbox { + void __iomem *base; + phys_addr_t phys_base; + struct irq_domain *device_domain; + struct irq_domain *msi_domain; + struct msi_domain_info domain_info; + /* Start and count of device mailboxes */ + unsigned int mb_count; + /* Bitmap of allocated bits in mailboxes */ + DECLARE_BITMAP(available, MAILBOXES_MAX_BIT_COUNT); + spinlock_t mailboxes_lock; + raw_spinlock_t mask_lock; + struct mb_data mb_data[MAILBOXES_MAX_COUNT]; +}; + +/** + * struct kvx_irq_data - per irq data + * @mb: Mailbox structure + */ +struct kvx_irq_data { + struct kvx_apic_mailbox *mb; +}; + +static void kvx_mailbox_get_from_hwirq(unsigned int hw_irq, + unsigned int *mailbox_num, + unsigned int *mailbox_bit) +{ + *mailbox_num = hw_irq / MAILBOXES_BIT_SIZE; + *mailbox_bit = hw_irq % MAILBOXES_BIT_SIZE; +} + +static void __iomem *kvx_mailbox_get_addr(struct kvx_apic_mailbox *mb, + unsigned int num) +{ + return mb->base + (num * KVX_MAILBOX_ELEM_SIZE); +} + +static phys_addr_t kvx_mailbox_get_phys_addr(struct kvx_apic_mailbox *mb, + unsigned int num) +{ + return mb->phys_base + (num * KVX_MAILBOX_ELEM_SIZE); +} + +static void kvx_mailbox_msi_compose_msg(struct irq_data *data, + struct msi_msg *msg) +{ + struct kvx_irq_data *kd = irq_data_get_irq_chip_data(data); + struct kvx_apic_mailbox *mb = kd->mb; + unsigned int mb_num, mb_bit; + phys_addr_t mb_addr; + + kvx_mailbox_get_from_hwirq(irqd_to_hwirq(data), &mb_num, &mb_bit); + mb_addr = kvx_mailbox_get_phys_addr(mb, mb_num); + + msg->address_hi = upper_32_bits(mb_addr); + msg->address_lo = lower_32_bits(mb_addr); + msg->data = mb_bit; + + iommu_dma_compose_msi_msg(irq_data_get_msi_desc(data), msg); +} + +static void kvx_mailbox_set_irq_enable(struct irq_data *data, + bool enabled) +{ + struct kvx_irq_data *kd = irq_data_get_irq_chip_data(data); + struct kvx_apic_mailbox *mb = kd->mb; + unsigned int mb_num, mb_bit; + void __iomem *mb_addr; + u64 mask_value, mb_value; + + kvx_mailbox_get_from_hwirq(irqd_to_hwirq(data), &mb_num, &mb_bit); + mb_addr = kvx_mailbox_get_addr(mb, mb_num); + + raw_spin_lock(&mb->mask_lock); + mask_value = readq(mb_addr + KVX_MAILBOX_MASK_OFFSET); + if (enabled) + mask_value |= BIT_ULL(mb_bit); + else + mask_value &= ~BIT_ULL(mb_bit); + + writeq(mask_value, mb_addr + KVX_MAILBOX_MASK_OFFSET); + + raw_spin_unlock(&mb->mask_lock); + + /** + * Since interrupts on mailboxes are edge triggered and are only + * triggered when writing the value, we need to trigger it manually + * after updating the mask if enabled. If the interrupt was triggered by + * the device just after the mask write, we can trigger a spurious + * interrupt but that is still better than missing one... + * Moreover, the mailbox is configured in OR mode which means that even + * if we write a single bit, all other bits will be kept intact. + */ + if (enabled) { + mb_value = readq(mb_addr + KVX_MAILBOX_VALUE_OFFSET); + if (mb_value & BIT_ULL(mb_bit)) + writeq(BIT_ULL(mb_bit), + mb_addr + KVX_MAILBOX_VALUE_OFFSET); + } +} + +static void kvx_mailbox_mask(struct irq_data *data) +{ + kvx_mailbox_set_irq_enable(data, false); +} + +static void kvx_mailbox_unmask(struct irq_data *data) +{ + kvx_mailbox_set_irq_enable(data, true); +} + +static void kvx_mailbox_set_cpu(struct kvx_apic_mailbox *mb, int mb_id, + int new_cpu) +{ + irq_set_affinity(mb->mb_data[mb_id].parent_irq, cpumask_of(new_cpu)); + mb->mb_data[mb_id].cpu = new_cpu; +} + +static void kvx_mailbox_free_bit(struct kvx_apic_mailbox *mb, int hw_irq) +{ + unsigned int mb_num, mb_bit; + + kvx_mailbox_get_from_hwirq(hw_irq, &mb_num, &mb_bit); + bitmap_clear(mb->available, hw_irq, 1); + + /* If there is no more IRQ on this mailbox, reset it to CPU 0 */ + if (mb->available[mb_num] == 0) + kvx_mailbox_set_cpu(mb, mb_num, 0); +} + +struct irq_chip kvx_apic_mailbox_irq_chip = { + .name = "kvx apic mailbox", + .irq_compose_msi_msg = kvx_mailbox_msi_compose_msg, + .irq_mask = kvx_mailbox_mask, + .irq_unmask = kvx_mailbox_unmask, +}; + +static int kvx_mailbox_allocate_bits(struct kvx_apic_mailbox *mb, int num_req) +{ + int first, align_mask = 0; + + /* This must be a power of 2 for bitmap_find_next_zero_area to work */ + BUILD_BUG_ON((MAILBOXES_BITS_PER_PAGE & (MAILBOXES_BITS_PER_PAGE - 1))); + + /* + * If user requested more than 1 mailbox, we must make sure it will be + * aligned on a page size for iommu_dma_prepare_msi to be correctly + * mapped in a single page. + */ + if (num_req > 1) + align_mask = (MAILBOXES_BITS_PER_PAGE - 1); + + spin_lock(&mb->mailboxes_lock); + + first = bitmap_find_next_zero_area(mb->available, + mb->mb_count * MAILBOXES_BIT_SIZE, 0, + num_req, align_mask); + if (first >= MAILBOXES_MAX_BIT_COUNT) { + spin_unlock(&mb->mailboxes_lock); + return -ENOSPC; + } + + bitmap_set(mb->available, first, num_req); + + spin_unlock(&mb->mailboxes_lock); + + return first; +} + +static int kvx_apic_mailbox_msi_alloc(struct irq_domain *domain, + unsigned int virq, + unsigned int nr_irqs, void *args) +{ + int i, err; + int hwirq = 0; + u64 mb_addr; + struct irq_data *d; + struct kvx_irq_data *kd; + struct kvx_apic_mailbox *mb = domain->host_data; + struct msi_alloc_info *msi_info = (struct msi_alloc_info *)args; + struct msi_desc *desc = msi_info->desc; + unsigned int mb_num, mb_bit; + + /* We will not be able to guarantee page alignment ! */ + if (nr_irqs > MAILBOXES_BITS_PER_PAGE) + return -EINVAL; + + hwirq = kvx_mailbox_allocate_bits(mb, nr_irqs); + if (hwirq < 0) + return hwirq; + + kvx_mailbox_get_from_hwirq(hwirq, &mb_num, &mb_bit); + mb_addr = (u64) kvx_mailbox_get_phys_addr(mb, mb_num); + err = iommu_dma_prepare_msi(desc, mb_addr); + if (err) + goto free_mb_bits; + + for (i = 0; i < nr_irqs; i++) { + kd = kmalloc(sizeof(*kd), GFP_KERNEL); + if (!kd) { + err = -ENOMEM; + goto free_irq_data; + } + + kd->mb = mb; + irq_domain_set_info(domain, virq + i, hwirq + i, + &kvx_apic_mailbox_irq_chip, + kd, handle_simple_irq, + NULL, NULL); + } + + return 0; + +free_irq_data: + for (i--; i >= 0; i--) { + d = irq_domain_get_irq_data(domain, virq + i); + kd = irq_data_get_irq_chip_data(d); + kfree(kd); + } + +free_mb_bits: + spin_lock(&mb->mailboxes_lock); + bitmap_clear(mb->available, hwirq, nr_irqs); + spin_unlock(&mb->mailboxes_lock); + + return err; +} + +static void kvx_apic_mailbox_msi_free(struct irq_domain *domain, + unsigned int virq, + unsigned int nr_irqs) +{ + int i; + struct irq_data *d; + struct kvx_irq_data *kd; + struct kvx_apic_mailbox *mb = domain->host_data; + + spin_lock(&mb->mailboxes_lock); + + for (i = 0; i < nr_irqs; i++) { + d = irq_domain_get_irq_data(domain, virq + i); + kd = irq_data_get_irq_chip_data(d); + kfree(kd); + kvx_mailbox_free_bit(mb, d->hwirq); + } + + spin_unlock(&mb->mailboxes_lock); +} + +static const struct irq_domain_ops kvx_apic_mailbox_domain_ops = { + .alloc = kvx_apic_mailbox_msi_alloc, + .free = kvx_apic_mailbox_msi_free +}; + +static struct irq_chip kvx_msi_irq_chip = { + .name = "KVX MSI", +}; + +static void kvx_apic_mailbox_handle_irq(struct irq_desc *desc) +{ + struct irq_data *data = irq_desc_get_irq_data(desc); + struct kvx_apic_mailbox *mb = irq_desc_get_handler_data(desc); + void __iomem *mb_addr = kvx_mailbox_get_addr(mb, irqd_to_hwirq(data)); + unsigned int irqn, cascade_irq, bit; + u64 mask_value, masked_its; + u64 mb_value; + /* Since we allocate 64 interrupts for each mailbox, the scheme + * to find the hwirq associated to a mailbox irq is the + * following: + * hw_irq = mb_num * MAILBOXES_BIT_SIZE + bit + */ + unsigned int mb_hwirq = irqd_to_hwirq(data) * MAILBOXES_BIT_SIZE; + + mb_value = readq(mb_addr + KVX_MAILBOX_LAC_OFFSET); + mask_value = readq(mb_addr + KVX_MAILBOX_MASK_OFFSET); + /* Mask any disabled interrupts */ + mb_value &= mask_value; + + /** + * Write all pending ITs that are masked to process them later + * Since the mailbox is in OR mode, these bits will be merged with any + * already set bits and thus avoid losing any interrupts. + */ + masked_its = (~mask_value) & mb_value; + if (masked_its) + writeq(masked_its, mb_addr + KVX_MAILBOX_LAC_OFFSET); + + for_each_set_bit(bit, (unsigned long *) &mb_value, BITS_PER_LONG) { + irqn = bit + mb_hwirq; + cascade_irq = irq_find_mapping(mb->device_domain, irqn); + generic_handle_irq(cascade_irq); + } +} + +static void __init +apic_mailbox_reset(struct kvx_apic_mailbox *mb) +{ + unsigned int i; + unsigned int mb_end = mb->mb_count; + void __iomem *mb_addr; + u64 funct_val = (KVX_MAILBOX_MODE_OR << KVX_MAILBOX_FUNCT_MODE_SHIFT) | + (KVX_MAILBOX_TRIG_DOORBELL << KVX_MAILBOX_FUNCT_TRIG_SHIFT); + + for (i = 0; i < mb_end; i++) { + mb_addr = kvx_mailbox_get_addr(mb, i); + /* Disable all interrupts */ + writeq(0ULL, mb_addr + KVX_MAILBOX_MASK_OFFSET); + /* Set mailbox to OR mode + trigger */ + writeq(funct_val, mb_addr + KVX_MAILBOX_FUNCT_OFFSET); + /* Load & Clear mailbox value */ + readq(mb_addr + KVX_MAILBOX_LAC_OFFSET); + } +} + +static struct msi_domain_ops kvx_msi_domain_ops = { +}; + +static struct msi_domain_info kvx_msi_domain_info = { + .flags = (MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS), + .ops = &kvx_msi_domain_ops, + .chip = &kvx_msi_irq_chip, +}; + +static int __init +kvx_init_apic_mailbox(struct device_node *node, + struct device_node *parent) +{ + struct kvx_apic_mailbox *mb; + unsigned int parent_irq, irq_count; + struct resource res; + int ret, i; + + mb = kzalloc(sizeof(*mb), GFP_KERNEL); + if (!mb) + return -ENOMEM; + + ret = of_address_to_resource(node, 0, &res); + if (ret) + return -EINVAL; + + mb->phys_base = res.start; + mb->base = of_io_request_and_map(node, 0, node->name); + if (!mb->base) { + ret = -EINVAL; + goto err_kfree; + } + + spin_lock_init(&mb->mailboxes_lock); + raw_spin_lock_init(&mb->mask_lock); + + irq_count = of_irq_count(node); + if (irq_count == 0 || irq_count > MAILBOXES_MAX_COUNT) { + ret = -EINVAL; + goto err_kfree; + } + mb->mb_count = irq_count; + + apic_mailbox_reset(mb); + + mb->device_domain = irq_domain_add_tree(node, + &kvx_apic_mailbox_domain_ops, + mb); + if (!mb->device_domain) { + pr_err("Failed to setup device domain\n"); + ret = -EINVAL; + goto err_iounmap; + } + + mb->msi_domain = platform_msi_create_irq_domain(of_node_to_fwnode(node), + &kvx_msi_domain_info, + mb->device_domain); + if (!mb->msi_domain) { + ret = -EINVAL; + goto err_irq_domain_add_tree; + } + + /* Chain all interrupts from gic to mailbox */ + for (i = 0; i < irq_count; i++) { + parent_irq = irq_of_parse_and_map(node, i); + if (parent_irq == 0) { + pr_err("unable to parse irq\n"); + ret = -EINVAL; + goto err_irq_domain_msi_create; + } + mb->mb_data[i].parent_irq = parent_irq; + + irq_set_chained_handler_and_data(parent_irq, + kvx_apic_mailbox_handle_irq, + mb); + } + + return 0; + +err_irq_domain_msi_create: + irq_domain_remove(mb->msi_domain); +err_irq_domain_add_tree: + irq_domain_remove(mb->device_domain); +err_iounmap: + iounmap(mb->base); +err_kfree: + kfree(mb); + + return ret; +} + +IRQCHIP_DECLARE(kvx_apic_mailbox, "kalray,kvx-apic-mailbox", + kvx_init_apic_mailbox); -- 2.37.2

Add process management support for kvx, including: thread info definition, context switch and process tracing. Co-developed-by: Clement Leger <clement(a)clement-leger.fr&gt; Signed-off-by: Clement Leger <clement(a)clement-leger.fr&gt; Co-developed-by: Guillaume Thouvenin <gthouvenin(a)kalray.eu&gt; Signed-off-by: Guillaume Thouvenin <gthouvenin(a)kalray.eu&gt; Co-developed-by: Julian Vetter <jvetter(a)kalray.eu&gt; Signed-off-by: Julian Vetter <jvetter(a)kalray.eu&gt; Co-developed-by: Marius Gligor <mgligor(a)kalray.eu&gt; Signed-off-by: Marius Gligor <mgligor(a)kalray.eu&gt; Co-developed-by: Vincent Chardon <vincent.chardon(a)elsys-design.com&gt; Signed-off-by: Vincent Chardon <vincent.chardon(a)elsys-design.com&gt; Co-developed-by: Yann Sionneau <ysionneau(a)kalray.eu&gt; Signed-off-by: Yann Sionneau <ysionneau(a)kalray.eu&gt; --- Notes: V1 -> V2: no change arch/kvx/include/asm/current.h | 22 +++ arch/kvx/include/asm/ptrace.h | 217 +++++++++++++++++++++++ arch/kvx/include/asm/switch_to.h | 21 +++ arch/kvx/include/asm/thread_info.h | 78 ++++++++ arch/kvx/include/uapi/asm/ptrace.h | 114 ++++++++++++ arch/kvx/kernel/process.c | 203 +++++++++++++++++++++ arch/kvx/kernel/ptrace.c | 276 +++++++++++++++++++++++++++++ arch/kvx/kernel/stacktrace.c | 173 ++++++++++++++++++ 8 files changed, 1104 insertions(+) create mode 100644 arch/kvx/include/asm/current.h create mode 100644 arch/kvx/include/asm/ptrace.h create mode 100644 arch/kvx/include/asm/switch_to.h create mode 100644 arch/kvx/include/asm/thread_info.h create mode 100644 arch/kvx/include/uapi/asm/ptrace.h create mode 100644 arch/kvx/kernel/process.c create mode 100644 arch/kvx/kernel/ptrace.c create mode 100644 arch/kvx/kernel/stacktrace.c diff --git a/arch/kvx/include/asm/current.h b/arch/kvx/include/asm/current.h new file mode 100644 index 000000000000..b5fd0f076ec9 --- /dev/null +++ b/arch/kvx/include/asm/current.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_CURRENT_H +#define _ASM_KVX_CURRENT_H + +#include <asm/percpu.h> +#include <asm/sfr.h> + +struct task_struct; + +static __always_inline struct task_struct *get_current(void) +{ + return (struct task_struct *) kvx_sfr_get(SR); +} + +#define current get_current() + +#endif /* _ASM_KVX_CURRENT_H */ diff --git a/arch/kvx/include/asm/ptrace.h b/arch/kvx/include/asm/ptrace.h new file mode 100644 index 000000000000..d1b1e0975d9e --- /dev/null +++ b/arch/kvx/include/asm/ptrace.h @@ -0,0 +1,217 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Marius Gligor + * Yann Sionneau + */ + +#ifndef _ASM_KVX_PTRACE_H +#define _ASM_KVX_PTRACE_H + +#include <asm/types.h> +#include <asm/sfr.h> +#include <uapi/asm/ptrace.h> + +#define GPR_COUNT 64 +#define SFR_COUNT 9 +#define VIRT_COUNT 1 + +#define ES_SYSCALL 0x3 + +#define KVX_HW_BREAKPOINT_COUNT 2 +#define KVX_HW_WATCHPOINT_COUNT 1 + +#define REG_SIZE sizeof(u64) + +/** + * When updating pt_regs structure, this size must be updated. + * This is the expected size of the pt_regs struct. + * It ensures the structure layout from gcc is the same as the one we + * expect in order to do packed load (load/store octuple) in assembly. + * Conclusion: never put sizeof(pt_regs) in here or we lose this check + * (build time check done in asm-offsets.c via BUILD_BUG_ON) + */ +#define PT_REGS_STRUCT_EXPECTED_SIZE \ + ((GPR_COUNT + SFR_COUNT + VIRT_COUNT) * REG_SIZE + \ + 2 * REG_SIZE) /* Padding for stack alignment */ + +/** + * Saved register structure. Note that we should save only the necessary + * registers. + * When you modify it, please read carefully the comment above. + * Moreover, you will need to modify user_pt_regs to match the beginning + * of this struct 1:1 + */ +struct pt_regs { + union { + struct user_pt_regs user_regs; + struct { + /* GPR */ + uint64_t r0; + uint64_t r1; + uint64_t r2; + uint64_t r3; + uint64_t r4; + uint64_t r5; + uint64_t r6; + uint64_t r7; + uint64_t r8; + uint64_t r9; + uint64_t r10; + uint64_t r11; + union { + uint64_t r12; + uint64_t sp; + }; + union { + uint64_t r13; + uint64_t tp; + }; + union { + uint64_t r14; + uint64_t fp; + }; + uint64_t r15; + uint64_t r16; + uint64_t r17; + uint64_t r18; + uint64_t r19; + uint64_t r20; + uint64_t r21; + uint64_t r22; + uint64_t r23; + uint64_t r24; + uint64_t r25; + uint64_t r26; + uint64_t r27; + uint64_t r28; + uint64_t r29; + uint64_t r30; + uint64_t r31; + uint64_t r32; + uint64_t r33; + uint64_t r34; + uint64_t r35; + uint64_t r36; + uint64_t r37; + uint64_t r38; + uint64_t r39; + uint64_t r40; + uint64_t r41; + uint64_t r42; + uint64_t r43; + uint64_t r44; + uint64_t r45; + uint64_t r46; + uint64_t r47; + uint64_t r48; + uint64_t r49; + uint64_t r50; + uint64_t r51; + uint64_t r52; + uint64_t r53; + uint64_t r54; + uint64_t r55; + uint64_t r56; + uint64_t r57; + uint64_t r58; + uint64_t r59; + uint64_t r60; + uint64_t r61; + uint64_t r62; + uint64_t r63; + + /* SFR */ + uint64_t lc; + uint64_t le; + uint64_t ls; + uint64_t ra; + + uint64_t cs; + uint64_t spc; + }; + }; + uint64_t sps; + uint64_t es; + + uint64_t ilr; + + /* "Virtual" registers */ + uint64_t orig_r0; + + /* Padding for stack alignment (see STACK_ALIGN) */ + uint64_t padding[2]; + + /** + * If you add some fields, please read carefully the comment for + * PT_REGS_STRUCT_EXPECTED_SIZE. + */ +}; + +#define pl(__reg) kvx_sfr_field_val(__reg, PS, PL) + +#define MODE_KERNEL 0 +#define MODE_USER 1 + +/* Privilege level is relative in $sps, so 1 indicates current PL + 1 */ +#define user_mode(regs) (pl((regs)->sps) == MODE_USER) +#define es_ec(regs) kvx_sfr_field_val(regs->es, ES, EC) +#define es_sysno(regs) kvx_sfr_field_val(regs->es, ES, SN) + +#define debug_dc(es) kvx_sfr_field_val((es), ES, DC) + +/* ptrace */ +#define PTRACE_GET_HW_PT_REGS 20 +#define PTRACE_SET_HW_PT_REGS 21 +#define arch_has_single_step() 1 + +#define DEBUG_CAUSE_BREAKPOINT 0 +#define DEBUG_CAUSE_WATCHPOINT 1 +#define DEBUG_CAUSE_STEPI 2 +#define DEBUG_CAUSE_DSU_BREAK 3 + +static inline void enable_single_step(struct pt_regs *regs) +{ + regs->sps |= KVX_SFR_PS_SME_MASK; +} + +static inline void disable_single_step(struct pt_regs *regs) +{ + regs->sps &= ~KVX_SFR_PS_SME_MASK; +} + +static inline bool in_syscall(struct pt_regs const *regs) +{ + return es_ec(regs) == ES_SYSCALL; +} + +int do_syscall_trace_enter(struct pt_regs *regs, unsigned long syscall); +void do_syscall_trace_exit(struct pt_regs *regs); + +static inline unsigned long get_current_sp(void) +{ + register const unsigned long current_sp __asm__ ("$r12"); + + return current_sp; +} + +extern char *user_scall_rt_sigreturn_end; +extern char *user_scall_rt_sigreturn; + +static inline unsigned long instruction_pointer(struct pt_regs *regs) +{ + return regs->spc; +} + +static inline long regs_return_value(struct pt_regs *regs) +{ + return regs->r0; +} + +static inline unsigned long user_stack_pointer(struct pt_regs *regs) +{ + return regs->sp; +} + +#endif /* _ASM_KVX_PTRACE_H */ diff --git a/arch/kvx/include/asm/switch_to.h b/arch/kvx/include/asm/switch_to.h new file mode 100644 index 000000000000..2b1fda06dea8 --- /dev/null +++ b/arch/kvx/include/asm/switch_to.h @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_SWITCH_TO_H +#define _ASM_KVX_SWITCH_TO_H + +struct task_struct; + +/* context switching is now performed out-of-line in switch_to.S */ +extern struct task_struct *__switch_to(struct task_struct *prev, + struct task_struct *next); + +#define switch_to(prev, next, last) \ + do { \ + ((last) = __switch_to((prev), (next))); \ + } while (0) + +#endif /* _ASM_KVX_SWITCH_TO_H */ diff --git a/arch/kvx/include/asm/thread_info.h b/arch/kvx/include/asm/thread_info.h new file mode 100644 index 000000000000..4ce0154813ef --- /dev/null +++ b/arch/kvx/include/asm/thread_info.h @@ -0,0 +1,78 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Guillaume Thouvenin + */ + +#ifndef _ASM_KVX_THREAD_INFO_H +#define _ASM_KVX_THREAD_INFO_H + +#include <asm/page.h> + +/* + * Size of the kernel stack for each process. + */ +#define THREAD_SIZE_ORDER 2 +#define THREAD_SIZE (PAGE_SIZE << THREAD_SIZE_ORDER) + +/* + * Thread information flags + * these are process state flags that various assembly files may need to + * access + * - pending work-to-be-done flags are in LSW + * - other flags in MSW + */ +#define TIF_SYSCALL_TRACE 0 /* syscall trace active */ +#define TIF_NOTIFY_RESUME 1 /* resumption notification requested */ +#define TIF_SIGPENDING 2 /* signal pending */ +#define TIF_NEED_RESCHED 3 /* rescheduling necessary */ +#define TIF_SINGLESTEP 4 /* restore singlestep on return to user mode */ +#define TIF_UPROBE 5 +#define TIF_SYSCALL_TRACEPOINT 6 /* syscall tracepoint instrumentation */ +#define TIF_SYSCALL_AUDIT 7 /* syscall auditing active */ +#define TIF_RESTORE_SIGMASK 9 +#define TIF_NOTIFY_SIGNAL 10 /* signal notifications exist */ +#define TIF_POLLING_NRFLAG 16 /* true if poll_idle() is polling TIF_NEED_RESCHED */ +#define TIF_MEMDIE 17 + +#define _TIF_SYSCALL_TRACE (1 << TIF_SYSCALL_TRACE) +#define _TIF_SYSCALL_TRACEPOINT (1 << TIF_SYSCALL_TRACEPOINT) +#define _TIF_SYSCALL_AUDIT (1 << TIF_SYSCALL_AUDIT) +#define _TIF_POLLING_NRFLAG (1 << TIF_POLLING_NRFLAG) +#define _TIF_NOTIFY_RESUME (1 << TIF_NOTIFY_RESUME) +#define _TIF_SIGPENDING (1 << TIF_SIGPENDING) +#define _TIF_NEED_RESCHED (1 << TIF_NEED_RESCHED) +#define _TIF_NOTIFY_SIGNAL (1 << TIF_NOTIFY_SIGNAL) + +#define _TIF_WORK_MASK \ + (_TIF_NOTIFY_RESUME | _TIF_SIGPENDING | _TIF_NEED_RESCHED) + +#define _TIF_SYSCALL_WORK \ + (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_TRACEPOINT | _TIF_SYSCALL_AUDIT) + +#ifndef __ASSEMBLY__ +/* + * We are using THREAD_INFO_IN_TASK so this struct is almost useless + * please prefer adding fields in thread_struct (processor.h) rather + * than here. + * This struct is merely a remnant of distant times where it was placed + * on the stack to avoid large task_struct. + * + * cf https://lwn.net/Articles/700615/ + */ +struct thread_info { + unsigned long flags; /* low level flags */ + int preempt_count; +#ifdef CONFIG_SMP + u32 cpu; /* current CPU */ +#endif +}; + +#define INIT_THREAD_INFO(tsk) \ +{ \ + .flags = 0, \ + .preempt_count = INIT_PREEMPT_COUNT, \ +} +#endif /* __ASSEMBLY__*/ +#endif /* _ASM_KVX_THREAD_INFO_H */ diff --git a/arch/kvx/include/uapi/asm/ptrace.h b/arch/kvx/include/uapi/asm/ptrace.h new file mode 100644 index 000000000000..f5febe830526 --- /dev/null +++ b/arch/kvx/include/uapi/asm/ptrace.h @@ -0,0 +1,114 @@ +/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Yann Sionneau + */ + +#ifndef _UAPI_ASM_KVX_PTRACE_H +#define _UAPI_ASM_KVX_PTRACE_H + +#include <linux/types.h> +/* + * User-mode register state for core dumps, ptrace, sigcontext + * + * This decouples struct pt_regs from the userspace ABI. + * The struct pt_regs must start with the same layout as struct user_pt_regs. + */ +struct user_pt_regs { + /* GPR */ + unsigned long r0; + unsigned long r1; + unsigned long r2; + unsigned long r3; + unsigned long r4; + unsigned long r5; + unsigned long r6; + unsigned long r7; + unsigned long r8; + unsigned long r9; + unsigned long r10; + unsigned long r11; + union { + unsigned long r12; + unsigned long sp; + }; + union { + unsigned long r13; + unsigned long tp; + }; + union { + unsigned long r14; + unsigned long fp; + }; + unsigned long r15; + unsigned long r16; + unsigned long r17; + unsigned long r18; + unsigned long r19; + unsigned long r20; + unsigned long r21; + unsigned long r22; + unsigned long r23; + unsigned long r24; + unsigned long r25; + unsigned long r26; + unsigned long r27; + unsigned long r28; + unsigned long r29; + unsigned long r30; + unsigned long r31; + unsigned long r32; + unsigned long r33; + unsigned long r34; + unsigned long r35; + unsigned long r36; + unsigned long r37; + unsigned long r38; + unsigned long r39; + unsigned long r40; + unsigned long r41; + unsigned long r42; + unsigned long r43; + unsigned long r44; + unsigned long r45; + unsigned long r46; + unsigned long r47; + unsigned long r48; + unsigned long r49; + unsigned long r50; + unsigned long r51; + unsigned long r52; + unsigned long r53; + unsigned long r54; + unsigned long r55; + unsigned long r56; + unsigned long r57; + unsigned long r58; + unsigned long r59; + unsigned long r60; + unsigned long r61; + unsigned long r62; + unsigned long r63; + + /* SFR */ + unsigned long lc; + unsigned long le; + unsigned long ls; + unsigned long ra; + + unsigned long cs; + unsigned long spc; +}; + +/* TCA registers structure exposed to user */ +struct user_tca_regs { + struct { + __u64 x; + __u64 y; + __u64 z; + __u64 t; + } regs[48]; +}; + +#endif /* _UAPI_ASM_KVX_PTRACE_H */ diff --git a/arch/kvx/kernel/process.c b/arch/kvx/kernel/process.c new file mode 100644 index 000000000000..2a1cd0509604 --- /dev/null +++ b/arch/kvx/kernel/process.c @@ -0,0 +1,203 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Guillaume Thouvenin + * Marius Gligor + * Yann Sionneau + */ + +#include <linux/elf.h> +#include <linux/sched.h> +#include <linux/ptrace.h> +#include <linux/printk.h> +#include <linux/sched/debug.h> +#include <linux/sched/task_stack.h> + +#include <asm/ptrace.h> +#include <asm/processor.h> +#include <asm/ptrace.h> +#include <asm/uaccess.h> +#include <asm/stacktrace.h> + +#if defined(CONFIG_STACKPROTECTOR) +#include <linux/stackprotector.h> +unsigned long __stack_chk_guard __read_mostly; +EXPORT_SYMBOL(__stack_chk_guard); +#endif + +#define SCALL_NUM_EXIT "0xfff" + +void arch_cpu_idle(void) +{ + wait_for_interrupt(); + local_irq_enable(); +} + +void show_regs(struct pt_regs *regs) +{ + + int in_kernel = 1; + unsigned short i, reg_offset; + void *ptr; + + show_regs_print_info(KERN_DEFAULT); + + if (user_mode(regs)) + in_kernel = 0; + + pr_info("\nmode: %s\n" + " PC: %016llx PS: %016llx\n" + " CS: %016llx RA: %016llx\n" + " LS: %016llx LE: %016llx\n" + " LC: %016llx\n\n", + in_kernel ? "kernel" : "user", + regs->spc, regs->sps, + regs->cs, regs->ra, regs->ls, regs->le, regs->lc); + + /* GPR */ + ptr = regs; + ptr += offsetof(struct pt_regs, r0); + reg_offset = offsetof(struct pt_regs, r1) - + offsetof(struct pt_regs, r0); + + /** + * Display all the 64 GPRs assuming they are ordered correctly + * in the pt_regs struct... + */ + for (i = 0; i < GPR_COUNT; i += 2) { + pr_info(" R%d: %016llx R%d: %016llx\n", + i, *(uint64_t *)ptr, + i + 1, *(uint64_t *)(ptr + reg_offset)); + ptr += reg_offset * 2; + } + + pr_info("\n\n"); +} + +/** + * Prepare a thread to return to userspace + */ +void start_thread(struct pt_regs *regs, + unsigned long pc, unsigned long sp) +{ + /* Remove MMUP bit (user is not privilege in current virtual space) */ + u64 clear_bit = KVX_SFR_PS_MMUP_MASK | KVX_SFR_PS_SME_MASK | + KVX_SFR_PS_SMR_MASK; + regs->spc = pc; + regs->sp = sp; + regs->sps = kvx_sfr_get(PS); + + regs->sps &= ~clear_bit; + + /* Set privilege level to +1 (relative) */ + regs->sps &= ~KVX_SFR_PS_PL_MASK; + regs->sps |= (1 << KVX_SFR_PS_PL_SHIFT); +} + +int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) +{ + struct pt_regs *regs, *childregs = task_pt_regs(p); + unsigned long clone_flags = args->flags; + unsigned long usp = args->stack; + unsigned long tls = args->tls; + + /* p->thread holds context to be restored by __switch_to() */ + if (unlikely(args->fn)) { + /* Kernel thread */ + memset(childregs, 0, sizeof(struct pt_regs)); + + p->thread.ctx_switch.r20 = (uint64_t)args->fn; /* fn */ + p->thread.ctx_switch.r21 = (uint64_t)args->fn_arg; + p->thread.ctx_switch.ra = + (unsigned long) ret_from_kernel_thread; + } else { + regs = current_pt_regs(); + + /* Copy current process registers */ + *childregs = *regs; + + /* Store tracing status in r20 to avoid computing it + * in assembly + */ + p->thread.ctx_switch.r20 = + task_thread_info(p)->flags & _TIF_SYSCALL_WORK; + p->thread.ctx_switch.ra = (unsigned long) ret_from_fork; + + childregs->r0 = 0; /* Return value of fork() */ + /* Set stack pointer if any */ + if (usp) + childregs->sp = usp; + + /* Set a new TLS ? */ + if (clone_flags & CLONE_SETTLS) + childregs->r13 = tls; + } + p->thread.kernel_sp = + (unsigned long) (task_stack_page(p) + THREAD_SIZE); + p->thread.ctx_switch.sp = (unsigned long) childregs; + + return 0; +} + +void release_thread(struct task_struct *dead_task) +{ +} + +void flush_thread(void) +{ +} + +/* Fill in the fpu structure for a core dump. */ +int dump_fpu(struct pt_regs *regs, elf_fpregset_t *fpu) +{ + /* + * On kvx, FPU uses standard registers + $cs which is a common register + * also needed for non-fpu execution, so there is no additional + * register to dump. + */ + return 0; +} + +static bool find_wchan(unsigned long pc, void *arg) +{ + unsigned long *p = arg; + + /* + * If the pc is in a scheduler function (waiting), then, this is the + * address where the process is currently stuck. Note that scheduler + * functions also include lock functions. This functions are + * materialized using annotation to put them is special text sections. + */ + if (!in_sched_functions(pc)) { + *p = pc; + return true; + } + + return false; +} + +/* + * __get_wchan is called to obtain "schedule()" caller function address. + */ +unsigned long __get_wchan(struct task_struct *p) +{ + unsigned long pc = 0; + struct stackframe frame; + + /* + * We need to obtain the task stack since we don't want the stack to + * move under our feet. + */ + if (!try_get_task_stack(p)) + return 0; + + start_stackframe(&frame, thread_saved_reg(p, fp), + thread_saved_reg(p, ra)); + walk_stackframe(p, &frame, find_wchan, &pc); + + put_task_stack(p); + + return pc; +} + diff --git a/arch/kvx/kernel/ptrace.c b/arch/kvx/kernel/ptrace.c new file mode 100644 index 000000000000..aacae5883cb5 --- /dev/null +++ b/arch/kvx/kernel/ptrace.c @@ -0,0 +1,276 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * derived from arch/riscv/kernel/ptrace.c + * + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Marius Gligor + * Clement Leger + */ + +#include <linux/sched.h> +#include <linux/sched.h> +#include <linux/audit.h> +#include <linux/irqflags.h> +#include <linux/thread_info.h> +#include <linux/context_tracking.h> +#include <linux/uaccess.h> +#include <linux/syscalls.h> +#include <linux/signal.h> +#include <linux/regset.h> +#include <linux/hw_breakpoint.h> + +#include <asm/dame.h> +#include <asm/ptrace.h> +#include <asm/syscall.h> +#include <asm/break_hook.h> +#include <asm/debug.h> +#include <asm/cacheflush.h> + +#define CREATE_TRACE_POINTS +#include <trace/events/syscalls.h> + +#define HW_PT_CMD_GET_CAPS 0 +#define HW_PT_CMD_GET_PT 1 +#define HW_PT_CMD_SET_RESERVE 0 +#define HW_PT_CMD_SET_ENABLE 1 + +#define FROM_GDB_CMD_MASK 3 +#define FROM_GDB_HP_TYPE_SHIFT 2 +#define FROM_GDB_HP_TYPE_MASK 4 +#define FROM_GDB_WP_TYPE_SHIFT 3 +#define FROM_GDB_WP_TYPE_MASK 0x18 +#define FROM_GDB_HP_IDX_SHIFT 5 + +#define hw_pt_cmd(addr) ((addr) & FROM_GDB_CMD_MASK) +#define hw_pt_is_bkp(addr) ((((addr) & FROM_GDB_HP_TYPE_MASK) >> \ + FROM_GDB_HP_TYPE_SHIFT) == KVX_HW_BREAKPOINT_TYPE) +#define get_hw_pt_wp_type(addr) ((((addr) & FROM_GDB_WP_TYPE_MASK)) >> \ + FROM_GDB_WP_TYPE_SHIFT) +#define get_hw_pt_idx(addr) ((addr) >> FROM_GDB_HP_IDX_SHIFT) +#define get_hw_pt_addr(data) ((data)[0]) +#define get_hw_pt_len(data) ((data)[1] >> 1) +#define hw_pt_is_enabled(data) ((data)[1] & 1) + +enum kvx_regset { + REGSET_GPR, +#ifdef CONFIG_ENABLE_TCA + REGSET_TCA, +#endif +}; + +void ptrace_disable(struct task_struct *child) +{ + /* nothing to do */ +} + +static int kvx_gpr_get(struct task_struct *target, + const struct user_regset *regset, + struct membuf to) +{ + struct user_pt_regs *regs = &task_pt_regs(target)->user_regs; + + return membuf_write(&to, regs, sizeof(*regs)); +} + +static int kvx_gpr_set(struct task_struct *target, + const struct user_regset *regset, + unsigned int pos, unsigned int count, + const void *kbuf, const void __user *ubuf) +{ + struct user_pt_regs *regs = &task_pt_regs(target)->user_regs; + + return user_regset_copyin(&pos, &count, &kbuf, &ubuf, regs, 0, -1); +} + +#ifdef CONFIG_ENABLE_TCA +static int kvx_tca_reg_get(struct task_struct *target, + const struct user_regset *regset, + struct membuf to) +{ + struct ctx_switch_regs *ctx_regs = &target->thread.ctx_switch; + struct tca_reg *regs = ctx_regs->tca_regs; + int ret; + + if (!ctx_regs->tca_regs_saved) + ret = membuf_zero(&to, sizeof(*regs)); + else + ret = membuf_write(&to, regs, sizeof(*regs)); + + return ret; +} + +static int kvx_tca_reg_set(struct task_struct *target, + const struct user_regset *regset, + unsigned int pos, unsigned int count, + const void *kbuf, const void __user *ubuf) +{ + struct ctx_switch_regs *ctx_regs = &target->thread.ctx_switch; + struct tca_reg *regs = ctx_regs->tca_regs; + int ret; + + if (!ctx_regs->tca_regs_saved) + ret = user_regset_copyin_ignore(&pos, &count, &kbuf, &ubuf, + 0, -1); + else + ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, regs, + 0, -1); + + return ret; +} +#endif + +static const struct user_regset kvx_user_regset[] = { + [REGSET_GPR] = { + .core_note_type = NT_PRSTATUS, + .n = ELF_NGREG, + .size = sizeof(elf_greg_t), + .align = sizeof(elf_greg_t), + .regset_get = &kvx_gpr_get, + .set = &kvx_gpr_set, + }, +#ifdef CONFIG_ENABLE_TCA + [REGSET_TCA] = { + .core_note_type = NT_KVX_TCA, + .n = TCA_REG_COUNT, + .size = sizeof(struct tca_reg), + .align = sizeof(struct tca_reg), + .regset_get = &kvx_tca_reg_get, + .set = &kvx_tca_reg_set, + }, +#endif +}; + +static const struct user_regset_view user_kvx_view = { + .name = "kvx", + .e_machine = EM_KVX, + .regsets = kvx_user_regset, + .n = ARRAY_SIZE(kvx_user_regset) +}; + +const struct user_regset_view *task_user_regset_view(struct task_struct *task) +{ + return &user_kvx_view; +} + +long arch_ptrace(struct task_struct *child, long request, + unsigned long addr, unsigned long data) +{ + return ptrace_request(child, request, addr, data); +} + +/* + * Allows PTRACE_SYSCALL to work. These are called from entry.S in + * {handle,ret_from}_syscall. + */ +int do_syscall_trace_enter(struct pt_regs *regs, unsigned long syscall) +{ + int ret = 0; + +#ifdef CONFIG_CONTEXT_TRACKING_USER + user_exit_callable(); +#endif + if (test_thread_flag(TIF_SYSCALL_TRACE)) + ret = ptrace_report_syscall_entry(regs); + +#ifdef CONFIG_HAVE_SYSCALL_TRACEPOINTS + if (test_thread_flag(TIF_SYSCALL_TRACEPOINT)) + trace_sys_enter(regs, syscall_get_nr(current, regs)); +#endif + + audit_syscall_entry(syscall, regs->r0, regs->r1, regs->r2, regs->r3); + + return ret; +} + +void do_syscall_trace_exit(struct pt_regs *regs) +{ + if (test_thread_flag(TIF_SYSCALL_TRACE)) + ptrace_report_syscall_exit(regs, 0); + + audit_syscall_exit(regs); + +#ifdef CONFIG_HAVE_SYSCALL_TRACEPOINTS + if (test_thread_flag(TIF_SYSCALL_TRACEPOINT)) + trace_sys_exit(regs, regs_return_value(regs)); +#endif + +#ifdef CONFIG_CONTEXT_TRACKING_USER + user_enter_callable(); +#endif +} + +static int kvx_bkpt_handler(struct break_hook *brk_hook, struct pt_regs *regs) +{ + /* Unexpected breakpoint */ + if (!(current->ptrace & PT_PTRACED)) + return BREAK_HOOK_ERROR; + + /* deliver the signal to userspace */ + force_sig_fault(SIGTRAP, TRAP_BRKPT, (void __user *) regs->spc); + + return BREAK_HOOK_HANDLED; +} + +static void kvx_stepi(struct pt_regs *regs) +{ + /* deliver the signal to userspace */ + force_sig_fault(SIGTRAP, TRAP_TRACE, (void __user *) regs->spc); +} + +void user_enable_single_step(struct task_struct *child) +{ + struct pt_regs *regs = task_pt_regs(child); + + enable_single_step(regs); +} + +void user_disable_single_step(struct task_struct *child) +{ + struct pt_regs *regs = task_pt_regs(child); + + disable_single_step(regs); +} + +/** + * Main debug handler called by the _debug_handler routine in entry.S + * This handler will perform the required action + * @es: Exception Syndrome register value + * @ea: Exception Address register + * @regs: pointer to registers saved when enter debug + */ +int ptrace_debug_handler(u64 ea, struct pt_regs *regs) +{ + + int debug_cause = debug_dc(regs->es); + + switch (debug_cause) { + case DEBUG_CAUSE_STEPI: + kvx_stepi(regs); + break; + default: + break; + } + + return DEBUG_HOOK_HANDLED; +} + +static struct debug_hook ptrace_debug_hook = { + .handler = ptrace_debug_handler, + .mode = MODE_USER, +}; + +static struct break_hook bkpt_break_hook = { + .id = BREAK_CAUSE_BKPT, + .handler = kvx_bkpt_handler, + .mode = MODE_USER, +}; + +static int __init arch_init_breakpoint(void) +{ + break_hook_register(&bkpt_break_hook); + debug_hook_register(&ptrace_debug_hook); + + return 0; +} + +postcore_initcall(arch_init_breakpoint); diff --git a/arch/kvx/kernel/stacktrace.c b/arch/kvx/kernel/stacktrace.c new file mode 100644 index 000000000000..85d52ba2d082 --- /dev/null +++ b/arch/kvx/kernel/stacktrace.c @@ -0,0 +1,173 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Vincent Chardon + */ + +#include <linux/context_tracking.h> +#include <linux/kallsyms.h> +#include <linux/printk.h> +#include <linux/init.h> + +#include <asm/stacktrace.h> +#include <asm/ptrace.h> + +#define STACK_SLOT_PER_LINE 4 +#define STACK_MAX_SLOT_PRINT (STACK_SLOT_PER_LINE * 8) + +static int notrace unwind_frame(struct task_struct *task, + struct stackframe *frame) +{ + unsigned long fp = frame->fp; + + /* Frame pointer must be aligned on 8 bytes */ + if (fp & 0x7) + return -EINVAL; + + if (!task) + task = current; + + if (!on_task_stack(task, fp)) + return -EINVAL; + + frame->fp = READ_ONCE_NOCHECK(*(unsigned long *)(fp)); + frame->ra = READ_ONCE_NOCHECK(*(unsigned long *)(fp + 8)); + + /* + * When starting, we set the frame pointer to 0, hence end of + * frame linked list is signal by that + */ + if (!frame->fp) + return -EINVAL; + + return 0; +} + +void notrace walk_stackframe(struct task_struct *task, struct stackframe *frame, + bool (*fn)(unsigned long, void *), void *arg) +{ + unsigned long addr; + int ret; + + while (1) { + addr = frame->ra; + + if (fn(addr, arg)) + break; + + ret = unwind_frame(task, frame); + if (ret) + break; + } +} + +#ifdef CONFIG_STACKTRACE +bool append_stack_addr(unsigned long pc, void *arg) +{ + struct stack_trace *trace; + + trace = (struct stack_trace *)arg; + if (trace->skip == 0) { + trace->entries[trace->nr_entries++] = pc; + if (trace->nr_entries == trace->max_entries) + return true; + } else { + trace->skip--; + } + + return false; +} + +/* + * Save stack-backtrace addresses into a stack_trace buffer. + */ +void save_stack_trace(struct stack_trace *trace) +{ + struct stackframe frame; + + trace->nr_entries = 0; + /* We want to skip this function and the caller */ + trace->skip += 2; + + start_stackframe(&frame, (unsigned long) __builtin_frame_address(0), + (unsigned long) save_stack_trace); + walk_stackframe(current, &frame, append_stack_addr, trace); +} +EXPORT_SYMBOL(save_stack_trace); +#endif /* CONFIG_STACKTRACE */ + +static bool print_pc(unsigned long pc, void *arg) +{ + unsigned long *skip = arg; + + if (*skip == 0) + print_ip_sym(KERN_INFO, pc); + else + (*skip)--; + + return false; +} + +void show_stacktrace(struct task_struct *task, struct pt_regs *regs) +{ + struct stackframe frame; + unsigned long skip = 0; + + /* Obviously, we can't backtrace on usermode ! */ + if (regs && user_mode(regs)) + return; + + if (!task) + task = current; + + if (!try_get_task_stack(task)) + return; + + if (regs) { + start_stackframe(&frame, regs->fp, regs->spc); + } else if (task == current) { + /* Skip current function and caller */ + skip = 2; + start_stackframe(&frame, + (unsigned long) __builtin_frame_address(0), + (unsigned long) show_stacktrace); + } else { + /* task blocked in __switch_to */ + start_stackframe(&frame, + thread_saved_reg(task, fp), + thread_saved_reg(task, ra)); + } + + pr_info("Call Trace:\n"); + walk_stackframe(task, &frame, print_pc, &skip); + + put_task_stack(task); +} + +/* + * If show_stack is called with a non-null task, then the task will have been + * claimed with try_get_task_stack by the caller. If task is NULL or current + * then there is no need to get task stack since it's our current stack... + */ +void show_stack(struct task_struct *task, unsigned long *sp) +{ + int i = 0; + + if (!sp) + sp = (unsigned long *) get_current_sp(); + + pr_info("Stack dump (@%p):\n", sp); + for (i = 0; i < STACK_MAX_SLOT_PRINT; i++) { + if (kstack_end(sp)) + break; + + if (i && (i % STACK_SLOT_PER_LINE) == 0) + pr_cont("\n\t"); + + pr_cont("%016lx ", *sp++); + } + pr_cont("\n"); + + show_stacktrace(task, NULL); +} -- 2.37.2

On Fri, Jan 20, 2023 at 03:09:51PM +0100, Yann Sionneau wrote: ... I believe tlb_entry is a better name and unless I've missed something, this struct is only used internally in arch/kvx/mm, so it'd be better to declare it in a header at arch/kvx/mm. Generally, it's better to move internal definitions out of include/asm and have them near the .c files that use them. For instance, I randomly picked several functions below, e.g. kvx_mmu_probetlb(), tlb_entry_overlaps(), and it seems they are only used in arch/kvx/mm. ... ... Please drop this comment, it does not add any information ... Looks like this entire header can be moved to arch/kvx/mm ... It seems this function is never called. Please drop these comments (here and for lower levels as well), they don't add information but only take space. ... ... With macro name pgd_ERROR, it's already clear that this is pgd_ERROR. Please drop the comment. Ditto Ditto I don't see HAVE_ARCH_TRANSPARENT_HUGEPAGE in arch/kvx/Kconfig. Please remove this part for now. Does kvx support holes between memory banks? If no, FLATMEM should do. It looks like this header can be moved to arch/kvx/mm ... There is also max_pfn that's used by various pfn walkers. In your case it should be the same as max_low_pfn. This may cut off the initrd memory. It's be better to cap the memory before setting up the initrd. Any reason not to use default implementation in init/main.c? This won't work because it relies on "System RAM" in the resource tree and you don't setup this. All occurrences of tlb_mk_entry() case paddr and vaddr parameters and then tlb_mk_entry() essentially treats them as unsigned longs. Isn't it better to pass declare tlb_mk_entry as tlb_mk_entry(phys_addr_t paddr, unsigned long vaddr, ... Please avoid having a parameter per line. Ditto -- Sincerely yours, Mike.

Add sigcontext definition and signal handling support for kvx. Co-developed-by: Clement Leger <clement(a)clement-leger.fr&gt; Signed-off-by: Clement Leger <clement(a)clement-leger.fr&gt; Co-developed-by: Jules Maselbas <jmaselbas(a)kalray.eu&gt; Signed-off-by: Jules Maselbas <jmaselbas(a)kalray.eu&gt; Co-developed-by: Julian Vetter <jvetter(a)kalray.eu&gt; Signed-off-by: Julian Vetter <jvetter(a)kalray.eu&gt; Co-developed-by: Yann Sionneau <ysionneau(a)kalray.eu&gt; Signed-off-by: Yann Sionneau <ysionneau(a)kalray.eu&gt; --- Notes: V1 -> V2: - use read_thread_flags() as suggested by Mark Rutland arch/kvx/include/uapi/asm/sigcontext.h | 16 ++ arch/kvx/kernel/signal.c | 265 +++++++++++++++++++++++++ arch/kvx/kernel/vdso.c | 87 ++++++++ 3 files changed, 368 insertions(+) create mode 100644 arch/kvx/include/uapi/asm/sigcontext.h create mode 100644 arch/kvx/kernel/signal.c create mode 100644 arch/kvx/kernel/vdso.c diff --git a/arch/kvx/include/uapi/asm/sigcontext.h b/arch/kvx/include/uapi/asm/sigcontext.h new file mode 100644 index 000000000000..97ab4f78152a --- /dev/null +++ b/arch/kvx/include/uapi/asm/sigcontext.h @@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _UAPI_ASM_KVX_SIGCONTEXT_H +#define _UAPI_ASM_KVX_SIGCONTEXT_H + +#include <asm/ptrace.h> + +struct sigcontext { + struct user_pt_regs sc_regs; +}; + +#endif /* _UAPI_ASM_KVX_SIGCONTEXT_H */ diff --git a/arch/kvx/kernel/signal.c b/arch/kvx/kernel/signal.c new file mode 100644 index 000000000000..5bd63d14f43e --- /dev/null +++ b/arch/kvx/kernel/signal.c @@ -0,0 +1,265 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#include <linux/bug.h> +#include <linux/syscalls.h> + +#include <asm/ucontext.h> +#include <asm/processor.h> +#include <asm/cacheflush.h> +#include <linux/resume_user_mode.h> + +struct rt_sigframe { + struct siginfo info; + struct ucontext uc; +}; + +int __init setup_syscall_sigreturn_page(void *sigpage_addr) +{ + unsigned int frame_size = (uintptr_t) &user_scall_rt_sigreturn_end - + (uintptr_t) &user_scall_rt_sigreturn; + + /* Copy the sigreturn scall implementation */ + memcpy(sigpage_addr, &user_scall_rt_sigreturn, frame_size); + + flush_icache_range((unsigned long) sigpage_addr, + (unsigned long) sigpage_addr + frame_size); + + return 0; +} + +static long restore_sigcontext(struct pt_regs *regs, + struct sigcontext __user *sc) +{ + long err; + + /* sc_regs is structured the same as the start of pt_regs */ + err = __copy_from_user(regs, &sc->sc_regs, sizeof(sc->sc_regs)); + + return err; +} + +long _sys_rt_sigreturn(void) +{ + struct pt_regs *regs = current_pt_regs(); + struct rt_sigframe __user *frame; + struct task_struct *task; + sigset_t set; + + current->restart_block.fn = do_no_restart_syscall; + + frame = (struct rt_sigframe __user *) user_stack_pointer(regs); + + /* + * Stack is not aligned but should be ! + * User probably did some malicious things. + */ + if (user_stack_pointer(regs) & STACK_ALIGN_MASK) + goto badframe; + + if (!access_ok(frame, sizeof(*frame))) + goto badframe; + + /* Restore sigmask */ + if (__copy_from_user(&set, &frame->uc.uc_sigmask, sizeof(set))) + goto badframe; + + set_current_blocked(&set); + + if (restore_sigcontext(regs, &frame->uc.uc_mcontext)) + goto badframe; + + if (restore_altstack(&frame->uc.uc_stack)) + goto badframe; + + return regs->r0; + +badframe: + task = current; + if (show_unhandled_signals) { + pr_info_ratelimited( + "%s[%d]: bad frame in %s: frame=%p pc=%p sp=%p\n", + task->comm, task_pid_nr(task), __func__, + frame, (void *) instruction_pointer(regs), + (void *) user_stack_pointer(regs)); + } + force_sig(SIGSEGV); + return 0; +} + + +static long setup_sigcontext(struct rt_sigframe __user *frame, + struct pt_regs *regs) +{ + struct sigcontext __user *sc = &frame->uc.uc_mcontext; + long err; + + /* sc_regs is structured the same as the start of pt_regs */ + err = __copy_to_user(&sc->sc_regs, regs, sizeof(sc->sc_regs)); + + return err; +} + +static inline void __user *get_sigframe(struct ksignal *ksig, + struct pt_regs *regs, size_t framesize) +{ + unsigned long sp; + /* Default to using normal stack */ + sp = regs->sp; + + /* + * If we are on the alternate signal stack and would overflow it, don't. + * Return an always-bogus address instead so we will die with SIGSEGV. + */ + if (on_sig_stack(sp) && !likely(on_sig_stack(sp - framesize))) + return (void __user __force *)(-1UL); + + /* This is the X/Open sanctioned signal stack switching. */ + sp = sigsp(sp, ksig) - framesize; + + /* Align the stack frame on 16bytes */ + sp &= ~STACK_ALIGN_MASK; + + return (void __user *)sp; +} + +/* TODO: Use VDSO when ready ! */ +static int setup_rt_frame(struct ksignal *ksig, sigset_t *set, + struct pt_regs *regs) +{ + unsigned long sigpage = current->mm->context.sigpage; + struct rt_sigframe __user *frame; + long err = 0; + + frame = get_sigframe(ksig, regs, sizeof(*frame)); + if (!access_ok(frame, sizeof(*frame))) + return -EFAULT; + + err |= copy_siginfo_to_user(&frame->info, &ksig->info); + + /* Create the ucontext. */ + err |= __put_user(0, &frame->uc.uc_flags); + err |= __put_user(NULL, &frame->uc.uc_link); + err |= __save_altstack(&frame->uc.uc_stack, user_stack_pointer(regs)); + err |= setup_sigcontext(frame, regs); + err |= __copy_to_user(&frame->uc.uc_sigmask, set, sizeof(*set)); + if (err) + return -EFAULT; + + /* + * When returning from the handler, we want to jump to the + * sigpage which will execute the sigreturn scall. + */ + regs->ra = sigpage; + /* Return to signal handler */ + regs->spc = (unsigned long)ksig->ka.sa.sa_handler; + regs->sp = (unsigned long) frame; + + /* Parameters for signal handler */ + regs->r0 = ksig->sig; /* r0: signal number */ + regs->r1 = (unsigned long)(&frame->info); /* r1: siginfo pointer */ + regs->r2 = (unsigned long)(&frame->uc); /* r2: ucontext pointer */ + + return 0; +} + +static void handle_signal(struct ksignal *ksig, struct pt_regs *regs) +{ + sigset_t *oldset = sigmask_to_save(); + int ret; + + /* Are we from a system call? */ + if (in_syscall(regs)) { + /* If so, check system call restarting.. */ + switch (regs->r0) { + case -ERESTART_RESTARTBLOCK: + case -ERESTARTNOHAND: + regs->r0 = -EINTR; + break; + case -ERESTARTSYS: + if (!(ksig->ka.sa.sa_flags & SA_RESTART)) { + regs->r0 = -EINTR; + break; + } + fallthrough; + case -ERESTARTNOINTR: + regs->r0 = regs->orig_r0; + regs->spc -= 0x4; + break; + } + } + + ret = setup_rt_frame(ksig, oldset, regs); + + signal_setup_done(ret, ksig, 0); +} + +asmlinkage void do_signal(struct pt_regs *regs) +{ + struct ksignal ksig; + + if (get_signal(&ksig)) { + handle_signal(&ksig, regs); + return; + } + + /* Are we from a system call? */ + if (in_syscall(regs)) { + /* + * If we are here, this means there is no handler + * present and we must restart the syscall. + */ + switch (regs->r0) { + case -ERESTART_RESTARTBLOCK: + /* Modify the syscall number in order to restart it */ + regs->r6 = __NR_restart_syscall; + fallthrough; + case -ERESTARTNOHAND: + case -ERESTARTSYS: + case -ERESTARTNOINTR: + /* We are restarting the syscall */ + regs->r0 = regs->orig_r0; + /* + * scall instruction isn't bundled with anything else, + * so we can just revert the spc to restart the syscall. + */ + regs->spc -= 0x4; + break; + } + } + + /* + * If there's no signal to deliver, we just put the saved sigmask + * back. + */ + restore_saved_sigmask(); +} + + +asmlinkage void do_work_pending(struct pt_regs *regs, + unsigned long thread_flags) +{ + /* We are called with IRQs disabled */ + trace_hardirqs_off(); + + do { + if (thread_flags & _TIF_NEED_RESCHED) { + schedule(); + } else { + local_irq_enable(); + if (thread_flags & (_TIF_SIGPENDING | _TIF_NOTIFY_SIGNAL)) + do_signal(regs); + + if (thread_flags & _TIF_NOTIFY_RESUME) { + clear_thread_flag(TIF_NOTIFY_RESUME); + resume_user_mode_work(regs); + } + } + local_irq_disable(); + thread_flags = read_thread_flags(); + } while (thread_flags & _TIF_WORK_MASK); +} + diff --git a/arch/kvx/kernel/vdso.c b/arch/kvx/kernel/vdso.c new file mode 100644 index 000000000000..1515de15eb31 --- /dev/null +++ b/arch/kvx/kernel/vdso.c @@ -0,0 +1,87 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#include <linux/mm.h> +#include <linux/vmalloc.h> +#include <linux/binfmts.h> + +#include <asm/syscall.h> + +static struct page *signal_page; + +static int __init init_sigreturn(void) +{ + struct page *sigpage; + void *mapped_sigpage; + int err; + + sigpage = alloc_page(GFP_KERNEL); + if (!sigpage) + panic("Cannot allocate sigreturn page"); + + mapped_sigpage = vmap(&sigpage, 1, 0, PAGE_KERNEL); + if (!mapped_sigpage) + panic("Cannot map sigreturn page"); + + clear_page(mapped_sigpage); + + err = setup_syscall_sigreturn_page(mapped_sigpage); + if (err) + panic("Cannot set signal return syscall, err: %x.", err); + + vunmap(mapped_sigpage); + + signal_page = sigpage; + + return 0; +} +arch_initcall(init_sigreturn); + +static int sigpage_mremap(const struct vm_special_mapping *sm, + struct vm_area_struct *new_vma) +{ + current->mm->context.sigpage = new_vma->vm_start; + return 0; +} + +static const struct vm_special_mapping sigpage_mapping = { + .name = "[sigpage]", + .pages = &signal_page, + .mremap = sigpage_mremap, +}; + +int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp) +{ + int ret = 0; + unsigned long addr; + struct mm_struct *mm = current->mm; + struct vm_area_struct *vma; + + mmap_write_lock(mm); + + addr = get_unmapped_area(NULL, STACK_TOP, PAGE_SIZE, 0, 0); + if (IS_ERR_VALUE(addr)) { + ret = addr; + goto up_fail; + } + + vma = _install_special_mapping( + mm, + addr, + PAGE_SIZE, + VM_READ|VM_EXEC|VM_MAYREAD|VM_MAYEXEC, + &sigpage_mapping); + if (IS_ERR(vma)) { + ret = PTR_ERR(vma); + goto up_fail; + } + + mm->context.sigpage = addr; + +up_fail: + mmap_write_unlock(mm); + return ret; +} -- 2.37.2

Add some misc common routines for kvx, including: asm-offsets routines, futex functions, i/o memory access functions. Co-developed-by: Clement Leger <clement(a)clement-leger.fr&gt; Signed-off-by: Clement Leger <clement(a)clement-leger.fr&gt; Co-developed-by: Guillaume Thouvenin <gthouvenin(a)kalray.eu&gt; Signed-off-by: Guillaume Thouvenin <gthouvenin(a)kalray.eu&gt; Co-developed-by: Jonathan Borne <jborne(a)kalray.eu&gt; Signed-off-by: Jonathan Borne <jborne(a)kalray.eu&gt; Co-developed-by: Julian Vetter <jvetter(a)kalray.eu&gt; Signed-off-by: Julian Vetter <jvetter(a)kalray.eu&gt; Co-developed-by: Julien Villette <jvillette(a)kalray.eu&gt; Signed-off-by: Julien Villette <jvillette(a)kalray.eu&gt; Co-developed-by: Yann Sionneau <ysionneau(a)kalray.eu&gt; Signed-off-by: Yann Sionneau <ysionneau(a)kalray.eu&gt; --- Notes: V1 -> V2: no changes arch/kvx/include/asm/futex.h | 141 ++++++++++++++++++++++++++++++ arch/kvx/include/asm/io.h | 34 ++++++++ arch/kvx/kernel/asm-offsets.c | 157 ++++++++++++++++++++++++++++++++++ arch/kvx/kernel/io.c | 96 +++++++++++++++++++++ 4 files changed, 428 insertions(+) create mode 100644 arch/kvx/include/asm/futex.h create mode 100644 arch/kvx/include/asm/io.h create mode 100644 arch/kvx/kernel/asm-offsets.c create mode 100644 arch/kvx/kernel/io.c diff --git a/arch/kvx/include/asm/futex.h b/arch/kvx/include/asm/futex.h new file mode 100644 index 000000000000..b71b52339729 --- /dev/null +++ b/arch/kvx/include/asm/futex.h @@ -0,0 +1,141 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2018-2023 Kalray Inc. + * Authors: + * Clement Leger <cleger(a)kalray.eu&gt; + * Yann Sionneau <ysionneau(a)kalray.eu&gt; + * Jonathan Borne <jborne(a)kalray.eu&gt; + * + * Part of code is taken from RiscV port + */ + +#ifndef _ASM_KVX_FUTEX_H +#define _ASM_KVX_FUTEX_H + +#ifdef __KERNEL__ + +#include <linux/futex.h> +#include <linux/uaccess.h> + +#define __futex_atomic_op(insn, ret, oldval, uaddr, oparg) \ +{ \ + __enable_user_access(); \ + __asm__ __volatile__ ( \ + " fence \n" \ + " ;;\n " \ + "1: lwz $r63 = 0[%[u]] \n" \ + " ;;\n " \ + " " insn " \n" \ + " ;;\n " \ + " acswapw 0[%[u]], $r62r63 \n" \ + " ;;\n " \ + " cb.deqz $r62? 1b \n" \ + " ;;\n " \ + " copyd %[ov] = $r63 \n" \ + " ;;\n " \ + "2: \n" \ + " .section .fixup,\"ax\" \n" \ + "3: make %[r] = 2b \n" \ + " ;;\n " \ + " make %[r] = %[e] \n" \ + " igoto %[r] \n" \ + " ;;\n " \ + " .previous \n" \ + " .section __ex_table,\"a\" \n" \ + " .align 8 \n" \ + " .dword 1b,3b \n" \ + " .dword 2b,3b \n" \ + " .previous \n" \ + : [r] "+r" (ret), [ov] "+r" (oldval) \ + : [u] "r" (uaddr), \ + [op] "r" (oparg), [e] "i" (-EFAULT) \ + : "r62", "r63", "memory"); \ + __disable_user_access(); \ +} + + +static inline int +arch_futex_atomic_op_inuser(int op, u32 oparg, int *oval, u32 __user *uaddr) +{ + int oldval = 0, ret = 0; + + if (!access_ok(uaddr, sizeof(u32))) + return -EFAULT; + switch (op) { + case FUTEX_OP_SET: /* *(int *)UADDR = OPARG; */ + __futex_atomic_op("copyd $r62 = %[op]", + ret, oldval, uaddr, oparg); + break; + case FUTEX_OP_ADD: /* *(int *)UADDR += OPARG; */ + __futex_atomic_op("addw $r62 = $r63, %[op]", + ret, oldval, uaddr, oparg); + break; + case FUTEX_OP_OR: /* *(int *)UADDR |= OPARG; */ + __futex_atomic_op("orw $r62 = $r63, %[op]", + ret, oldval, uaddr, oparg); + break; + case FUTEX_OP_ANDN: /* *(int *)UADDR &= ~OPARG; */ + __futex_atomic_op("andnw $r62 = %[op], $r63", + ret, oldval, uaddr, oparg); + break; + case FUTEX_OP_XOR: + __futex_atomic_op("xorw $r62 = $r63, %[op]", + ret, oldval, uaddr, oparg); + break; + default: + ret = -ENOSYS; + } + + if (!ret) + *oval = oldval; + + return ret; +} + +static inline int futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr, + u32 oldval, u32 newval) +{ + int ret = 0; + + if (!access_ok(uaddr, sizeof(u32))) + return -EFAULT; + __enable_user_access(); + __asm__ __volatile__ ( + " fence \n"/* commit previous stores */ + " copyd $r63 = %[ov] \n"/* init "expect" with ov */ + " copyd $r62 = %[nv] \n"/* init "update" with nv */ + " ;;\n " + "1: acswapw 0[%[u]], $r62r63 \n" + " ;;\n " + " cb.dnez $r62? 3f \n"/* if acswap ok -> return */ + " ;;\n " + "2: lws $r63 = 0[%[u]] \n"/* fail -> load old value */ + " ;;\n " + " compw.ne $r62 = $r63, %[ov] \n"/* check if equal to "old" */ + " ;;\n " + " cb.deqz $r62? 1b \n"/* if not equal, try again */ + " ;;\n " + "3: \n" + " .section .fixup,\"ax\" \n" + "4: make %[r] = 3b \n" + " ;;\n " + " make %[r] = %[e] \n" + " igoto %[r] \n"/* goto 3b */ + " ;;\n " + " .previous \n" + " .section __ex_table,\"a\" \n" + " .align 8 \n" + " .dword 1b,4b \n" + " .dword 2b,4b \n" + ".previous \n" + : [r] "+r" (ret) + : [ov] "r" (oldval), [nv] "r" (newval), + [e] "i" (-EFAULT), [u] "r" (uaddr) + : "r62", "r63", "memory"); + __disable_user_access(); + *uval = oldval; + return ret; +} + +#endif +#endif /* _ASM_KVX_FUTEX_H */ diff --git a/arch/kvx/include/asm/io.h b/arch/kvx/include/asm/io.h new file mode 100644 index 000000000000..c5e458c59bbb --- /dev/null +++ b/arch/kvx/include/asm/io.h @@ -0,0 +1,34 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_IO_H +#define _ASM_KVX_IO_H + +#include <linux/types.h> + +#include <asm/page.h> +#include <asm/pgtable.h> + +#define _PAGE_IOREMAP _PAGE_KERNEL_DEVICE + +/* + * String version of I/O memory access operations. + */ +extern void __memcpy_fromio(void *to, const volatile void __iomem *from, + size_t count); +extern void __memcpy_toio(volatile void __iomem *to, const void *from, + size_t count); +extern void __memset_io(volatile void __iomem *dst, int c, size_t count); + +#define memset_io(c, v, l) __memset_io((c), (v), (l)) +#define memcpy_fromio(a, c, l) __memcpy_fromio((a), (c), (l)) +#define memcpy_toio(c, a, l) __memcpy_toio((c), (a), (l)) + +#include <asm-generic/io.h> + +extern int devmem_is_allowed(unsigned long pfn); + +#endif /* _ASM_KVX_IO_H */ diff --git a/arch/kvx/kernel/asm-offsets.c b/arch/kvx/kernel/asm-offsets.c new file mode 100644 index 000000000000..3e79b6dd13bd --- /dev/null +++ b/arch/kvx/kernel/asm-offsets.c @@ -0,0 +1,157 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Guillaume Thouvenin + * Yann Sionneau + */ + +#include <linux/preempt.h> +#include <linux/thread_info.h> +#include <linux/kbuild.h> +#include <linux/stddef.h> +#include <linux/sched.h> +#include <linux/bug.h> + +#include <asm/processor.h> +#include <asm/ptrace.h> +#include <asm/page.h> +#include <asm/fixmap.h> +#include <asm/page_size.h> +#include <asm/pgtable.h> +#include <asm/ptrace.h> +#include <asm/tlb_defs.h> +#include <asm/mmu_stats.h> +#include <asm/stacktrace.h> + +int foo(void) +{ + BUILD_BUG_ON(sizeof(struct pt_regs) != PT_REGS_STRUCT_EXPECTED_SIZE); + /* + * For stack alignment purposes we must make sure the pt_regs size is + * a mutliple of stack_align + */ + BUILD_BUG_ON(!IS_ALIGNED(sizeof(struct pt_regs), STACK_ALIGNMENT)); + + /* Check that user_pt_regs size matches the beginning of pt_regs */ + BUILD_BUG_ON((offsetof(struct user_pt_regs, spc) + sizeof(uint64_t)) != + sizeof(struct user_pt_regs)); + + DEFINE(FIX_GDB_MEM_BASE_IDX, FIX_GDB_BARE_DISPLACED_MEM_BASE); + +#ifdef CONFIG_DEBUG_EXCEPTION_STACK + DEFINE(STACK_REG_SIZE, ALIGN(sizeof(uint64_t), STACK_ALIGNMENT)); +#endif + + /* + * We allocate a pt_regs on the stack when entering the kernel. This + * ensures the alignment is sane. + */ + DEFINE(PT_SIZE_ON_STACK, sizeof(struct pt_regs)); + DEFINE(TI_FLAGS_SIZE, sizeof(unsigned long)); + DEFINE(QUAD_REG_SIZE, 4 * sizeof(uint64_t)); + + /* + * When restoring registers, we do not want to restore r12 + * right now since this is our stack pointer. Allow to save + * only $r13 by using this offset. + */ + OFFSET(PT_R12, pt_regs, r12); + OFFSET(PT_R13, pt_regs, r13); + OFFSET(PT_TP, pt_regs, tp); + OFFSET(PT_R14R15, pt_regs, r14); + OFFSET(PT_R16R17, pt_regs, r16); + OFFSET(PT_R18R19, pt_regs, r18); + OFFSET(PT_FP, pt_regs, fp); + OFFSET(PT_SPS, pt_regs, sps); + + /* Quad description */ + OFFSET(PT_Q0, pt_regs, r0); + OFFSET(PT_Q4, pt_regs, r4); + OFFSET(PT_Q8, pt_regs, r8); + OFFSET(PT_Q12, pt_regs, r12); + OFFSET(PT_Q16, pt_regs, r16); + OFFSET(PT_Q20, pt_regs, r20); + OFFSET(PT_Q24, pt_regs, r24); + OFFSET(PT_Q28, pt_regs, r28); + OFFSET(PT_Q32, pt_regs, r32); + OFFSET(PT_Q36, pt_regs, r36); + OFFSET(PT_R38, pt_regs, r38); + OFFSET(PT_Q40, pt_regs, r40); + OFFSET(PT_Q44, pt_regs, r44); + OFFSET(PT_Q48, pt_regs, r48); + OFFSET(PT_Q52, pt_regs, r52); + OFFSET(PT_Q56, pt_regs, r56); + OFFSET(PT_Q60, pt_regs, r60); + OFFSET(PT_CS_SPC_SPS_ES, pt_regs, cs); + OFFSET(PT_LC_LE_LS_RA, pt_regs, lc); + OFFSET(PT_ILR, pt_regs, ilr); + OFFSET(PT_ORIG_R0, pt_regs, orig_r0); + + /* + * Flags in thread info + */ + OFFSET(TASK_TI_FLAGS, task_struct, thread_info.flags); + + /* + * Stack pointers + */ + OFFSET(TASK_THREAD_KERNEL_SP, task_struct, thread.kernel_sp); + + /* + * Offsets to save registers in switch_to using quads + */ + OFFSET(CTX_SWITCH_RA_SP_R18_R19, task_struct, thread.ctx_switch.ra); + OFFSET(CTX_SWITCH_Q20, task_struct, thread.ctx_switch.r20); + OFFSET(CTX_SWITCH_Q24, task_struct, thread.ctx_switch.r24); + OFFSET(CTX_SWITCH_Q28, task_struct, thread.ctx_switch.r28); + OFFSET(CTX_SWITCH_FP, task_struct, thread.ctx_switch.fp); + +#ifdef CONFIG_ENABLE_TCA + OFFSET(CTX_SWITCH_TCA_REGS, task_struct, thread.ctx_switch.tca_regs[0]); + OFFSET(CTX_SWITCH_TCA_REGS_SAVED, task_struct, + thread.ctx_switch.tca_regs_saved); + DEFINE(TCA_REG_SIZE, sizeof(struct tca_reg)); +#endif + + /* Save area offset */ + OFFSET(TASK_THREAD_SAVE_AREA, task_struct, thread.save_area); + + /* Fast tlb refill defines */ + OFFSET(TASK_ACTIVE_MM, task_struct, active_mm); + OFFSET(MM_PGD, mm_struct, pgd); +#ifdef CONFIG_KVX_DEBUG_ASN + OFFSET(MM_CTXT_ASN, mm_struct, context.asn); +#endif + +#ifdef CONFIG_KVX_MMU_STATS + DEFINE(MMU_REFILL_SIZE, sizeof(struct mmu_refill_stats)); + + OFFSET(MMU_STATS_REFILL_USER_OFF, mmu_stats, + refill[MMU_REFILL_TYPE_USER]); + OFFSET(MMU_STATS_REFILL_KERNEL_OFF, mmu_stats, + refill[MMU_REFILL_TYPE_KERNEL]); + OFFSET(MMU_STATS_REFILL_KERNEL_DIRECT_OFF, mmu_stats, + refill[MMU_REFILL_TYPE_KERNEL_DIRECT]); + OFFSET(MMU_STATS_CYCLES_BETWEEN_REFILL_OFF, mmu_stats, + cycles_between_refill); + OFFSET(MMU_STATS_LAST_REFILL, mmu_stats, last_refill); + + OFFSET(TASK_THREAD_ENTRY_TS, task_struct, thread.trap_entry_ts); +#endif + + DEFINE(ASM_PGDIR_SHIFT, PGDIR_SHIFT); + DEFINE(ASM_PMD_SHIFT, PMD_SHIFT); + + DEFINE(ASM_PGDIR_BITS, PGDIR_BITS); + DEFINE(ASM_PMD_BITS, PMD_BITS); + DEFINE(ASM_PTE_BITS, PTE_BITS); + + DEFINE(ASM_PTRS_PER_PGD, PTRS_PER_PGD); + DEFINE(ASM_PTRS_PER_PMD, PTRS_PER_PMD); + DEFINE(ASM_PTRS_PER_PTE, PTRS_PER_PTE); + + DEFINE(ASM_TLB_PS, TLB_DEFAULT_PS); + + return 0; +} diff --git a/arch/kvx/kernel/io.c b/arch/kvx/kernel/io.c new file mode 100644 index 000000000000..0922c1d6d0f7 --- /dev/null +++ b/arch/kvx/kernel/io.c @@ -0,0 +1,96 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * derived from arch/arm/kernel/io.c + * + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#include <linux/export.h> +#include <linux/types.h> +#include <linux/io.h> + +#define REPLICATE_BYTE_MASK 0x0101010101010101 + +/* + * Copy data from IO memory space to "real" memory space. + */ +void __memcpy_fromio(void *to, const volatile void __iomem *from, size_t count) +{ + while (count && !IS_ALIGNED((unsigned long)from, 8)) { + *(u8 *)to = __raw_readb(from); + from++; + to++; + count--; + } + + while (count >= 8) { + *(u64 *)to = __raw_readq(from); + from += 8; + to += 8; + count -= 8; + } + + while (count) { + *(u8 *)to = __raw_readb(from); + from++; + to++; + count--; + } +} +EXPORT_SYMBOL(__memcpy_fromio); + +/* + * Copy data from "real" memory space to IO memory space. + */ +void __memcpy_toio(volatile void __iomem *to, const void *from, size_t count) +{ + while (count && !IS_ALIGNED((unsigned long)to, 8)) { + __raw_writeb(*(u8 *)from, to); + from++; + to++; + count--; + } + + while (count >= 8) { + __raw_writeq(*(u64 *)from, to); + from += 8; + to += 8; + count -= 8; + } + + while (count) { + __raw_writeb(*(u8 *)from, to); + from++; + to++; + count--; + } +} +EXPORT_SYMBOL(__memcpy_toio); + +/* + * "memset" on IO memory space. + */ +void __memset_io(volatile void __iomem *dst, int c, size_t count) +{ + u64 qc = __builtin_kvx_sbmm8(c, REPLICATE_BYTE_MASK); + + while (count && !IS_ALIGNED((unsigned long)dst, 8)) { + __raw_writeb(c, dst); + dst++; + count--; + } + + while (count >= 8) { + __raw_writeq(qc, dst); + dst += 8; + count -= 8; + } + + while (count) { + __raw_writeb(c, dst); + dst++; + count--; + } +} +EXPORT_SYMBOL(__memset_io); -- 2.37.2

Coolidge v1 SoC has 5 clusters of 17 kvx cores: - 16 application cores aka PE - 1 privileged core, the Resource Manager, aka RM. Linux can run in SMP config on the 16 cores of a Cluster. Memory coherency between all cores is guarenteed by the L2 cache. Co-developed-by: Clement Leger <clement(a)clement-leger.fr&gt; Signed-off-by: Clement Leger <clement(a)clement-leger.fr&gt; Co-developed-by: Julian Vetter <jvetter(a)kalray.eu&gt; Signed-off-by: Julian Vetter <jvetter(a)kalray.eu&gt; Co-developed-by: Julien Hascoet <jhascoet(a)kalray.eu&gt; Signed-off-by: Julien Hascoet <jhascoet(a)kalray.eu&gt; Co-developed-by: Louis Morhet <lmorhet(a)kalray.eu&gt; Signed-off-by: Louis Morhet <lmorhet(a)kalray.eu&gt; Co-developed-by: Luc Michel <lmichel(a)kalray.eu&gt; Signed-off-by: Luc Michel <lmichel(a)kalray.eu&gt; Co-developed-by: Marius Gligor <mgligor(a)kalray.eu&gt; Signed-off-by: Marius Gligor <mgligor(a)kalray.eu&gt; Co-developed-by: Yann Sionneau <ysionneau(a)kalray.eu&gt; Signed-off-by: Yann Sionneau <ysionneau(a)kalray.eu&gt; --- Notes: V1 -> V2: - removed L2 cache driver - removed ipi and pwr-ctrl driver (splitted in their own patch) arch/kvx/include/asm/smp.h | 42 ++++++++++++ arch/kvx/kernel/smp.c | 110 ++++++++++++++++++++++++++++++++ arch/kvx/kernel/smpboot.c | 127 +++++++++++++++++++++++++++++++++++++ include/linux/cpuhotplug.h | 2 + 4 files changed, 281 insertions(+) create mode 100644 arch/kvx/include/asm/smp.h create mode 100644 arch/kvx/kernel/smp.c create mode 100644 arch/kvx/kernel/smpboot.c diff --git a/arch/kvx/include/asm/smp.h b/arch/kvx/include/asm/smp.h new file mode 100644 index 000000000000..e4fd4d001b2c --- /dev/null +++ b/arch/kvx/include/asm/smp.h @@ -0,0 +1,42 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_SMP_H +#define _ASM_KVX_SMP_H + +#include <linux/cpumask.h> +#include <linux/irqreturn.h> + +#include <asm/sfr.h> + +#ifdef CONFIG_SMP + +/* Hook for the generic smp_call_function_many() routine. */ +void arch_send_call_function_ipi_mask(struct cpumask *mask); + +/* Hook for the generic smp_call_function_single() routine. */ +void arch_send_call_function_single_ipi(int cpu); + +void __init setup_processor(void); + +void smp_init_cpus(void); + +irqreturn_t ipi_call_interrupt(int irq, void *dev_id); + +#define raw_smp_processor_id() ((int) \ + ((kvx_sfr_get(PCR) & KVX_SFR_PCR_PID_MASK) \ + >> KVX_SFR_PCR_PID_SHIFT)) + +#define flush_cache_vmap(start, end) do { } while (0) +#define flush_cache_vunmap(start, end) do { } while (0) + +#else + +void smp_init_cpus(void) {} + +#endif /* CONFIG_SMP */ + +#endif diff --git a/arch/kvx/kernel/smp.c b/arch/kvx/kernel/smp.c new file mode 100644 index 000000000000..ed4c35a8c4bc --- /dev/null +++ b/arch/kvx/kernel/smp.c @@ -0,0 +1,110 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#include <linux/smp.h> +#include <linux/cpu.h> +#include <linux/of_irq.h> +#include <linux/cpumask.h> +#include <linux/irq_work.h> +#include <linux/mm_types.h> +#include <linux/interrupt.h> + +#include <asm/ipi.h> +#include <asm/tlbflush.h> + +enum ipi_message_type { + IPI_RESCHEDULE, + IPI_CALL_FUNC, + IPI_IRQ_WORK, + IPI_MAX +}; + +/* A collection of single bit ipi messages. */ +static struct { + unsigned long bits ____cacheline_aligned; +} ipi_data[NR_CPUS] __cacheline_aligned; + +static void send_ipi_message(const struct cpumask *mask, + enum ipi_message_type operation) +{ + unsigned long flags; + int cpu; + + /* Set operation that must be done by receiver */ + for_each_cpu(cpu, mask) + set_bit(operation, &ipi_data[cpu].bits); + + /* Commit the write before sending IPI */ + smp_wmb(); + + local_irq_save(flags); + + kvx_ipi_send(mask); + + local_irq_restore(flags); +} + +void arch_send_call_function_ipi_mask(struct cpumask *mask) +{ + send_ipi_message(mask, IPI_CALL_FUNC); +} + +void arch_send_call_function_single_ipi(int cpu) +{ + send_ipi_message(cpumask_of(cpu), IPI_CALL_FUNC); +} + +#ifdef CONFIG_IRQ_WORK +void arch_irq_work_raise(void) +{ + send_ipi_message(cpumask_of(smp_processor_id()), IPI_IRQ_WORK); +} +#endif + +static void ipi_stop(void *unused) +{ + local_cpu_stop(); +} + +void smp_send_stop(void) +{ + struct cpumask targets; + + cpumask_copy(&targets, cpu_online_mask); + cpumask_clear_cpu(smp_processor_id(), &targets); + + smp_call_function_many(&targets, ipi_stop, NULL, 0); +} + +void smp_send_reschedule(int cpu) +{ + send_ipi_message(cpumask_of(cpu), IPI_RESCHEDULE); +} + +irqreturn_t ipi_call_interrupt(int irq, void *dev_id) +{ + unsigned long *pending_ipis = &ipi_data[smp_processor_id()].bits; + + while (true) { + unsigned long ops = xchg(pending_ipis, 0); + + if (ops == 0) + return IRQ_HANDLED; + + if (ops & (1 << IPI_RESCHEDULE)) + scheduler_ipi(); + + if (ops & (1 << IPI_CALL_FUNC)) + generic_smp_call_function_interrupt(); + + if (ops & (1 << IPI_IRQ_WORK)) + irq_work_run(); + + BUG_ON((ops >> IPI_MAX) != 0); + } + + return IRQ_HANDLED; +} diff --git a/arch/kvx/kernel/smpboot.c b/arch/kvx/kernel/smpboot.c new file mode 100644 index 000000000000..987a6f014163 --- /dev/null +++ b/arch/kvx/kernel/smpboot.c @@ -0,0 +1,127 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Julian Vetter + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include <linux/io.h> +#include <linux/of.h> +#include <linux/smp.h> +#include <linux/cpu.h> +#include <linux/sched.h> +#include <linux/cpumask.h> +#include <linux/sched/mm.h> +#include <linux/mm_types.h> +#include <linux/of_platform.h> +#include <linux/sched/task_stack.h> + +#include <asm/pwr_ctrl.h> +#include <asm/tlbflush.h> +#include <asm/ipi.h> + +void *__cpu_up_stack_pointer[NR_CPUS]; +void *__cpu_up_task_pointer[NR_CPUS]; + +void __init smp_prepare_boot_cpu(void) +{ +} + +int __cpu_up(unsigned int cpu, struct task_struct *tidle) +{ + __cpu_up_stack_pointer[cpu] = task_stack_page(tidle) + THREAD_SIZE; + __cpu_up_task_pointer[cpu] = tidle; + /* We need to be sure writes are committed */ + smp_mb(); + + kvx_pwr_ctrl_cpu_poweron(cpu); + while (!cpu_online(cpu)) + cpu_relax(); + + return 0; +} + +void __init smp_cpus_done(unsigned int max_cpus) +{ +} + +void __init smp_init_cpus(void) +{ + struct cpumask cpumask; + struct device_node *cpu; + const __be32 *reg; + u32 cpu_num; + unsigned int nr_cpus = 0; + + cpumask_clear(&cpumask); + + for_each_of_cpu_node(cpu) { + if (!of_device_is_available(cpu)) + continue; + + reg = of_get_property(cpu, "reg", NULL); + if (!reg) + continue; + + cpu_num = be32_to_cpup(reg); + if (cpu_num >= nr_cpu_ids) + continue; + + nr_cpus++; + cpumask_set_cpu(cpu_num, &cpumask); + } + + pr_info("%d possible cpus\n", nr_cpus); + init_cpu_possible(&cpumask); +} + +void __init smp_prepare_cpus(unsigned int max_cpus) +{ + if (num_present_cpus() <= 1) + init_cpu_present(cpu_possible_mask); +} + +int __init setup_smp(void) +{ + int ret; + + ret = kvx_pwr_ctrl_probe(); + if (ret) + panic("Failed to probe power controller !"); + + ret = kvx_ipi_ctrl_probe(ipi_call_interrupt); + if (ret) + panic("Failed to probe IPI controller !"); + + return 0; +} + +early_initcall(setup_smp); + +/* + * C entry point for a secondary processor. + */ +void __init start_kernel_secondary(void) +{ + struct mm_struct *mm = &init_mm; + unsigned int cpu = smp_processor_id(); + + setup_processor(); + kvx_mmu_early_setup(); + + /* All kernel threads share the same mm context. */ + mmgrab(mm); + current->active_mm = mm; + cpumask_set_cpu(cpu, mm_cpumask(mm)); + + notify_cpu_starting(cpu); + set_cpu_online(cpu, true); + trace_hardirqs_off(); + + local_flush_tlb_all(); + + local_irq_enable(); + cpu_startup_entry(CPUHP_AP_ONLINE_IDLE); +} diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h index f61447913db9..f5a484547b15 100644 --- a/include/linux/cpuhotplug.h +++ b/include/linux/cpuhotplug.h @@ -152,6 +152,7 @@ enum cpuhp_state { CPUHP_AP_IRQ_RISCV_STARTING, CPUHP_AP_IRQ_LOONGARCH_STARTING, CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING, + CPUHP_AP_IRQ_KVX_STARTING, CPUHP_AP_ARM_MVEBU_COHERENCY, CPUHP_AP_MICROCODE_LOADER, CPUHP_AP_PERF_X86_AMD_UNCORE_STARTING, @@ -189,6 +190,7 @@ enum cpuhp_state { CPUHP_AP_KVM_ARM_VGIC_INIT_STARTING, CPUHP_AP_KVM_ARM_VGIC_STARTING, CPUHP_AP_KVM_ARM_TIMER_STARTING, + CPUHP_AP_KVX_TIMER_STARTING, /* Must be the last timer callback */ CPUHP_AP_DUMMY_TIMER_STARTING, CPUHP_AP_ARM_XEN_STARTING, -- 2.37.2

On 20/01/2023 15:09, Yann Sionneau wrote: Are you sure this is the result of savedefconfig? Order looks a bit odd in several places, so I want to double check. Best regards, Krzysztof

Add support for cpuinfo on kvx arch. Co-developed-by: Clement Leger <clement(a)clement-leger.fr&gt; Signed-off-by: Clement Leger <clement(a)clement-leger.fr&gt; Co-developed-by: Guillaume Thouvenin <gthouvenin(a)kalray.eu&gt; Signed-off-by: Guillaume Thouvenin <gthouvenin(a)kalray.eu&gt; Co-developed-by: Julian Vetter <jvetter(a)kalray.eu&gt; Signed-off-by: Julian Vetter <jvetter(a)kalray.eu&gt; Signed-off-by: Jules Maselbas <jmaselbas(a)kalray.eu&gt; Signed-off-by: Yann Sionneau <ysionneau(a)kalray.eu&gt; --- Notes: V1 -> V2: no changes arch/kvx/kernel/cpuinfo.c | 96 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 96 insertions(+) create mode 100644 arch/kvx/kernel/cpuinfo.c diff --git a/arch/kvx/kernel/cpuinfo.c b/arch/kvx/kernel/cpuinfo.c new file mode 100644 index 000000000000..f44c46c1e4ba --- /dev/null +++ b/arch/kvx/kernel/cpuinfo.c @@ -0,0 +1,96 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Guillaume Thouvenin + */ + +#include <linux/seq_file.h> +#include <linux/delay.h> +#include <linux/clk.h> +#include <linux/cpu.h> +#include <linux/of.h> + +unsigned long elf_hwcap __read_mostly; + +static int show_cpuinfo(struct seq_file *m, void *v) +{ + int cpu_num = *(unsigned int *)v; + struct cpuinfo_kvx *n = per_cpu_ptr(&cpu_info, cpu_num); + + seq_printf(m, "processor\t: %d\nvendor_id\t: Kalray\n", cpu_num); + + seq_printf(m, + "copro enabled\t: %s\n" + "arch revision\t: %d\n" + "uarch revision\t: %d\n", + n->copro_enable ? "yes" : "no", + n->arch_rev, + n->uarch_rev); + + seq_printf(m, + "bogomips\t: %lu.%02lu\n" + "cpu MHz\t\t: %llu.%03llu\n\n", + (loops_per_jiffy * HZ) / 500000, + ((loops_per_jiffy * HZ) / 5000) % 100, + n->freq / 1000000, (n->freq / 10000) % 100); + + return 0; +} + +static void *c_start(struct seq_file *m, loff_t *pos) +{ + if (*pos == 0) + *pos = cpumask_first(cpu_online_mask); + if (*pos >= num_online_cpus()) + return NULL; + + return pos; +} + +static void *c_next(struct seq_file *m, void *v, loff_t *pos) +{ + *pos = cpumask_next(*pos, cpu_online_mask); + + return c_start(m, pos); +} + +static void c_stop(struct seq_file *m, void *v) +{ +} + +const struct seq_operations cpuinfo_op = { + .start = c_start, + .next = c_next, + .stop = c_stop, + .show = show_cpuinfo, +}; + +static int __init setup_cpuinfo(void) +{ + int cpu; + struct clk *clk; + unsigned long cpu_freq = 1000000000; + struct device_node *node = of_get_cpu_node(0, NULL); + + clk = of_clk_get(node, 0); + if (IS_ERR(clk)) { + printk(KERN_WARNING + "Device tree missing CPU 'clock' parameter. Assuming frequency is 1GHZ"); + goto setup_cpu_freq; + } + + cpu_freq = clk_get_rate(clk); + + clk_put(clk); + +setup_cpu_freq: + of_node_put(node); + + for_each_possible_cpu(cpu) + per_cpu_ptr(&cpu_info, cpu)->freq = cpu_freq; + + return 0; +} + +late_initcall(setup_cpuinfo); -- 2.37.2

From: Jules Maselbas <jmaselbas(a)kalray.eu&gt; The Power Controller (pwr-ctrl) control cores reset and wake-up procedure. Co-developed-by: Clement Leger <clement(a)clement-leger.fr&gt; Signed-off-by: Clement Leger <clement(a)clement-leger.fr&gt; Co-developed-by: Julian Vetter <jvetter(a)kalray.eu&gt; Signed-off-by: Julian Vetter <jvetter(a)kalray.eu&gt; Co-developed-by: Louis Morhet <lmorhet(a)kalray.eu&gt; Signed-off-by: Louis Morhet <lmorhet(a)kalray.eu&gt; Co-developed-by: Marius Gligor <mgligor(a)kalray.eu&gt; Signed-off-by: Marius Gligor <mgligor(a)kalray.eu&gt; Signed-off-by: Jules Maselbas <jmaselbas(a)kalray.eu&gt; Signed-off-by: Yann Sionneau <ysionneau(a)kalray.eu&gt; --- Notes: V1 -> V2: new patch arch/kvx/include/asm/pwr_ctrl.h | 45 ++++++++++++++++ arch/kvx/platform/Makefile | 6 +++ arch/kvx/platform/pwr_ctrl.c | 91 +++++++++++++++++++++++++++++++++ 3 files changed, 142 insertions(+) create mode 100644 arch/kvx/include/asm/pwr_ctrl.h create mode 100644 arch/kvx/platform/Makefile create mode 100644 arch/kvx/platform/pwr_ctrl.c diff --git a/arch/kvx/include/asm/pwr_ctrl.h b/arch/kvx/include/asm/pwr_ctrl.h new file mode 100644 index 000000000000..25f403ba935a --- /dev/null +++ b/arch/kvx/include/asm/pwr_ctrl.h @@ -0,0 +1,45 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Marius Gligor + */ + +#ifndef _ASM_KVX_PWR_CTRL_H +#define _ASM_KVX_PWR_CTRL_H + +#ifndef __ASSEMBLY__ + +int kvx_pwr_ctrl_probe(void); + +void kvx_pwr_ctrl_cpu_poweron(unsigned int cpu); + +#endif + +/* Power controller vector register definitions */ +#define KVX_PWR_CTRL_VEC_OFFSET 0x1000 +#define KVX_PWR_CTRL_VEC_WUP_SET_OFFSET 0x10 +#define KVX_PWR_CTRL_VEC_WUP_CLEAR_OFFSET 0x20 + +/* Power controller PE reset PC register definitions */ +#define KVX_PWR_CTRL_RESET_PC_OFFSET 0x2000 + +/* Power controller global register definitions */ +#define KVX_PWR_CTRL_GLOBAL_OFFSET 0x4040 + +#define KVX_PWR_CTRL_GLOBAL_SET_OFFSET 0x10 +#define KVX_PWR_CTRL_GLOBAL_SET_PE_EN_SHIFT 0x1 + +#define PWR_CTRL_WUP_SET_OFFSET \ + (KVX_PWR_CTRL_VEC_OFFSET + \ + KVX_PWR_CTRL_VEC_WUP_SET_OFFSET) + +#define PWR_CTRL_WUP_CLEAR_OFFSET \ + (KVX_PWR_CTRL_VEC_OFFSET + \ + KVX_PWR_CTRL_VEC_WUP_CLEAR_OFFSET) + +#define PWR_CTRL_GLOBAL_CONFIG_OFFSET \ + (KVX_PWR_CTRL_GLOBAL_OFFSET + \ + KVX_PWR_CTRL_GLOBAL_SET_OFFSET) + +#endif /* _ASM_KVX_PWR_CTRL_H */ diff --git a/arch/kvx/platform/Makefile b/arch/kvx/platform/Makefile new file mode 100644 index 000000000000..c7d0abb15c27 --- /dev/null +++ b/arch/kvx/platform/Makefile @@ -0,0 +1,6 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# Copyright (C) 2018-2023 Kalray Inc. +# + +obj-$(CONFIG_SMP) += pwr_ctrl.o diff --git a/arch/kvx/platform/pwr_ctrl.c b/arch/kvx/platform/pwr_ctrl.c new file mode 100644 index 000000000000..ee35d04845ae --- /dev/null +++ b/arch/kvx/platform/pwr_ctrl.c @@ -0,0 +1,91 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include <linux/io.h> +#include <linux/slab.h> +#include <linux/types.h> +#include <linux/module.h> +#include <linux/of_address.h> +#include <linux/of_platform.h> + +#include <asm/pwr_ctrl.h> +#include <asm/symbols.h> + +struct kvx_pwr_ctrl { + void __iomem *regs; +}; + +static struct kvx_pwr_ctrl kvx_pwr_controller; + +/** + * kvx_pwr_ctrl_cpu_poweron() - Wakeup a cpu + * @cpu: cpu to wakeup + */ +void kvx_pwr_ctrl_cpu_poweron(unsigned int cpu) +{ + /* Set PE boot address */ + writeq((unsigned long long)kvx_start, + kvx_pwr_controller.regs + KVX_PWR_CTRL_RESET_PC_OFFSET); + /* Wake up processor ! */ + writeq(1ULL << cpu, + kvx_pwr_controller.regs + PWR_CTRL_WUP_SET_OFFSET); + /* Then clear wakeup to allow processor to sleep */ + writeq(1ULL << cpu, + kvx_pwr_controller.regs + PWR_CTRL_WUP_CLEAR_OFFSET); +} + +static struct device_node * __init get_pwr_ctrl_node(void) +{ + const phandle *ph; + struct device_node *cpu; + struct device_node *node; + + cpu = of_get_cpu_node(raw_smp_processor_id(), NULL); + if (!cpu) { + pr_err("Failed to get CPU node\n"); + return NULL; + } + + ph = of_get_property(cpu, "power-controller", NULL); + if (!ph) { + pr_err("Failed to get power-controller phandle\n"); + return NULL; + } + + node = of_find_node_by_phandle(be32_to_cpup(ph)); + if (!node) { + pr_err("Failed to get power-controller node\n"); + return NULL; + } + + return node; +} + +int __init kvx_pwr_ctrl_probe(void) +{ + struct device_node *ctrl; + + ctrl = get_pwr_ctrl_node(); + if (!ctrl) { + pr_err("Failed to get power controller node\n"); + return -EINVAL; + } + + if (!of_device_is_compatible(ctrl, "kalray,kvx-pwr-ctrl")) { + pr_err("Failed to get power controller node\n"); + return -EINVAL; + } + + kvx_pwr_controller.regs = of_iomap(ctrl, 0); + if (!kvx_pwr_controller.regs) { + pr_err("Failed ioremap\n"); + return -EINVAL; + } + + return 0; +} -- 2.37.2

From: Jules Maselbas <jmaselbas(a)kalray.eu&gt; The Inter-Processor Interrupt Controller (IPI) provides a fast synchronization mechanism to the software. It exposes eight independent set of registers that can be use to notify each processor in the cluster. test Co-developed-by: Clement Leger <clement(a)clement-leger.fr&gt; Signed-off-by: Clement Leger <clement(a)clement-leger.fr&gt; Co-developed-by: Guillaume Thouvenin <gthouvenin(a)kalray.eu&gt; Signed-off-by: Guillaume Thouvenin <gthouvenin(a)kalray.eu&gt; Co-developed-by: Julian Vetter <jvetter(a)kalray.eu&gt; Signed-off-by: Julian Vetter <jvetter(a)kalray.eu&gt; Co-developed-by: Luc Michel <lmichel(a)kalray.eu&gt; Signed-off-by: Luc Michel <lmichel(a)kalray.eu&gt; Signed-off-by: Jules Maselbas <jmaselbas(a)kalray.eu&gt; Signed-off-by: Yann Sionneau <ysionneau(a)kalray.eu&gt; --- Notes: V1 -> V2: new patch arch/kvx/include/asm/ipi.h | 16 ++++++ arch/kvx/platform/Makefile | 1 + arch/kvx/platform/ipi.c | 108 +++++++++++++++++++++++++++++++++++++ 3 files changed, 125 insertions(+) create mode 100644 arch/kvx/include/asm/ipi.h create mode 100644 arch/kvx/platform/ipi.c diff --git a/arch/kvx/include/asm/ipi.h b/arch/kvx/include/asm/ipi.h new file mode 100644 index 000000000000..137407a075e6 --- /dev/null +++ b/arch/kvx/include/asm/ipi.h @@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_IPI_H +#define _ASM_KVX_IPI_H + +#include <linux/irqreturn.h> + +int kvx_ipi_ctrl_probe(irqreturn_t (*ipi_irq_handler)(int, void *)); + +void kvx_ipi_send(const struct cpumask *mask); + +#endif /* _ASM_KVX_IPI_H */ diff --git a/arch/kvx/platform/Makefile b/arch/kvx/platform/Makefile index c7d0abb15c27..27f0914e0de5 100644 --- a/arch/kvx/platform/Makefile +++ b/arch/kvx/platform/Makefile @@ -4,3 +4,4 @@ # obj-$(CONFIG_SMP) += pwr_ctrl.o +obj-$(CONFIG_SMP) += ipi.o diff --git a/arch/kvx/platform/ipi.c b/arch/kvx/platform/ipi.c new file mode 100644 index 000000000000..a471039b1643 --- /dev/null +++ b/arch/kvx/platform/ipi.c @@ -0,0 +1,108 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Luc Michel + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include <linux/io.h> +#include <linux/slab.h> +#include <linux/types.h> +#include <linux/module.h> +#include <linux/of_irq.h> +#include <linux/cpumask.h> +#include <linux/interrupt.h> +#include <linux/cpuhotplug.h> +#include <linux/of_address.h> +#include <linux/of_platform.h> + +#define IPI_INTERRUPT_OFFSET 0x0 +#define IPI_MASK_OFFSET 0x20 + +/* + * IPI controller can signal RM and PE0 -> 15 + * In order to restrict that to the PE, write the corresponding mask + */ +#define KVX_IPI_CPU_MASK (~0xFFFF) + +struct kvx_ipi_ctrl { + void __iomem *regs; + unsigned int ipi_irq; +}; + +static struct kvx_ipi_ctrl kvx_ipi_controller; + +/** + * @kvx_pwr_ctrl_cpu_poweron Wakeup a cpu + * + * cpu: cpu to wakeup + */ +void kvx_ipi_send(const struct cpumask *mask) +{ + const unsigned long *maskb = cpumask_bits(mask); + + WARN_ON(*maskb & KVX_IPI_CPU_MASK); + writel(*maskb, kvx_ipi_controller.regs + IPI_INTERRUPT_OFFSET); +} + +static int kvx_ipi_starting_cpu(unsigned int cpu) +{ + enable_percpu_irq(kvx_ipi_controller.ipi_irq, IRQ_TYPE_NONE); + + return 0; +} + +static int kvx_ipi_dying_cpu(unsigned int cpu) +{ + disable_percpu_irq(kvx_ipi_controller.ipi_irq); + + return 0; +} + +int __init kvx_ipi_ctrl_probe(irqreturn_t (*ipi_irq_handler)(int, void *)) +{ + struct device_node *np; + int ret; + unsigned int ipi_irq; + void __iomem *ipi_base; + + np = of_find_compatible_node(NULL, NULL, "kalray,kvx-ipi-ctrl"); + BUG_ON(!np); + + ipi_base = of_iomap(np, 0); + BUG_ON(!ipi_base); + + kvx_ipi_controller.regs = ipi_base; + + /* Init mask for interrupts to PE0 -> PE15 */ + writel(KVX_IPI_CPU_MASK, kvx_ipi_controller.regs + IPI_MASK_OFFSET); + + ipi_irq = irq_of_parse_and_map(np, 0); + of_node_put(np); + if (!ipi_irq) { + pr_err("Failed to parse irq: %d\n", ipi_irq); + return -EINVAL; + } + + ret = request_percpu_irq(ipi_irq, ipi_irq_handler, + "kvx_ipi", &kvx_ipi_controller); + if (ret) { + pr_err("can't register interrupt %d (%d)\n", + ipi_irq, ret); + return ret; + } + kvx_ipi_controller.ipi_irq = ipi_irq; + + ret = cpuhp_setup_state(CPUHP_AP_IRQ_KVX_STARTING, + "kvx/ipi:online", + kvx_ipi_starting_cpu, + kvx_ipi_dying_cpu); + if (ret < 0) { + pr_err("Failed to setup hotplug state"); + return ret; + } + + return 0; +} -- 2.37.2