SYSENTER on Intel CPUs

Question

So AFAIK the syscall instruction, is AMD's equivalent to sysenter . So in theory one should only find a syscall instruction on AMD chips right? Well, apparently that's not the case, as I was messing with ntdll.dll and ntdll.dll (WOW64 version), I found that the regular version uses syscall whereas the ntdll.dll from WOW64 uses sysenter . Why is that?

Answer 1

All x86-64 CPUs support syscall in 64-bit mode; it's the only way to make 64-bit system calls.

32-bit code uses whatever the CPU supports that's faster than int .

---

Your info about only AMD supporting syscall is true only in 32-bit user-space mode (legacy and compat modes).

Intel's sysenter became the primary choice for 32-bit user-space; Intel won that fight for dominance. Also, apparently AMD's legacy-mode syscall is a nightmare for the kernel to deal with; 32-bit Linux kernels don't even enable it. 64-bit Linux kernels do allow syscall from 32-bit user-space (compat mode) on AMD CPUs that support that. (Some links to the relevant comments on kernel asm entry points in this answer .)

Note that AMD CPUs don't support sysenter in compat mode, only legacy mode, so under a 64-bit kernel apparently you have to use syscall in 32-bit user-space if you want to avoid the slow int 0x80 on AMD.

---

AMD designed AMD64 (which became x86-64), and defined a new (fairly good) behaviour for how syscall works in 64-bit mode which is different from how it works in 32-bit mode. (eg in 64-bit userspace it saves the old RFLAGS into R11 , which doesn't exist in legacy mode and thus can't be what it does there.)

Intel adopted the 64-bit syscall as part of implementing their version of x86-64 in a way that's compatible with AMD's. (Modulo some implementation bugs, eg what happens if you attempt to sysret with a non-canonical RCX user-space return address; on Intel the fault is taken with privilege level = ring 0, but with RSP still the already-restored user-space stack => another thread can take over the kernel. So kernels can only use it safely if RCX is known safe.)

ie AMD's system call instruction won for x86-64 because they designed AMD64 while Intel was betting on IA-64 (Itanium); their syscall instruction became the only standard that anyone uses on x86-64 because there's no reason to use anything else. syscall is efficient and meets the needs of kernel devs.

Dispatching to pick an instruction that works on the current CPU is thus unnecessary.

---

https://reverseengineering.stackexchange.com/questions/16454/struggling-between-syscall-or-sysenter-windows explains more details.