X86-64 Instruction Encoding

This article describes how x86 and x86-64 instructions are encoded.

General Overview

An x86-64 instruction may be at most 15 bytes in length. It consists of the following components in the given order, where the prefixes are at the least-significant (lowest) address in memory:

• Legacy prefixes (1-4 bytes, optional)
• Opcode with prefixes (1-4 bytes, required)
• ModR/M (1 byte, if required)
• SIB (1 byte, if required)
• Displacement (1, 2, 4 or 8 bytes, if required)
• Immediate (1, 2, 4 or 8 bytes, if required)

Registers

The registers are encoded using the 4-bit values in the X.Reg column of the following table.

1: When any REX prefix is used, SPL, BPL, SIL and DIL are used. Otherwise, without any REX prefix AH, CH, DH and BH are used.

Legacy Prefixes

Each instruction can have up to four prefixes. Sometimes a prefix is required for the instruction while it loses it's original meaning (i.e. a 'mandatory prefix'). The following prefixes can be used, the order does not matter:

• Prefix group 1
  • 0xF0: LOCK prefix
  • 0xF2: REPNE/REPNZ prefix
  • 0xF3: REP or REPE/REPZ prefix
• Prefix group 2
  • 0x2E: CS segment override
  • 0x36: SS segment override
  • 0x3E: DS segment override
  • 0x26: ES segment override
  • 0x64: FS segment override
  • 0x65: GS segment override
  • 0x2E: Branch not taken
  • 0x3E: Branch taken
• Prefix group 3
  • 0x66: Operand-size override prefix
• Prefix group 4
  • 0x67: Address-size override prefix

When there are two or more prefixes from a single group, the behavior is undefined. Some processors ignore the subsequent prefixes from the same group, or use only the last prefix specified for any group.

LOCK prefix

With the LOCK prefix, certain read-modify-write instructions are executed atomically. The LOCK prefix can only be used with the following instructions or an Invalid Opcode Exception occurs: ADC, ADD, AND, BTC, BTR, BTS, CMPXCHG, CMPXCHG8B, CMPXCHG16B, DEC, INC, NEG, NOT, OR, SBB, SUB, XADD, XCHG and XOR.

REPNE/REPNZ, REP and REPE/REPZ prefixes

The repeat prefixes cause string handling instructions to be repeated.

The REP prefix will repeat the associated instruction up to CX times, decreasing CX with every repetition. It can be used with the INS, LODS, MOVS, OUTS and STOS instructions.

REPE and REPZ are synonyms and repeat the instruction until CX reaches 0 or when ZF is set to 0. It can be used with the CMPS, CMPSB, CMPSD, CMPSW, SCAS, SCASB, SCASD and SCASW instructions.

REPNE and REPNZ also are synonyms and repeat the instruction until CX reaches 0 or when ZF is set to 1. It can be used with the CMPS, CMPSB, CMPSD, CMPSW, SCAS, SCASB, SCASD and SCASW instructions

CS, SS, DS, ES, FS and GS segment override prefixes

Segment overrides are used with instructions that reference non-stack memory. The default segment is implied by the instruction, and using a specific override forces the use of the specified segment for memory operands.

In 64-bit the CS, SS, DS and ES segment overrides are ignored.

Branch taken/not taken prefixes

Branch hints may be used to lessen the impact of branch misprediction somewhat. The 'branch taken' hint is a strong hint, while the 'branch not taken' hint is a weak hint. The branch hints are only supported by Intel since the Pentium 4. Whether using them on AMD architectures has any (positive or negative) effect at all is not known.

Operand-size and address-size override prefix

The default operand-size and address-size can be overridden using these prefix. See the following table:

1: Certain instructions default to (or are fixed at) 64-bit operands and do not need the REX prefix for this, see this table.

NASM

NASM determines the operand size by looking at the MODRM.reg or (for a register) MODRM.rm fields. When they are both 32-bit, the operand size becomes 32-bit. Same for 16-bit and 64-bit. When they differ, an error occurs at compile time. The address size is determined by looking at (for a memory operand) the MODRM.rm field, or the SIB.base, SIB.index and displacement, in that order. So when SIB.base uses a 16-bit register (such as AX), the address size becomes 16-bit. Using a 32-bit displacement will result in the displacement being truncated.

Opcode

The x86-64 instruction set defines many opcodes and many ways to encode them, depending on several factors.

Legacy opcodes

Legacy (and x87) opcodes consist of, in this order:

• mandatory prefix;
• REX prefix;
• opcode.

Mandatory prefix

Certain instructions (most notably the SIMD instructions) require a mandatory prefix (0x66, 0xF2 or 0xF3), which looks like a normal modifier prefix. When a mandatory prefix is required, it is put with the modifier prefixes before the REX prefix (if any).

REX prefix

The REX prefix is only available in long mode.

Usage

A REX prefix must be encoded when:

• using 64-bit operand size and the instruction does not default to 64-bit operand size; or
• using one of the extended registers (R8 to R15, XMM8 to XMM15, YMM8 to YMM15, CR8 to CR15 and DR8 to DR15); or
• using one of the uniform byte registers SPL, BPL, SIL or DIL.

A REX prefix must not be encoded when:

• using one of the high byte registers AH, CH, BH or DH.

In all other cases, the REX prefix is ignored. The use of multiple REX prefixes is undefined, although processors seem to use only the last REX prefix.

Instructions that default to 64-bit operand size in long mode are:

Encoding

  7                           0
┌───┬───┬───┬───┬───┬───┬───┬───┐
│ 0 │ 1 │ 0 │ 0 │ W │ R │ X │ B │
└───┴───┴───┴───┴───┴───┴───┴───┘

Opcode

The opcode can be 1, 2 or 3 bytes in length. Depending on the opcode escape sequence, a different opcode map is selected. Possible opcode sequences are:

• <op>
• 0x0F <op>
• 0x0F 0x38 <op>
• 0x0F 0x3A <op>

Note that opcodes can specify that the REG field in the ModR/M byte is fixed at a particular value.

VEX/XOP opcodes

A VEX/XOP prefix must be encoded when:

• the instruction has only its VEX/XOP opcode and no legacy opcode; or
• 256-bit YMM registers are used; or
• more than three operands are used (e.g. nondestructive-source operations); or
• when using 128-bit XMM destination registers, bits 128-255 of the corresponding YMM register must be cleared.

A VEX/XOP prefix must not be encoded when:

• when using 128-bit XMM destination registers, bits 128-255 of the corresponding YMM register must not be changed.

There are many VEX and XOP instructions, all of which can be encoded using the three byte VEX/XOP escape prefix. The VEX and XOP escape prefixes use fields with the following semantics:

Three byte VEX escape prefix

The layout is as follows, starting with a byte with value 0xC4:

  7                           0       7                           0     7                           0
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+
| 1   1   0   0   0   1   0   0 |   |~R |~X |~B |     map_select    |   |W/E|     ~vvvv     | L |   pp  |
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+

A VEX instruction whose values for certain fields are VEX.~X == 1, VEX.~B == 1, VEX.W/E == 0 and map_select == b00001 may be encoded using the two byte VEX escape prefix.

Three byte XOP escape prefix

The layout is the same as the three-byte VEX escape prefix, but with initial byte value 0x8F:

  7                           0       7                           0     7                           0
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+
| 1   0   0   0   1   1   1   1 |   |~R |~X |~B |     map_select    |   |W/E|     ~vvvv     | L |   pp  |
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+

Two byte VEX escape prefix

A VEX instruction whose values for certain fields are VEX.~X == 1, VEX.~B == 1, VEX.W/E == 0 and map_select == b00001 may be encoded using the two byte VEX escape prefix. The layout is as follows:

  7                           0       7                           0
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+
| 1   1   0   0   0   1   0   1 |   |~R |     ~vvvv     | L |   pp  |
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+

3DNow! opcodes

3DNow! opcodes consist of, in this order:

• fixed opcode;
• (ModR/M, SIB, displacement);
• immediate opcode byte.

Fixed opcode

All 3DNow! opcodes have a fixed two-byte sequence equal to 0x0F 0x0F in the opcode position of the instruction.

Immediate opcode byte

3DNow! instructions encode the actual opcode as an 8-bit immediate value trailing the instruction (thus after the ModR/M, SIB and displacement).

ModR/M and SIB bytes

The ModR/M and SIB bytes are used to encode up to two operands of an instruction, each of which is a direct register or effective memory address.

ModR/M

The ModR/M byte encodes a register or an opcode extension, and a register or a memory address. It has the following fields:

  7                           0
+---+---+---+---+---+---+---+---+
|  mod  |    reg    |     rm    |
+---+---+---+---+---+---+---+---+

16-bit addressing

These are the meanings of the Mod (vertically) and REX/VEX/XOP.B and R/M bits (horizontally) for 16-bit addressing. B.R/M and Mod are in binary. The SIB-byte is not used in 16-bit addressing. In Long processing mode there is no way to specify 16-bit addresses.

32/64-bit addressing

These are the meanings of the Mod (vertically) and REX/VEX/XOP.B and R/M bits (horizontally) for 32 and 64-bit addressing. B.R/M and Mod are in binary.

1: In protected/compatibility mode, this is just disp32, but in long mode this is [RIP]+disp32 (for 64-bit addresses) or [EIP]+disp32 (for 32-bit addresses, i.e. with address-size override prefix, see here).
2: In long mode, to encode disp32 as in protected/compatibility mode, use the SIB byte.

RIP/EIP-relative addressing

Addressing in x86-64 can be relative to the current instruction pointer value. This is indicated with the RIP (64-bit) and EIP (32-bit) instruction pointer registers, which are not otherwise exposed to the program and may not exist physically. RIP-relative addressing allows object files to be location independent.

SIB

The SIB byte has the following fields:

  7                           0
+---+---+---+---+---+---+---+---+
| scale |   index   |    base   |
+---+---+---+---+---+---+---+---+

32/64-bit addressing

The meaning of the SIB byte while using 32 or 64-bit addressing is as follows. The ModR/M byte's Mod field and the SIB byte's index field are used vertically, the SIB byte's base field and REX/VEX/XOP.B bit horizontally. The s is the scaling factor. B.Base, X.Index and Mod are in binary.

1: No base register is encoded.
2: No index register is encoded.

Displacement

A displacement value is a 1, 2, 4, or 8 byte offset added to the calculated address. When an 8 byte displacement is used, no immediate operand is encoded.

The displacement value, if any, follows the ModR/M and SIB bytes discussed above. When the ModR/M or SIB tables state that a disp value is required, or without a ModR/M byte the use of moffset (AMD) or moffs (Intel) in the mnemonic syntax of the instruction, then the displacement bytes are required.

Immediate

Some instructions require an immediate value. The instruction (and the operand-size column in the above table) determine the length of the immediate value. The imm8 mnemonic (or 8-bit operand-size) means a one byte immediate value, imm16 (or 16-bit operand-size) means a two byte immediate value, imm32 (or 32-bit operand-size) a four byte value and imm64 (or 64-bit operand-size) an eight byte value. When an 8 byte immediate value is encoded, no displacement can be encoded.

See Also

External References

• AMD64 Architecture Programmer's Manual Volume 3: General-Purpose and System Instructions
• Intel 64 and IA-32 Architectures Software Developer's Manuals

• X86 CPU