Encoding x86 Instructions

• It is time to take a look that the actual machine instruction format of the x86 CPU family.
• They don't call the x86 CPU a Complex Instruction Set Computer (CISC) for nothing!
• Although more complex instruction encodings exist, no one is going to challenge that the x86 has a complex instruction encoding!

1. x86 Instructions Overview

The prefix bytes are not the opcode expansion prefix discussed earlier - they are special bytes to modify the behavior of existing instructions.

2. x86 Instruction Format Reference

• Intel x86 instructions by opcode
• Intel x86 instructions by mnemonic
• Brief Intel x86 instruction reference.

3. x86 Opcode Sizes

1. standard one-byte opcode
2. two-byte opcode consisting of a 0Fh opcode expansion prefix byte.
   The second byte then specifies the actual instruction.

• The x86 opcode bytes are 8-bit equivalents of iii field that we discussed in simplified encoding.
• This provides for up to 512 different instruction classes, although the x86 does not yet use them all.

3.1. x86 ADD Instruction Opcode

• If s = 0 then the operands are 8-bit registers and memory locations.
• If s = 1 then the operands are either 16-bits or 32-bits:
  • Under 32-bit operating systems the default is 32-bit operands if s = 1.
  • To specify a 16-bit operand (under Windows or Linux) you must insert a special operand-size prefix byte in front of the instruction (example of this later.)

4. Encoding x86 Instruction Operands, MOD-REG-R/M Byte

The MOD field specifies x86 addressing mode:

MOD	Meaning
00	Register indirect addressing mode or SIB with no displacement (when R/M = 100) or Displacement only addressing mode (when R/M = 101).
01	One-byte signed displacement follows addressing mode byte(s).
10	Four-byte signed displacement follows addressing mode byte(s).
11	Register addressing mode.

The REG field specifies source or destination register:

REG Value	Register if data size is eight bits	Register if data size is 16-bits	Register if data size is 32 bits
000	al	ax	eax
001	cl	cx	ecx
010	dl	dx	edx
011	bl	bx	ebx
100	ah	sp	esp
101	ch	bp	ebp
110	dh	si	esi
111	bh	di	edi

The R/M field, combined with MOD, specifies either

1. the second operand in a two-operand instruction, or
2. the only operand in a single-operand instruction like NOT or NEG.

• d=0: MOD R/M <- REG, REG is the source
• d=1: REG <- MOD R/M, REG is the destination

5. General-Purpose Registers

Since the processor accesses registers more quickly than it accesses memory, you can make your programs run faster by keeping the most-frequently used data in registers.

• The EAX, EDX, ECX, EBX, EBP, EDI, and ESI registers are 32-bit general-purpose registers, used for temporary data storage and memory access.
• The AX, DX, CX, BX, BP, DI, and SI registers are 16-bit equivalents of the above, they represent the low-order 16 bits of 32-bit registers.
• The AH, DH, CH, and BH registers represent the high-order 8 bits of the corresponding registers.
• Similarly, AL, DL, CL, and BL represent the low-order 8 bits of the registers.

6. REG Field of the MOD-REG-R/M Byte

Depending on the instruction, this can be either the source or the destination operand.

Many instructions have the d (direction) field in their opcode to choose REG operand role:

1. If d=0, REG is the source,
   MOD R/M <- REG.
2. If d=1, REG is the destination,
   REG <- MOD R/M.

9. MOD R/M Byte and Addressing Modes

MOD R/M Addressing Mode
=== === ================================
 00 000 [ eax ]
 01 000 [ eax + disp8 ]               (1)
 10 000 [ eax + disp32 ]
 11 000 register  ( al / ax / eax )   (2)
 00 001 [ ecx ]
 01 001 [ ecx + disp8 ]
 10 001 [ ecx + disp32 ]
 11 001 register  ( cl / cx / ecx )
 00 010 [ edx ]
 01 010 [ edx + disp8 ]
 10 010 [ edx + disp32 ]
 11 010 register  ( dl / dx / edx )
 00 011 [ ebx ]
 01 011 [ ebx + disp8 ]
 10 011 [ ebx + disp32 ]
 11 011 register  ( bl / bx / ebx )
 00 100 SIB  Mode                     (3)
 01 100 SIB  +  disp8  Mode
 10 100 SIB  +  disp32  Mode
 11 100 register  ( ah / sp / esp )
 00 101 32-bit Displacement-Only Mode (4)
 01 101 [ ebp + disp8 ]
 10 101 [ ebp + disp32 ]
 11 101 register  ( ch / bp / ebp )
 00 110 [ esi ]
 01 110 [ esi + disp8 ]
 10 110 [ esi + disp32 ]
 11 110 register  ( dh / si / esi )
 00 111 [ edi ]
 01 111 [ edi + disp8 ]
 10 111 [ edi + disp32 ]
 11 111 register  ( bh / di / edi )

8. SIB (Scaled Index Byte) Layout

• Scaled indexed addressing mode uses the second byte (namely, SIB byte) that follows the MOD-REG-R/M byte in the instruction format.
• The MOD field still specifies the displacement size of zero, one, or four bytes.
  • The MOD-REG-R/M and SIB bytes are complex, because Intel reused 16-bit addressing circuitry in the 32-bit mode, rather than simply abandoning the 16-bit format in the 32-bit mode.
  • There are good hardware reasons for this, but the end result is a complex scheme for specifying addressing modes in the opcodes.

8.1. Scaled Indexed Addressing Mode

[ reg32 + eax*n ] MOD = 00
[ reg32 + ebx*n ] 
[ reg32 + ecx*n ]
[ reg32 + edx*n ]
[ reg32 + ebp*n ]
[ reg32 + esi*n ]
[ reg32 + edi*n ]

[ disp + reg8 + eax*n ] MOD = 01
[ disp + reg8 + ebx*n ]
[ disp + reg8 + ecx*n ]
[ disp + reg8 + edx*n ]
[ disp + reg8 + ebp*n ]
[ disp + reg8 + esi*n ]
[ disp + reg8 + edi*n ]

[ disp + reg32 + eax*n ] MOD = 10
[ disp + reg32 + ebx*n ]
[ disp + reg32 + ecx*n ]
[ disp + reg32 + edx*n ]
[ disp + reg32 + ebp*n ]
[ disp + reg32 + esi*n ]
[ disp + reg32 + edi*n ]

[ disp + eax*n ] MOD = 00, and
[ disp + ebx*n ] BASE field = 101
[ disp + ecx*n ]
[ disp + edx*n ]
[ disp + ebp*n ]
[ disp + esi*n ]
[ disp + edi*n ]

    MOD R/M  Addressing Mode
    --- ---  --------------------------- 
     00 100  SIB
     01 100  SIB + disp8
     10 100  SIB + disp32

9. Examples

9.1. Encoding ADD Instruction Example

• The ADD opcode can be decimal 0, 1, 2, or 3, depending on the direction and size bits in the opcode:

  •  

• How could we encode various forms of the ADD instruction using different addressing modes?

9.2 Encoding ADD CL, AL Instruction

• Interesting side effect of the direction bit and the MOD-REG-R/M byte organization: some instructions can have two different opcodes, and both are legal!

• For example, encoding of

      add cl, al
  

  could be 00 C1 (if d=0), or 02 C8, if d bit is set to 1.

• The possibility of opcode duality issue here applies to all instructions with two register operands.

9.3. Encoding ADD ECX, EAX Instruction

•     add ecx, eax
  

•  

• Note that we could also encode ADD ECX, EAX using the bytes 03 C8.

9.4. Encoding ADD EDX, DISPLACEMENT Instruction

• Encoding the ADD EDX, DISP Instruction:

      add edx, disp
      
  

9.5. Encoding ADD EDI, [EBX] Instruction

• Encoding the ADD EDI, [ EBX ] instruction:

      add edi, [ebx]
      
  

9.6. Encoding ADD EAX, [ ESI + disp8 ] Instruction

• Encoding the ADD EAX, [ ESI + disp8 ] instruction:

      add eax, [ esi + disp8 ]
      
  

9.7. Encoding ADD EBX, [ EBP + disp32 ] Instruction

• Encoding the ADD EBX, [ EBP + disp32 ] instruction:

      add ebx, [ ebp + disp32 ]
      
  

9.8. Encoding ADD EBP, [ disp32 + EAX*1 ] Instruction

• Encoding the ADD EBP, [ disp32 + EAX*1 ] Instruction

      add ebp, [ disp32 + eax*1 ]
      
  

9.9. Encoding ADD ECX, [ EBX + EDI*4 ] Instruction

• Encoding the ADD ECX, [ EBX + EDI*4 ] Instruction

      add ecx, [ ebx + edi*4 ]
      
  

10. Encoding ADD Immediate Instruction

Instead, x86 uses a entirely different instruction format to specify instruction with an immediate operand.

There are three rules that apply:

1. If opcode high-order bit set to 1, then instruction has an immediate constant.
2. There is no direction bit in the opcode:
   • : indeed, you cannot specify a constant as a destination operand!
   • Therefore, destination operand is always the location encoded in the MOD-R/M bits of the the MOD-REG-R/M byte.
   • In place of the direction bit d, the opcode has a sign extension x bit instead:
     • For 8-bit operands, the CPU ignores x bit.
     • For 16-bit and 32-bit operands, x bit specifies the size of the Constant following at the end of the instruction:
       • If x bit contains zero, the Constant is the same size as the operand (i.e., 16 or 32 bits).
       • If x bit contains one, the Constant is a signed 8-bit value, and the CPU sign-extends this value to the appropriate size before adding it to the operand.
     • This little x trick often makes programs shorter, because adding small-value constants to 16 or 32 bit operands is very common.
3. The third difference between the ADD-immediate and the standard ADD instruction is the meaning of the REG field in the MOD-REG-R/M byte:
   • Since the instruction implies that
     • the source operand is a constant, and
     • MOD-R/M fields specify the destination operand,
     the instruction does not need to use the REG field to specify an operand.
   • Instead, the x86 CPU uses these three bits as an opcode extension.
   • For the ADD-immediate instruction the REG bits must contain zero.
   • Other bit patterns would correspond to a different instruction.

Note that when adding a constant to a memory location, the displacement (if any) immediately precedes the immediate (constant) value in the opcode sequence.

11. Encoding Eight, Sixteen, and Thirty-Two Bit Operands

• When Intel designed the 8086, one bit in the opcode, s, selected between 8 and 16 bit integer operand sizes.
• Later, when CPU added 32-bit integers to its architecture on 80386 chip, there was a problem:
  • three encodings were needed to support 8, 16, and 32 bit sizes.
• Solution was an operand size prefix byte.
• Intel studied x86 instruction set and came to the conclusion:
  • in a 32-bit environment, programs were more likely to use 8-bit and 32-bit operands far more often than 16-bit operands.
• So Intel decided to let the size bit s in the opcode select between 8- and 32-bit operands.

11.1. Encoding Sixteen Bit Operands

To allow for 16-bit operands, Intel added prefix a 32-bit mode instruction with the operand size prefix byte with value 66h.

This prefix byte tells the CPU to operand on 16-bit data rather than 32-bit data.

• There is nothing programmer has to do explicitly to put an operand size prefix byte in front of a 16-bit instruction:
  • the assembler does this automatically as soon as 16-bit operand is found in the instruction.
• However, keep in mind that whenever you use a 16-bit operand in a 32-bit program, the instruction is longer by one byte:

      Opcode     Instruction
      --------   ------------
      41h        INC ECX
      66h 41h    INC CX

• Be careful about using 16-bit instructions if size (and to a lesser extent, speed) are important, because
  1. instructions are longer, and
  2. slower because of their effect on the instruction cache.

12. x86 Instruction Prefix Bytes

• x86 instruction can have up to 4 prefixes.
• Each prefix adjusts interpretation of the opcode:
  1. Repeat/lock prefix byte guarantees that instruction will have exclusive use of all shared memory, until the instruction completes execution:

             F0h = LOCK

  2. String manipulation instruction prefixes

             F3h = REP, REPE
             F2h = REPNE

     where
     • REP repeats instruction the number of times specified by iteration count ECX.
     • REPE and REPNE prefixes allow to terminate loop on the value of ZF CPU flag.
     Related string manipulation instructions are:
     • MOVS, move string
     • STOS, store string
     • SCAS, scan string
     • CMPS, compare string, etc.
     See also string manipulation sample program: rep_movsb.asm
  3. Segment override prefix causes memory access to use specified segment instead of default segment designated for instruction operand.

             2Eh = CS
             36h = SS
             3Eh = DS
             26h = ES
             64h = FS
             65h = GS

  4. Operand override, 66h. Changes size of data expected by default mode of the instruction e.g. 16-bit to 32-bit and vice versa.
  5. Address override, 67h. Changes size of address expected by the instruction. 32-bit address could switch to 16-bit and vice versa.

13. Alternate Encodings for Instructions

• To shorten program code, Intel created alternate (shorter) encodings of some very commonly used instructions.
• For example, x86 provides a single byte opcode for

      add al, constant    ; one-byte opcode and no MOD-REG-R/M byte
      add eax, constant   ; one-byte opcode and no MOD-REG-R/M byte

  the opcodes are 04h and 05h, respectively. Also,
• These instructions are one byte shorter than their standard ADD immediate counterparts.
• Note that

      add ax, constant   ; operand size prefix byte + one-byte opcode, no MOD-REG-R/M byte

  requires an operand size prefix just as a standard ADD AX, constant instruction, yet is still one byte shorter than the corresponding standard version of ADD immediate.
• Any decent assembler will automatically choose the shortest possible instruction when translating program into machine code.
• Intel only provides alternate encodings only for the accumulator registers AL, AX, EAX.
• This is a good reason to use accumulator registers if you have a choice

  (also a good reason to take some time and study encodings of the x86 instructions.)

14. x86 Opcode Summary

• x86 opcodes are represented by one or two bytes.
• Opcode could extend into unused bits of MOD-REG-R/M byte.
• Opcode encodes information about
  • operation type,
  • operands,
  • size of each operand, including the size of an immediate operand.

14.1. MOD-REG-R/M Byte Summary

• MOD-REG-R/M byte follows one or two opcode bytes of the instruction
• It provides addressing mode information for one or two operands.
• If operand is in memory, or operand is a register:
  • MOD field (bits [7:6]), combined with the R/M field (bits [2:0]), specify memory/register operand, as well as its addressing mode.
  • REG field (bits [5:3]) specifies another register operand in of the two-operand instruction.

15. ISA Design Considerations

• Instruction set architecture design that can stand the test of time is a true intellectual challenge.
• It takes several compromises between space and efficiency to assign opcodes and encode instruction formats.
• Today people are using Intel x86 instruction set for purposes never intended by original designers.
• Extending the CPU is a very difficult task.
• The instruction set can become extremely complex.
• If x86 CPU was designed from scratch today, it would have a totally different ISA!
• Software developers usually don't have a problem adapting to a new architecture when writing new software...

  ...but they are very resistant to moving existing software from one platform to another.

• This is the primary reason the Intel x86 platform remains so popular to this day.

15.1. ISA Design Challenges

• Allowing for future expansion of the chip requires some undefined opcodes.
• From the beginning there should be a balance between the number of undefined opcodes and
  1. the number of initial instructions, and
  2. the size of your opcodes (including special assignments.)
• Hard decisions:
  • Reduce the number of instructions in the initial instruction set?
  • Increase the size of the opcode?
  • Rely on an opcode prefix byte(s), which makes later added instructions longer?
• There are no easy answers to these challenges for CPU designers!

16. Intel Architecture Software Developer's Manual

1. Volume 1 , Intel Basic Architecture: Order Number 243190 , PDF, 2.6 MB.
2. Volume 2 , Instruction Set Reference: Order Number 243191 , PDF, 6.6 MB.
3. Volume 3 , System Programing Guide: Order Number 243192 , PDF, 5.1 MB.

16.1. Intel Instruction Set Reference (Volume2)

• Chapter 3 of the Instruction Set Reference describes
  • each Intel instruction in detail
  • algorithmic description of each operation
  • effect on flags
  • operand(s), their sizes and attributes
  • CPU exceptions that may be generated.
• The instructions are arranged in alphabetical order.
• Appendix A provides opcode map for the entire Intel Architecture instruction set.

16.2. Chapter 3 of Intel Instruction Set Reference

• Chapter 3 begins with instruction format example and explains the Opcode column encoding.
• The Opcode column gives the complete machine codes as it is understood by the CPU.
• When possible, the actual machine code bytes are given as exact hexadecimal bytes, in the same order in which they appear in memory.
• However, there are opcode definitions other than hexadecimal bytes...

16.3. Intel Reference Opcode Bytes

• Fow example,

  

16.4. Intel Reference Opcode Bytes, Cont.

• /digit - A digit between 0 and 7 indicates that
  • The reg field of Mod R/M byte contains the instruction opcode extension.
  • The r/m (register or memory) operand of Mod R/M byte indicates

        R/M Addressing Mode
        === ===========================
        000 register ( al / ax / eax )
        001 register ( cl / cx / ecx )
        010 register ( dl / dx / edx )
        011 register ( bl / bx / ebx )
        100 register ( ah / sp / esp )
        101 register ( ch / bp / ebp )
        110 register ( dh / si / esi )
        111 register ( bh / di / edi )

• The size bit in the opcode specifies 8 or 32-bit register size.
• A 16-bit register requires a prefix byte:

      Opcode     Instruction
      --------   ------------
      41h        INC ECX
      66h 41h    INC CX

16.5. Intel Reference Opcode Bytes, Cont.

• /r - Indicates that the instruction uses the Mod R/M byte of the instruction.
• Mod R/M byte contains both
  • a register operand reg and
  • an r/m (register or memory) operand.

16.6. Intel Reference Opcode Bytes, Cont.

• cb, cw, cd, cp - A 1-byte (cb), 2-byte (cw), 4-byte (cd), or 6-byte (cp) value,
  following the opcode, is used to specify
  • a code offset,
  • and possibly a new value for the code segment register CS.

16.7. Intel Reference Opcode Bytes, Cont.

• ib, iw, id - A 1-byte (ib), 2-byte (iw), or 4-byte (id) indicates presence of the immediate operand in the instruction.
• Typical order of opcode bytes is
  • opcode
  • Mod R/M byte (optional)
  • SIB scale-indexing byte (optional)
  • immediate operand.
• The opcode determines if the operand is a signed value.
• All words and doublewords are given with the low-order byte first (little endian).

16.8. Intel Reference Opcode Bytes, Cont.

• +rb, +rw, +rd - A register code, from 0 through 7, added to the hexadecimal byte given at the left of the plus sign to form a single opcode byte.
• Register Encodings Associated with the +rb, +rw, and +rd:

  

  For example,

  

16.9. Intel Reference Instruction Column

• The Instruction column gives the syntax of the instruction statement as it would appear in a 386 Assembly program.
• For example,