diff --git a/Chapters/ObjectStructure/figures/32IndexableLastField.png b/Chapters/2-ObjectStructure/figures/32IndexableLastField.png similarity index 100% rename from Chapters/ObjectStructure/figures/32IndexableLastField.png rename to Chapters/2-ObjectStructure/figures/32IndexableLastField.png diff --git a/Chapters/ObjectStructure/figures/32bitsEmpty.graffle b/Chapters/2-ObjectStructure/figures/32bitsEmpty.graffle similarity index 100% rename from Chapters/ObjectStructure/figures/32bitsEmpty.graffle rename to Chapters/2-ObjectStructure/figures/32bitsEmpty.graffle diff --git a/Chapters/ObjectStructure/figures/32bitsImmediate.graffle b/Chapters/2-ObjectStructure/figures/32bitsImmediate.graffle similarity index 100% rename from Chapters/ObjectStructure/figures/32bitsImmediate.graffle rename to Chapters/2-ObjectStructure/figures/32bitsImmediate.graffle diff --git a/Chapters/ObjectStructure/figures/32bitsImmediate.pdf b/Chapters/2-ObjectStructure/figures/32bitsImmediate.pdf similarity index 100% rename from Chapters/ObjectStructure/figures/32bitsImmediate.pdf rename to Chapters/2-ObjectStructure/figures/32bitsImmediate.pdf diff --git a/Chapters/ObjectStructure/figures/64bitsFloatImmediate.graffle b/Chapters/2-ObjectStructure/figures/64bitsFloatImmediate.graffle similarity index 100% rename from Chapters/ObjectStructure/figures/64bitsFloatImmediate.graffle rename to Chapters/2-ObjectStructure/figures/64bitsFloatImmediate.graffle diff --git a/Chapters/ObjectStructure/figures/64bitsFloatImmediate.pdf b/Chapters/2-ObjectStructure/figures/64bitsFloatImmediate.pdf similarity index 100% rename from Chapters/ObjectStructure/figures/64bitsFloatImmediate.pdf rename to Chapters/2-ObjectStructure/figures/64bitsFloatImmediate.pdf diff --git a/Chapters/ObjectStructure/figures/64bitsImmediate.graffle b/Chapters/2-ObjectStructure/figures/64bitsImmediate.graffle similarity index 100% rename from Chapters/ObjectStructure/figures/64bitsImmediate.graffle rename to Chapters/2-ObjectStructure/figures/64bitsImmediate.graffle diff --git a/Chapters/ObjectStructure/figures/64bitsImmediate.pdf b/Chapters/2-ObjectStructure/figures/64bitsImmediate.pdf similarity index 100% rename from Chapters/ObjectStructure/figures/64bitsImmediate.pdf rename to Chapters/2-ObjectStructure/figures/64bitsImmediate.pdf diff --git a/Chapters/ObjectStructure/figures/Aligment2.graffle b/Chapters/2-ObjectStructure/figures/Aligment2.graffle similarity index 100% rename from Chapters/ObjectStructure/figures/Aligment2.graffle rename to Chapters/2-ObjectStructure/figures/Aligment2.graffle diff --git a/Chapters/ObjectStructure/figures/Aligment2.pdf b/Chapters/2-ObjectStructure/figures/Aligment2.pdf similarity index 100% rename from Chapters/ObjectStructure/figures/Aligment2.pdf rename to Chapters/2-ObjectStructure/figures/Aligment2.pdf diff --git a/Chapters/ObjectStructure/figures/Alignment.graffle b/Chapters/2-ObjectStructure/figures/Alignment.graffle similarity index 100% rename from Chapters/ObjectStructure/figures/Alignment.graffle rename to Chapters/2-ObjectStructure/figures/Alignment.graffle diff --git a/Chapters/ObjectStructure/figures/Alignment.pdf b/Chapters/2-ObjectStructure/figures/Alignment.pdf similarity index 100% rename from Chapters/ObjectStructure/figures/Alignment.pdf rename to Chapters/2-ObjectStructure/figures/Alignment.pdf diff --git a/Chapters/ObjectStructure/figures/DecodingTobeRedone.png b/Chapters/2-ObjectStructure/figures/DecodingTobeRedone.png similarity index 100% rename from Chapters/ObjectStructure/figures/DecodingTobeRedone.png rename to Chapters/2-ObjectStructure/figures/DecodingTobeRedone.png diff --git a/Chapters/ObjectStructure/figures/LittleEndian.graffle b/Chapters/2-ObjectStructure/figures/LittleEndian.graffle similarity index 100% rename from Chapters/ObjectStructure/figures/LittleEndian.graffle rename to Chapters/2-ObjectStructure/figures/LittleEndian.graffle diff --git a/Chapters/ObjectStructure/figures/LittleEndian.pdf b/Chapters/2-ObjectStructure/figures/LittleEndian.pdf similarity index 100% rename from Chapters/ObjectStructure/figures/LittleEndian.pdf rename to Chapters/2-ObjectStructure/figures/LittleEndian.pdf diff --git a/Chapters/ObjectStructure/figures/ObjectHeader.graffle b/Chapters/2-ObjectStructure/figures/ObjectHeader.graffle similarity index 100% rename from Chapters/ObjectStructure/figures/ObjectHeader.graffle rename to Chapters/2-ObjectStructure/figures/ObjectHeader.graffle diff --git a/Chapters/ObjectStructure/figures/ObjectHeader.pdf b/Chapters/2-ObjectStructure/figures/ObjectHeader.pdf similarity index 100% rename from Chapters/ObjectStructure/figures/ObjectHeader.pdf rename to Chapters/2-ObjectStructure/figures/ObjectHeader.pdf diff --git a/Chapters/ObjectStructure/figures/architecture32vs64.pdf b/Chapters/2-ObjectStructure/figures/architecture32vs64.pdf similarity index 100% rename from Chapters/ObjectStructure/figures/architecture32vs64.pdf rename to Chapters/2-ObjectStructure/figures/architecture32vs64.pdf diff --git a/Chapters/ObjectStructure/figures/architecture32vs64.svg b/Chapters/2-ObjectStructure/figures/architecture32vs64.svg similarity index 100% rename from Chapters/ObjectStructure/figures/architecture32vs64.svg rename to Chapters/2-ObjectStructure/figures/architecture32vs64.svg diff --git a/Chapters/ObjectStructure/figures/objectLayout.pdf b/Chapters/2-ObjectStructure/figures/objectLayout.pdf similarity index 100% rename from Chapters/ObjectStructure/figures/objectLayout.pdf rename to Chapters/2-ObjectStructure/figures/objectLayout.pdf diff --git a/Chapters/ObjectStructure/figures/objectLayout.svg b/Chapters/2-ObjectStructure/figures/objectLayout.svg similarity index 100% rename from Chapters/ObjectStructure/figures/objectLayout.svg rename to Chapters/2-ObjectStructure/figures/objectLayout.svg diff --git a/Chapters/ObjectStructure/figures/references.drawio b/Chapters/2-ObjectStructure/figures/references.drawio similarity index 100% rename from Chapters/ObjectStructure/figures/references.drawio rename to Chapters/2-ObjectStructure/figures/references.drawio diff --git a/Chapters/ObjectStructure/figures/references.pdf b/Chapters/2-ObjectStructure/figures/references.pdf similarity index 100% rename from Chapters/ObjectStructure/figures/references.pdf rename to Chapters/2-ObjectStructure/figures/references.pdf diff --git a/Chapters/ObjectStructure/objectStructure.md b/Chapters/2-ObjectStructure/objectStructure.md similarity index 100% rename from Chapters/ObjectStructure/objectStructure.md rename to Chapters/2-ObjectStructure/objectStructure.md diff --git a/Chapters/3-MethodsAndBytecode/contextReification.md b/Chapters/3-MethodsAndBytecode/contextReification.md new file mode 100644 index 0000000..08570d7 --- /dev/null +++ b/Chapters/3-MethodsAndBytecode/contextReification.md @@ -0,0 +1,13 @@ +### Contexts + + +Push, store, and jump bytecodes require only small changes to the state of the interpreter. +Objects may be moved to or from the stack, and the instruction pointer is always changed; but most of the state remains the same. Send and return bytecodes require much larger changes to the interpreter's state. + +When a new message is sent \(for example message `x` or `-` above\), all five parts of the interpreter's state may have to be changed to execute a different CompiledMethod in response to this new message. The interpreter's old state must be remembered because the bytecodes after the send must be executed after the value of the message is returned. + +The interpreter saves its state in objects called _contexts_. There will be many contexts in the system at any one time. The context that represents the current state of the interpreter is called the _active_ context. When a send bytecode in the active context's CompiledMethod requires a new compiled method to be executed, the active context becomes _suspended_ and a new context is created and made active. The suspended context retains the state associated with the original compiled method until that context becomes active again. A context must remember the context that it suspended so that the suspended context can be resumed when a result is returned. The suspended context is called the new context's _sender_. + + + +We extend the form used to show the interpreter's state. The active context will be indicated by the word **Active** in its top delimiter. Suspended contexts will say **Suspended**. diff --git a/Chapters/3-MethodsAndBytecode/figures/20220928_114230.jpg b/Chapters/3-MethodsAndBytecode/figures/20220928_114230.jpg new file mode 100644 index 0000000..1011aad Binary files /dev/null and b/Chapters/3-MethodsAndBytecode/figures/20220928_114230.jpg differ diff --git a/Chapters/CallingConventions/figures/AfterSend.graffle b/Chapters/3-MethodsAndBytecode/figures/AfterSend.graffle similarity index 100% rename from Chapters/CallingConventions/figures/AfterSend.graffle rename to Chapters/3-MethodsAndBytecode/figures/AfterSend.graffle diff --git a/Chapters/CallingConventions/figures/AfterSend.pdf b/Chapters/3-MethodsAndBytecode/figures/AfterSend.pdf similarity index 100% rename from Chapters/CallingConventions/figures/AfterSend.pdf rename to Chapters/3-MethodsAndBytecode/figures/AfterSend.pdf diff --git a/Chapters/CallingConventions/figures/AfterSend.png b/Chapters/3-MethodsAndBytecode/figures/AfterSend.png similarity index 100% rename from Chapters/CallingConventions/figures/AfterSend.png rename to Chapters/3-MethodsAndBytecode/figures/AfterSend.png diff --git a/Chapters/CallingConventions/figures/BeforeSend.graffle b/Chapters/3-MethodsAndBytecode/figures/BeforeSend.graffle similarity index 100% rename from Chapters/CallingConventions/figures/BeforeSend.graffle rename to Chapters/3-MethodsAndBytecode/figures/BeforeSend.graffle diff --git a/Chapters/CallingConventions/figures/BeforeSend.pdf b/Chapters/3-MethodsAndBytecode/figures/BeforeSend.pdf similarity index 100% rename from Chapters/CallingConventions/figures/BeforeSend.pdf rename to Chapters/3-MethodsAndBytecode/figures/BeforeSend.pdf diff --git a/Chapters/CallingConventions/figures/BeforeSend.png b/Chapters/3-MethodsAndBytecode/figures/BeforeSend.png similarity index 100% rename from Chapters/CallingConventions/figures/BeforeSend.png rename to Chapters/3-MethodsAndBytecode/figures/BeforeSend.png diff --git a/Chapters/CallingConventions/figures/GeneralArgument.graffle b/Chapters/3-MethodsAndBytecode/figures/GeneralArgument.graffle similarity index 100% rename from Chapters/CallingConventions/figures/GeneralArgument.graffle rename to Chapters/3-MethodsAndBytecode/figures/GeneralArgument.graffle diff --git a/Chapters/CallingConventions/figures/GeneralArgument.pdf b/Chapters/3-MethodsAndBytecode/figures/GeneralArgument.pdf similarity index 100% rename from Chapters/CallingConventions/figures/GeneralArgument.pdf rename to Chapters/3-MethodsAndBytecode/figures/GeneralArgument.pdf diff --git a/Chapters/CallingConventions/figures/GeneralArgument.png b/Chapters/3-MethodsAndBytecode/figures/GeneralArgument.png similarity index 100% rename from Chapters/CallingConventions/figures/GeneralArgument.png rename to Chapters/3-MethodsAndBytecode/figures/GeneralArgument.png diff --git a/Chapters/CallingConventions/figures/StackGrowingDown.graffle b/Chapters/3-MethodsAndBytecode/figures/StackGrowingDown.graffle similarity index 100% rename from Chapters/CallingConventions/figures/StackGrowingDown.graffle rename to Chapters/3-MethodsAndBytecode/figures/StackGrowingDown.graffle diff --git a/Chapters/CallingConventions/figures/StackGrowingDown.pdf b/Chapters/3-MethodsAndBytecode/figures/StackGrowingDown.pdf similarity index 100% rename from Chapters/CallingConventions/figures/StackGrowingDown.pdf rename to Chapters/3-MethodsAndBytecode/figures/StackGrowingDown.pdf diff --git a/Chapters/CallingConventions/figures/StackGrowingDown.png b/Chapters/3-MethodsAndBytecode/figures/StackGrowingDown.png similarity index 100% rename from Chapters/CallingConventions/figures/StackGrowingDown.png rename to Chapters/3-MethodsAndBytecode/figures/StackGrowingDown.png diff --git a/Chapters/3-MethodsAndBytecode/figures/frames.jpg b/Chapters/3-MethodsAndBytecode/figures/frames.jpg new file mode 100644 index 0000000..5d52804 Binary files /dev/null and b/Chapters/3-MethodsAndBytecode/figures/frames.jpg differ diff --git a/Chapters/3-MethodsAndBytecode/methodsbytecode.md b/Chapters/3-MethodsAndBytecode/methodsbytecode.md new file mode 100644 index 0000000..b83e10a --- /dev/null +++ b/Chapters/3-MethodsAndBytecode/methodsbytecode.md @@ -0,0 +1,369 @@ +## Methods, Bytecode and Primitives + + +This chapter explains the basics of Pharo execution. +The unit of execution of Pharo is a method. +Methods execute one after the other, and _call_ each other by means of message send. +Each method is made of a list of literals or constants, a list of instructions called bytecodes, and optionally a primitive operation. + +Methods execute under the hood using a stack machine. +A stack holds the current calls and their values. +Bytecode instructions and primitives manipulates this stack with push and pop operations. + +In this chapter we explain in detail how methods are modelled using the sista bytecode set. +We explain how bytecode and primitive instructions are executed using a conceptual stack. +The actual interpreter and the call stack are introduced in a later chapter. + +### Reminder + +Imagine the method `Rectangle>>center` defined as follows + +``` +Rectangle >> width + "Answer the width of the receiver." + ^ corner x - origin x +``` + + +A programmer does not interact directly with the compiler. When a new source method is added to a class \(`Rectangle` in this example\), the system asks the compiler for an instance of `CompiledMethod` containing the bytecode translation of the source method. +The class provides the compiler with some necessary information not given in the source method, including the names of the receiver's instance variables and the dictionaries containing accessible shared variables \(global, class, and pool variables\). +The compiler translates the source text into a `CompiledMethod` and the class stores the method in its message dictionary. + +### Bytecodes by example + + +Source methods are translated by the compiler into sequences of instructions for a stack-oriented interpreter. +The instructions are numbers called bytecodes. For example, the bytecodes corresponding to the source method shown above are: 1, 126, 0, 126, 97, and 92. + +``` +(Rectangle >> #width) bytecode +>>> #[1 126 0 126 97 92] +``` + + + +Pharo supports a simple description of the bytecode using the message `symbolicBytecodes`. + +``` +(Rectangle >> #width) symbolicBytecodes + 25 <01> pushRcvr: 1 + 26 <7E> send: x + 27 <00> pushRcvr: 0 + 28 <7E> send: x + 29 <61> send: - + 30 <5C> returnTop +``` + + + +A bytecode's value gives us little indication of its meaning to the interpreter. +In this chapter we follow the convention of the Blue Book and accompany lists of bytecodes with comments about their functions. + + +We wrap in parentheses any part of a bytecode's comment that depends on the context of the method in which it appears. +The unparenthesized part of the comment describes its general function. +For example, the bytecode 0 always instructs the interpreter to push the value of the receiver's first instance variable on to its stack. +The fact that the variable is named `origin` depends on the fact that this method is used by the class `Rectangle`, so `origin` is parenthesized. +The commented form of the bytecodes for Rectangle center is shown below: + + +Rectangle >> #width +- <01> pushRcvr: 1 - push the value of the receiver's second instance variable \(corner\) onto the stack +- <7E> send: x - send the unary message x +- <00> pushRcvr: 0 - push the value of the receiver's first instance variable \(origin\) onto the stack +- <7E> send: x - send the unary message x +- <61> send: - - send the binary message - +- <5C> returnTop - return the object on top of the stack as the value of the message \(width\) + + + +#### About the stack. + +The stack mentioned in some of the bytecodes is used for several purposes: +- In method `width`, it is used to hold the receiver, arguments, and results of the two messages that are sent. +- The stack is also used as the source of the result returned from the `width` method. + + + + +### A store bytecode + + +Another example of the bytecodes compiled from a source method illustrates the use of a store bytecode. +Let us define the method `extent:` as follows: + +``` +Rectangle >> extent: aPoint + corner := origin + aPoint +``` + + +The message `extent:` changes the receiver's `width` and `height` to be equal to the `x` and `y` coordinates of the argument \(`a Point`\). +The receiver's upper left corner \(`origin`\) is kept the same and the lower right corner \(`corner`\) is moved. + +``` +(Rectangle >> #extent:) bytecode +>>> #[0 64 96 201 88] +``` + + + +Rectangle >> #extent: +- <00> pushRcvr: 0 - push the value of the receiver's first instance variable \(origin\) onto the stack +- <40> pushTemp: 0 - push the argument \(aPoint\) onto the stack +- <60> send: `+` - send a binary message + +- popIntoRcvr: 1 - pop the top object off of the stack and store it in the receiver's second instance variable \(`corner`\) +- 29 <58> returnSelf - return the receiver as the value of the message \(`extent:`\) + + + + + + +### Compiled Methods + + +The compiler creates an instance of `CompiledMethod` to hold the bytecode translation of a source method. In addition to the bytecodes themselves, a `CompiledMethod` contains a set of objects called its _literal frame_. +The literal frame contains any objects that could not be referred to directly by bytecodes. + +All of the objects in the methods `Rectangle>>#extent:` and `Rectangle>>#width` were referred to directly by bytecodes, so the `CompiledMethod` of these methods do not need literal frames. + +As an example of a `CompiledMethod` with a literal frame, consider the method `Rectangle>>#squishedWithin:`. +The `squishedWithin:` message changes the receiver fits within `aRectangle` by reducing its size, not by changing its origin. + +``` +Rectangle >> squishedWithin: aRectangle + "Return an adjustment of the receiver that fits within aRectangle by reducing its size, not by changing its origin." + + ^ origin corner: (corner min: aRectangle bottomRight) +``` + + + +``` +(Rectangle >> #squishedWithin:) bytecode +>>> #[0 1 64 128 145 146 92] +``` + + + +The message selectors \(`bottomRight`, `min:`, `corner:`\) are not in the set that can be directly referenced by bytecodes. +These selectors are included in the `CompiledMethod`'s literal frame and the send bytecodes refer to the selectors by their position in the literal frame. +We show the compiledMethod's literal frame after its bytecodes. + +`Rectangle>>#squishedWithin:` +- <00> pushRcvr: 0 - push the value of the receiver's first instance variable \(`origin`\) onto the stack +- <01> pushRcvr: 1 - push the value of the receiver's second instance variable \(`corner`\) onto the stack +- <40> pushTemp: 0 - push the argument \(`aRectangle`\) +- <80> send: bottomRight - send a unary message with the selector in the literal frame location \(`bottomRight`\) +- <91> send: min: - send a binary message with the selector in the literal frame location \(`min:`\) +- <92> send: corner: - send a keyword message with the selector in the literal frame location \(`corner:`\) +- <5C> returnTop - return the object on top of the stack as the value of the message + +Literal frame +- 1 `#bottomRight` +- 2 `#min:` +- 3 `#corner:` + + +The categories of objects that can be referred to directly by bytecodes are: +- the receiver and arguments of the message +- the values of the receiver's instance variables +- the values of any temporary variables required by the method +- special constants \(true, false, nil, -1, 0, 1, and 2\) +- special message selectors + + +``` +BytecodeEncoder specialSelectors +>>> #(#+ #- #< #> #'<=' #'>=' #= #'~=' #* #/ #'\\' #@ #bitShift: #'//' #bitAnd: #bitOr: #at: #at:put: #size #next #nextPut: #atEnd #'==' nil "class" +#'~~' #value #value: #do: #new #new: #x #y) +``` + + + + Any objects referred to in a CompiledMethod's bytecodes that do not fall into one of the categories above must appear in its literal frame. The objects ordinarily contained in a literal frame are +- shared variables \(global, class, and pool\) +- most literal constants \(numbers, characters, strings, arrays, and symbols\) +- most message selectors \(those that are not special\) + +Objects of these three types may be intermixed in the literal frame. + + +% % we could add the following as in the blue book +% Two types of object that were referred to above, temporary variables and shared variables, have not been used in the example methods. The following example method for Rectangle merge: uses both types. The merge: message is used to find a Rectangle that includes the areas in both the receiver and the argument. +% merge: aRectangle +% | minPoint maxPoint | +% minPoint := origin min: aRectangle origin. +% maxPoint := corner max: aRectangle corner. +% ^Rectangle origin: minPoint corner: maxPoint +% When a CompiledMethod uses temporary variables (maxPoint and minPoint in this example), the number required is specified in %the first line of its printed form. When a CompiledMethod uses a shared variable (Rectangle in this example) an instance of Association is included in its literal frame. All CompiledMethods that refer to a particular shared variable's name include the same Association in their literal frames. + +% Rectangle merge: requires 2 temporary variables +% 0 push the value of the receiver's first instance variable (origin) onto the stack +% 16 push the contents of the first temporary frame location (the argument aRectangle) onto the stack +% 209 send a unary message with the selector in the second literal frame location (origin) +% 224 send the single argument message with the selector in the first literal frame location (min:) +% 105 pop the top object off of the stack and stare in the second temporary frame location (minPoint) +% 1 push the value of the receiver's second instance variable (corner) onto the stack +% 16 push the contents of the first temporary frame location (the argument aRectangle) onto the stack +% 211 send a unary message with the selector in the fourth literal frame location (corner) +% 226 send a single argument message with the selector in the third literal frame location (max:) +% 106 pop the top object off of the stack and store it in the third temporary frame location (maxPoint) +% 69 push the value of the shared variable in the sixth literal frame location (Rectangle) onto the stack +% 17 push the contents of the second temporary frame location (minPoint) onto the stack +% 18 push the contents of the third temporary frame location (maxPoint) onto the stack +% 244 send the two argument message with the selector in the fifth literal frame location (origin:corner:) +% 124 return the object on top of the stack as the value of the message (merge:) +% literal frame +% #min: +% #origin +% #max: +% #corner +% #origin:corner: +% Association: #Rectangle -> Rectangle + + +### The bytecodes + + +The interpreter understands bytecode instructions that fall into five categories: pushes, stores, sends, returns, and jumps. This section gives a general description of each type of bytecode without going into detail about which bytecode represents which instruction. + +Some of the bytecodes take extensions. An extension is one or two bytes following the bytecode that further specify the instruction. An extension is not an instruction on its own, it is only a part of an instruction. + +#### Push Bytecodes. + + A push bytecode indicates the source of an object to be added to the top of the interpreter's stack. The sources include + +- the receiver of the message that invoked the CompiledMethod +- the instance variables of the receiver +- the temporary frame \(the arguments of the message and the temporary variables\) +- the literal frame of the CompiledMethod +- the top of the stack \(i.e., this bytecode duplicates the top of stack\) + + +Examples of most of the types of push bytecode have been included in the examples. +The bytecode that duplicates the top of the stack is used to implement cascaded messages. + +Two different types of push bytecode use the literal frame as their source. +One is used to push literal constants and the other to push the value of shared variables. +Literal constants are stored directly in the literal frame, but the values of shared variables are stored in an Association that is pointed to by the literal frame. + + +% The following example method uses one shared variable and one literal constant. +% incrementIndex +% ^Index := Index + 4 + +% ExampleClass incrementIndex +% 64 push the value of the shared variable in the first literal frame location (Index) onto the stack +% 33 push the constant in the second literal frame location (4) onto the stack +% 176 send a binary message with the selector + +% 129,192 store the object on top of the stack in the shared variable in the first literal frame location (Index) +% 124 return the object on top of the stack as the value of the message (incrementIndex) +% literal frame +% Association: #Index -> 260 + +#### Store Bytecodes. + + +The bytecodes compiled from an assignment expression end with a store bytecode. The bytecodes before the store bytecode compute the new value of a variable and leave it on top of the stack. A store bytecode indicates the variable whose value should be changed. The variables that can be changed are +- the instance variables of the receiver +- temporary variables +- shared variables + + +Some of the store bytecodes remove the object to be stored from the stack, and others leave the object on top of the stack, after storing it. + + +#### Send Bytecodes. + +A send bytecode specifies the selector of a message to be sent and how many arguments it should have. The receiver and arguments of the message are taken off the interpreter's stack, the receiver from below the arguments. By the time the bytecode following the send is executed, the message's result will have replaced its receiver and arguments on the top of the stack. The details of sending messages and returning results is the subject of the next sections of this chapter. A set of 32 send bytecodes refer directly to the special selectors listed earlier. The other send bytecodes refer to their selectors in the literal frame. + +#### Return Bytecodes. + +When a return bytecode is encountered, the CompiledMethod in which it was found has been completely executed. Therefore a value is returned for the message that invoked that CompiledMethod. The value is usually found on top of the stack. Four special return bytecodes return the message receiver \(self\), true, false, and nil. + +#### Jump Bytecodes. + + +Ordinarily, the interpreter executes the bytecodes sequentially in the order they appear in a CompiledMethod. The jump bytecodes indicate that the next bytecode to execute is not the one following the jump. There are two varieties of jump, unconditional and conditional. The unconditional jumps transfer control whenever they are encountered. The conditional jumps will only transfer control if the top of the stack is a specified value. Some of the conditional jumps transfer if the top object on the stack is true and others if it is false. The jump bytecodes are used to implement efficient control structures. + + +The control structures that are so optimized by the compiler are conditional selection messages to Booleans \(`ifTrue:`, `ifFalse:`, and `ifTrue:ifFalse:`\), some of the logical operation messages to Booleans \(`and:` and `or:`\), and the conditional repetition messages to blocks \(`whileTrue:` and `whileFalse:`\). The jump bytecodes indicate the next bytecode to be executed relative to the position of the jump. In other words, they tell the interpreter how many bytecodes to skip. + +% The following method for Rectangle includesPoint: uses a conditional jump. +% includesPoint: aPoint +% origin <= aPoint +% ifTrue: [^aPoint < corner] +% ifFalse: [^false] +% Rectangle includesPoint: +% 0 push the value of the receiver's first instance variable (origin) onto the stack +% 16 push the contents of the first temporary frame location (the argument aPoint) onto the stack +% 180 send a binary message with the selector <= +% 155 jump ahead 4 bytecodes if the object on top of the stack is false +% 16 push the contents of the first temporary frame location (the argument aPoint) onto the stack +% 1 push the value of the receiver's second instance variable (corner) onto the stack +% 178 send a binary message with the selector < +% 124 return the object on top of the stack as the value of the message ( includesPoint:) +% 122 return false as the value of the message (includesPoint:) + + + + + + + + + + + + + + +### Handling blocks: inlined ones + + +``` +Rectangle >> containsPoint: aPoint + "Answer whether aPoint is within the receiver." + ^origin <= aPoint and: [aPoint < corner] +``` + + +- <00> pushRcvr: 0 +- <40> pushTemp: 0 +- <64> send: <= +- jumpFalse: 41 +- <40> pushTemp: 0 +- <01> pushRcvr: 1 +- <62> send: < +- jumpTo: 42 +- 41 <4E> pushConstant: false +- 42 <5C> returnTop\)" + + + +### Blocks + + +Context should have a receiver because executing `[ self foo ] value` should execute foo on self and not the block + + + +### Primitive Methods? + + +The interpreter's actions after finding a compiled method depend on whether or not the compiled method indicates that a primitive method may be able to respond to the message. If no primitive method is indicated, a new method context is created and made active as described in previous sections. If a primitive method is indicated in the compiled method, the interpreter may be able to respond to the message without actually executing the bytecodes. For example, one of the primitive methods is associated with the `+` message to instances of `SmallInteger`. + +``` +SmallInteger >> + addend + + ^super + addend +``` + + +- callPrimitive: 1 +- <4C> self - push the receiver on the stack +- <40> pushTemp: 0 - push the first argument +- superSend: `+` - send a message via super +- <5C> returnTop - return the object on top of the stack as the value of the message diff --git a/Chapters/3-MethodsAndBytecode/theInterpreter.md b/Chapters/3-MethodsAndBytecode/theInterpreter.md new file mode 100644 index 0000000..b4d5299 --- /dev/null +++ b/Chapters/3-MethodsAndBytecode/theInterpreter.md @@ -0,0 +1,178 @@ +### The Interpreter + + +The virtual machine contains one interpreter and a compiler \(that compiles to assembly code\). Here we talk about the interpreter. It executes the bytecode instructions found in CompiledMethods. The interpreter uses five pieces of information refered hereafter as the state of the interpreter and repeatedly performs a three-step cycle. + +#### The State of the Interpreter. + +- The CompiledMethod whose bytecodes are being executed. +- The location of the next bytecode to be executed in that CompiledMethod. This is the interpreter's _instruction pointer_. +- The receiver and arguments of the message that invoked the CompiledMethod. +- Any temporary variables needed by the CompiledMethod. +- A stack. + + +The execution of most bytecodes involves the interpreter's stack. Push bytecodes tell where to find objects to add to the stack. Store bytecodes tell where to put objects found on the stack. Send bytecodes remove the receiver and arguments of messages from the stack. When the result of a message is computed, it is pushed onto the stack. + +#### The Cycle of the Interpreter. + +- Fetch the bytecode from the CompiledMethod indicated by the instruction pointer. +- Increment the instruction pointer. +- Perform the function specified by the bytecode. + + +As an example of the interpreter's function, we will trace its execution of the CompiledMethod `Rectangle>>#width`. +The state of the interpreter will be displayed after each of its cycles. +The instruction pointer will be indicated by an arrow pointing at the next bytecode in the CompiledMethod to be executed. + +- **==>** <01> pushRcvr: 1 - push the value of the receiver's second instance variable \(corner\) onto the stack + + + +The receiver, arguments, temporary variables, and objects on the stack will be shown as normally printed. +For example, if a message is sent to a Rectangle, the receiver will be shown as: + +- **Receiver:** 50@50 corner: 200@200 + + + +At the start of execution, the stack is empty and the instruction pointer indicates the first bytecode in the CompiledMethod. This CompiledMethod does not require temporaries and the invoking message did not have arguments, so these two categories are also empty. + +#### Step 1. + +The interpreter is in an initial state. It points to the next instructions to be executed. It knows the receiver of the method to be executed. + +Rectangle >> #width +- **==>** <01> pushRcvr: 1 - push the value of the receiver's second instance variable \(corner\) onto the stack +- <7E> send: x - send the unaay message x +- <00> pushRcvr: 0 - push the value of the receiver's first instance variable \(origin\) onto the stack +- <7E> send: x - send the unary message x +- <61> send: - - send the binary message - +- <5C> returnTop - return the object on top of the stack as the value of the message \(width\) +- **Receiver:** 50@50 corner: 200@200 +- **Arguments:** +- **Temporary Variables:** +- **Stack:** + + +#### Step 2. + +Following one cycle of the interpreter, the value of the receiver's second instance variable has been copied onto the stack and the instruction pointer has been advanced. + +Rectangle >> #width +- <01> pushRcvr: 1 - push the value of the receiver's second instance variable \(corner\) onto the stack +- **==>** <7E> send: x - send the unray message with the selector x +- <00> pushRcvr: 0 - push the value of the receiver's first instance variable \(origin\) onto the stack +- <7E> send: x - send the unary message with the selector x +- <61> send: - - send the binary message with the selector - +- <5C> returnTop - return the object on top of the stack as the value of the message \(width\) +- **Receiver:** 50@50 corner: 200@200 +- **Arguments:** +- **Temporary Variables:** +- **Stack:** 200@200 + + + +#### Step 3. + +The interpreter encounters a send bytecode. +It removes one object from the stack and uses it as the receiver of a message with selector `x`. +Sending a message will not be described in detail here. +Sending messages will be described in later sections. +For the moment, it is only necessary to know that eventually the result of the `x` message will be pushed onto the stack. + +Rectangle >> #width +- <01> pushRcvr: 1 - push the value of the receiver's second instance variable \(corner\) onto the stack +- <7E> send: x - send the unray message with the selector x +- **==>** <00> pushRcvr: 0 - push the value of the receiver's first instance variable \(origin\) onto the stack +- <7E> send: x - send the unary message with the selector x +- <61> send: - - send the binary message with the selector - +- <5C> returnTop - return the object on top of the stack as the value of the message \(width\) +- **Receiver:** 50@50 corner: 200@200 +- **Arguments:** +- **Temporary Variables:** +- **Stack:** 200 + + +#### Step 4. + +In this cycle, the value of the first receiver's first instance variables \(origin\) onto the stack + +Rectangle >> #width +- <01> pushRcvr: 1 - push the value of the receiver's second instance variable \(corner\) onto the stack +- <7E> send: x - send the unray message with the selector x +- <00> pushRcvr: 0 - push the value of the receiver's first instance variable \(origin\) onto the stack +- **==>** <7E> send: x - send the unary message with the selector x +- <61> send: - - send the binary message with the selector - +- <5C> returnTop - return the object on top of the stack as the value of the message \(width\) +- **Receiver:** 50@50 corner: 200@200 +- **Arguments:** +- **Temporary Variables:** +- **Stack:** 200 +- **Stack:** 50@50 + + + +#### Step 5. + +In this cycle, the message x is sent: it removes one object from the stack and uses it as the receiver of a message with selector `x` and push back the result on the stack. + +Rectangle >> #width +- <01> pushRcvr: 1 - push the value of the receiver's second instance variable \(corner\) onto the stack +- <7E> send: x - send the unray message with the selector x +- <00> pushRcvr: 0 - push the value of the receiver's first instance variable \(origin\) onto the stack +- <7E> send: x - send the unary message with the selector x +- **==>** <61> send: - - send the binary message with the selector - +- <5C> returnTop - return the object on top of the stack as the value of the message \(width\) +- **Receiver:** 50@50 corner: 200@200 +- **Arguments:** +- **Temporary Variables:** +- **Stack:** 200 +- **Stack:** 50 + + +#### Step 6. + +The final bytecode returns a result to the width message. The result is found on the stack \(150\). It is clear by this point that a return bytecode must involve pushing the result onto another stack. +The details of returning a value to a message will be described after the description of sending a message. + + +Rectangle >> #width +- <01> pushRcvr: 1 - push the value of the receiver's second instance variable \(corner\) onto the stack +- <7E> send: x - send the unray message with the selector x +- <00> pushRcvr: 0 - push the value of the receiver's first instance variable \(origin\) onto the stack +- <7E> send: x - send the unary message with the selector x +- <61> send: - - send the binary message with the selector - +- **==>** <5C> returnTop - return the object on top of the stack as the value of the message \(width\) +- **Receiver:** 50@50 corner: 200@200 +- **Arguments:** +- **Temporary Variables:** +- **Stack:** 150 + + + +### Calling convention in the VM + +The interpreter and the JIT share the same calling conventions. The receiver and arguments are pushed on the stack. The instruction pointer is also pushed to the stack after the arguments. + +Figure *@stackGrowing@* shows that the stack is growing down from high addresses to low addresses. +This convention is important and has an impact of many aspect such as object allocation and different logic in garbage collector implementation. + +![Stack growing down.](figures/StackGrowingDown.pdf width=30&label=stackGrowing) + + +Figure *@beforesend@* shows that the before doing a call, the receiver and arguments are pushed to the stack. + +![Receiver and arguments are pushed to the stack.](figures/BeforeSend.pdf width=80&label=beforesend) + +Figure *@aftersend@* shows that the instruction pointer (IP) is also pushed to the stack. This way it is possible to find which instruction is the next one to execute on return. Notice also that the interpreter and the VM are __caller-saved__. It means that this is the caller responsibility to store information that should be recovered on return of the called function. + +![Caller saved: the IP is also pushed to make sure that the caller can know the next instruction on return.](figures/AfterSend.pdf width=80&label=aftersend) + +Figure *@generalArguments@* shows that the framepointer is used to compute +- method argument. Since the arguments are pushed on the stack before the new frame is allocated, a method argument is always computed as an addition to the framepointer (`arg1 = FP + arg1offset`). +- method. The method (with its metadata) is located at a fix offset from the frame pointer. Hence `method= FP- method offset`. + + +![arg1 = FP + offset and method = FP - method offset.](figures/GeneralArgument.pdf width=100&label=generalArguments) + diff --git a/Chapters/BasicsOnExecution/basicsOnExecution.md b/Chapters/BasicsOnExecution/basicsOnExecution.md deleted file mode 100644 index 71f97ed..0000000 --- a/Chapters/BasicsOnExecution/basicsOnExecution.md +++ /dev/null @@ -1,25 +0,0 @@ -## Basics on Execution: The interpreter point of view In this chapter we will explain how a compiled method is executed by the virtual machine interpreter. In a future chapter we will explain how the code is compiled by the virtual machine compiler. We will start with the source methods written by programmers. A method is compiled into sequences of instructions called _bytecodes_. The bytecodes produced by the compiler are instructions for an interpreter, which is described in the second section. We will present the logic of interpreting the method bytecodes using a dedicated and internal stack for bytecode. Then we present contexts that support the execution of messages. ### Reminder Imagine the method `Rectangle>>center` defined as follows ``` Rectangle >> width - "Answer the width of the receiver." - ^ corner x - origin x ``` A programmer does not interact directly with the compiler. When a new source method is added to a class \(`Rectangle` in this example\), the system asks the compiler for an instance of `CompiledMethod` containing the bytecode translation of the source method. The class provides the compiler with some necessary information not given in the source method, including the names of the receiver's instance variables and the dictionaries containing accessible shared variables \(global, class, and pool variables\). The compiler translates the source text into a `CompiledMethod` and the class stores the method in its message dictionary. ### Bytecodes by example Source methods are translated by the compiler into sequences of instructions for a stack-oriented interpreter. The instructions are numbers called bytecodes. For example, the bytecodes corresponding to the source method shown above are: 1, 126, 0, 126, 97, and 92. ``` (Rectangle >> #width) bytecode ->>> #[1 126 0 126 97 92] ``` Pharo supports a simple description of the bytecode using the message `symbolicBytecodes`. ``` (Rectangle >> #width) symbolicBytecodes - 25 <01> pushRcvr: 1 - 26 <7E> send: x - 27 <00> pushRcvr: 0 - 28 <7E> send: x - 29 <61> send: - - 30 <5C> returnTop ``` A bytecode's value gives us little indication of its meaning to the interpreter. In this chapter we follow the convention of the Blue Book and accompany lists of bytecodes with comments about their functions. We wrap in parentheses any part of a bytecode's comment that depends on the context of the method in which it appears. The unparenthesized part of the comment describes its general function. For example, the bytecode 0 always instructs the interpreter to push the value of the receiver's first instance variable on to its stack. The fact that the variable is named `origin` depends on the fact that this method is used by the class `Rectangle`, so `origin` is parenthesized. The commented form of the bytecodes for Rectangle center is shown below: Rectangle >> #width - <01> pushRcvr: 1 - push the value of the receiver's second instance variable \(corner\) onto the stack - <7E> send: x - send the unary message x - <00> pushRcvr: 0 - push the value of the receiver's first instance variable \(origin\) onto the stack - <7E> send: x - send the unary message x - <61> send: - - send the binary message - - <5C> returnTop - return the object on top of the stack as the value of the message \(width\) #### About the stack. The stack mentioned in some of the bytecodes is used for several purposes: - In method `width`, it is used to hold the receiver, arguments, and results of the two messages that are sent. - The stack is also used as the source of the result returned from the `width` method. ### A store bytecode Another example of the bytecodes compiled from a source method illustrates the use of a store bytecode. Let us define the method `extent:` as follows: ``` Rectangle >> extent: aPoint - corner := origin + aPoint ``` The message `extent:` changes the receiver's `width` and `height` to be equal to the `x` and `y` coordinates of the argument \(`a Point`\). The receiver's upper left corner \(`origin`\) is kept the same and the lower right corner \(`corner`\) is moved. ``` (Rectangle >> #extent:) bytecode ->>> #[0 64 96 201 88] ``` Rectangle >> #extent: - <00> pushRcvr: 0 - push the value of the receiver's first instance variable \(origin\) onto the stack - <40> pushTemp: 0 - push the argument \(aPoint\) onto the stack - <60> send: `+` - send a binary message + - popIntoRcvr: 1 - pop the top object off of the stack and store it in the receiver's second instance variable \(`corner`\) - 29 <58> returnSelf - return the receiver as the value of the message \(`extent:`\) ### Compiled Methods The compiler creates an instance of `CompiledMethod` to hold the bytecode translation of a source method. In addition to the bytecodes themselves, a `CompiledMethod` contains a set of objects called its _literal frame_. The literal frame contains any objects that could not be referred to directly by bytecodes. All of the objects in the methods `Rectangle>>#extent:` and `Rectangle>>#width` were referred to directly by bytecodes, so the `CompiledMethod` of these methods do not need literal frames. As an example of a `CompiledMethod` with a literal frame, consider the method `Rectangle>>#squishedWithin:`. The `squishedWithin:` message changes the receiver fits within `aRectangle` by reducing its size, not by changing its origin. ``` Rectangle >> squishedWithin: aRectangle - "Return an adjustment of the receiver that fits within aRectangle by reducing its size, not by changing its origin." - - ^ origin corner: (corner min: aRectangle bottomRight) ``` ``` (Rectangle >> #squishedWithin:) bytecode ->>> #[0 1 64 128 145 146 92] ``` The message selectors \(`bottomRight`, `min:`, `corner:`\) are not in the set that can be directly referenced by bytecodes. These selectors are included in the `CompiledMethod`'s literal frame and the send bytecodes refer to the selectors by their position in the literal frame. We show the compiledMethod's literal frame after its bytecodes. `Rectangle>>#squishedWithin:` - <00> pushRcvr: 0 - push the value of the receiver's first instance variable \(`origin`\) onto the stack - <01> pushRcvr: 1 - push the value of the receiver's second instance variable \(`corner`\) onto the stack - <40> pushTemp: 0 - push the argument \(`aRectangle`\) - <80> send: bottomRight - send a unary message with the selector in the literal frame location \(`bottomRight`\) - <91> send: min: - send a binary message with the selector in the literal frame location \(`min:`\) - <92> send: corner: - send a keyword message with the selector in the literal frame location \(`corner:`\) - <5C> returnTop - return the object on top of the stack as the value of the message Literal frame - 1 `#bottomRight` - 2 `#min:` - 3 `#corner:` The categories of objects that can be referred to directly by bytecodes are: - the receiver and arguments of the message - the values of the receiver's instance variables - the values of any temporary variables required by the method - special constants \(true, false, nil, -1, 0, 1, and 2\) - special message selectors ``` BytecodeEncoder specialSelectors ->>> #(#+ #- #< #> #'<=' #'>=' #= #'~=' #* #/ #'\\' #@ #bitShift: #'//' #bitAnd: #bitOr: #at: #at:put: #size #next #nextPut: #atEnd #'==' nil "class" -#'~~' #value #value: #do: #new #new: #x #y) ``` Any objects referred to in a CompiledMethod's bytecodes that do not fall into one of the categories above must appear in its literal frame. The objects ordinarily contained in a literal frame are - shared variables \(global, class, and pool\) - most literal constants \(numbers, characters, strings, arrays, and symbols\) - most message selectors \(those that are not special\) Objects of these three types may be intermixed in the literal frame. % % we could add the following as in the blue book % Two types of object that were referred to above, temporary variables and shared variables, have not been used in the example methods. The following example method for Rectangle merge: uses both types. The merge: message is used to find a Rectangle that includes the areas in both the receiver and the argument. % merge: aRectangle % | minPoint maxPoint | % minPoint := origin min: aRectangle origin. % maxPoint := corner max: aRectangle corner. % ^Rectangle origin: minPoint corner: maxPoint % When a CompiledMethod uses temporary variables (maxPoint and minPoint in this example), the number required is specified in %the first line of its printed form. When a CompiledMethod uses a shared variable (Rectangle in this example) an instance of Association is included in its literal frame. All CompiledMethods that refer to a particular shared variable's name include the same Association in their literal frames. % Rectangle merge: requires 2 temporary variables % 0 push the value of the receiver's first instance variable (origin) onto the stack % 16 push the contents of the first temporary frame location (the argument aRectangle) onto the stack % 209 send a unary message with the selector in the second literal frame location (origin) % 224 send the single argument message with the selector in the first literal frame location (min:) % 105 pop the top object off of the stack and stare in the second temporary frame location (minPoint) % 1 push the value of the receiver's second instance variable (corner) onto the stack % 16 push the contents of the first temporary frame location (the argument aRectangle) onto the stack % 211 send a unary message with the selector in the fourth literal frame location (corner) % 226 send a single argument message with the selector in the third literal frame location (max:) % 106 pop the top object off of the stack and store it in the third temporary frame location (maxPoint) % 69 push the value of the shared variable in the sixth literal frame location (Rectangle) onto the stack % 17 push the contents of the second temporary frame location (minPoint) onto the stack % 18 push the contents of the third temporary frame location (maxPoint) onto the stack % 244 send the two argument message with the selector in the fifth literal frame location (origin:corner:) % 124 return the object on top of the stack as the value of the message (merge:) % literal frame % #min: % #origin % #max: % #corner % #origin:corner: % Association: #Rectangle -> Rectangle ### Temporary Variables Temporary variables are created for a particular execution of a CompiledMethod and cease to exist when the execution is complete. The CompiledMethod indicates to the interpreter how many temporary variables are required. The arguments of the message and the values of the temporary variables are stored together in the temporary frame. The arguments are stored first and the temporary variable values immediately after. They are accessed by the same type of bytecode \(whose comments refer to a temporary frame location\). The compiler enforces the fact that the values of the argument names cannot be changed by never issuing a store bytecode referring to the part of the temporary frame inhabited by the arguments. #### Shared Variables. Shared variables are held in dictionaries or environments. - global variables in the system dictionary unique instance. - class variables in a dictionary held in the class declaring them. - pool variables in collection held in shared pool classes. The system represents associations in general, and shared variables in particular, with instances of `Association` \(representing key value pair of information\). When the compiler encounters the name of a shared variable in a source method, the `Association` with the same name is included in the CompiledMethod's literal frame. The bytecodes that access shared variables indicate the location of an `Association` in the literal frame. The actual value of the variable is stored in an instance variable of the `Association`. Note that the literal frame or dictionaries are holding an `Association` and not just the value because this way it does not force a recompilation of all the users of a shared variable when its value change. ### The bytecodes The interpreter understands bytecode instructions that fall into five categories: pushes, stores, sends, returns, and jumps. This section gives a general description of each type of bytecode without going into detail about which bytecode represents which instruction. Some of the bytecodes take extensions. An extension is one or two bytes following the bytecode that further specify the instruction. An extension is not an instruction on its own, it is only a part of an instruction. #### Push Bytecodes. A push bytecode indicates the source of an object to be added to the top of the interpreter's stack. The sources include - the receiver of the message that invoked the CompiledMethod - the instance variables of the receiver - the temporary frame \(the arguments of the message and the temporary variables\) - the literal frame of the CompiledMethod - the top of the stack \(i.e., this bytecode duplicates the top of stack\) Examples of most of the types of push bytecode have been included in the examples. The bytecode that duplicates the top of the stack is used to implement cascaded messages. Two different types of push bytecode use the literal frame as their source. One is used to push literal constants and the other to push the value of shared variables. Literal constants are stored directly in the literal frame, but the values of shared variables are stored in an Association that is pointed to by the literal frame. % The following example method uses one shared variable and one literal constant. % incrementIndex % ^Index := Index + 4 % ExampleClass incrementIndex % 64 push the value of the shared variable in the first literal frame location (Index) onto the stack % 33 push the constant in the second literal frame location (4) onto the stack % 176 send a binary message with the selector + % 129,192 store the object on top of the stack in the shared variable in the first literal frame location (Index) % 124 return the object on top of the stack as the value of the message (incrementIndex) % literal frame % Association: #Index -> 260 #### Store Bytecodes. The bytecodes compiled from an assignment expression end with a store bytecode. The bytecodes before the store bytecode compute the new value of a variable and leave it on top of the stack. A store bytecode indicates the variable whose value should be changed. The variables that can be changed are - the instance variables of the receiver - temporary variables - shared variables Some of the store bytecodes remove the object to be stored from the stack, and others leave the object on top of the stack, after storing it. #### Send Bytecodes. A send bytecode specifies the selector of a message to be sent and how many arguments it should have. The receiver and arguments of the message are taken off the interpreter's stack, the receiver from below the arguments. By the time the bytecode following the send is executed, the message's result will have replaced its receiver and arguments on the top of the stack. The details of sending messages and returning results is the subject of the next sections of this chapter. A set of 32 send bytecodes refer directly to the special selectors listed earlier. The other send bytecodes refer to their selectors in the literal frame. #### Return Bytecodes. When a return bytecode is encountered, the CompiledMethod in which it was found has been completely executed. Therefore a value is returned for the message that invoked that CompiledMethod. The value is usually found on top of the stack. Four special return bytecodes return the message receiver \(self\), true, false, and nil. #### Jump Bytecodes. Ordinarily, the interpreter executes the bytecodes sequentially in the order they appear in a CompiledMethod. The jump bytecodes indicate that the next bytecode to execute is not the one following the jump. There are two varieties of jump, unconditional and conditional. The unconditional jumps transfer control whenever they are encountered. The conditional jumps will only transfer control if the top of the stack is a specified value. Some of the conditional jumps transfer if the top object on the stack is true and others if it is false. The jump bytecodes are used to implement efficient control structures. The control structures that are so optimized by the compiler are conditional selection messages to Booleans \(`ifTrue:`, `ifFalse:`, and `ifTrue:ifFalse:`\), some of the logical operation messages to Booleans \(`and:` and `or:`\), and the conditional repetition messages to blocks \(`whileTrue:` and `whileFalse:`\). The jump bytecodes indicate the next bytecode to be executed relative to the position of the jump. In other words, they tell the interpreter how many bytecodes to skip. % The following method for Rectangle includesPoint: uses a conditional jump. % includesPoint: aPoint % origin <= aPoint % ifTrue: [^aPoint < corner] % ifFalse: [^false] % Rectangle includesPoint: % 0 push the value of the receiver's first instance variable (origin) onto the stack % 16 push the contents of the first temporary frame location (the argument aPoint) onto the stack % 180 send a binary message with the selector <= % 155 jump ahead 4 bytecodes if the object on top of the stack is false % 16 push the contents of the first temporary frame location (the argument aPoint) onto the stack % 1 push the value of the receiver's second instance variable (corner) onto the stack % 178 send a binary message with the selector < % 124 return the object on top of the stack as the value of the message ( includesPoint:) % 122 return false as the value of the message (includesPoint:) ### The Interpreter The virtual machine contains one interpreter and a compiler \(that compiles to assembly code\). Here we talk about the interpreter. It executes the bytecode instructions found in CompiledMethods. The interpreter uses five pieces of information refered hereafter as the state of the interpreter and repeatedly performs a three-step cycle. #### The State of the Interpreter. - The CompiledMethod whose bytecodes are being executed. - The location of the next bytecode to be executed in that CompiledMethod. This is the interpreter's _instruction pointer_. - The receiver and arguments of the message that invoked the CompiledMethod. - Any temporary variables needed by the CompiledMethod. - A stack. The execution of most bytecodes involves the interpreter's stack. Push bytecodes tell where to find objects to add to the stack. Store bytecodes tell where to put objects found on the stack. Send bytecodes remove the receiver and arguments of messages from the stack. When the result of a message is computed, it is pushed onto the stack. #### The Cycle of the Interpreter. - Fetch the bytecode from the CompiledMethod indicated by the instruction pointer. - Increment the instruction pointer. - Perform the function specified by the bytecode. As an example of the interpreter's function, we will trace its execution of the CompiledMethod `Rectangle>>#width`. The state of the interpreter will be displayed after each of its cycles. The instruction pointer will be indicated by an arrow pointing at the next bytecode in the CompiledMethod to be executed. - **==>** <01> pushRcvr: 1 - push the value of the receiver's second instance variable \(corner\) onto the stack The receiver, arguments, temporary variables, and objects on the stack will be shown as normally printed. For example, if a message is sent to a Rectangle, the receiver will be shown as: - **Receiver:** 50@50 corner: 200@200 At the start of execution, the stack is empty and the instruction pointer indicates the first bytecode in the CompiledMethod. This CompiledMethod does not require temporaries and the invoking message did not have arguments, so these two categories are also empty. #### Step 1. The interpreter is in an initial state. It points to the next instructions to be executed. It knows the receiver of the method to be executed. Rectangle >> #width - **==>** <01> pushRcvr: 1 - push the value of the receiver's second instance variable \(corner\) onto the stack - <7E> send: x - send the unaay message x - <00> pushRcvr: 0 - push the value of the receiver's first instance variable \(origin\) onto the stack - <7E> send: x - send the unary message x - <61> send: - - send the binary message - - <5C> returnTop - return the object on top of the stack as the value of the message \(width\) - **Receiver:** 50@50 corner: 200@200 - **Arguments:** - **Temporary Variables:** - **Stack:** #### Step 2. Following one cycle of the interpreter, the value of the receiver's second instance variable has been copied onto the stack and the instruction pointer has been advanced. Rectangle >> #width - <01> pushRcvr: 1 - push the value of the receiver's second instance variable \(corner\) onto the stack - **==>** <7E> send: x - send the unray message with the selector x - <00> pushRcvr: 0 - push the value of the receiver's first instance variable \(origin\) onto the stack - <7E> send: x - send the unary message with the selector x - <61> send: - - send the binary message with the selector - - <5C> returnTop - return the object on top of the stack as the value of the message \(width\) - **Receiver:** 50@50 corner: 200@200 - **Arguments:** - **Temporary Variables:** - **Stack:** 200@200 #### Step 3. The interpreter encounters a send bytecode. It removes one object from the stack and uses it as the receiver of a message with selector `x`. Sending a message will not be described in detail here. Sending messages will be described in later sections. For the moment, it is only necessary to know that eventually the result of the `x` message will be pushed onto the stack. Rectangle >> #width - <01> pushRcvr: 1 - push the value of the receiver's second instance variable \(corner\) onto the stack - <7E> send: x - send the unray message with the selector x - **==>** <00> pushRcvr: 0 - push the value of the receiver's first instance variable \(origin\) onto the stack - <7E> send: x - send the unary message with the selector x - <61> send: - - send the binary message with the selector - - <5C> returnTop - return the object on top of the stack as the value of the message \(width\) - **Receiver:** 50@50 corner: 200@200 - **Arguments:** - **Temporary Variables:** - **Stack:** 200 #### Step 4. In this cycle, the value of the first receiver's first instance variables \(origin\) onto the stack Rectangle >> #width - <01> pushRcvr: 1 - push the value of the receiver's second instance variable \(corner\) onto the stack - <7E> send: x - send the unray message with the selector x - <00> pushRcvr: 0 - push the value of the receiver's first instance variable \(origin\) onto the stack - **==>** <7E> send: x - send the unary message with the selector x - <61> send: - - send the binary message with the selector - - <5C> returnTop - return the object on top of the stack as the value of the message \(width\) - **Receiver:** 50@50 corner: 200@200 - **Arguments:** - **Temporary Variables:** - **Stack:** 200 - **Stack:** 50@50 #### Step 5. In this cycle, the message x is sent: it removes one object from the stack and uses it as the receiver of a message with selector `x` and push back the result on the stack. Rectangle >> #width - <01> pushRcvr: 1 - push the value of the receiver's second instance variable \(corner\) onto the stack - <7E> send: x - send the unray message with the selector x - <00> pushRcvr: 0 - push the value of the receiver's first instance variable \(origin\) onto the stack - <7E> send: x - send the unary message with the selector x - **==>** <61> send: - - send the binary message with the selector - - <5C> returnTop - return the object on top of the stack as the value of the message \(width\) - **Receiver:** 50@50 corner: 200@200 - **Arguments:** - **Temporary Variables:** - **Stack:** 200 - **Stack:** 50 #### Step 6. The final bytecode returns a result to the width message. The result is found on the stack \(150\). It is clear by this point that a return bytecode must involve pushing the result onto another stack. The details of returning a value to a message will be described after the description of sending a message. Rectangle >> #width - <01> pushRcvr: 1 - push the value of the receiver's second instance variable \(corner\) onto the stack - <7E> send: x - send the unray message with the selector x - <00> pushRcvr: 0 - push the value of the receiver's first instance variable \(origin\) onto the stack - <7E> send: x - send the unary message with the selector x - <61> send: - - send the binary message with the selector - - **==>** <5C> returnTop - return the object on top of the stack as the value of the message \(width\) - **Receiver:** 50@50 corner: 200@200 - **Arguments:** - **Temporary Variables:** - **Stack:** 150 ### Contexts Push, store, and jump bytecodes require only small changes to the state of the interpreter. Objects may be moved to or from the stack, and the instruction pointer is always changed; but most of the state remains the same. Send and return bytecodes require much larger changes to the interpreter's state. When a new message is sent \(for example message `x` or `-` above\), all five parts of the interpreter's state may have to be changed to execute a different CompiledMethod in response to this new message. The interpreter's old state must be remembered because the bytecodes after the send must be executed after the value of the message is returned. The interpreter saves its state in objects called _contexts_. There will be many contexts in the system at any one time. The context that represents the current state of the interpreter is called the _active_ context. When a send bytecode in the active context's CompiledMethod requires a new compiled method to be executed, the active context becomes _suspended_ and a new context is created and made active. The suspended context retains the state associated with the original compiled method until that context becomes active again. A context must remember the context that it suspended so that the suspended context can be resumed when a result is returned. The suspended context is called the new context's _sender_. We extend the form used to show the interpreter's state. The active context will be indicated by the word **Active** in its top delimiter. Suspended contexts will say **Suspended**. ### Example For example, consider a context representing the execution of the compiled method `Rectangle>>#rightCenter` with a receiver of `50@50 corner: 200@200`. The sender is some other context in the system. The source method is: ``` Rectangle >> rightCenter - ^self right @ self center y ``` #### After the execution of the first bytecode The following active context shows that state of the context after the execution of the first bytecode: **Active** Rectangle >> #rightCenter - <4C> self - push the receiver \(self\) onto the stack - **==>** <80> send: right - send a unary message with the selector in the first literal \(right\) - <4C> self push the receiver \(self\) onto the stack - <81> send: center - send a unary message with the selector in the first literal \(center\) - <7F> send: y - send a unary message with the selector in the first literal \(y\) - <6B> send: @ - send a unary message with the selector in the first literal \(@\) - <5C> returnTop - return the object on top of the stack as the value of the message \(rightCenter\) Literal frame - 1 `#right` - 2 `#center` - **Receiver:** 50@50 corner: 200@200 - **Arguments:** - **Temporary Variables:** - **Stack** 50@50 corner: 200@200 #### Execution of the second bytecode After the next bytecode is executed, that context will be suspended. The object pushed by the first bytecode has been removed to be used as the receiver of a new context, which becomes active. The new active context is shown above the suspended context for the method `right`: ``` Rectangle >> right - "Answer the position of the receiver's right vertical line." - ^ corner x ``` **Active** Rectangle >> #right - **==>**<01> pushRcvr: 1 - push the value of the receiver's second instance variable \(corner\) onto the stack - <7E> send: x - <5C> returnTop Literal frame - **Receiver:** 50@50 corner: 200@200 - **Arguments:** - **Temporary Variables:** - **Stack:** **Suspended** Rectangle >> #rightCenter - <4C> self - push the receiver \(self\) onto the stack - <80> send: right - send a unary message with the selector in the first literal \(right\) - **==>** <4C> self push the receiver \(self\) onto the stack - <81> send: center - send a unary message with the selector in the first literal \(center\) - <7F> send: y - send a unary message with the selector in the first literal \(y\) - <6B> send: @ - send a unary message with the selector in the first literal \(@\) - <5C> returnTop - return the object on top of the stack as the value of the message \(rightCenter\) Literal frame - 1 `#right` - 2 `#center` - **Receiver:** 50@50 corner: 200@200 - **Arguments:** - **Temporary Variables:** - **Stack:** #### Execution of the right message The next cycle of the interpreter advances the new context instead of the previous one. **Active** Rectangle >> #right - <01> pushRcvr: 1 - push the value of the receiver's second instance variable \(corner\) onto the stack - **==>**<7E> send: x - <5C> returnTop Literal frame - **Receiver:** 50@50 corner: 200@200 - **Arguments:** - **Temporary Variables:** - **Stack:** 200@200 **Suspended** Rectangle >> #rightCenter - <4C> self - push the receiver \(self\) onto the stack - <80> send: right - send a unary message with the selector in the first literal \(right\) - **==>** <4C> self push the receiver \(self\) onto the stack - <81> send: center - send a unary message with the selector in the first literal \(center\) - <7F> send: y - send a unary message with the selector in the first literal \(y\) - <6B> send: @ - send a unary message with the selector in the first literal \(@\) - <5C> returnTop - return the object on top of the stack as the value of the message \(rightCenter\) Literal frame - 1 `#right` - 2 `#center` - **Receiver:** 50@50 corner: 200@200 - **Arguments:** - **Temporary Variables:** - **Stack:** #### Execution of the right message In the next cycle, another message is sent, perhaps creating another context. Instead of following the response of this new message \(`x`\), we will skip to the point that this context returns a value \(to `right`\). When the result of `x` has been returned, the new context looks like this: **Active** Rectangle >> #right - <01> pushRcvr: 1 - push the value of the receiver's second instance variable \(corner\) onto the stack - <7E> send: x - **==>** <5C> returnTop Literal frame - **Receiver:** 50@50 corner: 200@200 - **Arguments:** - **Temporary Variables:** - **Stack:** 200 **Suspended** Rectangle >> #rightCenter - <4C> self - push the receiver \(self\) onto the stack - <80> send: right - send a unary message with the selector in the first literal \(right\) - **==>** <4C> self push the receiver \(self\) onto the stack - <81> send: center - send a unary message with the selector in the first literal \(center\) - <7F> send: y - send a unary message with the selector in the first literal \(y\) - <6B> send: @ - send a unary message with the selector in the first literal \(@\) - <5C> returnTop - return the object on top of the stack as the value of the message \(rightCenter\) Literal frame - 1 `#right` - 2 `#center` - **Receiver:** 50@50 corner: 200@200 - **Arguments:** - **Temporary Variables:** - **Stack:** #### Returning from right The next bytecode returns the value \(200\) on the top of the active context's stack as the value of the message that created the context \(`right`\). The active context's sender becomes the active context again and the returned value is pushed on its stack. It is ready to continue its execution. **Active** Rectangle >> #rightCenter - <4C> self - push the receiver \(self\) onto the stack - <80> send: right - send a unary message with the selector in the first literal \(right\) - **==>** <4C> self push the receiver \(self\) onto the stack - <81> send: center - send a unary message with the selector in the first literal \(center\) - <7F> send: y - send a unary message with the selector in the first literal \(y\) - <6B> send: @ - send a unary message with the selector in the first literal \(@\) - <5C> returnTop - return the object on top of the stack as the value of the message \(rightCenter\) Literal frame - 1 `#right` - 2 `#center` - **Receiver:** 50@50 corner: 200@200 - **Arguments:** - **Temporary Variables:** - **Stack:** 200 ### Handling blocks: inlined ones ``` Rectangle >> containsPoint: aPoint - "Answer whether aPoint is within the receiver." - ^origin <= aPoint and: [aPoint < corner] ``` - <00> pushRcvr: 0 - <40> pushTemp: 0 - <64> send: <= - jumpFalse: 41 - <40> pushTemp: 0 - <01> pushRcvr: 1 - <62> send: < - jumpTo: 42 - 41 <4E> pushConstant: false - 42 <5C> returnTop\)" ### Blocks Context should have a receiver because executing `[ self foo ] value` should execute foo on self and not the block ### Primitive Methods? The interpreter's actions after finding a compiled method depend on whether or not the compiled method indicates that a primitive method may be able to respond to the message. If no primitive method is indicated, a new method context is created and made active as described in previous sections. If a primitive method is indicated in the compiled method, the interpreter may be able to respond to the message without actually executing the bytecodes. For example, one of the primitive methods is associated with the `+` message to instances of `SmallInteger`. ``` SmallInteger >> + addend - - ^super + addend ``` - callPrimitive: 1 - <4C> self - push the receiver on the stack - <40> pushTemp: 0 - push the first argument - superSend: `+` - send a message via super - <5C> returnTop - return the object on top of the stack as the value of the message \ No newline at end of file diff --git a/Chapters/JIT/Jit-Introduction.md b/Chapters/JIT/Jit-Introduction.md new file mode 100644 index 0000000..04ffa5b --- /dev/null +++ b/Chapters/JIT/Jit-Introduction.md @@ -0,0 +1,370 @@ +# Introducing the Cogit JIT compiler + +JIT (Just-in-Time) compilers are an optimization technique often used for interpreted languages and virtual machines. +JIT compilers detect frequently used code and compile it to more efficient code, thus reaching a good balance between execution time and compilation time. In other words, the idea is that the application run-time spends too much time compiling but executing application code instead. +In this framework, non-frequent code falls back in slower execution engines such as interpreters. +For example, the Pharo and the Java VM run on a bytecode interpreter and eventually compile machine code for methods that are frequently called. + +The current Pharo JIT compiler, namely Cogit, implements a template-based of bytecode to object code compiler. +When a method is compiled, each bytecode is mapped to a template. +In its most basic mode, all templates are concatenated to form a single machine code method. +This generated machine code method removes lots of interpretation overhead such as bytecode fetching and decoding. + +This chapter starts by refreshing the reader's memory with how source code gets translated to bytecode. +Then it shows a general overview how bytecode are translated to machine code by the Cogit bytecode-to-machine-code JIT compiler. +This chapter discusses the Cogit template-based architecture. +Then it discusses the Cog register-transfer-language intermediate representation (IR), or CogRTL, used by the compiler to represent machine code before the actual machine code generation. + +Finally, this chapter introduces two different compilers defined with the Cog architecture. +First, the stack-based cogit is the simplest compiler in the Cogit family. +The stack-based cogit translates each bytecode on isolation of others, and data is exchanged through the actual memory stack at run-time. +Second, the stack-to-register-mapping compiler uses a simulated stack schema to expand the compilation context of bytecode translation. +At compile-time bytecodes read and write from a simulated stack, generating code to access the real stack only if necessary. +This schema avoids useless writes and reads from memory. + +## The Cogit Architecture + +The Pharo Cog VM is a bytecode-based execution engine. +In other words, Pharo code is first compiled into compact and architecture-independent code called bytecode. +The bytecode is then interpreted by a bytecode interpreter, and when required compiled to machine code on-the-fly. + +### The Bytecode Stack Machine + +To execute Pharo source code, each method is first compiled by a bytecode compiler. +The bytecode compiler generates a method made of machine-independent bytecode and a literal frame containing all literal objects used in that method. +The bytecodes are virtual machine code instructions. +Most instructions fit in a single byte (hence their name). +The main reason of their compactness is that they all have an implicit argument: a stack of operands. +The literals are the objects that represent fixed values used in the method. + +Consider for example the following method. +In this example, the method has two main operations, the message send `+` and an explicit return `^`. +This method also contains 2 literal objects: 1 and 17. +Although in this example both literals are numbers numbers, literals can be `'strings'`, literal arrays such as `#(a b c 1)` and characters `$A`. + +```smalltalk +MyClass >> foo + ^ 1 + 17 +``` + +To execute the method above, we need first to generate stack-based bytecode for it. +A bytecode method should first send the message `+` to the number `1` with argument `17`, and then return the result to the caller. +The send and the return operations exchange values through the operand stack. +The send operation takes the message receiver and the message arguments from the stack, executes the operation, and pushes the result to the stack. +Subsequent operations take this result from the stack if needed. +The following abstract bytecode illustrates this behaviour. + +``` +push 1 +push 17 +send + +returnTop a +``` + +Notice that since a send operation requires that the receiver and arguments are present in the stack, previous operations must take care of pushing such values before. In our case, this is done with explicit push operations. +In the example above, the first bytecode pushes a 1 into the stack and the second bytecode pushes a 2. +Then, the third bytecode has to send a message `+`, so it pops two elements from the stack: one to use as the receiver, and one to use as an argument. +When the execution of the message send finishes, the result is pushed to the stack. +Finally, the last bytecode takes the top of the stack (the result of the addition), and returns it to the caller. + +A very simple stack-based bytecode representation of a method can be obtained with a recursive depth-first post-order traversal of the method's abstract syntax tree. Such source-code to bytecode compilation is done in Pharo with Opal. +Creating and inspecting a method like the one above shows the following in an inspector. +The first number is the bytecode index, the second is the instruction byte(s), and then there is a mnemonic of the instruction and arguments. + +``` +25 <76> pushConstant: 1 +26 <20> pushConstant: 17 +27 send: + +28 <7C> returnTop +``` + +### Understanding the bytecode interpreter + +The `StackInterpreter` class implements Pharo's VM bytecode interpreter. +When a bytecode method is executed by the interpreter, the execution engine iterates over all bytecodes of a method and executes a VM routine for each of them. The interpreter contains a table mapping each bytecode to the routine implementing its behavior. +For example, the bytecode 0x76 is mapped to the routine `pushConstantOneBytecode`. +This routine pushes a 1 to the stack calling the `internalPush:` method. +Since pushing the value 1 is a very common operation, a special bytecode is used for it to avoid storing the 1 in the literal frame. + +```smalltalk +StackInterpreter >> pushConstantOneBytecode + + self fetchNextBytecode. + self internalPush: ConstOne. +``` + +For pushing less common values to the stack, bytecode 0x20 is mapped to the routine `pushLiteralConstantBytecode`. +This routine pushes a literal value stored in the method's literal frame to the stack also using `internalPush:`. +The value pushed is taken from the literal frame of the method, and the index is calculated from manipulating the `currentBytecode` variable. +Bytecode 33 pushes the first literal in the frame (33 bitAnd: 16r1F => 1), bytecode 34 pushes the second literal, and so on... + +```smalltalk +StackInterpreter >> pushLiteralConstantBytecode + + self + cCode: "this bytecode will be expanded so that refs to currentBytecode below will be constant" + [self fetchNextBytecode. + self pushLiteralConstant: (currentBytecode bitAnd: 16r1F)] + inSmalltalk: "Interpreter version has fetchNextBytecode out of order" + [self pushLiteralConstant: (currentBytecode bitAnd: 16r1F). + self fetchNextBytecode] +``` + +Finally, bytecodes such as the message send `+` are implemented as follows. +First this bytecode gets the top 2 values from the stack. +Then it checks if boths are integers, and if the result is an integer, in which case it pushes the value and finishes. +If they are not integers, it tries to add them as floats. +If that fails, it will perform a (slow) message send using the `normalSend` method. + +```smalltalk +StackInterpreter >> bytecodePrimAdd + | rcvr arg result | + rcvr := self internalStackValue: 1. + arg := self internalStackValue: 0. + (objectMemory areIntegers: rcvr and: arg) + ifTrue: [result := (objectMemory integerValueOf: rcvr) + (objectMemory integerValueOf: arg). + (objectMemory isIntegerValue: result) ifTrue: + [self internalPop: 2 thenPush: (objectMemory integerObjectOf: result). + ^ self fetchNextBytecode "success"]] + ifFalse: [self initPrimCall. + self externalizeIPandSP. + self primitiveFloatAdd: rcvr toArg: arg. + self internalizeIPandSP. + self successful ifTrue: [^ self fetchNextBytecode "success"]]. + + messageSelector := self specialSelector: 0. + argumentCount := 1. + self normalSend +``` + +### (Prelude) Understanding the existing Cogit JIT compiler + +When a bytecode method is executed a couple of times, the Pharo virtual machine decides to compile it to machine code. +Compiling the method to machine code avoids performance overhead due to instruction fetching, and allows one to perform several optimizations. +The compilation of a machine code method goes pretty similar to the interpretation of a method. +The JIT compiler iterates the bytecode method and for each of the bytecodes it executes a code generation routine. +This means that we will (almost) have a counterpart for each of the VM methods implementing bytecode interpretation. + +For example, the machine code generator implemented for `StackInterpreter>>pushLiteralConstantBytecode` is `Cogit>>genPushLiteralConstantBytecode`. + +```smalltalk +Cogit >> genPushLiteralConstantBytecode + ^self genPushLiteralIndex: (byte0 bitAnd: 31) + +StackToRegisterMappingCogit >> genPushLiteralIndex: literalIndex "" + "Override to avoid the BytecodeSetHasDirectedSuperSend check, which is unnecessary + here given the simulation stack." + + | literal | + literal := self getLiteral: literalIndex. + ^self genPushLiteral: literal +``` + +The JIT'ted version of the addition bytecode (`genSpecialSelectorArithmetic`) is slightly more complicated, but it pretty much matches what it is done in the bytecode. + +### Overview of Druid + +In Druid, a meta-interpreter analyzes the bytecode interpreter code and generates an intermediate representation from it. +A compiler interface then generates machine code from the intermediate representation. +The output of the intermediate representation should have in general terms the same behaviour as the existing Cogit JIT compiler. + +To verify the correctness of the compiler we use: + - a machine code simulator (Unicorn) + - a disassembler (llvm) + +(please see the links to these projects in the references) + +## The (meta-)interpreter + +The setup is the following: we have one Pharo AST interpreter that we call the meta-interpreter that executes the code code of the +`StackInterpreter` and generates the corresponding intermediate representation of `StackInterpreter` methods. +Check `Fun with interpreters` from the references to see more details on what an ASTs and abstract interpreters are. +In the code below, the meta-interpreter is called `DRASTInterpreter` (for the DruidAST interpreter) and it will analyse the methods +of the stack interpreter returned byt the expression `Druid new newBytecodeInterpreter`. +For this task, the meta-interpreter uses an IR builder that is responsible for encapsulating the logic of IR building. + +```smalltalk +builder := DRIRBuilder new. +builder isa: #X64. + +astInterpreter := DRASTInterpreter new. +astInterpreter vmInterpreter: Druid new newBytecodeInterpreter. +astInterpreter irBuilder: builder. +``` + +This AST interpreter then receives as input a list of bytecodes to analyze, it maps each bytecode to the routine to execute, +obtains the AST and interprets each of the instructions of the AST using a visitor pattern. + +```smalltalk +astInterpreter interpretBytecode: #[76]. +``` + +### Visiting the AST + +The `DRASTInterpreter` class implements a `visiting` protocol where the `visit` methods are grouped. +Most of the visit methods are simple, like the following ones: + +```smalltalk +DRASTInterpreter >> visitSelfNode: aRBSelfNode + + ^ self receiver + +DRASTInterpreter >> visitTemporaryNode: aRBTemporaryNode + + ^ currentContext temporaryNamed: aRBTemporaryNode name +``` + +### Context reification + +To properly analyze the scope of temporary variables, the AST interpret reifies the contexts/stack frames. +On each method or block activation, a new context is pushed to the stack. +The temporary variables are then read and written from the current context. +When a method or block returns, the current context is popped from the stack, to return to the caller context. + +### Interpreting Message sends + +The most important operation in a Pharo program are message sends. +Message sends are used not only for normal method invocations, but also for common operators such as additions (`+`) and multiplications (`*`) and control flow such as conditionals (`ifTrue:`) and loops (`whileTrue:`). +However, some of these special cases require special treatments when generating code. +For example, a multiplication should directly generate a normal multiplication and not require interpreting how that multiplication is implemented. + +To manage such special cases, we keep a table in the interpreter that maps (special selector -> special interpretation). +When we find a message send in the AST, we lookup the selector in the table. +If we find a special case in the entry, we invoke that special entry in the interpreter. +Otherwise, we lookup the method and activate the new method, which will recursively continue the interpretation + +```smalltalk +DRASTInterpreter >> visitMessageNode: aRBMessageNode + + | arguments astToInterpret receiver | + + "First interpret the arguments to generate instructions for them. + If this is a special selector, treat it specially with those arguments. + Otherwise, lookup and interpret the called method propagating the arguments" + receiver := aRBMessageNode receiver acceptVisitor: self. + arguments := aRBMessageNode arguments collect: [ :e | e acceptVisitor: self ]. + + specialSelectorTable + at: aRBMessageNode selector + ifPresent: [ :selfSelectorToInterpret | + ^ self perform: selfSelectorToInterpret with: aRBMessageNode with: receiver with: arguments ]. + + astToInterpret := self + lookupSelector: aRBMessageNode selector + receiver: receiver + isSuper: aRBMessageNode receiver isSuper. + ^ self interpretAST: astToInterpret withReceiver: receiver withArguments: arguments. +``` + +For example, the following code snippet shows how the `internalPush:` is mapped as just a normal push instruction in the intermediate representation. + +```smalltalk +DRASTInterpreter >> initialize + + super initialize. + irBuilder := DRIRBuilder new. + + specialSelectorTable := Dictionary new. + ... + specialSelectorTable at: #internalPush: put: #interpretInternalPushOn:receiver:arguments:. + +DRASTInterpreter >> interpretInternalPushOn: aRBMessageNode receiver: aStackInterpreterSimulatorLSB arguments: aCollection + + ^ irBuilder push: aCollection first +``` + +### The intermediate representation + +The intermediate representation is generated by a builder object, instance of `DRIRBuilder`. +The AST interpreter collaborates with it to create instructions, new basic blocks and so on. +The intermediate representation that is created is somewhat inspired on a low-level intermediate representation as described in [Linear Scan Register Allocation for the Java HotSpot™ Client Compiler]. + +### Some meta-interpretation of special cases + +Not all special cases generate instructions or interact with the DRIRBuilder. +Some of them actually modify the state of the AST interpreter. +For example, a special case is the `fetchNextBytecode` instruction, that makes the interpreter move to the next byte in the list of bytecodes. +To simulate the same behaviour in our interpreter, its special case is implemented as follows: + +```smalltalk +DRASTInterpreter >> interpretFetchNextBytecodeOn: aMessageSendNode receiver: aReceiver arguments: arguments + + self fetchNextInstruction + +DRASTInterpreter >> fetchNextInstruction + + currentBytecode := instructionStream next +``` + +### The compiler interface + +Once the interpreter finishes its job, the irBuilder will contain the intermediate representation instructions. +We can then generate machine code from them using the `DRIntermediateRepresentationToMachineCodeTranslator` class. + +```smalltalk + astInterpreter irBuilder assignPhysicalRegisters. + + mcTranslator := DRIntermediateRepresentationToMachineCodeTranslator + translate: builder instructions + withCompiler: cogit. + + address := cogit methodZone freeStart. + endAddress := mcTranslator generate. +``` + +`DRIntermediateRepresentationToMachineCodeTranslator` iterates all the instructions and uses the double-dispatch pattern to +generate code for each of them. +It uses a backend to generate the actual machine code. + +```smalltalk +aCollection do: [ :anIRInstruction | + anIRInstruction accept: self ]. + +DRIntermediateRepresentationToMachineCodeTranslator >> visitAdd: +DRIntermediateRepresentationToMachineCodeTranslator >> visitLoad: +DRIntermediateRepresentationToMachineCodeTranslator >> visitPush: +``` + +## About the Tests + +The whole process has the following steps: + +1. The bytecode interpreter is interpreted to generate the instructions in the IRBuilder +2. Then the physical registers will be allocated to the Druid instructions +3. The DRInstructions generates the AbstractInstructions (Cogit) that basically are one to one representations with the machine code +4. Compilation to machine code byte array + +So, we have different tests to test different stages in the transformation: + +- The class `DRIRBuilderTest` tests the construction of the ir. +- The class `DRASTInterpreterTest` tests the effect of interpreting ASTs +- The DRIntermediateRepresentationToMachineCodeTranslatorTest simulates that each DR instrucion is correctly translated to the corresponding machine code +- The class `DRSimulateGeneratedBytecodeTest` tests from end-to-end the generation of code of a bytecode and its execution in a machine code simulator + +Other test classes test a basic register allocation algorithm (`DRRegisterAllocationTest`), the generation of machine code from an IR without passing through the AST (`DRIntermediateRepresentationToMachineCodeTranslatorTest`) and the execution of machine code from an IR without passing through the AST (`DRSimulateGeneratedCodeTest`). + +## Little exercises +- In tbe book [https://github.com/SquareBracketAssociates/PatternsOfDesign/releases](https://github.com/SquareBracketAssociates/PatternsOfDesign/releases) +- Chapter 4: Die and DieHandle double Dispatch (if you want to make sure that Double Dispatch has been understood do the Stone Paper Scissor Chapter) +- Chapter 3 A little expression interpreter +- Chapter 6 Understanding visitor +- After reading [https://github.com/SquareBracketAssociates/Booklet-FunWithInterpreters](https://github.com/SquareBracketAssociates/Booklet-FunWithInterpreters) + + +## References + - Linear Scan Register Allocation for the Java HotSpot™ Client Compiler + http://www.ssw.uni-linz.ac.at/Research/Papers/Wimmer04Master/ + - Practical partial evaluation for high-performance dynamic language runtimes + https://dl.acm.org/doi/10.1145/3062341.3062381 + - Structure and Interpretation of Computer Programs + http://web.mit.edu/alexmv/6.037/sicp.pdf + - Fun with Interpreters + https://github.com/SquareBracketAssociates/Booklet-FunWithInterpreters/releases/download/continuous/fun-with-interpreters-wip.pdf + - [Trace-Based Register Allocation](https://gitlab.inria.fr/RMOD/vm-papers/-/blob/master/compilation+JIT/2016_Trace-based%20Register%20Allocation%20in%20a%20JIT%20Compiler.pdf) + - Paper explaining the motivations behind Sista Bytecode + https://github.com/SquareBracketAssociates/Booklet-PharoVirtualMachine/raw/master/bib/iwst2014_A%20bytecode%20set%20for%20adaptive%20optimizations.pdf + + - https://github.com/unicorn-engine/unicorn + - https://github.com/guillep/pharo-unicorn + - http://llvm.org/ + - https://github.com/guillep/pharo-llvmDisassembler \ No newline at end of file diff --git a/index.md b/index.md index 5b89500..cde5058 100644 --- a/index.md +++ b/index.md @@ -17,8 +17,8 @@ _Acknowledgements._ This work is supported by Ministry of Higher Education and R The work is supported by I-Site ERC-Generator Multi project 2018-2022. We gratefully acknowledge the financial support of the Métropole Européenne de Lille. This work is also supported by the Action Exploratoire Alamvic led by G. Polito and S. Ducasse. - - + +