Getting started with Instruments in GraalVM

Within the GraalVM platform Tools are also called Instruments. The Truffle instrumentation API is used to implement such instruments. Instruments can track very fine-grained VM-level runtime events to profile, inspect, and analyze the runtime behavior of applications running on GraalVM.

Examples

The following simple examples are intended show common use-cases that can be solved with the Instrumentation framework.

Instrumentation Event Listeners

The Truffle instrumentation API is defined in the com.oracle.truffle.api.instrumentation package. Instrumentation agents can be developed by extending the TruffleInstrument class, and can be attached to a running GraalVM instance using in the Instrumenter class. Once attached to a running language runtime, instrumentation agents remain usable as long as the language runtime is not disposed. Instrumentation agents on GraalVM can monitor a variety of VM-level runtime events, including any of the following:

  1. Source code-related events: the agent can be notified every time a new Source or SourceSection element is loaded by the monitored language runtime.
  2. Allocation events: the agent can be notified every time a new object is allocated in the memory space of the monitored language runtime.
  3. Language runtime and thread creation events: the agent can be notified as soon as a new execution context or a new thread for a monitored language runtime is created.
  4. Application execution events: the agent gets notified every time a monitored application executes a specific set of language operations. Examples of such operations include language statements and expressions, thus allowing an instrumentation agent to inspect running applications with very high precision.

For each execution event, instrumentation agents can define filtering criteria that will be used by the GraalVM instrumentation runtime to monitor only the relevant execution events. Currently, GraalVM instruments accept one of the following two filter types:

  1. AllocationEventFilter. To filter allocation events by allocation type.
  2. SourceSectionFilter. To filter source code locations in an application.

Filters can be created using the provided builder object. For example, the following builder creates a SourceSectionFilter:

SourceSectionFilter.newBuilder()
                   .tagIs(StandardTag.StatementTag)
                   .mimeTypeIs("x-application/js")
                   .build()

The filter in the example can be used to monitor the execution of all JavaScript statements in a given application. Other filtering options such as line numbers or file extensions can also be provided.

Source section filters like the one in the example can use Tags to specify a set of execution events to be monitored. Language-agnostic tags such as statements and expressions are defined in the com.oracle.truffle.api.instrumentation.Tag class, and are supported by all GraalVM languages. In addition to standard tags, GraalVM languages may provide other, language-specific, tags to enable fine-grained profiling of language-specific events (as an example, the Graal.js language runtime provides JavaScript-specific tags to track the usages of ECMA builtin objects such as Array, Map, or Math).

Monitoring execution events

Application execution events enable very precise and detailed monitoring. GraalVM supports two different types of instrumentation agents to profile such events, namely:

  1. Execution listener: an instrumentation agent that can be notified every time a given runtime event happens. Listeners implement the ExecutionEventListener interface, and cannot associate any state with source code locations.
  2. Execution event node: an instrumentation agent that can be expressed using Truffle AST nodes. Such agents extend the ExecutionEventNode class, have the same capabilities of an execution listener, but can associate state with source code locations.

Simple Instrumentation Agent

A simple example of a custom instrumentation agent to perform runtime code coverage can be found in the CoverageExample class. What follows is an overview of the agent, of its design, and of its capabilities.

All Truffle instruments extend the TruffleInstrument abstract class, and are registered in the GraalVM runtime through the @Registration annotation:

@Registration(id = CoverageExample.ID, services = Object.class)
public final class CoverageExample extends TruffleInstrument {

  @Override
  protected void onCreate(final Env env) {
  }

  /* Other methods omitted... */
}

Instruments override the onCreate(Env env) method to perform custom operations at instrument loading time. Typically, an instrument would use this method to register itself in the existing GraalVM execution environment. As an example, an instrument using Truffle AST nodes can be registered in the following way:

@Override
protected void onCreate(final Env env) {
  SourceSectionFilter.Builder builder = SourceSectionFilter.newBuilder();
  SourceSectionFilter filter = builder.tagIs(EXPRESSION).build();
  Instrumenter instrumenter = env.getInstrumenter();
  instrumenter.attachExecutionEventFactory(filter, new CoverageEventFactory(env));
}

The instrument connects itself to the running GraalVM using the attachExecutionEventFactory method, providing the following two arguments:

  1. SourceSectionFilter: a source section filter used to inform the GraalVM about specific code sections to be tracked.
  2. ExecutionEventNodeFactory: a Truffle AST factory that provides instrumentation AST nodes to be executed by the agent every time a runtime event (as specified by the source filter) is executed.

A basic ExecutionEventNodeFactory that instruments the AST nodes of an application can be implemented in the following way:

public ExecutionEventNode create(final EventContext ec) {
  return new ExecutionEventNode() {
    @Override
    public void onReturnValue(VirtualFrame vFrame, Object result) {
      /*
       * Code to be executed every time a filtered source code
       * element is evaluated by the guest language.
       */    
    }
  };
}

Execution event nodes can implement certain callback methods to intercept runtime execution events. Examples include:

  1. onEnter: executed before an AST node corresponding to a filtered source code element (e.g., a language statement or an expression) is evaluated.
  2. onReturnValue: executed after a source code element returns a value.
  3. onReturnExceptional: executed in case the filtered source code element throws an exception.

Execution event nodes are created on a per code location basis. Therefore, they can be used to store data specific to a given source code location in the instrumented application. As an example, an instrumentation node can simply keep track of all code locations that have already been visited using a node-local flag. Such a node-local boolean flag can be used to track the execution of AST nodes in the following way:

// To keep track of all source code locations executed
private final Set<SourceSection> coverage = new HashSet<>();

public ExecutionEventNode create(final EventContext ec) {
  return new ExecutionEventNode() {
    // Per-node flag to keep track of execution for this node
    @CompilationFinal private boolean visited = false;

    @Override
    public void onReturnValue(VirtualFrame vFrame, Object result) {
      if (!visited) {
        CompilerDirectives.transferToInterpreterAndInvalidate();
        visited = true;
        SourceSection src = ec.getInstrumentedSourceSection();
        coverage.add(src);
      }
    }
  };
}

As the above code shows, an ExecutionEventNode is a valid Truffle AST node. This implies that the instrumentation code will be optimized by the GraalVM together with the instrumented application, resulting in minimal instrumentation overhead. Furthermore, this allows instrument developers to use Truffle compiler directives directly from Truffle instrumentation nodes. In the example, compiler directives are used to inform the Graal compiler that visited can be considered compilation-final.

Each instrumentation node is bound to a specific code location. Such locations can be accessed by the agent using the provided EventContext object. The context object gives instrumentation nodes access to a variety of information about the current AST nodes being executed. Examples of query APIs available to instrumentation agents through EventContext include:

  1. hasTag to query an instrumented node for a certain node Tag (e.g., to check if a statement node is also a conditional node).
  2. getInstrumentedSourceSection to access the SourceSection associated with the current node.
  3. getInstrumentedNode to access the Node corresponding to the current instrumentation event.

Fine-grained expression profiling

Instrumentation agents can profile even fractional events such as language expressions. To this end, an agent needs to be initialized providing two source section filters:

// What source sections are we interested in?
SourceSectionFilter sourceSectionFilter = SourceSectionFilter.newBuilder().tagIs(JSTags.BinaryOperation.class).build();

// What generates input data to track?
SourceSectionFilter inputGeneratingLocations = SourceSectionFilter.newBuilder().tagIs(
                        StandardTags.ExpressionTag.class).build();

instrumenter.attachExecutionEventFactory(sourceSectionFilter, inputGeneratingLocations, factory);

The first source section filter (sourceSectionFilter, in the example) is a normal filter equivalent to other filters described before, and is used to identify the source code locations to be monitored. The second section filter, inputGeneratingLocations, is used by the agent to specify the intermediate values that should be monitored for a certain source section. Intermediate values correspond to all observable values that are involved in the execution of a monitored code element, and are reported to the instrumentation agent by means of the onInputValue callback. As an example, let’s assume an agent needs to profile all operand values provided to sum operations (i.e., +) in JavaScript:

var a = 3;
var b = 4;
// the '+' expression is profiled
var c = a + b;

By filtering on JavaScript Binary expressions, an instrumentation agent would be able to detect the following runtime events for the above code snippet:

  1. onEnter() for the binary expression at line 3.
  2. onInputValue() for the first operand of the binary operation at line 3. The value reported by the callback will be 3, that is, the value of the a local variable.
  3. onInputValue() for the second operand of the binary operation. The value reported by the callback will be 4, that is, the value of the b local variable.
  4. onReturnValue() for the binary expression. The value provided to the callback will be the value returned by the expression after it has completed its evaluation, that is, the value 7.

By extending the source section filters to all possible events, an instrumentation agent will observe something equivalent to the following execution trace (in pseudocode):

// First variable declaration
onEnter - VariableWrite
    onEnter - NumericLiteral
    onReturnValue - NumericLiteral
  onInputValue - (3)
onReturnValue - VariableWrite

// Second variable declaration
onEnter - VariableWrite
    onEnter - NumericLiteral
    onReturnValue - NumericLiteral
  onInputValue - (4)
onReturnValue - VariableWrite

// Third variable declaration
onEnter - VariableWrite
    onEnter - BinaryOperation
        onEnter - VariableRead
        onReturnValue - VariableRead
      onInputValue - (3)
        onEnter - VariableRead
        onReturnValue - VariableRead
      onInputValue - (4)      
    onReturnValue - BinaryOperation
  onInputValue - (7)
onReturnValue - VariableWrite

The onInputValue method can be used in combination with source section filters to intercept very fine-grained execution events such as intermediate values used by language expressions. The intermediate values that are accessible to the instrumentation framework greatly depend on the instrumentation support provided by each language. Moreover, languages may provide additional metadata associated with language-specific Tag classes.

Altering the execution flow of an application

The instrumentation capabilities that we have presented so far enable users to observe certain aspects of a running application. In addition to passive monitoring of an application’s behavior, the Truffle instrumentation API features support for actively altering the behavior of an application at runtime. Such capabilities can be used to write complex instrumentation agents that affect the behavior of a running application to achieve specific runtime semantics. For example, one could alter the semantics of a running application to ensure that certain methods or functions are never executed (e.g., by throwing an exception when they are called).

Instrumentation agents with such capabilities can be implemented by leveraging the onUnwind callback in execution event listeners and factories. As an example, let’s consider the following JavaScript code:

function inc(x) {
  return x + 1
}

var a = 10
var b = a;
// Let's call inc() with normal semantics
while (a == b && a < 100000) {
  a = inc(a);
  b = b + 1;
}
c = a;
// Run inc() and alter it's return type using the instrument
return inc(c)

An instrumentation agent that modifies the return value of inc to always be 42 can be implemented using an ExecutionEventListener, in the following way:

ExecutionEventListener myListener = new ExecutionEventListener() {

  @Override
  public void onReturnValue(EventContext context, VirtualFrame frame, Object result) {
    String callSrc = context.getInstrumentedSourceSection().getCharacters();
    // is this the function call that we want to modify?
    if ("inc(c)".equals(callSrc)) {
      CompilerDirectives.transferToInterpreter();
      // notify the runtime that we will change the current execution flow
      throw context.createUnwind(null);
    }
  }

  @Override
  public Object onUnwind(EventContext context, VirtualFrame frame, Object info) {
    // just return 42 as the return value for this node
    return 42;
  }
}

The event listener can be executed intercepting all function calls, for example using the following instrument:

@TruffleInstrument.Registration(id = "UniversalAnswer", services = UniversalAnswerInstrument.class)
public static class UniversalAnswerInstrument extends TruffleInstrument {

  @Override
  protected void onCreate(Env env) {
    env.registerService(this);
    env.getInstrumenter().attachListener(SourceSectionFilter.newBuilder().tagIs(CallTag.class).build(), myListener);
  }
}

When enabled, the instrument will execute its onReturnValue callback each time a function call returns. The callback reads the associated source section (using getInstrumentedSourceSection) and looks for a specific source code pattern (the function call inc(c), in this case). As soon as such code pattern is found, the instrument throws a special runtime exception, called UnwindException, that instructs the Truffle instrumentation framework about a change in the current application’s execution flow. The exception is intercepted by the onUnwind callback of the instrumentation agent, which can be used to return any arbitrary value to the original instrumented application. In the example, all calls to inc(c) will return 42 regardless of any application-specific data. A more realistic instrument might access and monitor several aspects of an application, and might not rely on source code locations, but rather on object instances or other application-specific data.