AFCL

Abstract Function Choreography Language

General overview

Serverless workflow applications or function choreographies (FCs), which connect serverless functions by data- and control-flow, have gained considerable momentum recently to create more sophisticated applications as part of Functionas-a-Service (FaaS) platforms.
AFCL is based on YAML which is a human-readable data serialization language. YAML is also the base language for several other platforms that support workflow applications. An AFCL FC consists of functions, which can be either base functions or compound functions. The former refers to a single computational task without further splitting it into smaller tasks, while the latter encloses some base functions or even nested compound functions. All base and compound functions can be connected by different control- and data-flow constructs. An FC is also a compound function. In order to create an FC, all its functions (base and compound) as well as control- and data- flow connections among them, must be specified. In order to facilitate optimized execution of FCs, a user optionally can specify properties and constraints for functions and data-flow connections. In order to simplify the reading of AFCL specifications, we use meta-syntax which extends YAML, such that YAML elements can be contained in { } and appended with wildcards “?” (0 or 1), “*” (0 or more), “+” (1 or more), and “|” (logical or).

Base function

A base function represents a computational or data processing task. The name of a function serves as a unique identifier. Functions are described by type - Function Types (FTs) which are abstract descriptions of their corresponding function implementations (FIs). An FI represents an actual implementation of an FT. The FIs of the same function type are semantically equivalent, but may expose different performance or cost behavior or may be implemented with different programming languages, etc. FTs shield implementation details from the FC developer. The selection of a specific FI for an FT is done by an underlying runtime system.

function: {
  name: "name",
  type: "type",
  dataIns: [ 
    {
      name: "name",
      type: "type",
      source: "source"?,
      value: "value"?
    }+
  ]?,
  dataOuts: [
    {
      name: "name",
      type: "type",
      saveto: "saveto"?
    }+
  ]?
}

The input and output data of a function are specified through dataIns and dataOuts ports of the function, respectively. The number and type attributes of the dataIns/Outs ports are uniquely determined by the chosen FT. A dataIns port can be specified by (i) setting its source attributed to the name of a data port of another function within the same FC (dataIns from a parent compound function or dataOuts from a predecessor function) or the name of a dataIns port of the FC; (ii) setting its source attribute to a specific URL referring to a file, or an ordered list of URLs referring to an ordered list of files; or (iii) specifying a constant or an ordered list of constants. The data associated with the dataOuts port can be stored in the location specified through its saveto attribute. Linking the data ports of different functions through the source attributes defines the data-flow of AFCL FCs. The value of a dataIns/Outs port can be used to define a constant. dataIns/Outs ports are optional as a function may perform some predefined action in which case there is no need to specify neither input nor output parameters. Every dataIns/Outs port is associated with a data type. The data types supported for AFCL are JSON datatypes: string, number, boolean, null/empty, object, and array, as well as two additional types, file and collection.
In the remainder we omit name, type, source, value, and saveto for simplicity. Instead, we will use only expressions dataIns[{}+]? and dataOuts[{}+]? for the list of data inputs and outputs of a function. Additionally, we will use the abbreviation function[{}+] to specify a list of base or compound functions (omitting name, type, dataIns, and dataOuts).

Compound function

AFCL introduces a rich set of control-flow constructs (compound functions) to simplify the specification of realistic FCs that are difficult to be composed with any current FC system without support by a skilled software developer. Compound functions contain inner functions, which can be base or compound and they are executed in the order defined by the compound function. The inner functions are called children functions of the compound function. The compound function is called the parent function of the inner functions. An inner function of a compound function fi can be another compound or base function fj . The term child function of fi refers to the entire compound function fj. AFCL introduces the following compound functions: sequence, if-then-else, switch, for, while, parallel, and parallelFor. The specifications for the name attribute, dataIns and dataOuts ports, along with the corresponding source and saveto attributes are similar as for a base function. In the remain der of this text, we will not separately explain the attributes of a compound function and when we use the term function, it can refer to either base function or compound function.

Sequence

The sequence compound function represents a sequential controlflow of all inner functions within the sequenceBody section. In order to simplify the AFCL, we assume that all base or compounds functions, which are specified one after the other, are sequential without specifying them into a sequence compound function. AFCL introduces source attribute for dataOuts ports for each compound function in order to specify internal data-flow from dataOuts ports of children functions to dataOuts ports of the parent compound function. The value of the source attribute is similar to that of the source attribute of a dataIns port, except that it refers to dataOut ports of children functions of a compound function.

sequence: {
  name: "name",
  dataIns: [{}+]?,
  sequenceBody: [
    {
      function: {}+
    }
  ],
  dataOuts: [
    {
      name: "name",
      type: "type",
      saveto: "saveto"?
      source: "source",
    }+
  ]?
}

If-then-else

The if-then-else compound function is one of two conditional compound functions of AFCL. The condition attribute describes a set of subconditions combined with the boolean combinedWith attribute. A sub-condition contains data1 and data2 which represent the data to be compared according to the value of the operator (==, <, >, ≤, ≥, and !=, contains, startsWith and endsWith) and optionally negation. The values of data1 and data2 can be constants or the output from previous functions. If the condition is satisfied then functions within the then part are executed, otherwise the else branch is executed.

if: {
  name: "name",
  dataIns: [{}+]?,
  condition: 
    {
      combinedWith: "and/or",
      conditions: [
        {
          data1: "data1",
          data2: "data2",
          operator: "operator",
          negation: "negation",
        }+
      ]
    },
  then: [{function: {}}+],
  else: [{function: {}}+]?,
  dataOuts: [{}+]?
}

Switch

The switch compound function can be used to select a single case or a default compound function depending on the value of the dataEval attribute, thereby acting as an XOR logical expression. In case break is not used, then the switch compound function acts as an OR logical expression and multiple case branches might be selected.

switch: {
  name: "name",
  dataIns: [{}+]?,
  dataEval: 
    {
      name: "name",
      type: "type",
      source: "source"?
    },
  cases: [
    {
      value: "value",
      break: "true"?,
      functions: [{function: {}}+]
    }+
  ],
  default: [{function: {}}+]?,
  dataOuts: [{}+]?
}

For

The for compound function executes its loopBody multiple times based on the specified loopCounter. The value of the loopCounter is initially set to the value specified by the attribute from and is then increased by the value of step until it reaches the value of to or larger. The attributes from, to, and step can be specified with a constant value or with data ports of other functions. To express dependencies across loop iterations the dataLoop ports are used. These ports get their initial value from the optional initSource field or a constant value from the value field. A loopSource field specifies a data-flow from the output of a function of the loop body which can be used as input to functions executed in the next loop iteration. name is an unique identifier of a DataLoop port and type specifies the data type of the value.

for: {
  name: "name",
  dataIns: [{}+]?,
  dataLoops: [
    {
      name: "name",
      type: "type",
      initSource: "source",?
      loopSource: "source",
      value: "constant"?
    }+
  ]?,
  loopCounter: 
    {
      name: "name",
      type: "type",
      from: "from",
      to: "to",
      step: "step"?,
    },
  loopBody: [{function: {}}+],
  dataOuts: [{}+]?
}

While

The while compound function is used to execute a loopBody zero or more times, depending on the specified condition. The condition has the same structure as in the if-then-else compound. The loopBody will be executed until the specified condition evaluates to false. Similarly as in the for compound function, dependencies across loop iterations in while can be expressed with dataLoop ports.

while: {
  name: "name",
  dataIns: [{}+]?,
  dataLoops: [
    {
      name: "name",
      type: "type",
      initSource: "source",?
      loopSource: "source",
      value: "constant"?
    }+
  ]?,
  condition: 
    {
      combinedWith: "and/or",
      conditions: [{}+]
    },
  loopBody: [{function: {}}+],
  dataOuts: [{}+]?
}

Parallel

The parallel compound function expresses the parallel execution of a set of sections. Each section within the parallelBody represents a list of base or a compound functions, which can run in parallel with other sections. The parallel compound function can have arbitrary many data input ports, whose associated data can be distributed among inner functions.

parallel: {
  name: "name",
  dataIns: [{}+]?,
  parallelBody: [
    {
      section: [{function: {}}+]
    }+
  ],
  dataOuts: [{}+]?
}

ParallelFor

The parallelFor compound function expresses the simultaneous execution of all loop iterations. It is assumed that there are no data dependen cies across loop iterations. All other elements of the constructs behave the same as in the for compound function.

parallelFor: {
  name: "name",
  dataIns: [{}+]?,
  loopCounter: 
    {
      name: "name", type: "type",
      from: "from", to: "to",
      step: "step"?,
    },
  loopBody: [{function: {}}+],
  dataOuts: [{}+]?
}

Properties and Constraints

Properties and constraints are optional attributes defined within the AFCL language, which provide additional information about dataIn ports, dataOut ports, and base and compound functions. Properties can be used to describe hints about the behaviour of functions, e.g. expected size of input data or memory required for execution. Constraints (e.g. finish execution time within a time limit, data distributions, fault tolerance settings) should be fulfilled by the runtime system on a best-effort basis.

function: {
  name: "name",
  type: "type",
  dataIns: [
    {
      name: "name", type: "type",
      source: "source"?, value: "value"?,
      properties: [{ name: "name", value: "value" }+]?,
      constraints: [{ name: "name", value: "value" }+]?
    }+
  ]?,
  properties: [{ name: "name", value: "value" }+]?,
  constraints: [{ name: "name", value: "value" }+]?,
  dataOuts: [
    {
      name: "name", type: "type", saveto: "saveto"?,
      properties: [{ name: "name", value: "value" }+]?,
      constraints: [{ name: "name", value: "value" }+]?
    }+
  ]?
}

Invocation type

The invoke-type is a built-in Property defined in AFCL. This Property can be used to specify whether a base function should run asynchronously (ASYNC) or synchronously (SYNC). An invoker of a synchronous function waits for the function to finish (if the function has output data then until the data arrived). With asynchronous invocation, the function will be queued without waiting for the function to be finished. For example, the runtime system can invoke a synchronous ”checker” function periodically to examine whether the output of an asynchronous function is stored in a specified storage (e.g. S3). By default, if the invoke-type property is defined within a compound function, all nested base functions within that compound function will inherit this property and are invoked with the specified invoke-type. Otherwise, the base function will be invoked as specified invoke-type property. If ASYNC is specified, the FC designer must guarantee that the FC still operates correctly. Without specifying any invoke-type in any of the parent compound functions, the base functions are executed synchronously in AFCL. The build-in function asyncHandler is used to handle ASYNC invoked functions. This function can be used the same way as a base function is used, while the type field specifies that it is a build-in function. The build-in function has one input parameter, representing a coma separated list of names of ASYNC invoked functions (e.g. FunctionName1) and one boolean output parameter which represents whether all of these invoked functions finished. asyncHandler can be invoked with invoke-type i) ASYNC, meaning that asyncHandler immediately returns with the output parameter set to true if all functions (specified in the input parameter) finished otherwise it is set to false, or ii) SYNC, meaning that asyncHandler waits for all functions (specified in the input parameter) to finish before it returns

Base function

function: {
  name: "name", type: "type",
  dataIns: [{}+]?,
  properties: [
    {
      name: "invoke-type",
      value: "ASYNC | SYNC"
    }+
  ]?,
  dataOuts: [{}+ ]?
}

Compound function

function: {
  name: "name", type: "build-in:asyncHandler",
  dataIns: [
    { 
      name: "name", type: "collection",
      value: "FunctionName1,FunctionName2,...,FunctionNameN" 
	}+
  ],
  properties: [
    {
      name: "invoke-type",
      value: "ASYNC | SYNC"
    }+
  ]?,
  dataOuts: [
    { 
      name: "name", type: "boolean" 
	}+
  ]
}