This specification was originally derived from the Starlark language specification. It was influenced by the Python specification, Copyright 1990-2017, Python Software Foundation.
An introduction to the language, and pointers to the module and context references, are on the overview.
TSL syntax (but not semantics) is a strict subset of Python syntax. Practically it means, tools working with Python AST can be used to work with TSL files.
A TSL program consists of one or more modules. Each module is defined by a single UTF-8-encoded text file.
TSL grammar is introduced gradually throughout this document as shown below, and a complete TSL grammar reference is provided at the end.
Grammar notation:
- lowercase and 'quoted' items are lexical tokens.
- Capitalized names denote grammar productions.
- (...) implies grouping.
- x | y means either x or y.
- [x] means x is optional.
- {x} means x is repeated zero or more times.
- The end of each declaration is marked with a period.The contents of a TSL file are broken into a sequence of tokens of five kinds: white space, punctuation, keywords, identifiers, and literals. Each token is formed from the longest sequence of characters that would form a valid token of each kind.
File = {Statement | newline} eof .White space consists of spaces (U+0020), tabs (U+0009), carriage returns (U+000D), and newlines (U+000A). Within a line, white space has no effect other than to delimit the previous token, but newlines, and spaces at the start of a line, are significant tokens.
Comments: A hash character (#) appearing outside of a string or bytes literal marks the start of a comment; the comment extends to the end of the line, not including the newline character. Comments are treated like other white space.
Punctuation: The following punctuation characters or sequences of characters are tokens:
+ - * / // % **
~ & | ^ << >>
. , = ; :
( ) [ ] { }
< > >= <= == !=
+= -= *= /= //= %=
&= |= ^= <<= >>=Keywords: The following tokens are keywords and may not be used as identifiers:
andasbreakcontinuedefelifelseexceptfinallyforifimportinlambdanotorpassraisereturntrywhile
The tokens below also may not be used as identifiers although they do not appear in the grammar; they are reserved as possible future keywords:
assertasyncawaitclassdelfromglobalisloadnonlocalwithyield
Identifiers: an identifier is a sequence of Unicode letters, decimal digits, and underscores (_), not starting with a digit. Identifiers are used as names for values.
Examples:
None True len
x index starts_with arg0Literals: literals are tokens that denote specific values. TSL has integer, floating-point, string, and bytes literals.
0 # int
123 # decimal int
0x7f # hexadecimal int
0o755 # octal int
0.0 0. .0 # float
1e10 1e+10 1e-10
1.1e10 1.1e+10 1.1e-10
"hello" 'hello' # string
'''hello''' """hello""" # triple-quoted string
r'hello' r"hello" # raw string literal
b"hello" b'hello' # bytes
b'''hello''' b"""hello""" # triple-quoted bytes
rb'hello' br"hello" # raw bytes literalInteger and floating-point literal tokens are defined by the following grammar:
int = decimal_lit | octal_lit | hex_lit | 0 .
decimal_lit = ('1' … '9') {decimal_digit} .
octal_lit = '0' ('o' | 'O') octal_digit {octal_digit} .
hex_lit = '0' ('x' | 'X') hex_digit {hex_digit} .
float = decimals '.' [decimals] [exponent]
| decimals exponent
| '.' decimals [exponent]
.
decimals = decimal_digit {decimal_digit} .
exponent = ('e'|'E') ['+'|'-'] decimals .
decimal_digit = '0' … '9' .
octal_digit = '0' … '7' .
hex_digit = '0' … '9' | 'A' … 'F' | 'a' … 'f' .It is a static error if a floating-point literal denotes a value whose magnitude is too large to be represented as a finite float value.
A TSL string literal denotes a string value. In its simplest form, it consists of the desired text surrounded by matching single- or double-quotation marks:
"abc"
'abc'Literal occurrences of the chosen quotation mark character must be escaped by a preceding backslash. So, if a string contains several of one kind of quotation mark, it may be convenient to quote the string using the other kind, as in these examples:
'Have you read "To Kill a Mockingbird?"'
"Yes, it's a classic."
"Have you read \"To Kill a Mockingbird?\""
'Yes, it\'s a classic.'Within a string literal, the backslash character \ indicates the start of an escape sequence, a notation for expressing things that are impossible or awkward to write directly.
The following traditional escape sequences represent the ASCII control codes 7-13:
\a \x07 alert or bell
\b \x08 backspace
\f \x0C form feed
\n \x0A line feed
\r \x0D carriage return
\t \x09 horizontal tab
\v \x0B vertical tabA literal backslash is written using the escape \\.
An escaped newline, that is, a backslash at the end of a line, is ignored, allowing a long string to be split across multiple lines of the source file.
"abc\
def" # "abcdef"An octal escape encodes a single string element using its octal value. It consists of a backslash followed by one, two, or three octal digits [0-7]. Similarly, a hexadecimal escape encodes a single string element using its hexadecimal value. It consists of \x followed by two hexadecimal digits [0-9a-fA-F]. It is an error if the value of an octal or hexadecimal escape is greater than decimal 127.
'\0' # "\x00" a string containing a single NUL element
'\12' # "\n" octal 12 = decimal 10
'\101-\132' # "A-Z"
'\119' # "\t9" = "\11" + "9"
'\x00' # "\x00" a string containing a single NUL element
'\x0A' # "\n" hexadecimal A = decimal 10
"\x41-\x5A" # "A-Z"A Unicode escape denotes the UTF-K encoding of a single, valid Unicode code point, where K is the implementation-defined number of bits in each string element (see strings). The \uXXXX form, with exactly four hexadecimal digits, denotes a 16-bit code point, and the \UXXXXXXXX, with exactly eight digits, denotes a 32-bit code point. It is an error if the value lies in the surrogate range (U+D800 to U+DFFF) or is greater than U+10FFFF.
'\u0041' # "A", an ASCII letter (U+0041)
'\u0414' # "Д", a Cyrillic capital letter (U+0414)
'\u754c' # "界", a Chinese character (U+754C)
'\U0001F600' # "😀", an Emoji (U+1F600)len returns the number of UTF-8 code points in a string.
String literals may contain any valid Unicode text:
len("A") # 1
len("Д") # 1
len("界") # 1
len("😀") # 1String literals can denote any valid Unicode text. When the repr function is applied to a string, its result is a valid string literal.
An ordinary string literal may not contain an unescaped newline, but a multiline string literal may spread over multiple source lines. It is denoted using three quotation marks at start and end. Within it, unescaped newlines and quotation marks (or even pairs of quotation marks) have their literal meaning, but three quotation marks end the literal. This makes it easy to quote large blocks of text with few escapes.
haiku = '''
Yesterday it worked.
Today it is not working.
That's computers. Sigh.
'''Regardless of the platform's convention for text line endings, for example, a linefeed (\n) on UNIX, or a carriage return followed by a linefeed (\r\n) on Microsoft Windows, an unescaped line ending in a multiline string literal always denotes a line feed (\n).
TSL also supports raw string literals, which look like an ordinary single- or double-quotation preceded by r. Within a raw string literal, there is no special processing of backslash escapes, other than an escaped quotation mark (which denotes a literal quotation mark), or an escaped newline (which denotes a backslash followed by a newline). This form of quotation is typically used when writing strings that contain many quotation marks or backslashes (such as regular expressions or shell commands) to reduce the burden of escaping:
"a\nb" # "a\nb" = 'a' + '\n' + 'b'
r"a\nb" # "a\\nb" = 'a' + '\\' + 'n' + 'b'
"a\
b" # "ab"
r"a\
b" # "a\\\nb"It is an error for a backslash to appear within a string literal other than as part of one of the escapes described above.
A TSL bytes literal denotes a bytes value, and looks like a string literal, in any of its various forms (single-quoted, double-quoted, triple-quoted, raw) preceded by the letter b.
b"abc" b'abc'
b"""abc""" b'''abc'''
br"abc" br'abc'
rb"abc" rb'abc'A raw bytes literal may be indicated by either a br or rb prefix.
Non-escaped text within a bytes literal denotes the UTF-8 encoding of that text. Bytes literals support the same escape sequences as text strings, with the following differences:
Octal and hexadecimal escapes may specify any byte value from zero (
\000or\x00) to 255 (\377or\xFF).A Unicode escape
\uXXXXor\UXXXXXXXXdenotes the byte sequence of the UTF-8 encoding of the specified 16- or 32-bit code point. (As with text strings, the code point value must not lie in the surrogate range.)
Any valid string literal that, with a b prefix, is also a valid bytes literal is equivalent in the sense that the bytes value is the UTF-8 encoding of the string value.
TSL is space-sensitive language, and indentation is used to denote a block of statements.
Unlike Python, indentation can only be composed of space characters (U+0020), not tabs.
TODO: define indent, outdent, semicolon, newline, eof
These are the main data types built in to the interpreter:
None # the type of None
bool # True or False
int # a signed 64-bit integer
float # an IEEE 754 double-precision floating-point number
str # a text string, with Unicode encoded as UTF-8
bytes # a byte string
list # a sequence of values, usually homogeneous in nature
tuple # an unmodifiable sequence of values
dict # an associative mapping from keys to values
set # a collection of unique values
function # a functionSome functions, such as the range function, return instances of special-purpose types that don't appear in this list. Additional data types may be defined by the host application into which the interpreter is embedded, and those data types may participate in basic operations of the language such as arithmetic, comparison, indexing, and function calls.
Some operations, like bool(x) or str(x), can be applied to any TSL value, while others, like integer indexing a[i], apply only to certain types. The value concepts section explains the groupings of types by the operators they support.
None is a distinguished value used to indicate the absence of any other value. For example, the result of a call to a function that contains no return statement is None.
None is equal only to itself. Its type is "None". The truth value of None is False.
There are two Boolean values, True and False, representing the truth or falsehood of a predicate. The type of a Boolean is "bool".
Boolean values are typically used as conditions in if statements, although any TSL value may be used in this way. Aside from False itself, the following core language values are considered false in a condition:
None- numerical 0 (
0,0.0and-0.0) - the empty string (
"") and empty bytes (b"") - the empty collections
[],(),{}, andset()
All other core language values are considered true. Application-defined types determine their own truth value.
Any value may be explicitly converted to a Boolean using the built-in bool function.
True and False may be converted to the values 1 and 0 using the int function, but Booleans are not numbers. Testing a non-Boolean value for equality to a Boolean using == returns False, regardless of whether they have the same truth value.
"A" if 1 + 1 else "B" # "A"
"A" if 0.0 else "B" # "B"
if not mylist:
empty = True
1 == True # False
bool(1) == True # TrueThe TSL integer type represents integers. Its type is "int".
Integers are restricted to signed 64-bit values, with a range of -2^63 to 2^63-1. Arithmetic that overflows this range is a dynamic error. Integers are totally ordered; comparisons follow mathematical tradition.
The + and - operators perform addition and subtraction, respectively. The * operator performs multiplication.
The // and % operations on integers compute floored division and remainder of floored division, respectively. If the signs of the operands differ, the sign of the remainder x % y matches that of the divisor, y. For all finite x and y (y ≠ 0), (x // y) * y + (x % y) == x. The / operator implements floating-point division, and yields a float result even when its operands are both of type int.
Integers, including negative values, may be interpreted as bit vectors. Negative values use two's complement representation. The |, &, and ^ operators implement bitwise OR, AND, and XOR, respectively. The unary ~ operator yields the bitwise inversion of its integer argument. The << and >> operators shift the first argument to the left or right by the number of bits given by the second argument.
Any bool, number, or string may be interpreted as an integer by using the int built-in function.
An integer used in a Boolean context is considered true if it is non-zero.
100 // 5 * 9 + 32 # 212
3 // 2 # 1
111111111 * 111111111 # 12345678987654321
int("0xffff", 16) # 65535The TSL floating-point data type represents an IEEE 754 double-precision floating-point number. Its type is "float".
Arithmetic on floats using the +, -, *, /, //, and % operators follows the IEEE 754 standard. However, computing the division or remainder of division by zero is a dynamic error.
An arithmetic operation applied to a mixture of float and int operands works as if the int operand were first converted to a float. For example, 3.141 + 1 is equivalent to 3.141 + float(1). The implicit conversion fails if the int value is too large to be represented as a float.
There are two floating-point division operators: x / y yields the floating-point quotient of x and y, whereas x // y yields floor(x / y), that is, the largest representable integer value not greater than x / y. Although the resulting number is integral, it is represented as a float if either operand is a float.
The % operation computes the remainder of floored division. As with the corresponding operation on integers, if the signs of the operands differ, the sign of the remainder x % y matches that of the divisor, y.
All float values are ordered, so they may be compared using operators such as == and <, and sorted using sorted.
IEEE 754 defines two zero values, +0.0 and -0.0. They compare equal to each other.
IEEE 754 defines two infinite float values +Inf and -Inf, which represent numbers greater/less than all finite float values.
IEEE 754 defines many "not a number" (NaN) values. They are non-finite, and represent the results of dubious operations such as Inf / Inf. All NaN values compare equal to each other, but greater than any non-NaN float value. (TSL does not follow the IEEE 754 standard for NaN comparisons, which requires that all comparisons with NaN are false, except NaN != NaN.)
Because all NaN values compare equal, == is how a value is tested against one. The Python idiom x != x reports False for a NaN in TSL and so detects nothing.
Neither an infinity nor a NaN has a literal spelling. Both are constructed by passing their name to the float built-in function, which accepts inf, infinity and nan case-insensitively with an optional leading sign. Note that str spells positive infinity with its sign.
inf = float("inf") # also float("+Inf"), float("INFINITY")
neginf = float("-inf")
nan = float("nan")
str(inf) # "+inf"
str(neginf) # "-inf"
str(nan) # "nan"
inf == inf # True
neginf == neginf # True
inf == neginf # False (the two infinities are distinct)
inf == -neginf # True
nan == nan # True
nan != nan # False (this does not detect a NaN)
nan > inf # True (NaN sorts above every other float)
int(inf) # error: cannot convert float infinity to integerDivision by zero is an error rather than an infinity; Inf / Inf yields a NaN.
A comparison operation may be applied to a mixture of int and float values. The result of such comparisons is mathematically exact, even if neither operand can be exactly represented by the type of the other.
(type(1.0), type(1)) # ("float", "int")
1.0 == 1 # True
big = (1<<53)+1 # first int value that is not exactly representable as float
(big + 0.0) == big # True (zero addition is a no-op)
(big + 1.0 - 1.0) == big # False (addition/subtraction caused rounding down)
((big + 0.0) - big) == 0.0 # True (both operands subject to rounding down)Any bool, number, or string may be interpreted as a floating-point number by using the float built-in function.
A float used in a Boolean context is considered true if it is non-zero (not equal to 0.0 or -0.0). A NaN value is thus considered true.
1.23e45 * 1.23e45 # 1.5129e+90
1.111111111111111 * 1.111111111111111 # 1.2345679012345676
3.0 / 2 # 1.5
3 / 2.0 # 1.5
float(3) / 2 # 1.5
3.0 // 2.0 # 1.0A string is an immutable array of elements that encode Unicode text. The type of a string is "str".
TSL uses UTF-8 encoding internally, so each string element is an 8-bit byte (code unit). The value of K (bits per string element) is therefore 8. A single Unicode code point may span one to four elements depending on its value.
The built-in len function returns the number of code points in a string.
Strings may be concatenated with the + operator.
Strings support indexing and slicing. The result of both operations is another string (with length 1, in the case of indexing). Indexing and slicing operate per-code-point, not per-byte. Users who need byte-level access should use bytes.
Strings are hashable, and thus may be used as keys in a dictionary.
Strings are totally ordered lexicographically, so strings may be compared using operators such as == and <.
Strings are not iterable, so they cannot be used as the operand of a for loop or of methods that expect iterables (such as the list() constructor). One must instead explicitly call a method of a string value to obtain an iterable view of its elements. Because strings are not iterable, they are not considered to be subtypes of Collection or Sequence. (TSL deviates from Python here to avoid a common pitfall in which a single string is mistakenly used where a list of strings was intended, resulting in exploding the string into its individual characters.)
Any value may formatted as a string using the str or repr built-in functions, the str % tuple operator, or the str.format method.
A string used in a Boolean context is considered true if it is non-empty.
Strings have several built-in methods. See the TSL Built-in API Reference for the full list.
A bytes is an immutable array of integers in the range 0-255. The type of a bytes is "bytes".
Unlike a string, which is intended for text, a bytes may represent binary data, such as the contents of an arbitrary file, without loss.
The built-in len function returns the number of elements (bytes) in a bytes.
Two bytes values may be concatenated with the + operator.
Bytes may be indexed and sliced. The result of indexing is the value of the byte at the given position, as an integer. The result of slicing is another bytes object.
The comparison x in b, where b is a bytes value and x is an integer in the range [0, 255], tests whether x's value is an element of b. If x is instead another bytes value, the comparison tests whether x is contained as a consecutive subsequence of b (analogous to substrings of a string). It is an error if x is an out-of-range integer, or any other type of value.
Like strings, bytes values are hashable, totally ordered, and considered True if they are non-empty. They are not iterable, and not a subtype of Collection nor Sequence.
See the TSL Built-in API Reference for bytes methods.
A list is a mutable sequence of values. The type of a list is "list".
Lists are a subtype of Sequence. The number of elements may be retrieved with the built-in len() function. Lists can be indexed to retrieve a single element, and sliced to produce a new list. Lists can be iterated over by for loops.
List may be constructed using bracketed list notation:
[] # an empty list
[1] # a 1-element list
[1, 2] # a 2-element listLists can also be constructed from any iterable by using the built-in list function.
List elements may be added using the append or extend methods, removed using the remove method, or reordered by assignments such as list[i] = list[j].
The concatenation operation x + y yields a new list containing all the elements of the two lists x and y.
For most types, x += y is equivalent to x = x + y, except that it evaluates x only once, that is, it allocates a new list to hold the concatenation of x and y. However, if x refers to a list, the statement does not allocate a new list but instead mutates the original list in place, similar to x.extend(y).
Lists are not hashable, so may not be used in the keys of a dictionary.
A list used in a Boolean context is considered true if it is non-empty.
A list comprehension creates a new list whose elements are the result of some expression applied to each element of another sequence.
[x*x for x in [1, 2, 3, 4]] # [1, 4, 9, 16]See the TSL Built-in API Reference for list methods.
A tuple is an immutable sequence of values. The type of a tuple is "tuple".
Like lists, tuple is a subtype of Sequence, and supports the same len(), indexing, slicing, and iteration operations. A slice of a tuple returns another tuple.
Tuples are constructed using parenthesized list notation:
() # the empty tuple
(1,) # a 1-tuple
(1, 2) # a 2-tuple ("pair")
(1, 2, 3) # a 3-tupleObserve that for the 1-tuple, the trailing comma is necessary to distinguish it from the parenthesized expression (1). 1-tuples are seldom used.
TSL, unlike Python, does not permit a trailing comma to appear in an unparenthesized tuple expression:
for k, v, in dict.items(): pass # syntax error at 'in'
_ = [(v, k) for k, v, in dict.items()] # syntax error at 'in'
sorted(3, 1, 4, 1,) # ok
[1, 2, 3, ] # ok
{1: 2, 3:4, } # okTuples can be constructed from any iterable by using the built-in tuple function.
Unlike lists, tuples cannot be modified. However, the mutable elements of a tuple may be modified.
Tuples are hashable if their elements are hashable, in which case they may be used as keys of a dictionary.
Tuples may be concatenated using the + operator. Note that it is not legal to directly concatenate a tuple with a list.
A tuple used in a Boolean context is considered true if it is non-empty.
A dictionary is a mutable mapping from keys to values. The type of a dictionary is "dict".
Dictionaries provide constant-time operations to insert an element, to look up the value for a key, or to remove an element. Dictionaries are implemented using hash tables, so keys must be hashable. Hashable values include None, Booleans, numbers, strings, and bytes, and tuples composed from hashable values. Most mutable values, such as lists, dictionaries, and sets, are not hashable, unless they are frozen.
Attempting to use a non-hashable value as a key in a dictionary results in a dynamic error.
A dictionary expression specifies a dictionary as a set of key/value pairs enclosed in braces:
coins = {
"penny": 1,
"nickel": 5,
"dime": 10,
"quarter": 25,
}The expression d[k], where d is a dictionary and k is a key, retrieves the value associated with the key. If the dictionary contains no such item, the operation fails:
coins["penny"] # 1
coins["dime"] # 10
coins["silver dollar"] # error: key not foundA key/value item may be added to a dictionary, or updated if the key is already present, by using d[k] on the left side of an assignment:
len(coins) # 4
coins["shilling"] = 20
len(coins) # 5, item was inserted
coins["shilling"] = 5
len(coins) # 5, existing item was updatedA dictionary can also be constructed using a dictionary comprehension, which evaluates a pair of expressions, the key and the value, for every element of another iterable such as a list. This example builds a mapping from each word to its length:
words = ["able", "baker", "charlie"]
{x: len(x) for x in words} # {"charlie": 7, "baker": 5, "able": 4}Dictionaries are a subtype of Mapping. len() returns the number of entries. Iteration yields the keys in the order that they were inserted; updating the value associated with an existing key does not affect the iteration order.
x = dict([("a", 1), ("b", 2)]) # {"a": 1, "b": 2}
x.update([("a", 3), ("c", 4)]) # {"a": 3, "b": 2, "c": 4}for name in coins:
print(name, coins[name]) # prints "quarter 25", "dime 10", ...Like all mutable values in TSL, a dictionary can be frozen, and once frozen, all subsequent operations that attempt to update it will fail.
A dictionary used in a Boolean context is considered true if it is non-empty.
The binary | operation may be applied to two dictionaries. It yields a new dictionary whose set of keys is the union of the sets of keys of the two operands. The corresponding values are taken from the operands, where the value taken from the right operand takes precedence if both contain a given key. Iterating over the keys in the resulting dictionary first yields all keys in the left operand in insertion order, then all keys in the right operand that were not present in the left operand, again in insertion order.
There is also an augmented assignment version of the | operation. For two dictionaries d1 and d2, the expression d1 |= d2 behaves similar to d1 = d1 | d2, but mutates d1 in-place rather than assigning a new dictionary to it.
Dictionaries may be compared for equality using == and !=. Two dictionaries compare equal if they contain the same number of items and each key/value item (k, v) found in one dictionary is also present in the other. Dictionaries are not ordered; it is an error to compare two dictionaries with <.
See the TSL Built-in API Reference for dictionary methods.
A set is a mutable collection of unique values - the set's elements. The type of a set is "set".
Sets provide constant-time operations to insert, remove, or check for the presence of a value. Sets are implemented using a hash table, and therefore, just like keys of a dictionary, elements of a set must be hashable. A value may be used as an element of a set if and only if it may be used as a key of a dictionary.
Sets may be constructed using the set() built-in function, which returns a set containing all the elements of its optional argument, which must be an iterable. Calling set() without an argument constructs an empty set. Sets have no literal syntax.
The in and not in operations check whether a value is (or is not) in a set:
s = set(["a", "b", "c"])
"a" in s # True
"z" in s # FalseA set is a subtype of Collection. Its size can be retrieved using the len() built-in function, and the order of iteration is the order in which elements were inserted. (Attempting to add an element that is already present is a no-op, and does not change the iteration order.)
s = set(["z", "y", "z", "y"])
len(s) # prints 2
s.add("x")
s.add("z")
len(s) # prints 3
for e in s:
print(e) # prints "z", "y", "x"A set used in Boolean context is true if and only if it is non-empty.
s = set()
"non-empty" if s else "empty" # "empty"
t = set(["x", "y"])
"non-empty" if t else "empty" # "non-empty"Sets may be compared for equality or inequality using == and !=. A set s is equal to t if and only if t is a set containing the same elements; iteration order is not significant. In particular, a set is not equal to the list of its elements. Sets are not ordered with respect to other sets, and an attempt to compare two sets using <, <=, >, >=, or to sort a sequence of sets, will fail.
set() == set() # True
set() != [] # True
set([1, 2]) == set([2, 1]) # True
set([1, 2]) != [1, 2] # TrueThe | operation on two sets returns the union of the two sets: a set containing the elements found in either one or both of the original sets.
set([1, 2]) | set([3, 2]) # set([1, 2, 3])The & operation on two sets returns the intersection of the two sets: a set containing only the elements found in both of the original sets.
set([1, 2]) & set([2, 3]) # set([2])
set([1, 2]) & set([3, 4]) # set()The - operation on two sets returns the difference of the two sets: a set containing the elements found in the left-hand side set but not the right-hand side set.
set([1, 2]) - set([2, 3]) # set([1])
set([1, 2]) - set([3, 4]) # set([1, 2])The ^ operation on two sets returns the symmetric difference of the two sets: a set containing the elements found in exactly one of the two original sets, but not in both.
set([1, 2]) ^ set([2, 3]) # set([1, 3])
set([1, 2]) ^ set([3, 4]) # set([1, 2, 3, 4])In each of the above operations, the elements of the resulting set retain their order from the two operand sets, with all elements that were drawn from the left-hand side ordered before any element that was only present in the right-hand side.
The corresponding augmented assignments, |=, &=, -=, and ^=, modify the left-hand set in place.
s = set([1, 2])
s |= set([2, 3, 4]) # s now equals set([1, 2, 3, 4])
s &= set([0, 1, 2, 3]) # s now equals set([1, 2, 3])
s -= set([0, 1]) # s now equals set([2, 3])
s ^= set([3, 4]) # s now equals set([2, 4])Like all mutable values in TSL, a set can be frozen, and once frozen, all subsequent operations that attempt to update it will fail.
See the TSL Built-in API Reference for set methods.
A function value represents a function defined in TSL. Its type is "function". A function value used in a Boolean context is always considered true.
Functions defined by a def statement are named; functions defined by a lambda expression are anonymous.
Function definitions may be nested, and an inner function may refer to a local variable of an outer function. TSL has no equivalent of Python's nonlocal keyword, and thus no way for an inner function to assign to a local variable of an outer function. However, the inner function may mutate the value of such variables until they become frozen.
A function definition defines zero or more named parameters. TSL has a rich mechanism for passing arguments to functions.
The example below shows a definition and call of a function of two required parameters, x and y.
def idiv(x, y):
return x // y
idiv(6, 3) # 2A call may provide arguments to function parameters either by position, as in the example above, or by name, as in first two calls below, or by a mixture of the two forms, as in the third call below. All the positional arguments must precede all the named arguments. Named arguments may improve clarity, especially in functions of several parameters.
idiv(x=6, y=3) # 2
idiv(y=3, x=6) # 2
idiv(6, y=3) # 2Optional parameters: A parameter declaration may specify a default value using name=value syntax; such a parameter is optional. The default value expression is evaluated during execution of the def statement, and the default value forms part of the function value. All optional parameters must follow all non-optional parameters. A function call may omit arguments for any suffix of the optional parameters; the effective values of those arguments are supplied by the function's parameter defaults.
def f(x, y=3):
return x, y
f(1, 2) # (1, 2)
f(1) # (1, 3)If a function parameter's default value is a mutable expression, modifications to the value during one call may be observed by subsequent calls. Beware of this when using lists or dicts as default values. If the function becomes frozen, its parameters' default values become frozen too.
# module a.sky
def f(x, mutable_var=[]):
mutable_var.append(x)
return mutable_var
f(4, [1,2,3]) # [1, 2, 3, 4]
f(1) # [1]
f(2) # [1, 2], not [2]!
# module b.sky
import a
a.f(3) # error: cannot append to frozen listVariadic functions: Some functions allow callers to provide an arbitrary number of arguments. After all required and optional parameters, a function definition may specify a variadic arguments list or varargs parameter, indicated by a star preceding the parameter name: *args. Any surplus positional arguments provided by the caller are formed into a tuple and assigned to the args parameter.
def f(x, y, *args):
return x, y, args
f(1, 2) # (1, 2, ())
f(1, 2, 3, 4) # (1, 2, (3, 4))Keyword-variadic functions: Some functions allow callers to provide an arbitrary sequence of name=value keyword arguments. A function definition may include a final keyword arguments dictionary or kwargs parameter, indicated by a double-star preceding the parameter name: **kwargs. Any surplus named arguments that do not correspond to named parameters are collected in a new dictionary and assigned to the kwargs parameter:
def f(x, y, **kwargs):
return x, y, kwargs
f(1, 2) # (1, 2, {})
f(x=2, y=1) # (2, 1, {})
f(x=2, y=1, z=3) # (2, 1, {"z": 3})It is a static error if any two parameters of a function have the same name.
Just as a function definition may accept an arbitrary number of positional or named arguments, a function call may provide an arbitrary number of positional or named arguments supplied by a list or dictionary:
def f(a, b, c=5):
return a * b + c
f(*[2, 3]) # 11
f(*[2, 3, 7]) # 13
f(*[2]) # error: f takes at least 2 arguments (1 given)
f(**dict(b=3, a=2)) # 11
f(**dict(c=7, a=2, b=3)) # 13
f(**dict(a=2)) # error: f takes at least 2 arguments (1 given)
f(**dict(d=4)) # error: f got unexpected keyword argument "d"Once the parameters have been successfully bound to the arguments supplied by the call, the sequence of statements that comprise the function body is executed.
It is a static error if a function call has two named arguments of the same name, such as f(x=1, x=2). A call that provides a **kwargs argument may yet have two values for the same name, such as f(x=1, **dict(x=2)). This results in a dynamic error.
Function arguments are evaluated in the order they appear in the call.
Unlike Python, TSL does not allow more than one *args argument in a call, and if a *args argument is present it must appear after all positional and named arguments. In particular, even though keyword-only arguments (see below) are declared after *args in a function's definition, they nevertheless must appear before *args in a call to the function.
A function call completes normally after the execution of either a return statement, or of the last statement in the function body. The result of the function call is the value of the return statement's operand, or None if the return statement had no operand or if the function completed without executing a return statement.
def f(x):
if x == 0:
return
if x < 0:
return -x
print(x)
f(1) # returns None after printing "1"
f(0) # returns None without printing
f(-1) # returns 1 without printingA built-in function is a function or method implemented by the interpreter or the application into which the interpreter is embedded. Its type is "builtin_function_or_method".
A built-in function value used in a Boolean context is always considered true.
Many built-in functions are predeclared in the environment; see Name Resolution. Some built-in functions such as len are universal, that is, available to all TSL programs. The host application may predeclare additional built-in functions in the environment of a specific module.
Except where noted, built-in functions accept only positional arguments.
The predeclared environment includes the constructible exception type Exception and a set of non-constructible exception type identifiers (RuntimeError, ValueError, TypeError, KeyError, IndexError, LookupError, IOError, ArithmeticError) usable only in except clauses. See Exception types.
After a TSL file is parsed, but before its execution begins, the TSL interpreter checks statically that the program is well formed. For example, break and continue statements may appear only within a for or while loop; return statements may appear only within a function; and import statements may appear only outside any function.
Name resolution is the static checking process that resolves names to variable bindings. During execution, names refer to variables. Statically, names denote places in the code where variables are created; these places are called bindings. A name may denote different bindings at different places in the program. The region of text in which a particular name refers to the same binding is called that binding's scope.
Four TSL constructs bind names, as illustrated in the example below: import statements (mylib), def statements (c), function parameters (d), and assignments (e, h, including the augmented assignment e += h). Variables may be assigned or re-assigned explicitly (e, h), or implicitly, as in a for statement (f) or comprehension for clause (g, i).
import mylib
def c(d):
e = 0
for f in d:
print([True for g in f])
e += 1
h = [2*i for i in mylib.items]The environment of a TSL program is structured as a tree of lexical blocks, each of which may contain name bindings. The tree of blocks is parallel to the syntax tree. Blocks are of five kinds.
At the root of the tree is the predeclared block, which binds several names implicitly. The set of predeclared names includes the universal constant values None, True, and False, and various built-in functions such as len and list; these functions are immutable and stateless. An application may pre-declare additional names to provide domain-specific functions to that file, for example. These additional functions may have side effects on the application. TSL programs cannot change the set of predeclared bindings or assign new values to them. Unlike standard Starlark, TSL does not allow shadowing of built-in or application-defined names in any scope.
Nested beneath the predeclared block is the module block, which contains the bindings of the current module. Bindings in the module block (such as mylib, c, and h in the example) are called global and may be visible to other modules. The module block is empty at the start of the file and is populated by top-level binding statements, but an application may pre-bind one or more global names, to provide domain-specific functions to that file, for example.
Nested beneath the module block is the file block, which contains bindings local to the current file. Names in this block are bound by import statements. The sets of names bound in the file block and in the module block do not overlap: it is an error for an import statement to bind the name of a global, or for a top-level statement to bind a name bound by an import statement.
A file block contains a function block for each top-level function, and a comprehension block for each top-level comprehension. Bindings in either of these kinds of block, and in the file block itself, are called local. (In the example, the bindings for e, f, g, and i are all local.) Additional functions and comprehensions, and their blocks, may be nested in any order, to any depth.
If name is bound anywhere within a block, all uses of the name within the block are treated as references to that binding, even if the use appears before the binding. This is true even at the top level, unlike Python. The binding of y on the last line of the example below makes y local to the function hello, so the use of y in the print statement also refers to the local y, even though it appears earlier.
y = "goodbye"
def hello():
for x in (1, 2):
if x == 2:
print(y) # prints "hello"
if x == 1:
y = "hello"It is a dynamic error to evaluate a reference to a local variable before it has been bound:
def f():
print(x) # dynamic error: local variable x referenced before assignment
x = "hello"The same is true for global variables:
print(x) # dynamic error: global variable x referenced before assignment
x = "hello"The same is also true for nested loops in comprehensions. In the (unnatural) examples below, the scope of the variables x, y, and z is the entire compehension block, except the operand of the first loop ([] or [1]), which is resolved in the enclosing environment. The second loop may thus refer to variables defined by the third (z), even though such references would fail if actually executed.
[1//0 for x in [] for y in z for z in ()] # [] (no error)
[1//0 for x in [1] for y in z for z in ()] # dynamic error: local variable z referenced before assignmentIt is a static error to refer to a name that has no binding at all.
def f():
if False:
g() # static error: undefined: g(This behavior differs from Python, which treats such references as global, and thus does not report an error until the expression is evaluated.)
Global variables may be re-assigned during module evaluation. This includes augmented assignments such as x += 1. Exported global variables are still frozen after module execution completes.
x = 1
x = 2 # valid in TSL: globals can be re-assigned
x += 3 # valid in TSL: augmented assignment on globals is allowedHowever, it is not permitted to shadow built-in or application-defined names in any scope, nor to reassign imported module names:
import fs
fs = 1234 # static error: cannot reassign imported module name
print = 1234 # static error: cannot shadow built-in nameInternally, TSL uses cell variables to support closures: when a local variable in an outer function is referenced by an inner function, the variable is converted to a cell that can be shared across scopes.
A name appearing after a dot, such as split in get_filename().split('/'), is not resolved statically. The dot expression .split is a dynamic operation on the value returned by get_filename().
TSL has over a dozen core data types. An application that embeds the TSL intepreter may define additional types that behave like TSL values. All values, whether core or application-defined, implement a few basic behaviors:
str(x) -- return a string representation of x
type(x) -- return a string describing the type of x
bool(x) -- convert x to a Boolean truth value
hash(x) -- return a hash code for xTSL is an imperative language: programs consist of sequences of statements executed for their side effects. For example, an assignment statement updates the value held by a variable, and calls to some built-in functions such as print change the state of the application that embeds the interpreter.
Values of some data types, such as None, bool, int, float, str, and bytes, are immutable; they can never change. Immutable values have no notion of identity: it is impossible for a TSL program to tell whether two integers, for instance, are represented by the same object; it can tell only whether they are equal.
Values of other data types, such as list and dict, are mutable: they may be modified by a statement such as a[i] = 0 or items.clear(). Although tuple and function values are not directly mutable, they may refer to mutable values indirectly, so for this reason we consider them mutable too. TSL values of these types are actually references to variables.
Copying a reference to a variable, using an assignment statement for instance, creates an alias for the variable, and the effects of operations applied to the variable through one alias are visible through all others.
x = [] # x refers to a new empty list variable
y = x # y becomes an alias for x
x.append(1) # changes the variable referred to by x
print(y) # "[1]"; y observes the mutationTSL uses call-by-value parameter passing: in a function call, argument values are assigned to function parameters as if by assignment statements. If the values are references, the caller and callee may refer to the same variables, so if the called function changes the variable referred to by a parameter, the effect may also be observed by the caller:
def f(y):
y.append(1) # changes the variable referred to by x
x = [] # x refers to a new empty list variable
f(x) # f's parameter y becomes an alias for x
print(x) # "[1]"; x observes the mutationAs in all imperative languages, understanding aliasing, the relationship between reference values and the variables to which they refer, is crucial to writing correct programs.
TSL has a feature unusual among imperative programming languages: a mutable value may be frozen so that all subsequent attempts to mutate it fail with a dynamic error; the value, and all other values reachable from it, become immutable.
Immediately after execution of a TSL module, all values in its top-level environment are frozen. Because all the global variables of an initialized TSL module are immutable, the module may be published to and used by other threads in a parallel program without the need for locks. For example, the Bazel build system loads and executes BUILD and .bzl files in parallel, and two modules being executed concurrently may freely access variables or call functions from a third without the possibility of a race condition.
Some values may be "unfreezeable" and therefore not available post module execution. Values that are not able to be frozen are likely holding OS resources and are designed to be short lived. It's likely that the attributes on that type are freezeable, and so you'd just need to grab a copy to that type to export (if needed). See the documentation for types to see if it's unfreezable. The behavior when trying to freeze an unfreezeable value is up to the runtime environment, it may cause a failure, or it may silently not export the value. If a container type contains an unfreezeable value, it should be treated as unfreezeable.
The dict data type is implemented using hash tables, so only hashable values are suitable as keys of a dict. Attempting to use a non-hashable value as the key in a dictionary results in a dynamic error.
The hash of a value is a 64-bit integer chosen so that two equal values have the same hash, in other words, x == y => hash(x) == hash(y). TSL uses 64-bit FNV-1a (FNV64-1a) as the hash algorithm. A hashable value has the same hash throughout its lifetime.
Values of the types None, bool, int, float, str, and bytes, which are all immutable, are hashable.
Values of mutable types such as list and dict are not hashable, unless they have become immutable due to freezing.
A tuple value is hashable only if all its elements are hashable. Thus ("localhost", 80) is hashable but ([127, 0, 0, 1], 80) is not.
Values of the types function and builtin_function_or_method are also hashable. Although functions are not necessarily immutable, as they may be closures that refer to mutable variables, instances of these types are compared by reference identity (see Comparisons), so their hash values are derived from their identity.
TSL defines several abstract data types. Although these are not directly referenced by name in TSL programs, they are useful concepts for understanding what operations a type supports and for documenting what types a function expects.
Collection: A type that is:- iterable (can appear on the right-hand side of a
forloop, whether that be aforstatement or comprehensionforclause), - can have its length taken via the
len()function, and - supports testing for membership via the
inoperator.
Examples include
list,tuple,set, anddict(its elements are its keys), but notstrorbytessince they are not iterable.- iterable (can appear on the right-hand side of a
Sequence: ACollectionthat additionally supports subscript and slicing expressions, following the semantics of indexing.listandtupleareSequences, but notsetordict.Mapping: ACollectionof keys associated with values, where a value can be retrieved by subscripting with the key.dictis the only example of aMappingin the core language.
Applications may define additional data structures that may or may not have a particular abstract data type. If it does have the type, it must support all of its required operations, but the converse is not necessarily true.
It is a dynamic error to mutate a collection such as a list, set, or dictionary while iterating over it. This helps avoid a common source of errors and implementation pitfalls. For example, the following code snippet is syntactically valid in both Python and TSL:
a = [5, 11, 12, 13, 8]
for x in a:
if x > 10:
a.remove(x)In Python, the result is [5, 12, 8] -- the 12 is missed due to the implicit iterator's internal integer indexing. But in TSL, an error is raised at the attempt to mutate a inside the loop. The same code would succeed with the correct result in both languages if the loop were rewritten as for x in list(a).
Unlike Python 3, the result of certain methods like enumerate() or zip() is a new list rather than an iterator object. Loops over such lists will not prevent modifying the original list.
Many TSL operators and functions require an index operand i, such as a[i] or list.insert(i, x). Others require two indices i and j that indicate the start and end of a subsequence, such as a[i:j], list.index(x, i, j), or string.find(x, i, j). All such operations follow similar conventions, described here.
Indexing in TSL is zero-based. The first element of a string or list has index 0, the next 1, and so on. The last element of a sequence of length n has index n-1.
"hello"[0] # "h"
"hello"[4] # "o"
"hello"[5] # error: index out of rangeFor subsequence operations that require two indices, the first is inclusive and the second exclusive. Thus a[i:j] indicates the sequence starting with element i up to but not including element j. The length of this subsequence is j-i. This convention is known as half-open indexing.
"hello"[1:4] # "ell"Either or both of the index operands may be omitted. If omitted, the first is treated equivalent to 0 and the second is equivalent to the length of the sequence:
"hello"[1:] # "ello"
"hello"[:4] # "hell"It is permissible to supply a negative integer to an indexing operation. The effective index is computed from the supplied value by the following two-step procedure. First, if the value is negative, the length of the sequence is added to it. This provides a convenient way to address the final elements of the sequence:
"hello"[-1] # "o", like "hello"[4]
"hello"[-3:-1] # "ll", like "hello"[2:4]Second, for subsequence operations, if the value is still negative, it is replaced by zero, or if it is greater than the length n of the sequence, it is replaced by n. In effect, the index is "truncated" to the nearest value in the range [0:n].
"hello"[-1000:1000] # "hello"This truncation step does not apply to indices of individual elements:
"hello"[-6] # error: index out of range
"hello"[-5] # "h"
"hello"[4] # "o"
"hello"[5] # error: index out of rangeAn expression specifies the computation of a value.
The TSL grammar defines several categories of expression. An operand is an expression consisting of a single token (such as an identifier or a literal), or a bracketed expression. Operands are self-delimiting. An operand may be followed by any number of dot, call, or slice suffixes, to form a primary expression. In some places in the TSL grammar where an expression is expected, it is legal to provide a comma-separated list of expressions denoting a tuple. The grammar uses Expressions where a multiple-component expression is allowed, and Expression where it accepts an expression of only a single component.
Expressions = Expression {',' Expression} .
Expression = IfExpr | PrimaryExpr | UnaryExpr | BinaryExpr | LambdaExpr .
PrimaryExpr = Operand
| PrimaryExpr DotSuffix
| PrimaryExpr CallSuffix
| PrimaryExpr SubscriptSuffix
.
Operand = identifier
| int | float | string | bytes
| ListExpr | ListComp
| DictExpr | DictComp
| '(' [Expressions] [,] ')'
.
DotSuffix = '.' identifier .
CallSuffix = '(' [Arguments [',']] ')' .
SubscriptSuffix = '[' [Expressions] [':' Expression [':' Expression]] ']'
| '[' Expressions ']'
.Operand = identifierAn identifier is a name that identifies a value.
Lookup of locals and globals may fail if not yet defined.
TSL supports literals of four different kinds:
Operand = int | float | string | bytesEvaluation of an int, float, string, or bytes literal yields the value of that literal. See Literals for details.
Operand = '(' [Expressions] ')'A single expression enclosed in parentheses yields the result of that expression. Explicit parentheses may be used for clarity, or to override the default association of subexpressions.
1 + 2 * 3 + 4 # 11
(1 + 2) * (3 + 4) # 21If the parentheses are empty, or contain a single expression followed by a comma, or contain two or more expressions, the expression yields a tuple.
() # (), the empty tuple
(1,) # (1,), a tuple of length 1
(1, 2) # (1, 2), a 2-tuple or pair
(1, 2, 3) # (1, 2, 3), a 3-tuple or tripleIn some contexts, such as a return or assignment statement or the operand of a for statement, a tuple may be expressed without parentheses.
x, y = 1, 2
return 1, 2
for x in 1, 2:
print(x)TSL (like Python 3) does not accept an unparenthesized tuple or lambda expression as the operand of a for clause in a comprehension:
[2*x for x in 1, 2, 3] # parse error: unexpected ','
[2*x for x in lambda: 0] # parse error: unexpected 'lambda'A dictionary expression is a comma-separated list of colon-separated key/value expression pairs, enclosed in curly brackets, and it yields a new dictionary object. An optional comma may follow the final pair.
DictExpr = '{' [Entries [',']] '}' .
Entries = Entry {',' Entry} .
Entry = Expression ':' Expression .Examples:
{}
{"one": 1}
{"one": 1, "two": 2,}The key and value expressions are evaluated in left-to-right order. Evaluation fails if the same key is used multiple times.
Only hashable values may be used as the keys of a dictionary.
A list expression is a comma-separated list of element expressions, enclosed in square brackets, and it yields a new list object. An optional comma may follow the last element expression.
ListExpr = '[' [Expressions [',']] ']' .Element expressions are evaluated in left-to-right order.
Examples:
[] # [], empty list
[1] # [1], a 1-element list
[1, 2, 3,] # [1, 2, 3], a 3-element listThere are four unary operators, all appearing before their operand: +, -, ~, and not.
UnaryExpr = '+' Expression
| '-' Expression
| '~' Expression
| 'not' Expression
.+ number unary positive (int, float)
- number unary negation (int, float)
~ number unary bitwise inversion (int)
not x logical negation (any type)The + and - operators may be applied to any number: + yields the operand unchanged, and - yields its negation. The + operator is never necessary in a correct program but may serve as an assertion that its operand is a number, or as documentation.
if x > 0:
return +1
elif x < 0:
return -1
else:
return 0The not operator returns the negation of the truth value of its operand.
not True # False
not False # True
not [1, 2, 3] # False
not "" # True
not 0 # TrueThe ~ operator yields the bitwise inversion of its integer argument. The bitwise inversion of x is defined as -(x+1).
~1 # -2
~-1 # 0
~0 # -1TSL has the following binary operators, arranged in order of increasing precedence:
or
and
== != < > <= >= in not in
|
^
&
<< >>
- +
* / // %Comparison operators, in, and not in are non-associative, so the parser will not accept 0 <= i < n. All other binary operators of equal precedence associate to the left.
BinaryExpr = Expression {Binop Expression} .
Binop = 'or'
| 'and'
| '==' | '!=' | '<' | '>' | '<=' | '>=' | 'in' | 'not' 'in'
| '|'
| '^'
| '&'
| '<<' | '>>'
| '-' | '+'
| '*' | '%' | '/' | '//'
.The or and and operators yield, respectively, the logical disjunction and conjunction of their arguments, which need not be Booleans. The expression x or y yields the value of x if its truth value is True, or the value of y otherwise.
False or False # False
False or True # True
True or False # True
True or True # True
0 or "hello" # "hello"
1 or "hello" # 1Similarly, x and y yields the value of x if its truth value is False, or the value of y otherwise.
False and False # False
False and True # False
True and False # False
True and True # True
0 and "hello" # 0
1 and "hello" # "hello"These operators use "short circuit" evaluation, so the second expression is not evaluated if the value of the first expression has already determined the result, allowing constructions like these:
len(x) > 0 and x[0] == 1 # x[0] is not evaluated if x is empty
x and x[0] == 1
len(x) == 0 or x[0] == ""
not x or not x[0]The == operator reports whether its operands are equal; the != operator is its negation.
The operators <, >, <=, and >= perform an ordered comparison of their operands. It is an error to apply these operators to operands of unequal type, unless one of the operands is an int and the other is a float. Of the built-in types, only the following support ordered comparison, using the ordering relation shown:
bool # False < True
int # mathematical
float # as defined by IEEE 754, except NaN > +Inf
string # lexicographical
bytes # lexicographical
tuple # lexicographical
list # lexicographicalComparison of floating-point values follows the IEEE 754 standard for finite values (including -0.0) and for positive and negative infinity, but not for NaN values, for which the standard behavior would break several mathematical identities. Thus:
-Inf < -1e50 < -1.0 < -1e-50 < 0.0 < 1e-50 < 1.0 < 1e50 < +Inf < NaN
+0.0 == -0.0
NaN == NaNApplications may define additional types that support ordered comparison. The application-defined comparison relation must be a strict weak ordering.
The remaining built-in types support only equality comparisons. Values of type dict compare equal if their elements compare equal, and values of type function are equal only to themselves.
dict # equal contents
function # identityThe following table summarizes the binary arithmetic operations available for built-in types:
Arithmetic (int or float; result has type float unless both operands have type int)
number + number # addition
number - number # subtraction
number * number # multiplication
number / number # floating-point division (result is always a float)
number // number # floored division
number % number # remainder of floored division
Bitwise operations:
int ^ int # bitwise XOR
int & int # bitwise AND
int | int # bitwise OR
int << int # bitwise left shift
int >> int # bitwise right shift (arithmetic)
Set operations:
set | set # set union
set & set # set intersection
set - set # set difference
set ^ set # set symmetric difference
Concatenation
string + string
bytes + bytes
list + list
tuple + tuple
Repetition (string/bytes/list/tuple)
int * sequence
sequence * int
String interpolation
string % any # see String Interpolation
Set or dictionary union
dict | dict # see Dictionaries, SetsThe operands of the arithmetic operators +, -, *, //, and %, must both be numbers (int or float) but need not have the same type. The type of the result has type int only if both operands have that type. The result of floating-point division / always has type float.
The & operator requires two operands of type int, and yields the bitwise intersection (AND) of its operands. The | operator likewise computes bitwise union, and the ^ operator bitwise XOR (exclusive OR).
The << and >> operators require two operands of type int. They shift the first operand to the left or right by the number of bits given by the second operand. Right shifts are arithmetic, not logical: they fill the vacated bits with copies of the sign bit. It is a dynamic error if the second operand is negative. Left shifts will also throw an error if the result is not representable using a signed 64-bit integer.
0x12345678 & 0xFF # 0x00000078
0x12345678 | 0xFF # 0x123456FF
0b01011101 ^ 0b110101101 # 0b111110000
0b01011101 >> 2 # 0b010111
0b01011101 << 2 # 0b0101110100
-1 >> 100 # error; shifts over 63 are not allowedThe + operator may be applied to non-numeric operands of the same type, such as two lists, two tuples, two strings, or two bytes, in which case it computes the concatenation of the two operands and yields a new value of the same type.
"Hello, " + "world" # "Hello, world"
(1, 2) + (3, 4) # (1, 2, 3, 4)
[1, 2] + [3, 4] # [1, 2, 3, 4]The * operator may be applied to an integer n and a value of type str, bytes, list, or tuple, in which case it yields a new value of the same sequence type consisting of n repetitions of the original sequence. The order of the operands is immaterial. Negative values of n behave like zero.
'mur' * 2 # 'murmur'
3 * (True, "a") # (True, "a", True, "a", True, "a")Applications may define additional types that support any subset of these operators.
any in sequence (list, tuple, dict, set, string, bytes, range*)
any not in sequenceThe in operator reports whether its first operand is a member of its second operand, which must be a list, tuple, dict, set, string, bytes, or range. (*range is a special-purpose type returned by the built-in range function; see the Built-in API Reference.) The not in operator is its negation. Both return a Boolean.
The meaning of membership varies by the type of the second operand: the members of a list or tuple are its elements; the members of a dict are its keys; the members of a string or bytes are all its substrings. Additionally, the members of a bytes include the int values of its (byte) elements.
1 in [1, 2, 3] # True
4 not in (1, 2, 3) # True
d = {"one": 1, "two": 2}
"one" in d # True
"three" in d # False
1 in d # False
"nasty" in "dynasty" # True
"a" in "banana" # True
"f" not in "way" # True
b"nasty" in b"dynasty" # True
97 in b"abc" # True (97 = 'a')The expression format % args performs string interpolation, a simple form of template expansion. The format string is interpreted as a sequence of literal portions and conversions. Each conversion, which starts with a % character, is replaced by its corresponding value from args. The characters following % in each conversion determine which argument it uses and how to convert it to a string.
Each % character marks the start of a conversion specifier, unless it is immediately followed by another %, in which cases both characters together denote a single literal percent sign.
The conversion's operand is the next element of args, which must be a tuple with exactly one component per conversion, unless the format string contains only a single conversion, in which case args itself is its operand.
TSL does not support the flag, width, and padding specifiers supported by Python's % and other variants of C's printf.
After the % comes a single letter indicating what operand types are valid and how to convert the operand x to a string:
% none literal percent sign
s any as if by str(x)
r any as if by repr(x)
d number signed integer decimal
o number signed octal, no 0o prefix
x number signed hexadecimal, lowercase, no 0x prefix
X number signed hexadecimal, uppercase, no 0x prefix
e number float exponential format, lowercase (1.230000e+12)
E number float exponential format, uppercase (1.230000E+12)
f number float decimal format (1230000000000.000000)
F number same as %f
g number compact format, lowercase (0.0, 1.1, 1200, 1e+45, 1.2e+12)
G number compact format, uppercase (0.0, 1.1, 1200, 1e+45, 1.2E+12)TSL only supports the conversion specifiers listed above; %c and other specifiers found in some implementations are not available.
str(float) is not %g: where %g renders six significant digits, str uses the least precision that still reads back as the same value, so str(1.2345679012345676) keeps every digit while "%g" of it gives 1.23457. str omits unnecessary trailing zeros in the significand (along with the decimal point itself if the significand has no fraction), and always contains a decimal point or an exponent and thus unambiguously denotes a float, not an int. Non-finite values render as +inf, -inf and nan under both.
It is an error if the argument does not have the type required by the conversion specifier, except that ints may be converted to floats and floats may be truncated to ints. A Boolean argument is not considered a number.
Examples:
"Hello %s" % "Bob" # "Hello Bob"
"Hello %s, your score is %d" % ("Bob", 75) # "Hello Bob, your score is 75"One subtlety: to use a tuple as the operand of a conversion in format string containing only a single conversion, you must wrap the tuple in a singleton tuple:
"coordinates=%s" % (40, -74) # error: too many arguments for format string
"coordinates=%s" % ((40, -74),) # "coordinates=(40, -74)"A conditional expression has the form a if cond else b. It first evaluates the condition cond. If it's true, it evaluates a and yields its value; otherwise it yields the value of b.
IfExpr = Expression 'if' Expression 'else' Expression .Example:
"yes" if enabled else "no"During parsing, the if operator, considered as a postfix operator on the "true" expression, has higher precedence than else (a prefix operator on the "false" expression), which in turn has higher precedence than the lambda prefix operator.
a if b else (c if d else e) # parens are redundant
(a if b else c) if d else e # parens are required
lambda: (a if b else c) # parens are redundant
(lambda: a) if b else c # parens are required
a if b else lambda: (c if d else e) # parens are redundant
a if b else (lambda: c if d else e) # parens are required
(a if b else lambda: c) if d else e # parens are requiredA comprehension constructs new list or dictionary value by looping over one or more iterables and evaluating a body expression that produces successive elements of the result.
A list comprehension consists of a single expression followed by one or more clauses, the first of which must be a for clause. Each for clause resembles a for statement, and specifies an iterable operand and a set of variables to be assigned by successive values of the iterable. An if clause resembles an if statement, and specifies a condition that must be met for the body expression to be evaluated. A sequence of for and if clauses acts like a nested sequence of for and if statements.
ListComp = '[' Expression {CompClause} ']'.
DictComp = '{' Entry {CompClause} '}' .
CompClause = 'for' LoopVariables 'in' Expression
| 'if' Expression .
LoopVariables = PrimaryExpr {',' PrimaryExpr} .Examples:
[x*x for x in range(5)] # [0, 1, 4, 9, 16]
[x*x for x in range(5) if x%2 == 0] # [0, 4, 16]
[(x, y) for x in range(5)
if x%2 == 0
for y in range(5)
if y > x] # [(0, 1), (0, 2), (0, 3), (0, 4), (2, 3), (2, 4)]A dict comprehension resembles a list comprehension, but its body is a pair of expressions, key: value, separated by a colon, and its result is a dictionary containing the key/value pairs for which the body expression was evaluated. Evaluation fails if the value of any key is unhashable.
As with a for statement, the loop variables may exploit compound assignment:
[x*y+z for (x, y), z in [((2, 3), 5), (("o", 2), "!")]] # [11, 'oo!']TSL, following Python 3, does not accept an unparenthesized tuple as the operand of a for clause:
[x*x for x in 1, 2, 3] # parse error: unexpected commaComprehensions in TSL, again following Python 3, define a new lexical block, so assignments to loop variables have no effect on variables of the same name in an enclosing block:
x = 1
_ = [x for x in [2]] # new variable x is local to the comprehension
print(x) # 1CallSuffix = '(' [Arguments [',']] ')' .
Arguments = Argument {',' Argument} .
Argument = Expression | identifier '=' Expression | '*' Expression | '**' Expression .A value f of type function may be called using the expression f(...). Applications may define additional types whose values may be called in the same way.
A method call such as filename.endswith(".sky") is the composition of two operations, m = filename.endswith and m(".sky"). The first, a dot operation, yields a bound method, a function value that pairs a receiver value (the filename string) with a choice of method (string.endswith).
Only built-in or application-defined types may have methods.
See Functions for an explanation of function parameter passing.
A dot expression x.f selects the attribute f (a field or method) of the value x.
Fields are possessed by none of the main TSL data types, but some application-defined types have them. Methods belong to the built-in types str, list, dict, and set and to many application-defined types.
DotSuffix = '.' identifier .A dot expression fails if the value does not have an attribute of the specified name.
Use the built-in function hasattr(x, "f") to ascertain whether a value has a specific attribute, or dir(x) to enumerate all its attributes. The getattr(x, "f") function can be used to select an attribute when the name "f" is not known statically.
A dot expression that selects a method typically appears within a call expression, as in these examples:
["able", "baker", "charlie"].index("baker") # 1
"banana".count("a") # 3
"banana".reverse() # error: string has no .reverse field or methodBut when not called immediately, the dot expression evaluates to a bound method, that is, a method coupled to a specific receiver value. A bound method can be called like an ordinary function, without a receiver argument:
f = "banana".count
f # <built-in method count of string value>
f("a") # 3
f("n") # 2A subscript expression a[x] retrieves a value identified by x from the value a.
SubscriptSuffix = '[' Expressions ']' .There are two main cases: indexing a sequence, and retrieving values of a mapping.
When a is a Sequence (such as a list or tuple), or another indexable object like str or bytes, the expression a[i] yields the ith element of a. Here, i must be an int value in the range -n ≤ i < n, where n is len(a); any other index results in an error.
A valid negative index i behaves like the non-negative index n+i, allowing for convenient indexing relative to the end of the sequence.
"abc"[0] # "a"
"abc"[1] # "b"
"abc"[-1] # "c"
("zero", "one", "two")[0] # "zero"
("zero", "one", "two")[1] # "one"
("zero", "one", "two")[-1] # "two"For a subscript expression m[k], where the value of m is a Mapping (most commonly a dict) or any other mapping-like object, the expression retrieves the value that m associates with k. It is an error if m contains no such key k.
A subscript expression appearing on the left side of an assignment causes the specified sequence element or mapping value to be updated:
a = list(range(10, 13)) # a == [10, 11, 12]
a[2] = 7 # a == [10, 11, 7]
coins = {} # coins == {}
coins["suzie b"] = 100 # coins == {"suzie b": 100}It is a dynamic error to attempt to update an element of an immutable type, such as a tuple or string, or a frozen value of a mutable type.
TSL does not have a del statement like Python. Deleting the value associated with an index or key requires calling a method on the containing object.
The slice expression acts as a sort of indexing subscript expression that retrieves a range of elements rather than a single element. It applies to types that can be indexed with an integer as above, such as Sequences, str, and bytes.
SubscriptSuffix = '[' [Expressions] [':' Expression [':' Expression]] ']' .A slice expression a[start:stop:stride] yields a copy of a containing exactly the values whose indices are in the specified range. Conceptually, the range begins at start, progresses in increments of stride, and ends before reaching stop, without ever going out-of-bounds.
Each of the start, stop, and stride operands may be None or omitted entirely; otherwise, they must evaluate to integers. If stride is omitted then the second colon may optionally also be omitted. (The first colon is syntactically required in order to distinguish from a subscript expression.)
If stride is omitted its value defaults to 1. It is an error to specify a stride of 0.
The effective start and stop indices are computed from the three operands as follows. Let n be the length of the sequence.
If the stride is positive: If the start operand was omitted, it defaults to -infinity. If the end operand was omitted, it defaults to +infinity. For either operand, if a negative value was supplied, n is added to it. The start and end values are then "clamped" to the nearest value in the range 0 to n, inclusive.
If the stride is negative: If the start operand was omitted, it defaults to +infinity. If the end operand was omitted, it defaults to -infinity. For either operand, if a negative value was supplied, n is added to it. The start and end values are then "clamped" to the nearest value in the range -1 to n-1, inclusive.
"abc"[1:] # "bc" (remove first element)
"abc"[:-1] # "ab" (remove last element)
"abc"[1:-1] # "b" (remove first and last element)
"banana"[1::2] # "aaa" (select alternate elements starting at index 1)
"banana"[4::-2] # "nnb" (select alternate elements in reverse, starting at index 4)Unlike Python, TSL does not allow a slice expression to be the target of an assignment, although it may appear as a subexpression in the target.
Slicing a tuple, string, or bytes may be more efficient than slicing a list because tuple, string, and bytes values are immutable, so the result of the operation can share the underlying representation of the original operand (when the stride is 1). By contrast, slicing a list requires the creation of a new list and copying of the necessary elements.
A lambda expression yields a new function value.
LambdaExpr = 'lambda' [LambdaParameters] ':' Expression .Lambda parameters use the same forms as def parameters except that they do not accept type annotations. See Type annotations.
Syntactically, a lambda expression consists of the keyword lambda, followed by a parameter list like that of a def statement but unparenthesized, then a colon :, and a single expression, the function body.
Example:
def map(f, arr):
return [f(x) for x in arr]
map(lambda x: 2*x, range(3)) # [2, 4, 6]As with functions created by a def statement, a lambda function captures the syntax of its body, the default values of any optional parameters, a reference to each free variable appearing in its body, and the global dictionary of the current module.
The name of a function created by a lambda expression is "lambda".
The two statements below are essentially equivalent, but the function created by the def statement is named twice and the function created by the lambda expression is named lambda.
def twice(x):
return x * 2
twice = lambda x: x * 2Statement = DefStmt | IfStmt | ForStmt | WhileStmt | TryStmt | SimpleStmt .
SimpleStmt = SmallStmt {';' SmallStmt} [';'] '\n' .
SmallStmt = ReturnStmt
| BreakStmt | ContinueStmt | PassStmt
| AssignStmt
| ExprStmt
| ImportStmt
| RaiseStmt
.A pass statement does nothing. Use a pass statement when the syntax requires a statement but no behavior is required, such as the body of a function that does nothing.
PassStmt = 'pass' .Example:
def noop():
pass
def list_to_dict(items):
# Convert list of tuples to dict
m = {}
for k, m[k] in items:
pass
return mAn assignment statement has the form lhs = rhs. It evaluates the expression on the right-hand side then assigns its value (or values) to the variable (or variables) on the left-hand side.
AssignStmt = Expressions '=' Expressions .The expression on the left-hand side is called a target. The simplest target is the name of a variable, but a target may also have the form of a subscript expression to update the element of a list or dictionary, or the form of a dot expression to update the field of an object. These forms may be nested.
k = 1
a[i] = v
o.f = ""
o.g(1)[2] = NoneCompound targets may consist of a comma-separated list of subtargets, optionally surrounded by parentheses or square brackets. These too may be nested. An assignment to a compound target checks that the right-hand value is a sequence with the same number of elements as the target. Each element of the sequence is then assigned to the corresponding element of the target, recursively applying the same logic.
a, b = 2, 3
(x, y) = f()
[zero, one, two] = range(3)
[] = ()The same process for assigning a value to a target expression is used in for statements and in comprehension for clauses.
A simple variable assignment may also carry an optional type annotation:
AnnotatedAssignStmt = identifier ':' Type ['=' Expressions] .age: int = 30
names: list[str] = ["alice", "bob"]
flag: bool # bare annotation, no assignmentThe type annotation is parsed but otherwise has no runtime effect; the right-hand side, when present, is assigned to the target exactly as in a regular assignment statement. The target of an annotated assignment must be a single identifier - compound targets are not supported in this form. See Type annotations for the grammar of Type.
An augmented assignment, which has the form lhs op= rhs updates the variable lhs by applying a binary arithmetic operator op (one of +, -, *, /, //, %, &, |, ^, <<, >>) to the previous value of lhs and the value of rhs.
AssignStmt = Expressions ('=' | '+=' | '-=' | '*=' | '/=' | '//=' | '%=' | '&=' | '|=' | '^=' | '<<=' | '>>=') Expressions .The left-hand side must be a simple target: a name, a subscript expression, or a dot expression.
x -= 1
x.filename += ".sky"
a[index()] *= 2Any subexpressions in the target on the left-hand side are evaluated exactly once, before the evaluation of rhs. The first two assignments above are thus equivalent to:
x = x - 1
x.filename = x.filename + ".sky"and the third assignment is similar in effect to the following two statements but does not declare a new temporary variable i:
i = index()
a[i] = a[i] * 2A def statement creates a named function and assigns it to a variable.
DefStmt = 'def' identifier '(' [Parameters [',']] ')' ['->' Type] ':' Suite .A function's parameters and return value may carry optional type annotations. See Type annotations for details.
Example:
def twice(x):
return x * 2
str(twice) # "<function f>"
twice(2) # 4
twice("two") # "twotwo"The function's name is preceded by the def keyword and followed by the parameter list (which is enclosed in parentheses), a colon, and then an indented block of statements which form the body of the function.
The parameter list is a comma-separated list whose elements are of several kinds. First come zero or more required parameters, which are simple identifiers; all calls must provide an argument value for these parameters.
The required parameters are followed by zero or more optional parameters, of the form name=expression. The expression specifies the default value for the parameter for use in calls that do not provide an argument value for it.
The required parameters are optionally followed by a single parameter name preceded by a *. This is the called the varargs parameter, and it accumulates surplus positional arguments specified by a call. It is conventionally named *args.
The varargs parameter may be followed by zero or more parameters, again of the forms name or name=expression, but these parameters differ from earlier ones in that they are keyword-only: if a call provides their values, it must do so as keyword arguments, not positional ones.
Note that even though keyword-only arguments are declared after *args in a function's definition, they nevertheless must appear before *args in a call to the function.
def g(a, *args, b=2, c):
print(a, b, c, args)
g(1, 3) # error: function g missing 1 argument (c)
g(1, *[4, 5], c=3) # error: keyword argument c may not follow *args
g(1, 4, c=3) # "1 2 3 (4,)"
g(1, c=3, *[4, 5]) # "1 2 3 (4, 5)"A non-variadic function may also declare keyword-only parameters, by using a bare * in place of the *args parameter. This form does not declare a parameter but marks the boundary between the earlier parameters and the keyword-only parameters. This form must be followed by at least one keyword-only parameter.
def f(a, *, b=2, c):
print(a, b, c)
f(1) # error: function f missing 1 argument (c)
f(1, 3) # error: function f accepts 1 positional argument (2 given)
f(1, c=3) # "1 2 3"Finally, there may be an optional parameter name preceded by **. This is called the keyword arguments parameter, and accumulates in a dictionary any surplus name=value arguments that do not match a prior parameter. It is conventionally named **kwargs.
Here are some example parameter lists:
def f(): pass
def f(a, b, c): pass
def f(a, b, c=1): pass
def f(a, b, c=1, *args): pass
def f(a, b, c=1, *args, **kwargs): pass
def f(**kwargs): pass
def f(a, b, c=1, *, d=1): passExecution of a def statement creates a new function object. The function object contains: the syntax of the function body; the default value for each optional parameter; a reference to each free variable appearing within the function body; and the global dictionary of the current module.
def f(x):
res = []
def get_x():
res.append(x)
get_x()
x = 2
get_x()
f(1) # returns [1, 2]TSL accepts optional type annotations on function parameters, function return types, and variable assignments. Annotations are parsed but otherwise have no runtime effect today. They serve as documentation and a foothold for future linting.
Parameter = identifier
| identifier '=' Expression
| identifier ':' Type
| identifier ':' Type '=' Expression
| '*'
| '*' identifier
| '**' identifier
.
Type = TypeAtom
| Type '|' TypeAtom
.
TypeAtom = identifier
| TypeAtom '.' identifier
| TypeAtom '[' TypeArgs ']'
.
TypeArgs = Type {',' Type} .def greet(name: str, count: int = 1) -> str:
return name * countThe type sublanguage is intentionally narrower than Expression. Permitted forms are:
- Bare identifiers (
int,str,None,MyType) - Dotted names referring to a member of an imported module (
mod.Foo) - Index expressions (
list[int],dict[str, int]) - Unions with
|(int | str,int | str | None)
Arbitrary expressions are not allowed in type position: def f() -> 1 + 1: and def f(x: foo()): are syntax errors.
Identifiers used in type annotations are not required to resolve to anything in scope. Writing x: NotDefined is legal even when NotDefined has no binding. This lets users adopt a project's type vocabulary incrementally without import churn.
Lambda parameters do not accept type annotations: lambda x: int: x is a syntax error (the second : would be ambiguous with the lambda body separator).
A return statement ends the execution of a function and returns a value to the caller of the function.
ReturnStmt = 'return' [Expressions] .A return statement may have zero, one, or more result expressions separated by commas. With no expressions, the function has the result None. With a single expression, the function's result is the value of that expression. With multiple expressions, the function's result is a tuple.
return # returns None
return 1 # returns 1
return 1, 2 # returns (1, 2)An expression statement evaluates an expression and discards its result.
ExprStmt = Expressions .Any expression may be used as a statement, but an expression statement is most often used to call a function for its side effects.
my_list.append(1)An if statement evaluates an expression (the condition), then, if the truth value of the condition is True, executes a list of statements.
IfStmt = 'if' Expression ':' Suite {'elif' Expression ':' Suite} ['else' ':' Suite] .Example:
if score >= 100:
print("You win!")
returnAn if statement may have an else block defining a second list of statements to be executed if the condition is false.
if score >= 100:
print("You win!")
return
else:
print("Keep trying...")
continueIt is common for the else block to contain another if statement. To avoid increasing the nesting depth unnecessarily, the else and following if may be combined as elif:
if x > 0:
result = 1
elif x < 0:
result = -1
else:
result = 0An if statement is permitted at the top level of a module, in the global scope, as well as inside function definitions.
A for statement evaluates its operand, which must be an iterable value. Then, for each element of the iterable, the loop assigns the successive element values to one or more variables and executes a list of statements, the loop body.
ForStmt = 'for' LoopVariables 'in' Expressions ':' Suite .Example:
for x in range(10):
print(10)The assignment of each value to the loop variables follows the same rules as an ordinary assignment. In this example, two-element lists are repeatedly assigned to the pair of variables (a, i):
for a, i in [["a", 1], ["b", 2], ["c", 3]]:
print(a, i) # prints "a 1", "b 2", "c 3"TSL does not guarantee that iterable types are finite. It is up to the runtime implementation to impose bounds on any scripts run, either through iteration bounds or through time bounds.
Within the body of a for loop, break and continue statements may be used to stop the execution of the loop or advance to the next iteration.
A for statement is permitted at the top level of a module as well as inside function definitions.
The break and continue statements terminate the current iteration of a for or while loop. Whereas the continue statement resumes the loop at the next iteration, a break statement terminates the entire loop.
BreakStmt = 'break' .
ContinueStmt = 'continue' .Example:
for x in range(10):
if x%2 == 1:
continue # skip odd numbers
if x > 7:
break # stop at 8
print(x) # prints "0", "2", "4", "6"Both statements affect only the innermost lexically enclosing for or while loop. It is a static error to use a break or continue statement outside a loop.
A while statement executes its body repeatedly as long as the condition expression evaluates to a true value.
WhileStmt = 'while' Expression ':' Suite .Example:
x = 0
while x < 5:
print(x)
x += 1 # prints "0", "1", "2", "3", "4"Within the body of a while loop, break and continue statements may be used to stop the execution of the loop or advance to the next iteration.
Unlike for loops, while loops are not guaranteed to terminate. It is the programmer's responsibility to ensure that the loop condition eventually becomes false, or that the loop body executes a break statement.
A while statement is permitted at the top level of a module as well as inside function definitions.
The import statement loads another TSL module and binds it to a name in the current module. All symbols from the imported module must be accessed through the module name, although the module name may be aliased.
ImportStmt = 'import' (identifier | string) ['as' identifier] .An import statement names a single module to load. An optional as clause provides an alias for the module name in the current scope.
import os # binds the module to the name "os"
import fs as filesystem # binds the module to the name "filesystem"Unlike Python, TSL does not support from X import Y syntax. You must import a whole module and always qualify symbol references through the module name (e.g., os.stat(path)). You also cannot import multiple modules on a single line; each import must appear as its own statement.
During execution, the application determines what action to take for an import statement. A typical implementation locates and executes a TSL file, populating a cache of files executed so far to avoid duplicate work, to obtain a module, which is a mapping from global names to values.
An import statement within a function is a static error. An import statement can be within a globally-scoped if statement:
import os
if os.platform == os.PLATFORM_WINDOWS:
import windowsA try statement runs a block of code with one or more handlers for exceptions raised during execution. It is a Tanium extension to Starlark.
TryStmt = 'try' ':' Suite ExceptClause {ExceptClause} ['finally' ':' Suite]
| 'try' ':' Suite 'finally' ':' Suite .
ExceptClause = 'except' [Expression ['as' identifier]] ':' Suite .A try statement evaluates the body. If the body raises an exception, the exception is matched in order against each except clause; the first matching clause's body is executed, and the exception is considered handled. If no clause matches, the exception continues propagating.
A bare except: (with no type expression) matches every exception. An except clause with a type expression matches only exceptions whose type is that type or one of its descendants in the exception hierarchy. The type expression may also be a tuple of exception types; the clause matches if the exception is an instance of any of them.
try:
rows = tanium.query_sql(sql)
except IOError:
rows = []
except (KeyError, ValueError):
rows = []
except:
log.error("unexpected failure")
rows = []Unlike Python, TSL has no else clause on try statements. Code that should run only after a successful try body can simply follow the try statement (when no other paths exist) or be guarded by an explicit success flag.
The optional finally clause runs unconditionally - after a successful body, after a handled exception, and even when an unhandled exception is propagating. A try statement may pair a body with finally: alone, with no except clauses.
f = fs.open(path)
try:
process(f)
finally:
f.close()If finally itself raises, the new exception replaces any in-flight exception. A return, break, or continue inside finally discards any in-flight exception or pending control transfer from the try or except body. These behaviors mirror Python.
It is a static error to use continue inside a finally block.
try, except, and finally bodies do not introduce a new lexical scope. Variables assigned anywhere inside a try statement are visible in subsequent except clauses, in the finally clause, and after the try statement ends, just as they would be for if or for bodies.
A consequence: a variable assigned only inside the try body may be unbound when the finally clause runs, because the exception could have fired before the assignment completed. Reading an unbound variable is a runtime error. The typical idiom is to acquire the resource before entering the try, so the binding is unconditionally established by the time finally executes:
f = fs.open(path) # acquire before try
try:
process(f)
finally:
f.close() # f is always bound hereThe except T as name: binding is the one exception to this rule - it is scoped to the except clause body. See Exception binding.
An except T as name: clause binds the caught exception value to name for the duration of the clause body. The binding is removed when the clause exits (matching Python's behavior - the name does not survive the block). Referring to name outside the clause is a static error within the same function.
try:
parse(input)
except ValueError as e:
log.warn("parse failed: " + e.message)
# e is not in scope hereThe expression in an except clause is evaluated when the clause is reached. It must yield an exception type (or a tuple of exception types). If it does not, a TypeError is raised at the except site.
Because the expression is evaluated normally, module-exported exception types work without special syntax, including under aliasing imports:
import wmi as winmgmt
try:
rows = winmgmt.query(q)
except winmgmt.QueryError as e:
log.warn(e.message)Inside an except clause, a bare raise re-raises the currently active exception. Outside an except clause, bare raise is a static error. See Raise statements.
A raise statement signals an exception. It is a Tanium extension to Starlark.
RaiseStmt = 'raise' [Expression] .User code may only raise the base Exception type:
raise Exception("something went wrong")Specialized builtin types (KeyError, ValueError, etc.) and module-exported exception types are visible only as match targets in except clauses. They are not constructible from script code; attempting to call them raises a TypeError at the call site. This is a deliberate departure from Python and discourages exceptions-as-control-flow patterns: the language exposes specific types for handling stdlib errors, not for user-defined signaling.
A bare raise (no expression) re-raises the currently active exception. It is a static error to use bare raise outside an except clause.
try:
risky()
except IOError as e:
log.warn("retry: " + e.message)
raise # re-raise the IOErrorThe legacy fail(value) builtin remains and is equivalent to raise Exception(value).
TSL provides a fixed hierarchy of builtin exception types:
Exception
├── RuntimeError
├── ValueError
├── TypeError
├── KeyError
├── IndexError
├── LookupError
├── IOError
└── ArithmeticErrorException is the only builtin that is constructible from script code (via raise Exception("message") or as a callable: Exception("message")). All other names in the hierarchy are bound in the predeclared environment for the sole purpose of appearing in except clauses.
Native modules may export additional exception types via the host registration API; such types are documented as attributes of the module. Module-exported types are subtypes of Exception (or, where appropriate, of a more specific builtin) and are not constructible from script code. Scripts match on them just like builtin types:
import fs
try:
fs.list_dir(path)
except fs.NotFound:
log.info("missing directory; skipping")
except IOError as e:
log.warn(e.message)User-defined exception classes are not supported; there is no class statement in TSL.
A raised exception produces an exception value, distinct from the exception type. Exception values have the following attributes:
| Attribute | Description |
|---|---|
message | Human-readable description (string). |
args | Tuple of arguments passed at construction. |
traceback | Captured TSL call stack at the point the exception was raised. |
Exception values are non-freezable: they cannot be assigned to a module-level name that survives module evaluation, and therefore cannot be exported from a module. Attempting to freeze a module that retains a reference to an exception value is a runtime error reported at freeze time.
This is in contrast to exception types, which are ordinary frozen module attributes intended to be exported by native modules.
Each TSL file defines a module, which is a mapping from the names of global variables to their values. When a TSL file is executed, whether directly by the application or indirectly through an import statement, a new TSL thread is created, and this thread executes all the top-level statements in the file. TSL allows if, for, and while statements at the top level, so control flow at the module level may include branching and looping.
If execution reaches the end of the file, module initialization is successful. At that point, the value of each of the module's global variables is frozen, rendering subsequent mutation impossible. The module is then ready for use by another TSL thread, such as one executing an import statement. Such threads may access values or call functions defined in the loaded module.
A TSL thread may carry state on behalf of the application into which it is embedded, and application-defined functions may behave differently depending on this thread state. Because module initialization always occurs in a new thread, thread state is never carried from a higher-level module into a lower-level one. The initialization behavior of a module is thus independent of whichever module triggered its initialization.
If a TSL thread encounters an error, execution stops and the error is reported to the application, along with a backtrace showing the stack of active function calls at the time of the error. If an error occurs during initialization of a TSL module, any active import statements waiting for initialization of the module also fail.
TSL provides no mechanism by which errors can be handled within the language.
A script may terminate early by calling the built-in exit(code) function, where code is a required integer argument in the int32 range (-2147483648 to 2147483647). Calling exit immediately stops execution of the entire script, including unwinding through nested function calls, and sets the script's exit code to code.
A script that completes normally without calling exit has an implicit exit code of 0.
exit is a controlled termination, not an error. It cannot be intercepted by try/except statements or the catch() built-in. This distinguishes it from fail(), which raises a catchable error.
exit(0) # terminate successfully
exit(1) # terminate with failure code
def cleanup():
exit(42) # terminates the entire script, not just this function
catch(lambda: exit(1)) # script terminates with exit code 1; catch does not interceptThe exit code is visible to the caller of the script execution API. When a TSL script is executed as a package action, the exit code is reported in the package execution result and evaluated against the action's exit code criteria.
See the TSL Built-in API Reference for documentation of all built-in constants, functions, and type methods.
File = {Statement | newline} eof .
Statement = DefStmt | IfStmt | ForStmt | WhileStmt | TryStmt | SimpleStmt .
DefStmt = 'def' identifier '(' [Parameters [',']] ')' ['->' Type] ':' Suite .
Parameters = Parameter {',' Parameter}.
Parameter = identifier
| identifier '=' Expression
| identifier ':' Type
| identifier ':' Type '=' Expression
| '*'
| '*' identifier
| '**' identifier
.
Type = TypeAtom | Type '|' TypeAtom .
TypeAtom = identifier | TypeAtom '.' identifier | TypeAtom '[' TypeArgs ']' .
TypeArgs = Type {',' Type} .
IfStmt = 'if' Expression ':' Suite {'elif' Expression ':' Suite} ['else' ':' Suite] .
ForStmt = 'for' LoopVariables 'in' Expressions ':' Suite .
WhileStmt = 'while' Expression ':' Suite .
TryStmt = 'try' ':' Suite ExceptClause {ExceptClause} ['finally' ':' Suite]
| 'try' ':' Suite 'finally' ':' Suite .
ExceptClause = 'except' [Expression ['as' identifier]] ':' Suite .
Suite = [newline indent {Statement} outdent] | SimpleStmt .
SimpleStmt = SmallStmt {';' SmallStmt} [';'] '\n' .
# NOTE: '\n' optional at EOF
SmallStmt = ReturnStmt
| BreakStmt | ContinueStmt | PassStmt
| AssignStmt
| AnnotatedAssignStmt
| ExprStmt
| ImportStmt
| RaiseStmt
.
ReturnStmt = 'return' [Expressions] .
BreakStmt = 'break' .
ContinueStmt = 'continue' .
PassStmt = 'pass' .
AssignStmt = Expressions ('=' | '+=' | '-=' | '*=' | '/=' | '//=' | '%=' | '&=' | '|=' | '^=' | '<<=' | '>>=') Expressions .
AnnotatedAssignStmt = identifier ':' Type ['=' Expressions] .
ExprStmt = Expressions .
RaiseStmt = 'raise' [Expression] .
ImportStmt = 'import' ( identifier | string ) ['as' identifier] .
Expression = IfExpr | PrimaryExpr | UnaryExpr | BinaryExpr | LambdaExpr .
IfExpr = Expression 'if' Expression 'else' Expression .
PrimaryExpr = Operand
| PrimaryExpr DotSuffix
| PrimaryExpr CallSuffix
| PrimaryExpr SubscriptSuffix
.
Operand = identifier
| int | float | string | bytes
| ListExpr | ListComp
| DictExpr | DictComp
| '(' [Expressions [',']] ')'
.
DotSuffix = '.' identifier .
SubscriptSuffix = '[' [Expressions] [':' Expression [':' Expression]] ']'
| '[' Expressions ']'
.
CallSuffix = '(' [Arguments [',']] ')' .
Arguments = Argument {',' Argument} .
Argument = Expression | identifier '=' Expression | '*' Expression | '**' Expression .
ListExpr = '[' [Expressions [',']] ']' .
ListComp = '[' Expression {CompClause} ']'.
DictExpr = '{' [Entries [',']] '}' .
DictComp = '{' Entry {CompClause} '}' .
Entries = Entry {',' Entry} .
Entry = Expression ':' Expression .
CompClause = 'for' LoopVariables 'in' Expression | 'if' Expression .
UnaryExpr = '+' Expression
| '-' Expression
| '~' Expression
| 'not' Expression
.
BinaryExpr = Expression {Binop Expression} .
Binop = 'or'
| 'and'
| '==' | '!=' | '<' | '>' | '<=' | '>=' | 'in' | 'not' 'in'
| '|'
| '^'
| '&'
| '<<' | '>>'
| '-' | '+'
| '*' | '%' | '/' | '//'
.
LambdaExpr = 'lambda' [LambdaParameters] ':' Expression .
# LambdaParameters mirrors Parameters but does not permit type annotations.
LambdaParameters = LambdaParameter {',' LambdaParameter} .
LambdaParameter = identifier
| identifier '=' Expression
| '*'
| '*' identifier
| '**' identifier
.
Expressions = Expression {',' Expression} .
# NOTE: trailing comma permitted only when within [...] or (...).
LoopVariables = PrimaryExpr {',' PrimaryExpr} .Tokens:
- spaces: newline, eof, indent, outdent.
- identifier.
- literals: string, bytes, int, float.
- plus all quoted tokens such as '+=', 'return'.
Notes:
- Ambiguity is resolved using operator precedence.
- The grammar does not enforce the legal order of params and args, nor that the first CompClause must be a 'for'.