Next: 5. Command Reference Manual
Up: 4. Miscellaneous Concepts
Previous: 4. Miscellaneous Concepts
Subsections
4.1 An Introduction to Condor's ClassAd Mechanism
ClassAds are a flexible mechanism for representing the characteristics and
constraints of machines and jobs in the Condor system. ClassAds are used
extensively in the Condor system to represent jobs, resources, submitters
and other Condor daemons. An understanding of this mechanism is required
to harness the full flexibility of the Condor system.
A ClassAd is is a set of uniquely named expressions. Each named expression
is called an attribute. Figure 4.1 shows an example
of a ClassAd with ten attributes.
Figure 4.1:
An example ClassAd
 |
ClassAd expressions look very much like expressions in C, and are composed
of literals and attribute references composed with operators. The difference
between ClassAd expressions and C expressions arise from the fact that ClassAd
expressions operate in a much more dynamic environment. For example, an
expression from a machine's ClassAd may refer to an attribute in a job's
ClassAd, such as TARGET.Owner
in the above example. The value and type
of the attribute is not known until the expression is evaluated in an
environment which pairs a specific job ClassAd with the machine ClassAd.
ClassAd expressions handle these uncertainties by defining all operators
to be total operators, which means that they have well defined
behavior regardless of supplied operands. This functionality is provided
through two distinguished values, UNDEFINED and ERROR,
and defining all operators so that they can operate on all possible values
in the ClassAd system. For example, the multiplication operator which usually
only operates on numbers, has a well defined behavior if supplied with values
which are not meaningful to multiply. Thus, the expression
10 * "A string"
evaluates to the value ERROR. Most operators
are strict with respect to ERROR, which means that they evaluate
to ERROR if any of their operands are ERROR. Similarly,
most operators are strict with respect to UNDEFINED.
ClassAd expressions are formed by composing literals, attribute references and
other sub-expressions with operators.
Literals in the ClassAd language may be of integer, real, string, undefined or
error types. The syntax of these literals is as follows:
- Integer
- A sequence of continuous digits (i.e.,
[0-9]
).
Additionally, the keywords TRUE
and FALSE
(case
insensitive) are syntactic representations of the integers 1 and 0
respectively.
- Real
- Two sequences of continuous digits separated by a period
(i.e.,
[0-9]+.[0-9]+
).
- String
- A double quote character, followed by an list of characters
terminated by a double quote character. A backslash character inside
the string causes the following character to be considered as part of
the string, irrespective of what that character is.
- Undefined
- The keyword UNDEFINED (case insensitive)
represents the UNDEFINED value.
- Error
- The keyword ERROR (case insensitive)
represents the ERROR value.
Every expression in a ClassAd is named by an attribute name. Together,
the (name,expression) pair is called an attribute. An attributes may be
referred to in other expressions through its attribute name.
Attribute names are sequences of alphabetic characters, digits and underscores,
and may not begin with a digit. All characters in the name are significant,
but case is not significant. Thus, Memory
, memory
and
MeMoRy
all refer to the same attribute.
An attribute reference consists of the name of the attribute being
referenced, and an optional scope resolution prefix. The three
prefixes that may be used are MY., TARGET. and ENV..
The semantics of supplying a prefix are discussed in
Section 4.1.2.
The operators that may be used in ClassAd expressions are similar to those
available in C. The available operators and their relative precedence is shown
in figure 4.2.
Figure 4.2:
Relative precedence of ClassAd expression
operators
 |
The operator with the highest precedence is the unary minus operator. The
only operators which are unfamiliar are the =?=
and =!=
operators, which are discussed in Section 4.1.2.
4.1.2 Evaluation Semantics
The ClassAd mechanism's primary purpose is for matching entities who supply
constraints on candidate matches. The mechanism is therefore defined to
carry out expression evaluations in the context of two ClassAds which are
testing each other for a potential match. For example, the condor_negotiator
evaluates the Requirements expressions of machine and job ClassAds to
test if they can be matched. The semantics of evaluating such constraints
is defined below.
Literals are self-evaluating, Thus, integer, string, real, undefined and
error values evaluate to themselves.
Since the expression evaluation is being carried out in the context of two
ClassAds, there is a potential for namespace ambiguities. The following
rules define the semantics of attribute references made by ad A that is being
evaluated in a context with another ad B:
- 1.
- If the reference is prefixed by a scope resolution prefix,
- If the prefix is MY., the attribute is looked up in
ClassAd A. If the named attribute does not exist in A, the
value of the reference is UNDEFINED. Otherwise, the
value of the reference is the value of the expression bound to
the attribute name.
- Similarly, if the prefix is TARGET., the attribute is
looked up in ClassAd B. If the named attribute does not exist in
B, the value of the reference is UNDEFINED. Otherwise,
the value of the reference is the value of the expression bound to
the attribute name.
- Finally, if the prefix is ENV., the attribute is
evaluated in the ``environment.'' Currently, the only attribute
of the environment is CurrentTime, which evaluates to the
integer value returned by the system call time(2).
- 2.
- If the reference is not prefixed by a scope resolution prefix,
- If the attribute is defined in A, the value of the reference
is the value of the expression bound to the attribute name in A.
- Otherwise, if the attribute is defined in B, the value of the
reference is the value of the expression bound to the attribute
name in B.
- Otherwise, if the attribute is defined in the environment, the
value of the reference is the evaluated value in the environment.
- Otherwise, the value of the reference is UNDEFINED.
- 3.
- Finally, if the reference refers to an expression that is itself in
the process of being evaluated, there is a circular dependency in the
evaluation. The value of the reference is ERROR.
4.1.2.3 Operators
All operators in the ClassAd language are total, and thus have well
defined behavior regardless of the supplied operands. Furthermore, most
operators are strict with respect to ERROR and
UNDEFINED, and thus evaluate to ERROR (or UNDEFINED)
if either of their operands have these exceptional values.
- Arithmetic operators:
- 1.
- The operators
*
, /
, +
and -
operate
arithmetically only on integers and reals.
- 2.
- Arithmetic is carried out in the same type as both operands,
and type promotions from integers to reals are performed if one operand
is an integer and the other real.
- 3.
- The operators are strict with respect to both UNDEFINED
and ERROR.
- 4.
- If either operand is not a numerical type, the value of the
operation is ERROR.
- Comparison operators:
- 1.
- The comparison operators
==
, !=
, <=
,
<
, >=
and >
operate on integers, reals and strings.
- 2.
- Comparisons are carried out in the same type as both operands,
and type promotions from integers to reals are performed if one operand
is a real, and the other an integer. Strings may not be converted to
any other type, so comparing a string and an integer results in
ERROR.
- 3.
- The operators
==
, !=
, <=
, <
and
>=
>
are strict with respect to both UNDEFINED
and ERROR.
- 4.
- In addition, the operators
=?=
and =!=
behave
similar to ==
and !=
, but are not strict. Semantically,
the =?=
tests if its operands are ``identical,'' i.e., have
the same type and the same value. For example, 10 == UNDEFINED
and UNDEFINED == UNDEFINED
both evaluate to UNDEFINED,
but 10 =?= UNDEFINED
and UNDEFINED =?= UNDEFINED
evaluate to FALSE and TRUE respectively. The
=!=
operator test for the ``is not identical to'' condition.
- Logical operators:
- 1.
- The logical operators
&&
and ||
operate on
integers and reals. The zero value of these types are considered
FALSE and non-zero values TRUE.
- 2.
- The operators are not strict, and exploit the
``don't care'' properties of the operators to squash UNDEFINED
and ERROR values when possible. For example,
UNDEFINED && FALSE
evaluates to FALSE, but
UNDEFINED || FALSE
evaluates to UNDEFINED.
- 3.
- Any string operand is equivalent to an ERROR operand.
The simplicity and flexibility of ClassAds is heavily exploited in the Condor
system. ClassAds are not only used to represent machines and jobs in the
Condor pool, but also other entities that exist in the pool such as
checkpoint servers, submitters of jobs and master daemons. Since arbitrary
expressions may be supplied and evaluated over these ads, users have a uniform
and powerful mechanism to specify constraints over these ads. These constraints
may take the form of Requirements expressions in resource and job ads,
or queries over other ads.
This is the mechanism by which users specify the constraints over machines
and jobs respectively. Requirements for machines are specified through
configuration files, while requirements for jobs are specified through the
submit command file.
In both cases, the Requirements expression specifies the correctness
criterion that the match must meet, and the Rank expression specifies
the desirability of the match (where higher numbers mean better matches).
For example, a job ad may contain the following expressions:
Requirements = Arch=="SUN4u" && OpSys == "SOLARIS251"
Rank = TARGET.Memory + TARGET.Mips
In this case, the customer requires an UltraSparc computer running the Solaris
2.5.1 operating system. Among all such computers, the customer prefers those
with large physical memories and high MIPS ratings. Since the Rank is
a user specified metric, any expression may be used to specify the
perceived desirability of the match. The condor_negotiator runs algorithms
to deliver the ``best'' resource (as defined by the Rank expression)
while satisfying other criteria.
Similarly, owners of resources may place constraints and preferences on
their machines. For example,
Friend = Owner == "tannenba" || Owner == "wright"
ResearchGroup = Owner == "jbasney" || Owner == "raman"
Trusted = Owner != "rival" && Owner != "riffraff"
Requirements = Trusted && ( ResearchGroup || LoadAvg < 0.3 &&
KeyboardIdle > 15*60 )
Rank = Friend + ResearchGroup*10
The above policy states that the computer will never run jobs owned by
users ``rival'' and ``riffraff,'' while the computer will always run a
job submitted by members of the research group. Furthermore, jobs
submitted by friends are preferred to other foreign jobs, and jobs submitted
by the research group are preferred to jobs submitted by friends.
Note: Because of the dynamic nature of ClassAd expressions, there
is no a priori notion of an integer valued expression, a real valued
expression, etc. However, it is intuitive to think of the Requirements
and Rank expressions as integer valued and real valued expressions
respectively. If the actual type of the expression is not of the expected
type, the value is assumed to be zero.
The flexibility of this system may also be used when querying ClassAds
through the condor_status and condor_q tools which allow users to
supply ClassAd constraint expressions from the command line.
For example, to find all computers which have had their keyboards idle for
more than 20 minutes and have more than 100 MB of memory:
% condor_status -const 'KeyboardIdle > 20*60 && Memory > 100'
Name Arch OpSys State Activity LoadAv Mem ActvtyTime
amul.cs.wi SUN4u SOLARIS251 Claimed Busy 1.000 128 0+03:45:01
aura.cs.wi SUN4u SOLARIS251 Claimed Busy 1.000 128 0+00:15:01
balder.cs. INTEL SOLARIS251 Claimed Busy 1.000 1024 0+01:05:00
beatrice.c INTEL SOLARIS251 Claimed Busy 1.000 128 0+01:30:02
...
...
Machines Owner Claimed Unclaimed Matched Preempting
SUN4u/SOLARIS251 3 0 3 0 0 0
INTEL/SOLARIS251 21 0 21 0 0 0
SUN4x/SOLARIS251 3 0 3 0 0 0
SGI/IRIX6 1 0 0 1 0 0
INTEL/LINUX 1 0 1 0 0 0
Total 29 0 28 1 0 0
The similar flexibility exists in querying job queues in the Condor system.
Next: 5. Command Reference Manual
Up: 4. Miscellaneous Concepts
Previous: 4. Miscellaneous Concepts
condor-admin@cs.wisc.edu