2.3 Condor Matchmaking with ClassAds

Before you start learning about how to submit a job, it is important to understand how Condor performs resource allocation. Understanding the unique framework by which Condor matches submitted jobs with machines is the key to getting the most from Condor's scheduling algorithm.

Condor is unlike most other batch systems, which typically involve submitting a job to one of several pre-defined job queues. These environments typically become both constrained and complicated, as system administrators scramble to add more queues in response to the variety of user needs. Likewise, the user is asked to make compromises and is left with the burden of not only keeping track of which queues have which properties, but also deciding which queue would be the optimal one to use for their jobs.

Instead, Condor simply acts as a matchmaker of ClassAds. Condor's ClassAds are analogous to the classified advertising section of the newspaper. Sellers advertise specifics about what they have to sell, hoping to attract a buyer. Buyers may advertise specifics about what they wish to purchase. Both buyers and sellers may have constraints that need to be satisfied. For instance, perhaps the buyer is not willing to spend more than X dollars, and the seller requires to receive a minimum of Y dollars. Furthermore, both want to rank requests from the other in such a fashion that is to their advantage. Certainly a seller would rank a buyer offering $50 dollars for a service higher than a different buyer offering $25 for the same service. In Condor, users submitting jobs can be thought of as buyers of compute resources and machine owners are the sellers.

All machines in the Condor pool advertise their attributes, such as available RAM memory, CPU type and speed, virtual memory size, current load average, and many other static and dynamic properties, in a machine ClassAd. The machine ClassAd also advertises under what conditions it is willing to run a Condor job and what type of job it would prefer. These policy attributes can reflect the individual terms and preferences by which all the different owners have graciously allowed their machine to be part of the Condor pool. For example, John Doe's machine may advertise that it is only willing to run jobs at night and when there is nobody typing at the keyboard. In addition, John Doe's machines may advertise a preference (rank) for running jobs submitted by either John Doe or one of his co-workers whenever possible.

Likewise, when submitting a job, you can specify many different job attributes, including whatever requirements and preferences for whatever type of machine you'd like this job to use. For instance, perhaps you're looking for the fastest floating-point machine available, i.e. you want to rank matches based upon floating-point performance. Or perhaps you only care that the machine has a minimum of 128 Meg of RAM. Or perhaps you'll just take any machine you can get! These job attributes and requirements are bundled up into a job ClassAd and ``published'' to Condor.

Condor then plays the role of a matchmaker by continuously reading through all of the job ClassAds and all of the machine ClassAds and matching and ranking job ads with machine ads, while making certain that all requirements in both ads are satisfied.

2.3.1 Inspecting Machine ClassAds with condor_status

Now would be a good time to try the condor_status command to get a feel for what a ClassAd actually looks like. condor_status displays summarizes and displays information from ClassAds about the resources available in your pool. If you just type condor_status and hit enter, you will see a summary of all the machine ClassAds similar to the following:

Name       Arch     OpSys        State      Activity   LoadAv Mem  ActvtyTime

adriana.cs INTEL    SOLARIS251   Claimed    Busy       1.000  64    0+01:10:00
alfred.cs. INTEL    SOLARIS251   Claimed    Busy       1.000  64    0+00:40:00
amul.cs.wi SUN4u    SOLARIS251   Owner      Idle       1.000  128   0+06:20:04
anfrom.cs. SUN4x    SOLARIS251   Claimed    Busy       1.000  32    0+05:16:22
anthrax.cs INTEL    SOLARIS251   Claimed    Busy       0.285  64    0+00:00:00
astro.cs.w INTEL    SOLARIS251   Claimed    Busy       0.949  64    0+05:30:00
aura.cs.wi SUN4u    SOLARIS251   Owner      Idle       1.043  128   0+14:40:15

...

condor_status can summarize machine ads in a wide variety of ways. For example, condor_status -available shows only machines which are willing to run jobs now, and condor_status -run shows only machines which are currently running jobs. condor_status can also display other types of ads other than just machine ads; condor_status -help lists all the options, and/or refer to the condor_status command reference page located on page .

To get a feel for what a typical machine ClassAd looks like in its entirety, use the condor_status -l command. Figure 2.1 shows the complete machine ad for workstation alfred.cs.wisc.edu. Some of these attributes are used by Condor itself for scheduling. Other attributes are simply informational. But the important point is that any of these attributes in the machine ad can be utilized at job submission time as part of a request or preference on what machine to use. Furthermore, additional attributes can be easily added; for example, perhaps your site administrator has added a physical location attribute to your machine ClassAds.

  

Figure 2.1: Sample output from condor_status -l alfred