Changes Ahead for Allocations

By Herbert Morgan, NCSA

This winter, the new partnership between NCSA and SDSC will bring changes in the way computational resources are allocated.

Beginning with the December board meetings, the Alliance Allocations Board (AAB), run by NCSA, and the Partnership Resource Allocations Committee (PRAC), run by SDSC, will be combined into a medium-level allocations board, yet to be named. Users will just go to one place instead of SDSC or NCSA. This newly formed board will accept requests for service units (SUs) that fall between 10,000 (30,000 for TeraGrid) and 200,000 SUs.

The medium-level allocations board will continue to meet quarterly: March, June, September, and December. The National Resource Allocations Committee (NRAC)—which is the large-level allocations board—will be renamed, as well. It will continue to meet twice a year: March and December.

How Proposals become Awards

Currently, proposals are submitted through the PACI Online Proposal System (POPS). John Towns, Persistent Infrastructure Director, and Melissa Johnson, Allocations Manager, read through them and assign the reviewers. Johnson notifies the selected reviewers who then go to the review section of the online system where they find their assignments. All reviews have to be completed and submitted approximately a week before the board meeting.

On the night before the meeting, reviewers caucus and come to an agreement on the number of SUs per proposal. The next day at the meeting, one reviewer is selected to represent a proposal, to state its agreed upon award, and to summarize the rationale for how the award was decided.

Johnson writes an award letter to the principal investigator (PI) based on the meeting notes, explaining the results. If the PI did not receive the full amount of SUs requested, he/she can send in a justification. The award letter contains instructions on that procedure.

Often there are not enough SUs to meet the requests of all the proposals. So reviewers also evaluate awards based on the amount of SUs actually available on a machine. If a PI doesn't get the full award, it might mean that not enough SUs were available on their requested machine. Some machines are more popular than others. Typically though, a less-than-full award is due to insufficient justification for the requested SUs.

About Board Reviewers

A reviewer is picked for a three-year term on the board, which is an NSF recommendation. The scientific field represented by board membership fluctuates based on the proposals' subject matter. "For a number of years," says Johnson, "we received many chemistry proposals; so, proportionately, we had many chemists who served as reviewers." Johnson strives to strike a balance so that there are not too many reviewers who never get a proposal or some who get too many.

The former PRAC and the former AAB each received about 30 proposals per meeting. These amounts mean that, combined, the new medium-level board could probably receive about 60 proposals every quarter. To handle this work, Johnson would like to increase the size of the review board.

Each reviewer is responsible for three to ten proposals—three being the average. Ten is a very heavy load because a proposal can be 15 pages plus attached addenda. The amount of requested SUs can also be a factor. In the past, a number of PIs have requested more than three million SUs. Johnson tries to assign five reviewers to NRAC proposals that large.

The Allocations boards have strict, well-defined conflict-of-interest regulations. Reviewers may also be PIs submitting proposals. If so, they receive booklets printed especially for them that do not contain the reviews for their proposals. They are not allowed in the room when their proposal is being discussed and reviewed. Conflict-of-interest rules may also apply to reviewers who collaborate on submitted proposals or who are members of the same institution as the PI.

All reviewers are invited to the board meeting. If they cannot attend, they must submit their review via POPS so that everyone can evaluate it. Reviewers who attend the meetings receive food and lodging, but are not compensated for being on the board.