Guidelines for the Review and Evaluation of Annual Meeting Program Units

For the sake of transparency, accountability, and to facilitate efficient and accurate record-keeping, the Program Committee has adopted these guidelines for its Program Unit review process. These guidelines are provided to units and all reviewers prior to the review process so that assessment criteria are uniformly accessible.

Rationale: The review and evaluation process represents the chief, though not the only, means by which the Academy assesses the work and functionality of its constituent Annual Meeting program units. It is also intended to serve as a way of ensuring that the Academy is responsive to important changes and developments in the academic study of religion and thus remains representative of the interests and concerns of its members.

Assumptions: The review of program units is undertaken for the purpose of determining which units shall be continued. Beyond continuation considerations, the Program Committee uses these reviews as a primary mechanism for allocating the limited number of programming slots at the Annual Meeting. The Program Committee wishes to emphasize that competition for program slots has become increasingly intense in recent years. For renewal, there needs to be a compelling argument for continuation, a healthy procedural structure, and persuasive evidence of conceptually rigorous plans for another term. Seminars are nonrenewable.

Procedures: The review is based on at least three forms of evidence, both qualitative and quantitative:

• The program unit chair’s annual reports.
• A self-review stating the aims of the unit, its procedures, its programming accomplishments, and a rationale for the unit’s continued existence (due in the executive office and to the unit’s reviewer by October 15th of the review year).
• Quantitative data, such as the number of proposals a unit receives, the number of proposals it accepted or rejected, the number of members who attended their sessions, and the number of sessions it sponsored or cosponsored. Some of this data is derived from the PAPERS System, but also from the Annual Reports.

Self-Review: Among the criteria deemed relevant to the self-review, include:

• The extent to which the field of interest represented by the unit continues to reflect a major area of interest and work for a significant portion of the Academy's membership;  one possible metric by which this may be measured is the number and quality of proposals the unit receives.
• The intellectual rigor, imagination, conceptual richness, and distinction of the work carried on by the unit, whether through the presentation of papers, the sponsorship of discussions, or the publication of proceedings.
• The degree of commitment that the unit's constituency exhibits to the ongoing life of the unit; one possible metric by which this may be measured is the attendance at the unit’s Annual Meeting sessions.
• The procedural health of the unit, including leadership practices, such as mechanisms for selecting new chairs and steering committee members; communication within the unit, between units, and with the AAR; and the unit’s proposal review process and other decision making practices.
• The extent to which the unit's constituency has been afforded an opportunity to participate in the sessions (with attention to diversity of seniority, race, ethnicity, nationality, and gender among participants).
• The unit's range of appeal to those members of the Academy whose own fields of specialization do not typically fall within the field of interest represented by the unit and the unit's ability to involve such people periodically in its programs; one possible metric by which this may be measured is the number of sessions which the unit cosponsors with other Program Units.
• Over the course of the next five years, the promise the unit offers for advancing the academic study of religion, or the relation of that study to other disciplines.
• Additional considerations expressed by the Program Unit leadership, the External Reviewer, and/or the Program Committee’s Lead Reader.

External Reviewers: If a Unit requests a change of status, additional sessions, requests an external review, or if the Program Committee, after its examination of the Program Unit’s self-review, decides to follow-up with an external review, a reviewer will be selected by the Program Committee. Efforts are made to locate a member who has expertise in the field and who is able to play the role of participant-observer in the unit’s review.

The reviewer’s written report, is based on (1) attending as many sessions of the unit as possible during the Annual Meeting, including the unit’s business meeting(s); (2) personal interviews with the unit chairs, members of the steering committee and a cross-section of participants at Annual Meetings both current and previous (if that can be arranged); and (3) the unit’s written self-review.

Program Unit Chair’s Responsibilities during External Review: The following is a list of program unit chair responsibilities to guide you as you prepare your unit’s proposal for an external review. Program unit chairs should:

• Cooperate fully with the reviewer in supplying all requested information in a timely fashion.
• Assist the reviewer in arranging a mutually acceptable time or times at the Annual Meeting to meet with the steering committee of the unit.
• Prepare any supplemental material the unit wishes to submit to the Program Committee in light of the reviewer’s report.
• Indicate where you may be reached by telephone by the Program Committee during its meeting (held in early January) in the event there is need to do so.
• Submit your self-review to the Reviewer and Director of Meetings. This should review the history and activities of the unit since it was last authorized, present a plan for the future work of the unit if it is continued, and indicate the contribution of the work done under the unit’s auspices to advancing the academic study of religion.
• Indicate committed leadership for the future of the unit, and discuss mechanisms for selecting new chairs and steering committee members.

Previewing the Reviewer’s Report: The reviewer will arrange to meet with the chairs and steering committee of the unit under review near the conclusion of the Annual Meeting and will indicate to them the substance of the report that will be made to the Program Committee.

The chairs and steering committee of the unit may choose to submit additional materials responsive to the reviewer’s evaluation of the unit. The supplement will be due at the same time as the Program Unit Report, but should be submitted separately as an e-mail or e-mail attachment.

The Program Committee Meeting: The Program Committee considers all review reports and related documents in its meeting. The Committee must consider the case for renewal or change of status in relation to a range of other considerations.

To navigate the enormous mass of material and to introduce/inform the Program Committee’s deliberation, it is the Committee’s custom to assign a lead reader for each unit’s review. The Lead Reader’s comments and recommendations, along with the rest of the Committee’s considerations and concerns, will be recorded and summarized by the official notetaker to give guidance to the Program Unit. This summary, along with the Program Committee’s decision, will be given to the Program Unit Chairs after the meeting.

The decisions of the committee regarding the future of a program unit are final, but no unit will be discontinued without an external review. Should the Committee vote to have a unit re-reviewed with an external reviewer, the unit will be continued on a one-year basis. The subsequent deliberation of the Committee the following year will involve at least two new members due to its rotational schedule, thus changing the makeup of the deliberative body.