Previously, an application was 'triaged' (not discussed at the committee meeting) if both internal reviewers rated the application 2.9 or below. This procedure resulted in a relatively low level of applications not being discussed (approx. 10%), while typically 70 - 75% of applications per competition have not been funded.
The new system focuses on an 'assessment of fundability' independent of the initial rating. Prior to the committee meeting, reviewers will be asked on ResearchNet to identify the applications they reviewed that they feel are the most competitive; i.e., that not only deserve to be funded but that have a good probability of being funded. As a guideline, reviewers should aim at placing approximately 50% of the applications they reviewed in this category, although this may be higher or lower depending on the overall quality of the pool of applications they reviewed. At the meeting, if an application is flagged as having a low funding probability (not considered to be in top 50% of applications by either reviewer), it does not have to be discussed as long as all committee members are in agreement, and the average of the initial ratings is <3.50 (i.e., not in the fundable range). If either of these conditions is not met, the application must be discussed, and committee members vote. If both conditions are met, the application is not discussed, no budget is recommended, and reviewers do not vote; the final rating is calculated as the mean of the internal reviewers' initial ratings. The new procedures are also outlined in section 5 of the CIHR Peer Review Process on the CIHR web site.
Extensive modeling of previous competitions indicates that the new system will result in up to 25 - 30% of applications being flagged as not requiring discussion. Note that for an application not to be discussed at a committee meeting, both reviewers must place the application in their bottom half, the average rating must be <3.5, and there must be agreement from the rest of the committee members. In keeping with current practice, applicants whose proposals are not discussed will receive written reviews from the two internal reviewers but no Scientific Officer notes, and their rating will be the average of the two internal reviewers' initial ratings.
These changes are part of SMIPR's ongoing efforts to ensure the highest quality peer review process. We welcome your comments and feedback on these changes or on any peer review issues.
Contact: Greg Huyer at ghuyer@cihr-irsc.gc.ca or (613) 954-1816.
