What is an NSF Site Visit and why is it important?

Site Visit Best Practice

Prepared by Jamie Burns, Quyen Wickham, and Faye Farmer

This document provides best practices to assist Arizona State University (ASU) teams in their R/SV preparation. Information has been sourced from the following resources: (1) an aggregation of nine responses to a National Organization of Research Development Professionals (NORDP) listserv question regarding pre-award National Science Foundation (NSF) RSVs, (2) a 2013 NORDP Presentation on NSF Science and Technology Center (STC) SV best practices, (3) a blog post from Faye Farmer, Executive Director of ASU Research Development (RD), who has participated in successful R/SVs, and (4) interviews with NSF Artificial Intelligence (AI) Institute awardees regarding their R/SV strategies.

Download the PDF HERE.

What is an R/SV?

Large, center-level, cooperative awards from the NSF often include a post-submission set of events that allow for greater interaction between program representatives and candidate teams. These are called site visits when the NSF visits either virtually or in person the lead institution and reverse site visits when the candidate team, usually a select few of the leaders, visits NSF. The site visit precedes the reverse site visit, with usually months between each. A reverse site visit is not always included after a site visit. There is often extensive coordination of information between the SV and RSV, and subsequent to the RSV. This communication is usually confidential.

What is the goal of an R/SV?

The purpose of the R/SV is to convince the review team, comprised of external NSF selected reviewers and internal program representatives from NSF, that the project should be funded. Key to this convincing is ensuring reviewers understand the project, are excited about the project, understand the impact and transformative nature of the project, believe the project can be accomplished, and believe this is the right team to tackle the project.

Who does NSF want to hear from?

There are typically a specific number of participants NSF wants to hear from. The list is generally longer for the SV and much more select for the RSV. This will include the PIs/PDs as well as additional team members and support personnel, which could be as little as four people or as many as 20 for very large projects. In these presentations, the PI needs to show they are a strong/competent leader, and the full team must appear to be cohesive. The NSF generally provides an agenda or states an expectation for the presentation order and/or content.

How long is an R/SV?

Site visit presentations are usually a full day, while reverse site visits can be four hours to a full day. NSF typically provides a structure for the presentations with detailed and specific questions. This content is usually tied to the reviews provided in the summary statement. The structure may include time allotments to address specific topics. Most importantly, program personnel want to be able to participate and ask questions. Build in adequate time for this during your preparation. Be sure to develop a strategy for limiting your team’s responses if they go too long or stray from the topic at hand. The presentation is typically followed by a Q&A session with the review panel for twenty minutes to an hour (or a designated time per presentation). This may include breaks for the review panel to formulate questions. There is usually a period at the end of the day where the reviewers provide additional questions to the presenters to respond to in a formal document within a prescribed window of time.

What is expected in addition to the presentation?

In addition to presentations, a revised budget, state of work, or management plan may be requested. Possible questions to consider in R/SV preparation include:

• Is the budget appropriate for the proposed activities?
• Is the proposed management plan likely to be effective?
• Are intellectual property issues adequately addressed?

R/SV Strategies for Success

Read your proposal again.

Proposals at this level and stage are highly polished, well developed, articulate documents that demonstrate your initial (and winning) program design. This content should form the foundation of any presentation. Reusing graphics, language, and narrative structure provides a familiar starting point for your team and the reviewers. It is highly likely that the reviewers have read your proposal, as they will be responsible for finding gaps in logic, resources, or risk to performance. You have a final budget and you should resist the urge to change your program in response to reviews, as this may stretch your resources too thin.

Purposely design the interaction.

Your seating arrangement, order of presentations, and who speaks should be scrutinized. If a virtual visit, names can be connected to institutions and background images can be coordinated. This is an opportunity to visually demonstrate the diversity and cohesion of your team. Be aware of how you have distributed everyone (audience and presenters) according to ethnicity, race, age, discipline, career level, and gender. Ensure compliance to all requirements provided by the sponsor. If they say you have 30 minutes on a specific topic, you have only 30 minutes on that topic.

• A previous NSF STC awardee noted that they prepared binders for the reviews. Their binders included an agenda, a copy of the presentation, a copy of the proposal, notes from their review, their response to review notes, a pouch with pens, highlighters, page tags, and a USB with all media files.
• A virtual background that includes the NSF logo and a signature graphic element can be effective to creating a sense of unity. However, having two or three background options allows team members feel more unique. Consider adding the logos/art of collaborator institutions to the background.

Invest in the presentation.

You should invest both time and money in the presentation. Research Development (RD) can assist with this preparation. Focus on your team’s overall message, walk through anticipated questions, preparing the team, and debriefing after the R/SV. Consider contracting a professional coach to provide guidance on presentation and messaging. A professional presentation coach can ensure clarity and an approachable format, be able to comment on the sponsor agency expectations, and review slide content.

• NORDP experts note their recommended inclusion of RD personnel. Send a representative from the RD office, although that person may not be in the actual meeting (due to limited seats). If RD staff can attend, have them take notes and provide reflections.

Practice makes perfect.

Across the board, it was recommended that more than three practice sessions made the difference between a flailing versus polished presentation. In person (or virtual) practices are good for content development and immediate feedback. Each practice should have at least one new, unbiased audience member to provide constructive comments. Add senior faculty with sponsor experience to play the role of the review panel. Encourage questions from the audience. You may also consider inviting previous proposal reviewers, e.g., pink and red team reviewers.

• A previous NSF AI Institute awardee noted they had four planning sessions as well as four to five practice sessions. In the practice sessions, they invited people from all over their campus to listen. This included Associate Deans for Research (ADRs), people who were PIs of successful large proposals and/or had been through an NSF RSV. These groups listened and provided honest feedback. The awardee noted that this feedback was very helpful.
• A previous NSF STC awardee also emphasized the need to conduct a red team review. This awardee recommended teams plan for a red team review three weeks before the RSV and noted that the red team should include science experts, education experts, NSF experts, and campus leadership.

Hone your strongest themes.

Craft a consistent message that emphasizes the importance beyond the research. Be sure to emphasize that the Center or Institute is about more than just the research. Use your proposal themes to describe the project's impact quickly and easily. In addition, each presentation section's theme should reflect and reiterate an overarching message. Polish your slides so that they look crisp, clean, uncluttered and message driven.

• A previous NSF AI institute awardee noted that teams must be able to show how the research will advance foundational knowledge of AI while also simultaneously advancing knowledge in the thematic discipline. Be sure to highlight more than the research by discussing the broadening participation, education, workforce, and social science aspects.

Cohesion is important.

The R/SV provides an opportunity to demonstrate team cohesion and understanding of roles within the institute. There must be a team approach throughout. Show how the team will function well together. You can accomplish this through frequent team and subgroup meetings to review the proposed plan.

• A previous NSF STC awardee noted that design elements make the team look cohesive. Consider logos, PI pictures, group pictures, and PowerPoint templates. Professionally developed graphics are essential for messaging, presentation, logos, banners, and conveying data.
• A previous NSF AI Institute awardee noted to be sure that the minority partner/HBCU is involved in all aspects of the institute and a part of the RSV team. This shows that partners are engaged and valued participants.

Accept any and all sponsor feedback.

If they mention it, address it fully and politely. Your response to questions and suggestions are part of the site visit evaluation.

• A previous NSF STC awardee noted that on the night after the first day, teams should plan to spend the evening answering questions. The awardee noted that their team was given 12 questions and prepared a 10-page response to present to reviewers the next morning.

Present the obvious, plus the less obvious, minus the distracting.

Your presentation should balance existing content (the proposal) and new content (adjustments based on the reviews), and introduce new information that bolsters your project, but doesn't distract from it. For example: Program A is innovative but reviewers say it lacks X factor. Your team met for a month to find the X factor and discovered that Y was neat too. Present your original program A, bring up added X factor along with Y, but don’t distract with details that you had to meet for a grueling month to discover X and Y.

Conclusion

By purposefully designing your R/SV interaction, meticulously planning and preparing, and focusing on clear messaging and team cohesion, you’ll have the best chance for success in highlighting why your project merits funding and why your team is best suited to tackle the project. Good luck!