Coordinated Tribal Assistance Solicitation Training: Performance Area 4

GABRIELA CASTELLANOS: Here is our agenda. As you can see, it is very packed. So without further ado, I'm going to pass it on to my colleague, Jeannine, to get things started. 

JEANNINE BULBULIA: Hi, everyone. Welcome. Apologies. I'm going to be cameras off today. My camera was giving some issues earlier, so just so you guys can hear me okay, I'm going to be cameras off for this session.

So, what is performance management? Performance management is the process by which programs regularly collect data on their grant activities to determine whether they're implementing activities as intended and achieving their desired goals and objectives. Using performance measures that capture ongoing data to understand how a program changes over time enables pre- and post-comparisons that can be used to assess change. Reviewing the data to identify trends, patterns, and insights can inform program decisions and improvements. Program data can be shared with funders to demonstrate the success of grant activities and compliance to regulations. You can find additional information and several resources on the Office of Justice Programs Grant Performance Measurement and Progress Reporting Information portal at the BJA performance website listed on this slide. 

Performance measures have many purposes for BJA. BJA uses performance measures to understand how funds are being distributed in use. BJA regularly tracks progress towards goals and develops various reports using annual key performance indicators. Results inform budget, strategic plan, and future funding. BJA also tracks grant activity and progress towards program goals. Performance measures may involve BJA to respond to external requests—for example, congressional inquiries and media requests. BJA also uses performance measures to identify areas of success and potential areas of improvement. Performance measures identify needs for TTA or revisions to program designs and implementation. Finally, BJA uses performance measures to comply with the law and complete BJA's required reporting. The tracking of progress through performance measurement is required by the Government Performance and Results Modernization Act of 2010, the Digital Accountability and Transparency Act of 2014, and the Grant Reporting Efficiency and Agreements Transparency Act of 2019. Some data is also required to be reported to various government agencies and Congress or to respond to FOIA requests. 

Not only does BJA benefit from performance measurements, but there are many benefits to you and your programs as well. You should be systematically monitoring performance measures as well as your own data to help identify what's working well and what might need to change. This helps you make data-driven decisions to manage and improve your program. Additionally, the data you provide allows BJA to provide targeted training and technical assistance resources to grantees that need them the most. Performance measurement data can also be used as evidence that you are meeting your goals. This can help with program sustainability and help you advocate for additional resources for your programs. 

Now let's talk about the what, where, when, and how of reporting requirements. We will review each of these next. The performance measures for CTAS purpose area four can be found at the link on this slide. In January 2025—excuse me—CTAS fiscal year 2020 grantees and later will report using new performance measures. We encourage you to be familiar with the new performance measures before the next reporting period. Additionally, you can use the questionnaire as a guide to track relevant data in your files and time intervals that is easiest for you. When it's time to report, you can add those figures to get your reporting period totals. (coughing loudly) Excuse me. 

There are two required sources of data that can be used to report: performance measures and narrative questions. The performance measure questions for your program are a series of questions that measure outcome of grant activities and demonstrate the accomplishment of goals and objectives of BJA programs. Narrative questions are also provided to grantees to be used for data reporting. These questions are a series of questions related to grantee-specific goals, objectives, barriers, and successes. 

BJA has two online tools to facilitate performance measurement reporting: the performance measurement tool, also known as the PMT, and JustGrants. BJA is in the process of transitioning reporting to the JustGrants system. Eventually, all programs will report performance measures data in JustGrants. The CTAS program grantees from fiscal year 2020 and later will begin reporting data in JustGrants during the next reporting period in January 2025. The great news is you already have access to JustGrants because this is the same system you were logging into to upload your performance reports. We have the PMT and JustGrants resources here on this slide to help you with your transition. If you have any specific questions related to your program's performance measures or reporting requirements, please reach out to the PMT help desk. If you have JustGrants system-related questions, please contact the JustGrants support team. We will provide the contact information for both these teams at the end of this presentation. Depending on the year of your grant, you'll have [a] different reporting process. I'll explain the different processes in the next few slides. 

For FY 2019 and earlier grantees, there will be no change. Grantees with FY 2019 and earlier grants will continue reporting performance measures directly in the PMT on a quarterly basis until the end of their grant. In January and July, grantees will need to upload their performance reports to JustGrants. Again, just for clarity, FY 2019 and earlier grants, there will be no change. Next slide. 

For fiscal year 2020 and later grantees, they will be transitioning to JustGrants in January 2025, reporting using new performance measures. Grantees will be reporting performance measures directly in JustGrants on a semi-annual basis in January and July. Grantees will have to report data collected from July 1 through December 31 by the January 30 deadline. No reporting will occur in the PMT, so we understand that there will be some double reporting since there is some crossover assistance and grantees have already submitted performance reporting for July 1 and September 30. That is okay. We want you to report the data in the correct reporting period in the correct system. If you have any questions at any time while you're reporting in January, please reach out to the help desk. They're available to assist you with any questions you have while you're reporting. Next slide. 

I know that was a lot of information, so we've created a visual for you to better understand where and when you should be reporting data. On this slide, if your organization has a fiscal year 2019 or earlier grant that will be active and collecting data, you fall into the category of grantees on the left. For the January 30 deadline, you will submit performance measures in the PMT for the October 1 through December 31 reporting period. You will continue reporting in the PMT until the end of your grant. This means you will not be using the new reporting measures we will go over today. If your organization has a fiscal year 2020 or later grant that will be active and collecting data, you will submit performance measures using the new performance measures in January of 2025. You fall into the category on the right. 

Performance measures are semi-annual in JustGrants, so grantees will begin submitting performance measures twice a year. Grantees will need to submit performance measures using data they collected during the July 1 through December 31 and not just the October 1 through December 31 reporting period. Please keep in mind, if your organization has multiple active grants which fall within both the left and the right side on this slide, you will have to submit performance reports in both systems for the respective fiscal year of the grant. So please keep in mind if you have different grants, just, you will be reporting in both the PMT and in JustGrants. Next slide. 

I will now turn the presentation over to Eulana from the JustGrants team to provide you with an overview of JustGrants. 

EULANA WILLIAMS: Alright, well good afternoon everyone, and just gimme one moment while I share my screen. All right, if I can get a quick thumbs up to see, make sure you're seeing the correct screen. 

GABRIELA CASTELLANOS: All good. 

EULANA WILLIAMS: All right, awesome. Perfect. All right, let me just move this over. Alright, so again, thank you so much for having me. My name is Eulana Williams. I’m from the JustGrants training team and I am here along with some other JustGrants training team members to show you how to submit your performance measures in JustGrants rather than the performance management tool, the PMT system, that you've been previously using. 

Now today's presentation is going to cover an overview of performance reports. It's also going to highlight the process of question sets and the steps to complete and submit your reports. Then lastly, we'll show you where to find some helpful resources. 

Alright, so we're going to begin with a few key reminders about performance reports. Alright, now all required performance reporting will now be completed directly within the JustGrants system. Now, what does that mean? Well, that means that rather than uploading external files, you will fill out structured performance measured question sets inside of each performance report. Now this change should streamline reporting by guiding you through specific questions, ensuring that all the relevant data and details are captured. 

Now, some other quick reminders that I want to make you aware of is that delinquent reports may result in a suspension of grant funds. Now, once the delinquent report is submitted, the suspension will be lifted within a 24-hour period. The reporting cadence is going to be determined by the solicitation and is based on the program and funder. Reports are either going to be regular or final, and the grant award administrator can mark a report as the final performance report, signifying that all the grant activities are complete, funds are obligated, and the award is ending early. Now, final reports are due no later than 120 days after the end of the award, and performance reports are generated based on frequency, so the due date is determined by the program's award. Performance reports generated for an award's last reporting period will automatically be final reports. Now, all of the others are going to be considered regular performance reports. 

Alright, so now we're going to show you how to submit your performance measures in JustGrants. Now, once you log into JustGrants as the grant award administrator, you're now going to select the appropriate performance report from your My Work list. All right, you can see that here in green. Now if you're not familiar with using the filters, there is filtering options to help you find things easier. There's also an upside-down triangle and that's located on each of the column headers to open up the filters for you. 

Now, the reporting type is listed here and it should always be regular unless the very last performance report for the award, alright? Now, if it is the final report, then JustGrants is going to automatically change it. So, under the program description, all right, so you can see here under the project description section, you will find a list of the question sets to complete. Alright? And here is, you're going to see this link. That's what you're going to click. You're going to select that link to open up the question set, so that way, you can enter in your performance measures. 

Now, in the upper-right corner, alright, you're going to select the Begin link and this is going to allow you to edit the performance measures. So, I do want to give you a quick little note there, but the Begin option will only appear for a user that can actually take action. Now, the performance report will display in read-only mode, but you can print the questions by selecting the Action menu and selecting the Print option. Alright, so here's the Action menu, but once you click on that, you will see that print option there. Just remember, changes cannot be made to the question set while it is in a read-only mode. 

Alright, so now you can begin responding to the questions on the page. Now, the continue button, which is here on the lower-right corner of the screen, is going to allow you to continue to the next page, or you can select the Save option, alright, which is over here, which will allow you to come back to finish the questions later. Now, just also be aware that questions are going to vary among awards, but if you see questions that have a red asterisk, those questions must be answered because you won't be able to submit the report if the required questions are not answered. 

On the last page of the question set, the Continue button will be replaced with a Finish button, so you're going to select the Finish button when the question set is complete, or you still have the option to press the Save button if you're not finished answering or responding to the question sets and you want to come back to it later. 

Now, once you have selected the Finish button, you will return to the performance report and the question set is now in a Resolved-Completed status. Now, you'll also notice that the performance report is still in a draft status, although your question set has been completed. So, to edit the question set, you can select Reopen. You're going to then click the Begin to edit the question set. You will need to select the Finish button again to return the status to a Resolved-Completed. The question set must be in a Resolved-Completed status to submit the performance report. Now, if the Resolved-Completed status is not displayed, what you can do is go up to that Actions menu that I showed you before, up here in the upper-right corner, and you can select the refresh button so that way, you can see it. 

Now in the performance report, you can add relevant comments right here in this section, just like you did when you were uploading your PMT reports. That was going a little fast. There's also a section for attachments, so this is where you can upload additional required documents, but this should be familiar because this is where you used to upload your PMT reports, but now, you know, all of that data is going to be included in the question sets. But if you would like to upload additional documentation, you can certainly do that here. 

Alright, now you're going to select the submit button to complete the performance report, or again, if you're not finished, then you can select the Save button to return back to it later. But once it is submitted, the performance report is going to be routed to the grant manager for review. Remember, you will not be able to submit if the status of the question set is not Resolved-Completed. Now, if the report type is final, the confirmation page will appear after selecting Submit. You're going to go ahead and select the Confirm button, and just as a reminder, the report should only be final if it is the last report of the award. Now, selecting the Cancel is going to return you back to the performance report, so that way the grant award administrator can go ahead and make any changes to the performance report before submitting it. 

Alright, so then, sure, I got, here we go. So now you're going to see here up at the top, the displayed status. Alright, so the status should be pending repository upload. Now there's also another notice that reads, “Thank you. The next step in this case has been routed appropriately,” and so, this is where the grant award administrator is going to be receiving an email confirming that that performance report was submitted. 

Now, there's an audit history available as well, so the audit page is going to display an up-to-date historical description of all of the actions that have taken place on the performance report. I do want you to notice that the audit indicates that the performance report was sent to the grant manager reviewer for approval or rejection. Now, I know that's going to be a little bit hard for you to see, but what that is is this top entry here. Hopefully, you can see my mouse. That is what that entry is stating, that the performance report was sent to the grant manager for review, for approval or rejection. Okay? 

All right. Now before I conclude for today, I would like to show you some additional resources that you have available to help you as you go through this new process. All right, so we have the JustGrants website, which has a ton of information about JustGrants, including topic-specific training resources, user support information, news, and updates. Now, there's also the email for our JustGrants training support and we do monitor that, so if you have a question or anything like that, you can certainly reach out to us via email. We do monitor that email and we'll provide a response to you. 

All right. We also have some additional training session, training webinars to help assist you as well, and these, we offer award management sessions on the first and third Mondays of each month. We also have entity management, which is held on the second and fourth Tuesday of the month. On Wednesdays, the second Wednesday of the month, we have our application submission session. That's basically for—excuse me—for submitting an application. We have that on Wednesdays. On Thursdays of every week, we offer the grant award acceptance. Now all of the times that you see listed here on the screen are Eastern Standard Time, but these sessions are really repeated sessions. They don't cover any new material or updated material, but, and if you've attended one of these sessions before, you don't necessarily have to come to attend these sessions, but if you would like a refresher, we are there. We actually facilitate these sessions, so you can come here and that's just another opportunity for you to be able to ask some questions pertaining to that topic. 

Alright, and that concludes my section and I will turn this presentation back over to Jeannine. 

JEANNINE BULBULIA: Thank you, Eulana. Before we move on to the performance measures, I would like to note that the changes we have discussed today are for BJA CTAS programs. In the next few slides, I'll be reviewing the new performance measures for CTAS Purpose Area Four. These new performance measures will be reported by FY 2020 grantees and later in January. CTAS Purpose Area Four consists of seven sections: general awards administration, baseline, project impact, renovation or expansion grants, new permanent facility grants, and narrative questions. 

Although this is not your first time reporting under this award, in order for us to have some baseline data in JustGrants, we will need all grantees to respond to the baseline questions. These baseline questions will need to be answered during the first reporting period reported in JustGrants. Please ensure these responses are correct, as they will not be available to change in future reporting periods. 

The project impact section will guide grantees to the correct questions. Grantees funding a new permanent facility project will only be required to respond to the new permanent facility section. Grantees funding a renovation or expansion facility project will only be required to respond to the renovation or expansion section. JustGrants will skip questions based on grantees’ responses to the first question in the project impact section, so please ensure you're selecting the correct option. I recommend reviewing the CTAS PA Four performance measures PDF ahead of time, so you can see the skip patterns. 

When grantees select that their grant is funding a renovation or expansion project, the renovation or expansion section questions will appear. The renovation or expansion grant section consists of subsections: project planning and project implementation. Grantees will need to select all the planning activities they have completed during the life of their award and not just within the reporting period. If your project has not begun renovating or expanding, you will skip the project implementation section. Only grantees who have begun renovating or expanding will answer all questions in that section. Don't worry. Again, JustGrants will skip those questions for you that do not apply as long as you select the right response. 

The new permanent facility section consists of subsections: project planning, and project implementation. Grantees will need to select all the planning activities they have completed during the life of their award. Only grantees who have begun construction will answer all questions in the planning implementation section. Sorry, if your project has not begun construction, you will skip the project implementation section. Only grantees who have begun construction will answer all questions in this section. Next slide. 

Lastly, the performance measures end with narrative questions. The narrative questions at the end of the performance measures are also new, so please review them within the PDF when completing your performance measures. Since grantees will be reporting semi-annually now in JustGrants, narrative questions will be required during each reporting period in January and in July. 

We want to make sure that you're answering the narrative questions correctly. When responding to your narrative questions, grantees should include a description of their program goals when responding to the accomplishments and challenges questions. We have provided an example on this slide. For the first narrative question, which reads, “What were your accomplishments, including any progress made towards achieving your grant-funded program goals during the reporting period?” The grantee's response states their program's goal, followed by the accomplishment of that particular goal. 

For the second narrative question, which reads, “What challenges did you encounter, if any, within the reporting period that prevented you from achieving your goals or milestones?” Grantees should restate their goal and then state the challenge. If grantees have more than one goal, each goal should be listed with a corresponding accomplishment and challenge. Please feel free to number goals and their corresponding accomplishment and challenges. 

Finally, we would like to share some tips to help improve data quality for your program. It's recommended that a designated staff person coordinate all performance measure data collection and entry to ensure consistency. If this is the first time you will report data for your organization, make sure you're familiar with the data you will need to collect and report. Do this by reviewing the PDF version of the questionnaire. Ensure a backup person is aware of the data collection and reporting process so they can fill in if the designated person is unavailable or leaves their role. Consider available data collection methods, including case management systems or other databases, spreadsheets, tracking intake forms, and other methods. If partner organizations are included in your program design, be sure to engage them from the start in the planning process. Determine if formalized agreements are needed to ensure the necessary data is collected and meets the program's goals and objectives. 

It is also important to know which help desk to contact so your issues or questions can be resolved quickly. You can contact the PMT help desk if you have any questions on the following: questions on performance measures, to schedule a TTA session for staff on data collection, PMT reporting or understanding performance measures, PMT questions or account creations, unlocking previously submitted reports, creating semi-annual PMT reports. You can contact the JustGrants help desk if you have questions on the following: account Creations in JustGrants, navigating JustGrants to find progress reports, unlocking account or help with passwords, roles and permissions, error messages, and uploading and attaching documents. The contact information for both these offices and help desks are at the bottom of the slide. 

On this slide are a few websites that are worth bookmarking so you can go back and reference them. Specifically, I want to highlight the BJA performance measures webpage, as it has a lot of information to help you with your reporting as well as your performance measures. There's also various training videos on the BJA YouTube channel that might be useful to you.