Coordinated Tribal Assistance Solicitation Training: Performance Area 3

JEANNINE BULBULIA: Thank you, Gabriela. Let's start with, “What is performance management?” Performance management is a process by which programs regularly collect data on their grant activities to determine whether they're implementing activities as intended and achieving their desired goals and objectives. Using performance measures that capture ongoing data to understand how a program changes over time enables pre- and post-comparisons that can be used to assess change. Reviewing the data to identify trends, patterns, and insights can inform program decisions and improvements. Program data can be shared with funders to demonstrate the success of grant activities and compliance to regulations. You can find additional information and several resources on the Office of Justice Programs Grant Performance Measurement and Progress Reporting Information portal at the BJA performance website listed on this slide. 

Performance measures have many purposes for BJA. BJA uses performance measures to understand how funds are being distributed and used. BJA regularly tracks progress towards goals and develops various reports using annual key performance indicators. Results inform budget, strategic plan, and future funding. BJA also tracks grant activity and progress towards program goals. Performance measures enable BJA to respond to external requests, for example, congressional inquiries and media requests. BJA uses performance measures to identify areas of success and potential areas of improvement. Performance measures identify needs for TTA or revisions to program designs or implementation. Finally, BJA uses performance measures to comply with the law and complete BJA required reporting. 

The tracking of progress through performance measures is required by the Government Performance and Results Modernization Act of 2010, the Digital Accountability and Transparency Act of 2014, and the Grant Reporting Efficiency and Agreement Transparency Act of 2019. Some data is also required to be reported to various government agencies and Congress, or to respond to FOIA requests. 

Not only does BJA benefit from performance measurement, but there are many benefits to you and your program as well. You should be systematically monitoring performance measures as well as your own data to help identify what's working well and what might need some changes. This helps you make data-driven decisions to manage and improve your program. Additionally, the data you provide allows BJA to provide targeted training and technical assistance resources to grantees that need it most. Performance measurement data can also be used as evidence that you are meeting your goals. This can help you with program sustainability and help you advocate for additional resources for your program. 

Now let's get into the what, where, when, and how of reporting requirements. We'll review each of these next. 

The performance measures for CTAS Purpose Area Three can be found at the link on the screen. In January 2025, CTAS fiscal year 2020 grantees and later will be reporting using new performance measures. We encourage you to become familiar with the new performance measures before the next reporting period. Additionally, you can use the questionnaire as a guide to track relevant data in your files in time intervals that is easiest for you. When it’s time to report, you can add those figures to get your reporting period totals. 

There are two required sources of data that you will use to report: performance measures and narrative questions. The performance measure questions for your program are a series of questions that measure outcomes of grant activities and demonstrate the accomplishments of goals and objectives of BJA programs. Narrative questions are also provided to grantees to be used for data reporting. These questions are a series of questions related to grantee-specific goals, objectives, barriers, and successes. 

BJA has two online tools to facilitate performance measurement reporting: the performance measurement tool, the PMT, and JustGrants. BJA is in the process of transitioning grantees’ reporting to the JustGrants system. Eventually, all programs will be reporting in JustGrants. The CTAS program grantees from fiscal year 2020 and later will begin reporting data in JustGrants during the next reporting period in January 2025. The great news is you already have access to JustGrants because this is the same system you were logging into to upload your performance reports. We have the PMT and JustGrants resources here on this slide to help you with your transition. 

If you have specific questions related to your program's performance measures or reporting requirements, please reach out to the PMT help desk. If you have JustGrants system-related questions, please contact the JustGrants support team. We will provide contact information again at the end of the presentation. Depending on the year of your grant, you will have a different reporting process. I'll explain the different processes in the next few slides. 

[For] FY 2019 and earlier grantees, there will be no change. Grantees with FY 2019 and earlier grants will continue to report performance measures directly in the PMT on a quarterly basis until the end of their grant. In January and July, grantees will need to upload their performance reports to JustGrants. Again, for clarity, FY 2019 and earlier, there will be no change. 

FY 2020 and later grantees will transition to reporting in JustGrants in January 2025, reporting using the new performance measures. Grantees will be reporting performance measures directly in JustGrants on a semi-annual basis in January and in July. Grantees have to report data collected from July 1 to December 31 by [the] January 30 deadline. No reporting occurs in the PMT for the FY 2020 and later grantees. 

We do understand that there will be some double reporting since there is a crossover of systems, and grantees will have to submit performance reporting for July 1 through December—grantees have already submitted performance reporting for July 1 through September 30. That is okay, we want you to report in the correct system. The help desk is available to assist grantees at any time when reporting if you have any questions on what or where you should be reporting. 

So, I know that was a lot of information, so we created a visual to better understand when and where you should be reporting. On this slide, if your organization has an FY 2019 or earlier grant that will be active and collecting data, you fall into the category on the left. For the January 30 deadline, you will submit performance measures in the PMT for the October 1 through December 31 reporting period. You will continue reporting in the PMT until the end of your grant. This means you will not be using the new performance measures we will go over today. 

If your organization has an FY 2020 or later grant that will be active and collecting data using FY 2020 or later funding, you will submit performance measures using the new performance measures in January. You fall into the category on the right. Performance measures are semi-annual in JustGrants, so grantees will begin submitting performance measures twice a year. Grantees will need to submit performance measures using data they collected during July 1 through December 31 and not just the October to December reporting period. Please keep in mind if your organization has multiple active grants, which fall within both left and right side on this slide, you will have to submit performance reports in both systems for the respective fiscal year grant. 

I would now like to turn the presentation over to Eulana from the JustGrants team to provide you with an overview of JustGrants.

EULANA WILLIAMS: Alright, well good afternoon, everyone. Thank you so much for having me. Again, my name is Eulana Williams and I'm from the JustGrants training team, and I’m here to show you how to submit your performance measures in JustGrants rather than using the performance management tool (PMT) system that you've been used to using. 

Now, today's presentation is going to cover an overview of the performance reports. It’s going to highlight the process of question sets and the steps to complete and submit your reports. And then lastly, we're going to show you where to find some helpful resources.

Alright, so now what I want to do is go over a few key reminders about performance reports. Now, all required performance reporting will now be completed directly within the JustGrants system. Now, what that means is that rather than uploading external files, you will fill out [a] structured performance measure question set inside each performance report. Now, this change should streamline reporting by guiding you through specific questions, ensuring that all relevant details have been captured. 

Now, a few reminders that I want you just to be aware of is that delinquent reports may result in a suspension of funds. Now, once the delinquent report is submitted, then that suspension should be lifted within the 24-hour period. The reporting cadence is also determined by the solicitation and is based on the program and funder. Reports are going to either be regular or final, and the Grant Award Administrator can mark a report as the final performance report, signifying that all grant activities are complete, funds are obligated, and the award is ending early. Final reports are due no later than 120 days after the end of the award. And then performance reports are generated based on frequency. The due date is determined by the program's award, and then the performance reports, those are generated for an award's last reporting period. 

So, now we're going to show you how to submit a performance report, performance measured in JustGrants using the Question Set. So, the report type should always be regular, unless this is the very last performance report for the award. Now, if it is the final report, then JustGrants is going to automatically change it. Now under the program description, you're going to find a list of question sets to complete. You're going to select the link, alright, to open up that question set to enter in your performance measures. 

Now, in the upper-right corner, you're going to select the Begin link, right? So you can see that over here. I tried to enlarge that for you. This will allow you to edit the performance measure questions. Now, just remember also the Begin option will only appear for a user that can take action. The performance report will display in a read-only mode, but you can print the questions by selecting the Actions menu, the Actions menu over here up in upper-right corner, and select the Print option. Just remember, though, that changes cannot be made to the question set while it is in read-only mode. 

So now you can begin responding to the questions on the page. The Continue button allows you to move to the next page, the Continue button here, and the Save option allows you to come back later to finish it. Alright, now just, questions are going to be... they may vary among awards, alright? So they're not always going to be the same questions, but if you see any questions that have a red asterisk, those questions must be answered, right? You will not be able to submit the report if the questions, those that are required or have that red asterisk, if they're not answered. 

On the last page of the question set, alright, then, the Continue button is going to be replaced with a Finish button. You're going to select the Finish button when the question set is completed, or if you're, you know, if you still need to come back and finish some things, you can do the Save button, alright, and then come back to it later. Once you have selected the Finish button, you will return to the performance report. And now the question set is now set to Resolved-Completed, right? You'll see that in green down at the bottom. Now you'll notice that the performance report is still in draft status, although your question set has been completed. Now, to edit the question set, you can select the Reopen button and that way you can go back, edit the question set, and once you're finished you can select the Finish button again to return to the status. 

I just have a quick question. Alright, is everybody still able to see my slides? 

PARTICIPANT: Yes, we are. 

GABRIELA CASTELLANOS: Yes.

EULANA WILLIAMS: You can still see them. Okay, just wanted to make sure. Alright, now the question set must be in a Resolved-Completed status to submit the performance report. And if the Resolved-Completed status is not displayed, open the Actions menu and you can select the Refresh button. 

GABRIELA CASTELLANOS: Eulana. 

EULANA WILLIAMS: Yes. 

GABRIELA CASTELLANOS: It seems that you're sharing it, but you're also sharing your presentation notes. 

EULANA WILLIAMS: Okay, so you're, okay. So for some reason, it looks like the closed captioning from the WebEx is covering my notes. Let's see here. Okay, let me try it from here now. 

GABRIELA CASTELLANOS: Now I can see it. 

EULANA WILLIAMS: All right, how about now, can you see it? 

GABRIELA CASTELLANOS: Yes. 

EULANA WILLIAMS: Okay, let's see if I can close the closed captioning. There we go. Alright, awesome. Sorry about that everybody. Alright.

Alright, so in the Performance report you can add any relevant comments like you did when you were uploading your PMT reports. You can see that there. And then in the Attachments section, upload, you can attach additional required documents there. This is where you used to upload your PMT reports, but now all of your data is going to be included in those question sets. But if you still want to add additional documentation, you can certainly do that by uploading it here. 

Now you're going to select Submit to complete the performance report, or you can select Save if you need to come back and return to it later. But once submitted, the performance report is routed to the grant manager for review. Now, you will not be able to submit if the status of the question set is not Resolved-Completed. So, if the report type is Final, the Confirmation page will appear after selecting Submit. You're going to select Confirm, but then just as a reminder, the report should only be Final if it is the last report of the award. But if you need to select Cancel, Cancel is going to return the performance report back to the grant award administrator, so that way they will be able to make any changes before submitting that performance report. 

Alright, so you can review the displayed status, and the status should be Pending Repository Upload. Alright, now there's also another notification that'll say, “Thank you. The next step in this case has been routed appropriately.” That's what you want to see. Alright? That's the happy path. The Grant Award Administrator will receive an email confirming that the performance report was submitted. 

Now, there's also an Audit page that displays and it gives you a historical description of all of the actions that have taken place in the performance report. Now, it's a little bit hard to read, but up at the top, that last entry there just indicates that the performance report was sent to the grant manager reviewer for approval or rejection. 

Now, before I conclude for today, I want to show you some additional resources that we've created to help you as you manage these different processes in JustGrants. Alright, so we do have the Just Grants website, and this has a ton of information about Just Grants, including topic-specific training resources. There's user support information, news, and updates. There's also, here listed on the screen, the email address to our Just Grants training support. Erica and I, who are on the call, and Lisa, we all monitor that email, so if you have any questions, please don't hesitate to send us an email. 

Now, we also have some upcoming sessions that are helpful if you ever needed some support. We offer Award Management sessions on the first and third Monday of each month. We have Entity Management on the second and fourth Tuesdays of each month, and then on the second Wednesday of each month, we hold our Application Submission, and we also have, on Thursdays, actually every Thursday, every week, the Grant Award Acceptance session, and that's from 2 to 3 on every Thursday. So, if you ever have any questions or you just need to, you know, go over that information again, please come out, join us. It's a great opportunity for you to ask questions and things like that. Just know that the information, it's repeated information. It's not any new information that we're going over each week. But if it's something that if you need a refresher, these sessions are available to you. And again, Erica and I will be there, (laughing) training those sessions as well. 

All right, now I believe that is all that I have for you today, so I will go ahead and turn it back over to Jeannine. 

JEANNINE BULBULIA: Thank you, Elena. Let's talk about the new performance measures FY 2020 and later grantees we'll be reporting in January. CTAS Purpose Area Three is comprised of seven sections: General Award Administration, Baseline, Expenditures, Strategic Planning, Training, Participant Services, Planning Activities, and Narrative Questions. 

The JustGrants system does not allow for carry-forward questions. So instead, grantees will need to respond to the baseline questions. These baseline questions will need to be answered during the first reporting period reported in JustGrants. Please ensure these responses are correct, as they will not be available to change in future reporting periods. 

In the Expenditure section, grantees will need to respond ‘Yes’ or ‘No’ to whether they purchased computer equipment/software, general office supplies and equipment, medical, and/or other items with award funds during the reporting period. If grantees answer ‘Yes,’ they'll be prompted to describe the items, provide total quantity purchased, and the total award funds spent. JustGrants will automatically skip to the next question when grantees respond ‘No.’ So you don't have to worry about going through and skipping questions. Please review the PDF version of the performance measures ahead of time so you're aware of the skip patterns and the questions you'll be responding to. 

In the Participant Services section, grantees will respond ‘Yes’ or ‘No’ to providing specific services or activities with this award, and then select all types of services provided for each specific service/activity. These questions will follow a skip logic as well in JustGrants. 

At the end of the Participant Services section, grantees will be asked a series of questions related to the Healing to Wellness/Adult Treatment course. If grantees answer ‘Yes’ to admitting, graduating, or having participants leave the program, they will need to provide numeric responses. When providing these responses, the numeric response to the yellow highlighted questions should be less than or equal to the numeric responses of the green highlighted questions on this slide. Grantees should refer to the Performance Measure PDF for assistance on navigating this section. 

Lastly, the Performance measures end with Narrative Questions. The Narrative Questions at the end of the Performance measures are also new, so please review them within the PDF when completing your Performance measures. Since grantees will be reporting semi-annually now in JustGrants, Narrative Questions will be required during each reporting period in January and in July. 

We want to make sure that you're answering the Narrative Questions correctly. When you're responding to Narrative Questions, grantees should include description of their program goals when responding to the Accomplishments and Challenges question. We have provided an example on this slide for you. The first Narrative Question, which reads, “What were your accomplishments, including any progress made toward achieving your grant-funded program goals during the reporting period?” The grantee's response states their program goals followed by the accomplishment of that particular goal. 

For the second Narrative Question, which reads, “What challenges did you encounter, if any, within the reporting period that prevented you from reaching your goals or milestones?” Grantees should restate their goal and then state the challenge. If grantees have more than one goal, each goal should be listed with the corresponding accomplishment and challenge. Please feel free to number the goals and their corresponding accomplishment and challenges. 

Finally, we would like to share some tips to improve data quality that your program can do from the very beginning. It is recommended that a designated staff person coordinate all performance measure data collection and entry to ensure consistency. If this is the first time you will report data for your organization, make sure you are familiar with the data you will need to collect and report. You can do this by reviewing the PDF version of the questionnaire. Ensure a backup person is aware of the data collection and reporting process so they can fill in if the designated person is unavailable or leaves their role. Consider available data collection methods, including case management systems or other databases, spreadsheets, tracking intake forms, or other methods. If partner organizations are included in your program design, be sure to engage them from the start in the planning progress. Determine if formalized agreements are needed to ensure the necessary data is collected and the program meets its goals and objectives. 

It's important to know which Helpdesk to contact so that your issues or questions can be resolved quickly. On this slide, we have provided the PMT Helpdesk information for you. The PMT will be able to answer your questions on performance measures so you can schedule a TTA session for staff on data collection, PMT reporting, or understanding any performance measures, any PMT questions, and account creations. They can also unlock previously submitted reports and help you with creating your semi-annual PMT report that will need to be uploaded in JustGrants. On the right, we have provided the JustGrants Helpdesk information. Please contact the JustGrants Helpdesk if you need account creation in JustGrants, navigating JustGrants to find your progress report, unlocking account or help with passwords in JustGrants, roles and permissions, error messages, uploading and attaching documents. 

On this slide, there are a few websites that are worth bookmarking so you can go back to reference them. Specifically, I want to highlight the BJA Performance Measure webpage that has a lot of resources to help you with your reporting as well as your performance measures. There are also various training videos on the BJA YouTube channel that may be useful.