Harold Rogers Prescription Drug Monitoring Program New Performance Measures Training

Gabriela Castellanos: Okay let's get things started. Hello, good afternoon or good morning depending on where you are in the country, and welcome to today's Harold Rogers prescription drug monitoring program, otherwise known as PDMP New performance measures training. My name is Gabriella Gastianos, and I am the TTA specialist on the BJA planning, performance and impact team. We are so happy that you've joined us today and we are excited to present this information as you, are in the January submission period. For your awareness, this session is being recorded and will be posted on the BJA youtube and BJA multimedia pages in the coming weeks.

But today we're gonna have presentations from the PDMP team. A couple of announcements before we get into our content, again, the session is being recorded and a copy of the presentation slides will be given to all participants following this training. If you experience any technical difficulty.

During the session, please reach out to my colleague Angela Beebe, through the Q and A function, which I will explain in just a 2nd. One common technical difficulties that users experience is access to Webex via browser versus the app. So if you have access to the Webex app, we recommend that you use that platform instead. Alright, into the Q and A, we encourage you to ask questions as they arise throughout the presentation. However, please note that for today's session, given the volume of participants and interest in the session, we will only be taking questions directly through the Q and A function. So if you'd like to submit a question, please use the Q and A function on the bottom right hand side of the screen. We will answer questions as they come in. But for more complex questions are those that require additional explanation. We will address those at the end during the open Q and A portion of the session or we will follow up with you via email. We will also, just a reminder, once again, we will send out the presentation slides after the session. And if after today's training, you still have questions or think that you need additional help, we can provide you with one on one assistance. We will provide you on who to contact for that towards the end of this presentation. The chat box will be used by panelists to share links and resources with you, the participants. Here is our agenda today, and with further ado here are our presenters. There is me, like I said Gabriella, I'm the TTA specialist, and here are my colleagues, There's Shanise, and Armonee they're gonna, they're so excited to present the information to you. So without further ado, I'm gonna hand it over to Armonee. Take it away Armonee.

Armonee Avent: Thank you Gabriella. Yes, my name is Armonee and I will be starting the presentation with the performance management section. Next slide.

Performance management at BJA involves the process by which programs and grantees regularly collect data on their grant activities using performance measures, this will help to determine whether grantees are implementing activities as intended and achieving their desired goals and objectives. It also helps to capture outputs and outcomes over time and to enable pre and post comparisons that can be used to assess change.

The performance measures create data that supports the analysis to identify trends, patterns, and insights that can inform the program for decisions and improvements. It also demonstrates the success of the grant activities and compliance to regulations. You can find additional information using the several resources on the OJP performance measurement and progress supporting information portal on the BJA performance websites listed below. Next slide.

BJA uses performance measures for these focus areas. To understand how funds are being distributed, BJA and OJP regularly track progress towards goals and develop various reports using annual key performance indicators.

When it comes to tracking grant activity as well as progress towards those program goals, these will enable BJA to respond to external requests and data calls, whether it's a congressional inquiry media request, to identify areas for a success and potential areas for improvement. Performance measures can help to identify the need for training and technical assistance or revision to program design or implementation. And finally BJA uses performance measures to comply with the law and complete bja's required reporting.

The tracking of your progress through these performance measures are required by various legislative acts such as the government performance and results Modernization Act of 2010 or the Transparency Act of 2014. There is some data that is also required to be reported to various government agencies and Congress or in response to a four year request. Next slide.

This serves as a benefit to the grantees as well because BJA can use the data to develop performance reports. These are the snapshots that highlight performance accomplishments by the program as well as those successes. We encourage grantees to check these out on our website. To support data requests, BJA supports many types of requests that include internal BJA inquiries, congressional requests as stated before or even those four year requests. Then to inform funding programs and TTA decisions, these results can help inform the budget, strategic plans, and future funding. This helps to let BJA assess the priorities, NOFOs, and informational resources and guides. Next, so now we can get into the discussion when it comes to reporting the data out using the three W's. It is important to understand where, when, and what data is needed during the reporting period, so we'll cover that now. So we can start with where to report that data. The performance measurement tool, also known as the PMT and just grants, are the two online platforms that are used for data reporting. As a PDMP grantee, you will be reporting your performance measure data into the PMT system on a quarterly basis. This image just demonstrates what you will see when you log in to begin reporting into the PMT system. There is a report that is generated by the PMT that you must upload into the just grant system on a semi annual basis, and that will occur in January as well as in July. These links will lead you to the login platforms for both the PNC and just grant system. As you plan out for your reporting of the data, we have provided a table with some important information. This table shows the dates for each reporting period. For each reporting period, those dates are needed to be, sorry, those days that need to be met in order to submit your final reports. So whether that's in the PNC system or the jess grant system are both, the deadlines for each reporting period is detailed as well. For those PDF generated reports, there are certain requirements that are needed, so e.g. example as of January 2025, there will be two PMC reports that would have needed to be uploaded into the jess grant system. And that is possible when the July to September 2024 period as well as the October to December, 2024 period are complete. Next slide.

To prepare for that data that is needed for the PMT to report quarterly, we have attached a copy of the performance measurement questionnaire to the slide. Here's an image.

With the questionnaire looks like in PDF format that covers the questions that will be located on the PMC. And just as a quick tip, this guide can be used to begin the tracking of your data, so you can start to prepare in advance for the necessary data needed for your, for each reporting period before the deadline.

The data that you are reporting on is unique to your program using the PDMP questionnaire. The questionnaire is organized into three sections. The 1st question section covers general award administration information, which is used for the activity status of your grant that has taken place during a specific period. The project specific questions are for the activities, products, as well as the outcomes of your project. And the last section for the narrative questions are used to summarize the grantees, goals, barriers, and overall successes.

I would like to mention that this is a special note. If you are a grantee with multiple awards that are active, then you are responsible for collecting all of that data on the PDMP questionnaire for each award. This image is just an example, quick screenshot of what it looks like when there are multiple active awards and each active award is required to have its own narrative and closeout report associated with that award individually. I will now transition the questionnaire presentation to my colleague Sarashaniece to go over PDMP specific performance measures.

Sarashaniece Coleman: Thank you Remindy for the introduction. Yes, let's continue to talk more about program specific performance measures for PDMP. The revised and current PDMP questionnaire contains new or updated performance measures in language, simplified data tables, updated award categories and new sections. There are three general award questions grantees then have an option to manually click which category they fall under category one or category two. Category one contains a total of eight questions while category two contains twelve questions. The questionnaire will still have a closeout and narrative section. Here grantees will see four closeout questions and six narrative questions. Next slide please.

On this slide, we can see that general award administration activity will start with the well known question. Was their grant activity during the reporting period? If yes is selected, the PMT system will dropdown question number three and allow you to select the appropriate category based on your award.

Category one is for grantees who have category one awards. You select this option if you are the authorized state, territory or tribal agency that administers your PDMP. Category two is for grantees who have category two awards. So you select only if you receive project based awards in FY 2023 and support of data german responses, multi state projects or advancement of PDMP. Category two was funded in FY 2023 and supplemented in FY 2024.

The category one awards section consists of two subsections, which are PDMP users and system capabilities and PDMP data use and investigations. In the 1st section PDMPM user PDMP users and system capabilities grantees will answer questions under two additional sections, the baseline prescriber, the in user data and data sharing, and you can see an example of that to the right. Category one award sections does include carry forward questions. Now carry forward indicates that the grantees response will carry forward and pre populate in subsequent reporting periods. Grantees will not have to respond to the question each reporting period.

But are able to make changes and updates if needed. This functionality only works in the PMT system. Please note that you may encounter various types of questions as you report on your program's grant activity. In general while going through your questions there are additional questions that you might come across. For instance, grantees will see multiple choice questions where multiple response options are provided and grantees should select the response that.

That's reflects their project's activity during the reporting period. Multiple response questions allow grantees to select all responses that reflect their project activities during that reporting period. Open text questions may request numeric or text responses. If you don't, if you do not know the data point, it is acceptable to report 9999 for unknown or NA. Finally, the system has a built in skip pattern logic based on your responses to other questions. Therefore, please ensure you answer every que.

Question that appears.

Within the subsection PDMP data use and investigations, you'll find updated questions and simplified inputs. If you look to the right, you can see an update to the specific question regarding how many reports your program's produced. Please be aware that the questionnaire in the PMP system will look slightly different and that the PD.

PDF is meant to use for guideline purposes.

The category two words section consists of subsections, advancement of PDMP capabilities, PDMP data policies, procedures and rules, data driven decision making, and multi state collaborative efforts. As I mentioned on the previous slide, you will come across various types of questions.

Specifically in category two, you can see how subsections contain both open text and multiple choice questions. An example of open text is question number 17 to the right or, and for multiple choice, question number 20. An example of built in skip pattern logic is shown in question number 22 at the bottom of the attached image.

Additionally, for some multiple choice questions, if a specific answer is selected an additional additional question could trigger a dropdown question. Question number 20 triggers a dropdown open text question when yes is, is selected.

Now, closeout questions will appear in the last reporting period before your narrative section. A grantee's responses are specific to each individual federal award. Grantees only respond to the closeout questions when activities have been completed, and the word has come to an end. Complete the closeout section if you are conducting an.

Evaluation on the effectiveness of your program. Please note, if you are not conducting an evaluation, select NO to skip to the narrative section.

Now the semi annual narrative questions. Grantees are required to complete the narrative questions in January, July, and at the end of the close of their award. In the narrative section grantees can highlight their accomplishments and discuss their challenges. Previously, a similar section was called goals and objectives. These questions have been updated.

To better reflect goals and corresponding accomplishments and challenges.

When responding to your narrative questions, grantees should include a description of their program goals when you're responding to accomplishments and challenge questions. We have provided an example on this slide. The programmed goal to the left is, as stated, within the award period, develop three training videos and provide two live.

Webinars to educate users on PDMP best practices. The grantee stated that during their reporting period, they gauged in working groups to develop training and reached out potential users and 80 % of attendees had an increase in knowledge and post event surveys. That was their accomplishment for the stated goal.

For the challenges, the grantee provided the goal and stated due to delay in funding during COVID-19 pandemic, potential training and outreach regarding drug monitoring has been delayed for health care providers. If grantees have more than one goal each, each goal should be listed with the corresponding accomplishment and.

Challenge.

Now, we will transition over to the topic of data reporting and best practices. 1st, let's talk about why it's important for the data to be accurate. Grantee data is used in a number of ways for critical BJ and congressional functions, including but not limited to legislatively mandated.

Reporting, informing and justifying federal budgets for future funding, responding to questions from legislators and other policy makers publicly reported statistics, testifying before Congress, identifying promising practices in the field and responsible use of federal funds. You are the best person to identify.

If something is incorrect in the reporting.

We would also like to share some tips to improve data quality that your program can do from the very beginning. It is recommended that a designated staff person coordinates all performance measured data collection and entry to ensure consistency. Please start as soon as possible. Reporting begins the day your awards starts. If.

This is the 1st time you will report data. Make sure you are familiar with the data you will need to collect and report. Do this by reviewing the PDF version of the questionnaire. Ensure a backup person or personnel are aware of the data collection and reporting process so they can fill in if the designated staff person is unavailable or leaves the role.

Do not duplicate reported activities, outcomes across reporting periods or awards if project is funded by multiple awards. And again, 9999 can be used if data is unavailable and provided and you would need to provide an explanation or plan for adjusting unavailable data. Please note.

Though that 9999 should be used as a last resort because it is a requirement to provide all requested data if possible. Please ensure to build data collection and tracking into standard routines, scan other existing resources within your organizations that support related data collection activities, as well as make a plan for your data collection method and processes. Lastly, please ensure all sub recipient data or data you receive from contractors service providers is reviewed and validated before completing a data entry.

I will now turn it back to Gabriella, where she will speak more about helpful resources and how to stay connected shortly after. She will jump in to the Q and A session to give you a chance to ask any questions you might have regarding PDMP new measured reporting.

Gabriela Castellanos: Thanks Sarah. Okay, so it's important to know which help desk to contact so that your issue or question can be resolved quickly. So you can contact the PMT help desk if you have questions on the following. Questions on performance measures.

If you'd like to schedule a TTA session for staff on data collection PMT reporting or understanding performance measures, PMT questions and account creations, unlocking previously submitted reports and creating a semi annual PMT report.

You can then contact the just grants help desk with the following. Account creation in just grants, navigating just grants to find progress reports, unlocking account or help with passwords, roles and permissions such as grant award administrator or entity administrator, error messages you may see or uploading and attaching documents.

On this slide are on this slide excuse me, are a few websites that are worth bookmarking, so you can go back and reference them. Specifically, I want to highlight the BJA performance measures webpage that has a lot of resources to help you report as well as your performance measures.

There are also various training videos on the BJA youtube channel that maybe useful to you as well. Lastly, before we go into Q and A, I'd like to show that you can stay connected with us, and find us on Facebook, X, youtube, and subscribe to our newsletters, which I believe is in that QR code. You can also.

Also find more resources via the BJA website listed on the screen and included in the chat. And please stay connected. We hope that you enjoyed the session in this training and we're excited to see what kind of questions that you have based on what we've presented today. So without further ado I'm going to turn it over to my colleague Angela to start reading out the questions that are in the Q and A go ahead and.

Angela Beebe: Good afternoon, thank you. So we have a few questions in the Q and A So I know that we have some PDMP panelists that if you would like to jump in to answer any of these questions, please feel free so that we can get as accurate information as possible.

The 1st question that was given to us was from Leonard. It had to do with, it says number one measure six A, how many licensed pharmacies are registered to use the PDMP system? Particularly question A, where pharmacies are not necessarily licensed to use the PDMP, individual pharmacists register with their credentials to use the PDMP and their employee has an entity license for each individual pharmacist to query the PDMP through integration.

What their specific question was, how would BJA like states to distinguish individual pharmacies? I was not sure with that one, that maybe when we need to address more specifically.

After the we have joined, after the presentation I do have a question about can subrecipients be added to category one?

Okay yes, the slides will be given at the end of the presentation. We will send that out with a thank you email and it is being recorded.

Eric Smith: Yes for that.

Angela Beebe: Oh Yep.

Eric Smith: I'm, I'll jump in here for the questions related to six. I think there's a couple related to registered pharmacists, and how we want, want the group to, to answer these, we're not saying the incorrect thing out loud to everyone and make sure we're right. We'll get back to you and reach out to you to reach out to you directly on our response for those ones.

Just want to make sure we're saying the right thing and it's aligned with what we have in our questionnaire and what we're hoping to see on the back end, so we'll follow up with those who asked that question directly.

Angela Beebe: Okay, thank you. Their other question, another question was, can subrecipients be added to category one? I wasn't sure if we've addressed that one or not.

Okay. Okay.

Sarashaniece Coleman: Yeah, Angela, from my understanding, sub recipients can just be added directly into whatever category they're in. Is that correct? I don't know if into the PMT system. So whatever the the main grantee where wherever they put them at for category one or two, they will automatically have their own, sign in portion for the PMT and they can just fill in their own information, or is that incorrect? I wasn't I just wanted to double check.

Angela Beebe: Okay. So one of the question was the responses Yes, we will get back to everyone. If there's not a question that we can answer today, we are, as we said at the beginning, we will make sure.

Or that we reach via email and reach back out. So we will be addressing some of these questions individually after the presentation is over. Couple of questions I do want to read them out so that everyone hears them. We did have another one that is if our award is under the 2022, BJ 8th fiscal 22, Harold Rogers prescription drug monitoring program, does that mean we are category one questions?

Eric Smith: So I would have to look at the, we'd have to know the specific award number. It is likely that you are category one as there are not many awardies who fall into category two, but we'd have to look at your specific award to confirm what that would be. Okay.

Angela Beebe: And then one of the other questions that was asked is what is meant by category cat 1Q10 option H, the very specific drug treatment program. Does this include substance abuse treatment programs? This was something Lisa wanted to know.

Eric Smith:  Yeah Angela I'm sorry, can you hear.

Angela Beebe: Repeat that one. Yeah I can, I'm sorry. It's, it's very specific to a cap category one question ten and it says option eight H drug treatment program. Does this include substance abuse treatment programs?

Angela Balchi: I can jump in here Angela, this is Angela. Yep yep, sure. I can't see that we've had this question before, so we may also want to go back and double check and get back to you, but all any drug treatment programs including maybe inpatient or outpatient treatment programs would be included there. We don't have, specific examples or definitions of any of these end users, but anything that would fall under a drug treatment program, so substance abuse would apply there.

Angela Beebe: Okay, thank you one another question from Jamie with pharmacies are not necessary licensed to use PDMP? PDMP usage is either done by registered pharmacists or via an entity license.

This one kind of I think we've already, we mentioned earlier in letters that this is something we will get back to you on, so we make sure that we give the correct information for. So I did want to make sure that it was addressed out loud, but it does go along with a previous question that we have already said we will reach back out to give more information.

Jamie did have another question about measure nine and hiies have the potential to skew the count of entities integrated with a PDMP system as one HIE could connect multiple healthcare organization.

If an HIE is counted as a single integration, the actual number of individual entities utilizing the PDMP via the HIE, maybe under represented. How should states factor this into their calculations?

Okay, and then he had a follow up of what is the time frame state should consider when they're evaluating these entities, and he gave an example like should they go twelve, a twelve month look back?

Angela Balchi: This is Angie again. So for question nine, which is reads how many entities such as or including hospitals, pharmacies, facilities clinics are integrated, e.g. electronic health records, pharmacies.

See dispensing, software, health information exchanges with your PDMP system. So we recognize that there maybe some integration specifically with hiies, so if you are able to provide a total number and then this is open text, so you if you're able to provide more details.

On what your count or breakdown includes, like such as the breakdown of, X number of hospitals or pharmacies, and you can also provide that specific note about having those integrated systems as well. Question nine would be.

Cumula it's a carry forward question, so that response that you enter will carry forward each reporting period and you only need to provide updates to that if you have maybe additional entities that you need to add or updates to that question.

Angela Beebe: Okay, thank you Angie. Our other question, last question that I have right now that hasn't been read is from Justin, and he said Colorados grant was issued to the administrating entity of the PDMP, from the division of professions, occupations.

Department of regulatory agencies, but a hundred percent of the grant goes to a sub recipient, which is the University of Colorado for implementation and evaluation of PDMP, use mandates for the CDS tool. None of the grant funding goes to the support.

Reporting PDMP operations for the administrating entity. We are seeking clarification as to which category we should be reporting under category one, category two or both.

Angela Balchi: Yeah, this is another question, Justin, that we may have to reach out to you directly in order to get your federal award number to see where you, which award you were or category you were funded under, and it is most likely category one, but recognizing with an evaluation aspect, not all the questions maybe relevant to you, so we may have to reach out and work with you directly to get some more information and provide you with more specific guidance.

Angela Beebe: Okay, thank you. Missy just typed to questions that for our user totals, are those for the just for those just the reporting period or a grand total?

Angela Balchi: Missy, I think you, are you still referring to question ten, with the end user total table? So those totals are the count for the specific reporting period for which you.

Angela Beebe: Reporting on. Perfect.

Angela Balchi: Okay, correct. Yep. Not cumulative for that one. Okay.

Angela Beebe: Right now that seems to be all that has come in. I'm waiting for.

Eric Smith: And just to reiterate for those who asked about if they're category one or two, we'll follow up with directly to get your war number and then provide you additional guidance from there and we'll share out responses that maybe relevant to the group, specifically as it relates to the integration of, or the, the pharmacies and pharmacists and the license and how to report those.

Angela Beebe: Yeah. At this time I'm not seeing any other new questions.

Sorry about that, just got one. Lisa wanted to know just to clarify on Q nine, are we to make our best guests and then explain there could be more?

Angela Balchi: Yeah, so, Lisa, if you want to reach out to us directly on where you know what data you have available and what you think you would like to report on that and we can work with your situation on how it would be best to report that. But yeah, making any clarification patients as necessary with text to explain any, you know, totals on, total entities that you're working with. So I'm just having that descriptive information in text.

Angela Beebe: Okay, thank you. Jamie said, who specifically do we need to reach out to for clarifications on nine?

Angela Balchi: So you can contact the PMT help desk. We will be, you know, reaching back out to people specifically, that had that question and trying to provide, more general guidance to the group on that one, as the IC, we're all in the same boat here exactly. So, but if you have, you feel of a very specific or unique situation you want to provide maybe more information.

Angela Beebe: Okay, this bouncing. I do have from Nancy, Alabama would also like to clarify if awards are category one or two and they included their award numbers so we will be getting back in touch with you to let you know. Thank you.

Okay, uh. Okay Lisa did put in a question that says that we just need to provide a response and then be consistent. Is that the best guidance?

123 "Angela Beebe: Okay.

Eric Smith: So generally, yeah, we want people to provide their best guess, and best response for that and then stay consistent with how they're reporting on that. So Lisa yes, that's generally our approach. Okay, uh.

Angela Beebe: And just as a reminder, we do have everyone's email, so we do know who was here today, so we do know who to reach back out to, so do not worry that. That information is not provided. We do have that.

Gabriela Castellanos: Alright, we're keeping the Q and A, open for another 5 min. We really appreciate all the great questions that you guys have, and we cannot.

Reiterate enough, that we appreciate your questions and that we'll follow up not only with the resources shared in the session, but with the slides and then eventually a recording of the session so you can go back and reference it. Feel free.

To contact my colleague, she's put her email in the [email protected] if you would like to extend those questions to us, but again we have your information, and we will be reaching out to you in the coming days.

Otherwise, unless you have a question that you'd like to insert into the Q and A, we will be concluding the session. We really appreciate your attendance and we hope that you have a good rest of your day and your weekend. Thanks everyone.