Q&A from PPS Operator Assisted Call on July 17, 2015

Q 1: The PPS would like to address mitigating factors that will resolve the risks identified by the Independent Assessor in the Implementation Plan´s Risk Communication section. There is not sufficient space to include both the Ia´s feedback and the PPS´s response. Should the PPS reduce the size of the communications for each work stream?

A: Space has been provided for 3900 characters within the narrative box. The PPS is advised to reduce the length of the response.


Q 2: The PPS would like to know whether the text entered into the narrative box can be edited.

A: The DSRIP team confirms that text can be edited, but anything the PPS includes must stay within the 3900 character range to avoid truncation of answers. This limitation applies to answers that are cut and pasted as well. Any box can be edited up until the report is submitted.


Q 3: Is it true that PPSs do not need to fill out the provider ramp-up tables, and that zeroes will be sufficient for the Q1 submission?

A: No, that is not accurate. All tables where fields are shown, extending through DY5 Q4, should be filled in with an accurate projection of the PPS´s numerical provider ramp-up. The table must be filled out in its entirety, but the numbers are for informational purposes only.


Q 4: To follow up, the PPS was told that it is okay to change those numbers with each quarterly submission. As an example, if the report is being submitted in Q3 can the numbers for Q2 be changed to reflect a more accurate picture, as long as it is explained in the text box?

A: In every quarter from DY1 Q2 forward the modules will be restructured so that the PPS can provide an update on the actual activities from the previous quarter, and a sense of the real-time activity. However, there will not be an opportunity to re-project over the five-year term. Any gap between the Q1 baseline projection and the actual activity should be explained in the text box for the benefit of the Independent Assessor.


Q 5: There are two modules that have a ramp-up for every project except for Domain 4. The first module outlines provider types and a set deadline for speed and scale. The other table associated with the Domain 1 Metrics and Milestones has a different timeline. Do the ramp-ups have to align?

A: For provider types specified in the Project Requirements, the ramp-ups have to match. E.g., if the ramp-up is associated with the systems transformation base which will be completed by DY2, then the earlier over-arching timeframe trumps the speed and scale commitment. The 2 ramp-up shown for a PCP level requirement needs to be accurate to the total numbers of PCPs committed to in the original Project Implementation Plan speed module. But the timeline should match the requirement itself as described in the Domain 1 Project Requirement Document from June 18th. The only time the ramp-up won´t match is if the report calls for a ramp-up for the second module that is specific to a subset of the total provider classification, such as safety net providers that are a subset of the total primary care physicians on the initial module. The safety net ramp-up would need to meet the commitment that was submitted specifically for safety nets in the speed and scale tables in January.


Q 6: Can the DSRIP team confirm that there is no risk mitigation for Domain 4 projects? Further, will there ever be one?

A: Confirmed. Those projects are designated for PPS engagement in existing prevention agenda activities, so there is no risk mitigation module for Domain 4. None is being planned.


Q 7: Can the DSRIP team confirm that there is nothing required for the workforce section with regard to Q1?

A: MAPP does not contain a section for workforce in Q1. It will exist in Q2, but is not required for submission until Q3.


Q 8: PPSs previously submitted speed and scale at the project level by provider type. Do they now have to provide that same information at the requirement level for those providers?

A: Information is only required for those providers specified in the second module. Clarification was provided 6 to 8 weeks ago by alleviating the requirements on speed and scale that would only measure certain project requirements specific to certain provider classifications.


Q 9: It was unclear prior to reviewing the format of the MAPP tool that the PPS had to submit a quarterly number for each of the provider level requirements. In addition to that, can the DSRIP team confirm that the PPS must also provide projected speed and scale for each one of those?

A: Confirmed.


Q 10: The PPSs have not previously been asked to submit expectations of provider commitments. Is that a purposeful inclusion for the submission due July 31st?

A: As far back as the project plan application provider speed and scale and safety net provider speed and scale commitments were submitted. Those correlate to the Project Implementation speed module in MAPP.


Q 11: The numbers in the total committed column have changed since January, when the PPSs were given the detail of each provider type. Will that level of detail be provided again so that the PPSs will know exactly who is counted in those numbers?

A: Yes, that effort is ongoing. Within the MAPP and Implementation Tool that will be released in October, there will be drill-down capability so that the PPS can understand where providers fit in the various classifications consistent with the table for speed and scale. The table is being updated in an effort to address the feedback received by the DOH from the PPSs. Further, if changes occur in the number of providers in a given category, the expected target percentage will be liberalized commensurate with those changes. There will be drop-down capability to select providers within categories so the PPS will be able to identify which providers are hitting the various categories, among which are the safety net, non-safety net, PCP, and non-PCP. E.g. If the PPS committed to 75 out of 100 providers, and the number of providers drops to 80, the PPS will not then be held to 75 out of 80. The percentage would be adjusted to reflect the suppression in volume.

To the first point of changing the numbers that are in MAPP for the module for total commitments, don´t miss the January speed and scale commitment. Contact your Performance Facilitator to have that double-checked.


Q 12: Will PPSs have an opportunity to revise the projections given that they won´t know who the players are until October? Should they use their best estimates for the ramp-up of numbers based on their timeline and Implementation Plan?

A: Use the best estimate for now and any necessary large scale revisions will be done on the back-end, likely in Q2.


Q 13: What flexibility will be afforded to PPSs given that their providers have until October to commit, which could impact the provider ramp-up tables? The PPS is concerned that the fluidity of commitment could result in duplicative work in the next two weeks to make the tables accurate.

A: The DSRIP team acknowledges the issue and is prepared to be flexible given that re-categorizing providers may change or reduce the actual number of available providers in a given category. If that happens it will not adversely affect the PPS. However, if there are no adjustments in that aspect, and the providers by category remains the same, the PPS will be held accountable for the commitments set forth in the application. The amount of points the PPS received in the application was a direct reflection of the PPS´s commitment around speed and scale, so there will be no flexibility in that regard.

It is important to note that the PPS will not be selecting individual providers until the next quarterly report in October, when there will be drop-down capability. That selection will be based on a new categorization from a new master list that will be maintained through the life of the project. New targets will be recalibrated in advance of that selection, the PPS will be cognizant of the target, will be getting a new set of providers, and will be selecting at that point. There is some time to think about the providers with whom they will partner.

As was previously stated, networks may be opened to allow PPSs to add providers, meaning they were not included in the PPSs´ initial network. The DSRIP team could explore the possibility with an eye to an October timeframe to ensure that speed and scale requirements will be met. The idea that providers can join a PPS at a later point has been anticipated, but not the reverse, where providers leave. The exception is structural changes which are going to be allowed at the midpoint set.


Q 14: On the table for budget and funds, it appears that only the waiver revenue line has been pre-populated, and it seems to reflect only net project valuation. In some cases, this is different from what the PPSs had filled in for the June 1st Implementation Plan. Could you give us all some guidance on how you prefer us to modify our budget and funds for the table to reflect any differences in that waiver revenue? For example, would you suggest changing all the table lines proportionately?

A: There is a discrepancy between the funding that has been displayed for waiver revenue and the federal fund. That is a known defect, and the DSRIP IT team is working to resolve it. For now, the information provided in "Budget and Fund Flow" should reconcile with the waiver revenue displayed. The amounts will be updated, but given the validations in the system you are required to reconcile your budget and your fund flow to the numbers populated in the tool. To be clear, that number is accurate for the purpose of performance money. It just doesn´t address all the equity polls, the high performance, etc. It will be updated to include those all-in dollar amounts on a performance basis. For now, it´s strictly project valuation, i.e. the net federal funds coming to the performance program.


Q 15: Some PPSs have included safety net equity guaranteed funding in the table. Please provide an update on the timing of when those guaranteed funds and the performance funds are planned to be distributed.

A: The mechanics are still being worked on so there is no firm date, but the hope is October 1st. It is understood that despite outstanding program integrity issues, the PPS leads have contractual issues to resolve with downstream providers about flow of funds. If there is a cash-flow issue PPSs should contact the DOH directly.


Q 16: On the Implementation Plan, if a work stream section has no feedback from the independent assessor, can the PPS assume that the plan outline was sufficient?

A: That is correct.


Q 17: Are there sections in MAPP that are supposed to be pre-populated?

A: In addition to the organizational milestones and project requirements, the PPS should be able to see the speed and scale commitment in the project implementation speed and patient engagement speed module. If those areas are not pre-populated or there is a conflict, contact your Performance Facilitator.


Q 18: Back in June it sounded like PPSs were only doing pre-completion reporting for the organizational section. Is that required for the July 31st reporting period or only subsequent periods?

A: The initial Implementation Plan did focus on the organizational sections and not all the projects. For the submission of the quarterly report that is due on July 31st, pre-completion reporting for both the project and organizational components of the prescribed milestones is required.


Q 19: Are PPSs required to submit pre-completion reporting for both the organization milestones and the project requirements or only the end progress requirements?

A: For organizational milestones adhering to the 4/21 Domain 1 Achievement Values Webinar, pre-completion reporting is required for all pertinent organizational prescribed milestones. For project requirements pursuant to filling out your projection and filling out all of the fields that are available, there are no expectations for additional reporting until the PPS gets to the point of completing the project requirements.


Q 20: Where should the PPSs add the subtext for organizational and project sections? Should they be added into the prescribed milestones modules of the PPS-defined milestones module?

A: If the subsets align and are associated under existing prescribed milestones, they should be added in with the associated milestone in the prescribed milestone module. If there is separate activity in terms of a milestone task subset that a PPS is conducting outside of the purview of the prescribed milestone, that should be reported within the PPS defined modules.


Q 21: What type of information should be added into the description field for those sub-steps?

A: There is a name field and a description field. The name field should be briefer than the description field.


Q 22: Has the state considered delaying the transition to dual factor identification until after this quarterly report is complete?

A: As included in the announcement, that date has been moved from July 29th to August 3rd.


Q 23: For pre-completion reporting, should the PPS be populating those updates in the narrative boxes tied to each milestone?

A: For pre-completion reporting that goes extends beyond the visible field, the PPS should use either the narrative box or the file upload functionality.


Q 24: Is the description field required for every task if the PPS does a sufficient job of describing the task within the task box?

A: Yes, the description field is required. The PPS could copy-paste the same information from the name field into the description field.


Q 25: The unit level requirements in the Project Engagement tables are tied to specific Domain 1 milestones which are part of the other module. Is this high level table just a roll-up, meaning that if there isn´t a unit level requirement in the project it would be left as "zero-engaged," like pharmacy for INTERACT for example?

A: No, the project implementation speed is the "big brother" of the provider level requirements. It adheres to the original provider classifications that were committed to for speed and scale on January 14th. For all the categories displayed, that table should show that actual ramp-up to hit the total commitment within the time frame that is displayed.


Q 26: How will the DOH be assessing and looking at the engagement requirements for some of the unit levels that are identified unless we show a data source or are not tied to project engagement numbers?

A: It is expected that for the partner requirements that were identified as project level, the PPS would report as a project-wide data source or project wide evidence. Whereas for the provider 7 requirements, provider unit level reporting would be through the individual providers as demonstrated in the Doman 1 Milestones and Metrics Document.


Q 27: Re. Project 2.d.i., "PAM Provider" is a unit level for the PPS to identify as a spread. Does training satisfy the requirement for PAM provider?

A: In January, PPSs identified how many individuals would be trained in the PAM active patient-engagement technique during the life of the project or by the time that was indicated. The DSRIP team expects that ramp-up to be consistent with the training totals to which the PPS committed in the January Project Application.


Q 28: Can some of the start dates that are populated be prior to April 1st?

A: No, the start date is April 1, 2015.


Q 29: The workforce section is not in the MAPP tool, but the PPSs receive feedback on it from the independent assessor. May it be assumed that feedback updates will not be addressed until the next submission period?

A: Yes, that is accurate.


Q 30: With regard to Domain 4 projects, the PPS will submit the milestones pre-defined in its December project plan application. May any changes be made, such as milestones being added or withdrawn, and is a fully developed set of tasks required for those milestones?

A: The answer to all three questions is yes. Provide the most accurate picture of the activities being planned and already accomplished. Reflect any changes to the December report in the module itself. It would be prudent to include a brief explanation of what has changed and why. Include any further milestones that are planned and the pertinent task associated with each. If the prescribed milestones encompass the entirety of the planned activities, then the module itself is sufficient.


Q 31: In the more detailed milestones provider types are listed, but Module 2 is more of a higher-level roll-up. How would the PPS be judged as having met all milestones, and what evidence should be provided?

A: Project-wide requirements often deal with training protocols, certifications, policies and procedures, or governance models that have been adopted. It would also include evidence of training that was conducted across all partners. Provider-Level requirements would include a list of PCPs achieving patients and a medical home level pre-certification, or other types of projects specific to provider units.


Q 32: If the PPS has a PCP-type provider and multiple milestones, but there is only one place for PCP in the higher level, is it appropriate to only provide the ramp-up that is most aggressive in that level?

A: The project implementation model should reflect the ramp-up aligned to the original, higher-level speed and scale commitment. The project requirement module should reflect more aggressive timelines.


Q 33: The description text box was not in the original template for the sections submitted on June 1st. If no remediation was noted by the Independent Assessor may those sections be left blank?

A: The description field is required. If they are left blank in the draft the document can be saved, but it cannot be saved and then submitted with blank required fields.


Q 34: Without the description box it is difficult to fill in the task box, especially if the solution is to cut and paste the description into the task. Has the DSRIP team considered this exercise from a work process flow?

A: The issue will be taken under advisement. At this point it is a required field.


Q 35: What is the differentiation between the statuses "on-hold" and "in-progress?"

A: If the date of the future activity is known, select the "in-progress" status, enter the actual date, and information will be saved in the document. Placing an activity "on-hold" causes the program to use the default start date of "prior to January 2015" and the default end date of "March 31, 2020." Those dates are indicators to the Independent Assessor that the activity is planned for some future date and that there is some blocker that prevents entering a specific "known date." Reports will be viewed from that perspective, with the idea that specific dates will be entered as they become clearer.


Q 36: May the tables for Roles and Stakeholders be uploaded as Excel spreadsheets or Word documents so that the work that went into creating them does not have to be duplicated?

A: Uploading documents is not currently an option. The issue will be taken under advisement.


Q 37: What time does the MAPP portal close on July 31st?

A: The MAPP portal will close at 11:59 PM.


Q 38: Will there be support personnel or a contact person available to answer questions in real-time? In addition, the MAPP portal is extremely slow. Is there someone who can address that issue?

A: Feedback on MAPP functionality is extremely valuable. Document any issues for the DSRIP team so that CMA can be contacted and asked to address them. CMA has been adding bandwidth resources to MAPP, as well as servers and process capability in an effort to improve the system. All PPSs are encouraged to submit reports as early as possible so that any problems can be resolved expeditiously.


Q 39: In the Budget and Funds Flow module the categories don´t match up with those submitted in the Implementation Plan. The PPS´s KPMG representative said that a "crosswalk" document is being prepared to explain discrepancies and the terminology used between MAPP and the IP. When will that be ready?

A: The DSRIP team is unaware of any such document. The categories do align, however there are two additional categories from DOH that will include the PPSs´ cost of non-covered services and "Other."


Q 40: In the webinar, for project 3.a.i. there is a list of ten sub-steps. Should the PPS use that level of detail as a guide for what the Independent Assessor would like to see as a clear action plan and include those sub-steps underneath the milestone?

A: No, those will be tasks under the milestone, not added in the narrative or as part of an uploaded document.


Q 41: Once a task is added, can it be changed or deleted over the five-year period?

A: PPSs will be able to change task names and descriptions as necessary over the five years, but tasks cannot be deleted. If necessary, a task can be put in "on-hold" status, but the Independent Assessor will require an explanation in the narrative box as to why that activity will not be completed. Tasks can be added as needed. The idea is to have an ongoing record over time that regularly updates the evolution of the workplan.


Q 42: The deliverables were added as sub-steps to the milestone in the MAPP tool, but the PPS did not originally prepare that in the Implementation Plan. Is the expectation that the PPS will provide a timeline for the deliverables for the specific milestone?

A: The metrics and deliverables have been loaded under each project requirement and a timeline should be provided by the PPS.


Q 43: Is the DSRIP team aware that there is an issue with saving numbers to the ramp-up table and other facets of the MAPP tool?

A: If PPSs notice any problems with MAPP functionality a ticket should immediately be submitted to the CMA Help Desk. Make sure to also inform the Performance Facilitators or Account Managers. Providing a short video or snippet of the screen may help to get the issue resolved more quickly.


Q 44: What is the best way to contact CMA?

A: The CMA Help Desk information, phone number and email address appear on the MAPP homepage when you log in.


Q 45: In the pre-completion reporting for July 31st, is the PPS required to submit a narrative for each organizational requirement that is in progress?

A: The submission due in July is the baseline implementation plan for projects, so no pre-completion would be expected.


Q 46: Please provide further clarification: based on earlier answers, the PPS not only provides all of the milestones and steps, but is also required to report on progress for anything that is pre-completion. Is the DSRIP team saying that the PPS does not need to do that in this initial baseline submission?

A: There may be some confusion as to terms in this instance. If the PPS is further along in work steps for a particular milestone in terms of meaningful use development, then the Independent Assessor would like that information included in the July 31st Implementation Plan. Many comments that came back from the original Implementation Plan stated that more information was needed from the PPS. The Independent Assessor needs evidence supporting any claim that the PPS has already embarked on the project and a description of what steps have been completed. That process is different from the "pre-completion" referred to in various webinars.


Q 47: It is understood that work that has been completed should be reflected in either uploaded documents or in the narrative section. However, the majority of milestones are in pre-completion status. Does the PPS need to provide narrative updates for each within the organizational section in this submission, or should that wait until October when they will be used as the baseline?

A: The Independent Assessor will expect to see that the organizational plan as well as 2.a.i. Implementation Plan would be revised to reflect some of the guidance or address the comments provided by the review team in the June evaluations. Documentation does not necessarily need to be provided, but communicate about any progress that has been made.


Q 48: There are project specific reporting requirements for number or roll-out commitments that are not phased in, but are more binary in nature and may happen all at once for a large group of providers. Early on PPSs were advised not to enter continuous quarters of zeroes for provider level commitments. Is that still correct, given that PCPs could come online all at once for RHIO connectivity or PCMH certification?

A: Work steps and sub-steps within each milestone were instituted to allow the PPSs to demonstrate a logical and appropriate fast-forward to meet those requirements, specifically in cases where all the medical villages "turn their lights on" simultaneously. The PPS can enter zeroes in the ramp-up for that type of project requirement, but it is expected that the entire ramp-up will be completed in accordance with the time frame, and consistent with the PPS´s commitments in speed and scale.


Q 49: In addition to "on-hold" and "in-progress," will there be another project status added to reflect "not started?"

A: The DSRIP team is aware of this issue or defect and is working toward a solution.


Q 50: Is it acceptable for the PPS to enter future dates and use "in process" to get the timeline started?

A: It is acceptable for the PPS to enter "in process" to delineate future activities through the start and end date.


Q 51: In the Patient Engagement module of MAPP tools there is an option to attach or upload a document. Will the DSRIP team please clarify what that document might be and whether it is mandatory?

A: If necessary, the PPS would provide documentation to attest to or provide evidence of reaching a certain level of patient engagement, which will come into play in future quarters. It will be mandatory to provide evidence demonstrating that patient engagement based off the definition previously provided in the Project Application.


Q 52: What are the requirements or what must be provided to document that a PPS has reached its target for Patient Engagement?

A: Technical guidance will be forthcoming around the types of reports and the template of how the Patient Engagement achievement will be demonstrated by the PPSs. Each project has a definition explaining how the PPS would determine which patients from your attribute population would be impacted or served by the project you are selecting. The second piece of that project requirement is that the PPS must establish the EHR or some type of IT platform that tracks patients engaged in each project.


Q 53: Is there a timeline established for when the PPSs may expect guidance on items like uploads, file formats, and other reporting tools referenced in this call?

A: The expectation is that those clarifying elements will be ready by mid-August. The data sources and types of activities that meet particular requirements for both organizational milestones and speed and scale have already been articulated.


Q 54: Can the DSRIP team be more specific about the technical changes to MAPP, specifically with regard to functionality and changes to the process? In particular, templates for on-boarding staff and allocating resources would simplify the process for PPSs, but the August timeframe is concerning.

A: PPSs´ input and requests for modifications to the MAPP tool are duly noted and appreciated. The DSRIP team will circle back with CMA to address them. However, the changes should not be expected by the July 31st reporting deadline.


Q 55: Though it was referred to earlier, there is not an option for a narrative or a description of task on the project side. Please clarify.

A: Throughout the application, on the organizational or project side, any information detail that needs to be clarified in narrative fashion or by uploading documents regarding tasks should be communicated at the milestone level.


Q 56: A lot of planning on many levels has gone into Patient Engagement and partnerships. Providing technical guidance in mid-August deters those organizational efforts. Can KPMG and PCG please begin to answer questions around those issues in a timely manner?

A: Every effort will be made to expedite the process.


Q 57: Which should be given more weight: comments from the Independent Assessor, or the nine-step example in the workforce section under the detailed gap analysis?

A: Give more weight to the example in the presentation.


Q 58: The dates in the detailed gap analysis extend out to DY2 Q3. Does that timing align with the DSRIP team´s perspective?

A: The dates were intended to be illustrative and not prescribed, so they need to be consistent with the PPS´s prior commitments.


Q 59: Since the workforce section does not currently exist in MAPP, are the modifications to the Implementation Plan for workforce still due July 31st?

A: No, they are not. The workforce section will become available in MAPP in Q2 and will require submission in Q3.


Q 60: The quarterly report submission has an associated Achievement Value. If the PPS accidentally skips a section and is not alerted to that fact by the MAPP tools internal edit check, will there be an opportunity to rectify that mistake before imperiling the associated AV?

A: The application itself should prevent submissions of reports with unfinished or blank sections at the point of the PPS lead attesting and submitting. However, failing that process, after 30 days of independent Assessor review, the PPS has 15 days for remediation.


Q 61: Will there be a checklist provided to PPSs for this report similar to the one the IAs or support team provided for the June report?

A: No, everything needs to be completed and the system has embedded tools to alert PPSs to errors in the reporting.


Q 62: How should the PPS handle speed commitments when the dates do not align with the required timeframe to meet the specific milestone that is part of the project? For example, if our commitment is later in time than the one mandated by the DOH, we must meet that earlier deadline. What happens if our commitment precedes the one DOH has mandated; may we adjust our timeline to the latter date?

A: Your own commitment to speed and scale trumps everything, regardless of prescribed milestones that may occur at a later date.


Q 63: In terms of activating our patients in 2.d.i., administering the PAM, scoring them, and engaging the patient in primary care community services, is it not necessary to report on a "re-PAM?" Further, in terms of overall activation, may we simply report where we believe that patient is within our system after one, or two, or three years?

A: No, according to CMS and to the 2.d.i. description, the emphasis is on ongoing engagement with these patients. It is not a one-time outreach. When the Patient Activation Measure is completed it is loaded into the PPS´s database as well as the large Insignia Database. That measurement will be monitored over time for the level of engagement and the change in activation score. This issue will be discussed further with the PPSs.


Q 64: How should tasks be entered into MAPP? It reverses the order, meaning that the second task is added above the first task and so on. Should they be in calendar order? Is that something that can be changed in MAPP?

A: As long as the start and end date of the activity reflect the correct chronological implementation of the activity, there will be no issue.


Q 65: Some tasks overlap, so should they be numbered as they are entered into MAPP to facilitate the Independent Assessor´s understanding?

A: Numbering tasks would work well. In addition, if the PPS knows how many steps will be added, the plus key can be hit as many times as necessary to enter individual lines per task, and the tasks may then be input in the appropriate order. The DSRIP team will review that procedure for future functionality.