EVALUATION TOOLKIT
About this toolkit
The research team of the Patient and Public Engagement Evaluation Toolkit project proposes to various health system stakeholders a wide range of tools to assist them in the evaluation of patient and public engagement initiatives, both in health research and in health care.
The Evaluation Toolkit is a resource designed for practitioners of the health sector, produced after the completion of a rigorous systematic review of patient and public engagement evaluation tools.
Choose a category to start
Q&A
How is this toolkit organized?
In a few clicks, the browser is designed to help users find the most relevant tool(s) to support their evaluation process.
The Table view provides a complete overview of the 27 tools for users who would like to compare tools or get a full picture of the toolkit content.
The List view presents all tools in a list or can be filtered to navigate a smaller number of tools sorted by context of engagement, namely health research or health care. Users can also sort tools by level of measurement, at the organization, project or participant level.
What can you find in the tool’s descriptive sheets ?
1) Icons help identify the type of tool. Below is the list of icons used to classify the tools:
 |
Checklist |
 |
Survey |
 |
Portfolio |
 |
Questionnaire |
 |
Scale |
 |
Toolkit |
 |
Index |
 |
Scorecard |
 |
Framework |
 |
Spider-gram |
2) The name, full reference, and objectives of the tool are simply outlined.
3) The tool assessment highlights the strengths and the weaknesses of the tools. An assessment grid can be found at the end of each descriptive sheet.
Tools that are available and open access can be downloaded directly from the toolkit website.
Unfortunately, many tools included in the Toolkit refer to articles describing the development or use of an evaluation tool but do not necessarily provide a copy of the usable tool itself. A note to that effect can be found on the descriptive sheet of these tools with reference to contact original authors, if necessary.
Also, some tools are available in a usable format but need to be found within longer reports or as an appendix of an article. A note mentioning the page or the exact location of the tool can be found on the descriptive sheet of the related tool.
This Toolkit is not…
Also, some tools are available in a usable format but need to be found within longer reports or as an appendix of an article. A note mentioning the page or the exact location of the tool can be found on the descriptive sheet of the related tool.
We created this toolkit because…
In Canada, opportunities for patient and public engagement are multiplying locally, provincially and across the country both in research and health care. With increased commitments to patient and public engagement have come calls for evidence-based engagement practices informed by robust evaluation. Engagement initiatives can help stakeholders to better understand the key conditions for success needed for such initiatives to transform research practices and health outcomes. In this context, the Patient and Public Evaluation Toolkit Project aims to improve engagement evaluation capacity at all levels of the health system.
Developed by
Health Care
Click on “List view” to enable filters (organization, participant or project)
Click on “Table view” to sort tools by features
| Title | Type | S | P | C | U |
| Patients as Partners in Research Surveys | Survey | 2 | 4 | 5 | 4 |
| Engaging patients as partners in practice improvement | Questionnaire | 3 | 3 | 2 | 3 |
| Scottish Health Council Participation Evaluation Toolkit | Toolkit | 1 | 4 | 4 | 5 |
| Organisational Self-Assessment and Planning (OSAP) Tool | Questionnaire | 3 | 3 | 5 | 4 |
| Scoresheet for the Tangible Effects of Patient Participation (STEPP) | Scorecard | 4 | 4 | 1 | 3 |
| Rifkin spider-gram | Spider-gram | 2 | 1 | 3 | 2 |
| Public and Patient Engagement Evaluation Tool (PPEET) | Questionnaire | 4 | 4 | 5 | 4 |
| Well Connected – a self-assessment tool on community involvement | Spider-gram | 4 | 3 | 3 | 2 |
| Checklist for attitudes for patients and families as advisors | Checklist | 0 | 0 | 1 | 4 |
| The Involvement Portfolio | Portfolio | 1 | 3 | 3 | 4 |
| PEI Engagement Toolkit | Toolkit | 2 | 3 | 4 | 4 |
| Measuring Organisational Readiness for patient Engagement (MORE) | Scale | 2 | 2 | 1 | 1 |
| A Scorecard for evaluating engagement | Scorecard | 1 | 4 | 3 | 2 |
| A resource toolkit for engaging patient and families at the planning table | Scale | 1 | 2 | 2 | 5 |
Health Research
Click on “List view” to enable filters (organization, participant or project)
Click on “Table view” to sort tools by features
Assessment Grid
We provide a five-point rating for each of the following four criteria:
- Scientific Rigour: Was the development of the evaluation tool scientifically rigorous and based on existing evidence on patient and public engagement?
- Patient and Public Perspective: Does the evaluation tool take into account the views of patients and the public (both in its development and use)?
- Comprehensiveness: Is the tool comprehensive in evaluating the context, process, outcomes and impacts of patient and public engagement?
- Usability: Is the evaluation tool easy to use?
Each criterion has 5 question-items. We gave 1 point per item if the answer to the question was YES, 0 points if the answer was NO or CANNOT ANSWER.
- Is the evaluation tool based on a comprehensive literature review on patient and public engagement research?
- Is the evaluation tool based on the experience/expertise of key stakeholders?
- Is the evaluation tool based on a conceptual/theoretical framework of patient and public engagement?
- Was the evaluation tool tested for validity (i.e., the tool evaluates what it is purported to evaluate)?
- Was the evaluation tool tested for reliability (i.e., the tool produces stable and consistent results)?
- Were patients and/or the public involved in the development of the evaluation tool?
- Is the tool designed to be completed by patients and/or members of the public (self-administered)?
- Does the tool explicitly state that the evaluation results must be reported back to patients and the public?
- Was the tool specifically designed to evaluate patient and public engagement activities?
- Does the tool capture the influence of patients and the public? (e.g., on the engagement process, on the final decisions, etc.)
- Does the tool document the context of engagement?
- Does the tool document the process of engagement?
- Does the tool document the outcome/impact of engagement?
- Does the tool monitor the engagement process at multiple moments?
- Does the tool consist of a set of open and closed questions?
- Is the purpose of the evaluation tool stated?
- Is the evaluation tool freely available?
- Is the evaluation tool available in an applicable format?
- Is the evaluation tool easy to read and understand?
- Is the tool accompanied by instructions for use?
Co-principal investigators
Antoine Boivin, MD PhD
Canada Research Chair on Patient and Public Partnership, CRCHUM
Co-director, CEPPP
Julia Abelson, PhD
Centre for Health Economics and Policy Analysis (CHEPA)
McMaster University
Research associate
Audrey L’Espérance, PhD
CEPPP
François-Pierre Gauvin, PhD
CEPPP
Research team members
Pascale Lehoux (Université de Montréal)
Vincent Dumez (Université de Montréal)
Ann C. Macaulay (McGill University)
Acknowledgment
This project was made possible by the financial support of the following regional Support for People and Patient-Oriented Research and Trials (SUPPORT) Units within the Strategy for Patient-Oriented Research (SPOR) of the Canadian Institutes of Health Research (CIHR) and partners: British Columbia, Alberta, Saskatchewan, Manitoba, Ontario, Québec, Maritime, and Newfoundland and Labrador. Thanks to Sujane Kandasamy, Benjamin Mathiot, Laura Tripp, and Archie Zhang for their assistance during the systematic review process and the toolkit development. Web design and conception of the online platform was done by Frédéric Tousignant.
Contact Us
Partners
An initiative of the Université de Montréal
Supported by
Direction collaboration partenariat patient
Faculté de médecine de l'Université de Montréal
Canada Research Chair in Patient and Public Partnership
CHUM Research Centre
Institut de recherche en santé publique de l'Université de Montréal
Canadian Foundation for Innovation