Patient and public engagement

EVALUATION TOOLKIT

About this toolkit

The research team of the Patient and Public Engagement Evaluation Toolkit project proposes to various health system stakeholders a wide range of tools to assist them in the evaluation of patient and public engagement initiatives, both in health research and in health care.

The Evaluation Toolkit is a resource designed for practitioners of the health sector, produced after the completion of a rigorous systematic review of patient and public engagement evaluation tools.

Choose a category to start

Q&A

How is this toolkit organized?
The research team, with the help of many patients and professional collaborators, organized information about the tools so users can easily navigate the complete list of tools and identify the characteristics most relevant for their work

In a few clicks, the browser is designed to help users find the most relevant tool(s) to support their evaluation process.

The Table view provides a complete overview of the 27 tools for users who would like to compare tools or get a full picture of the toolkit content.

The List view presents all tools in a list or can be filtered to navigate a smaller number of tools sorted by context of engagement, namely health research or health care. Users can also sort tools by level of measurement, at the organization, project or participant level.

What can you find in the tool’s descriptive sheets ?
A descriptive sheet was developed for each tool. When clicking on a tool of interest, the descriptive sheet gives users a broad overview of the tool to help them choose the tool(s) the most appropriate to their needs.

1) Icons help identify the type of tool. Below is the list of icons used to classify the tools:

 Checklist icon Checklist
 Survey icon Survey
 Portfolio icon Portfolio
 Questionnaire icon Questionnaire
 Scale icon Scale
 Toolkit icon Toolkit
 Index icon Index
 Scorecard icon Scorecard
 Framework icon Framework
 Spider-gram icon Spider-gram

2) The name, full reference, and objectives of the tool are simply outlined.

3) The tool assessment highlights the strengths and the weaknesses of the tools. An assessment grid can be found at the end of each descriptive sheet.

Tools that are available and open access can be downloaded directly from the toolkit website.

Unfortunately, many tools included in the Toolkit refer to articles describing the development or use of an evaluation tool but do not necessarily provide a copy of the usable tool itself. A note to that effect can be found on the descriptive sheet of these tools with reference to contact original authors, if necessary.

Also, some tools are available in a usable format but need to be found within longer reports or as an appendix of an article. A note mentioning the page or the exact location of the tool can be found on the descriptive sheet of the related tool.

This Toolkit is not…
Unfortunately, many tools included in the Toolkit refer to articles describing the development or use of an evaluation tool but do not necessarily provide a copy of the usable tool itself. A note to that effect can be found on the descriptive sheet of these tools with reference to contact original authors, if necessary.

Also, some tools are available in a usable format but need to be found within longer reports or as an appendix of an article. A note mentioning the page or the exact location of the tool can be found on the descriptive sheet of the related tool.

We created this toolkit because…

In Canada, opportunities for patient and public engagement are multiplying locally, provincially and across the country both in research and health care. With increased commitments to patient and public engagement have come calls for evidence-based engagement practices informed by robust evaluation. Engagement initiatives can help stakeholders to better understand the key conditions for success needed for such initiatives to transform research practices and health outcomes. In this context, the Patient and Public Evaluation Toolkit Project aims to improve engagement evaluation capacity at all levels of the health system.

Developed by

Health Care

Click on “List view” to enable filters (organization, participant or project)
Click on “Table view” to sort tools by features

PCORI ENGAGEMENT ACTIVITY INVENTORY (Net-ENACT AND WE-ENACT)

PCORI ENGAGEMENT ACTIVITY INVENTORY (Net-ENACT AND WE-ENACT)

The Involvement Portfolio

The Involvement Portfolio

An Evaluation of In-Person and Online Engagement in Central Newfoundland

An Evaluation of In-Person and Online Engagement in Central Newfoundland

Patients as Partners in Research Surveys

Patients as Partners in Research Surveys

Patients as Partners in Research Surveys

Patients as Partners in Research Surveys

Community engagement and participation in research measures

Community engagement and participation in research measures

Community Engagement in Research Index (CERI)

Community Engagement in Research Index (CERI)

Health Democracy Index

Health Democracy Index

Survey of Lay members of research ethics committees

Survey of Lay members of research ethics committees

Engaging patients as partners in practice improvement

Engaging patients as partners in practice improvement

Scottish Health Council Participation Evaluation Toolkit

Scottish Health Council Participation Evaluation Toolkit

Kroutil Checklist

Kroutil Checklist

Organisational Self-Assessment and Planning (OSAP) Tool

Organisational Self-Assessment and Planning (OSAP) Tool

ReseArch with Patient and Public invOlvement: a RealisT evaluation (RAPPORT)

ReseArch with Patient and Public invOlvement: a RealisT evaluation (RAPPORT)

Scoresheet for the Tangible Effects of Patient Participation (STEPP)

Scoresheet for the Tangible Effects of Patient Participation (STEPP)

Scoresheet for the Tangible Effects of Patient Participation (STEPP)

Scoresheet for the Tangible Effects of Patient Participation (STEPP)

Quality Involvement Questionnaire

Quality Involvement Questionnaire

Rifkin spider-gram

Rifkin spider-gram

Public and Patient Engagement Evaluation Tool (PPEET)

Public and Patient Engagement Evaluation Tool (PPEET)

Public Involvement Impact Assessment Framework (PiiAF)

Public Involvement Impact Assessment Framework (PiiAF)

Well Connected – a self-assessment tool on community involvement

Well Connected – a self-assessment tool on community involvement

Evaluating the participatory process in a community-based heart health project

Evaluating the participatory process in a community-based heart health project

Partnership Assessment In community-based Research (PAIR)

Partnership Assessment In community-based Research (PAIR)

Checklist for attitudes for patients and families as advisors

Checklist for attitudes for patients and families as advisors

The Involvement Portfolio

The Involvement Portfolio

PEI Engagement Toolkit

PEI Engagement Toolkit

Measuring Organisational Readiness for patient Engagement (MORE)

Measuring Organisational Readiness for patient Engagement (MORE)

A Scorecard for evaluating engagement

A Scorecard for evaluating engagement

A resource toolkit for engaging patient and families at the planning table

A resource toolkit for engaging patient and families at the planning table

Survey on consumers’ involvement in NHS research

Survey on consumers’ involvement in NHS research

Health Research

Click on “List view” to enable filters (organization, participant or project)
Click on “Table view” to sort tools by features

PCORI ENGAGEMENT ACTIVITY INVENTORY (Net-ENACT AND WE-ENACT)

PCORI ENGAGEMENT ACTIVITY INVENTORY (Net-ENACT AND WE-ENACT)

The Involvement Portfolio

The Involvement Portfolio

An Evaluation of In-Person and Online Engagement in Central Newfoundland

An Evaluation of In-Person and Online Engagement in Central Newfoundland

Patients as Partners in Research Surveys

Patients as Partners in Research Surveys

Patients as Partners in Research Surveys

Patients as Partners in Research Surveys

Community engagement and participation in research measures

Community engagement and participation in research measures

Community Engagement in Research Index (CERI)

Community Engagement in Research Index (CERI)

Health Democracy Index

Health Democracy Index

Survey of Lay members of research ethics committees

Survey of Lay members of research ethics committees

Engaging patients as partners in practice improvement

Engaging patients as partners in practice improvement

Scottish Health Council Participation Evaluation Toolkit

Scottish Health Council Participation Evaluation Toolkit

Kroutil Checklist

Kroutil Checklist

Organisational Self-Assessment and Planning (OSAP) Tool

Organisational Self-Assessment and Planning (OSAP) Tool

ReseArch with Patient and Public invOlvement: a RealisT evaluation (RAPPORT)

ReseArch with Patient and Public invOlvement: a RealisT evaluation (RAPPORT)

Scoresheet for the Tangible Effects of Patient Participation (STEPP)

Scoresheet for the Tangible Effects of Patient Participation (STEPP)

Scoresheet for the Tangible Effects of Patient Participation (STEPP)

Scoresheet for the Tangible Effects of Patient Participation (STEPP)

Quality Involvement Questionnaire

Quality Involvement Questionnaire

Rifkin spider-gram

Rifkin spider-gram

Public and Patient Engagement Evaluation Tool (PPEET)

Public and Patient Engagement Evaluation Tool (PPEET)

Public Involvement Impact Assessment Framework (PiiAF)

Public Involvement Impact Assessment Framework (PiiAF)

Well Connected – a self-assessment tool on community involvement

Well Connected – a self-assessment tool on community involvement

Evaluating the participatory process in a community-based heart health project

Evaluating the participatory process in a community-based heart health project

Partnership Assessment In community-based Research (PAIR)

Partnership Assessment In community-based Research (PAIR)

Checklist for attitudes for patients and families as advisors

Checklist for attitudes for patients and families as advisors

The Involvement Portfolio

The Involvement Portfolio

PEI Engagement Toolkit

PEI Engagement Toolkit

Measuring Organisational Readiness for patient Engagement (MORE)

Measuring Organisational Readiness for patient Engagement (MORE)

A Scorecard for evaluating engagement

A Scorecard for evaluating engagement

A resource toolkit for engaging patient and families at the planning table

A resource toolkit for engaging patient and families at the planning table

Survey on consumers’ involvement in NHS research

Survey on consumers’ involvement in NHS research

Assessment Grid

We provide a five-point rating for each of the following four criteria:
  • Scientific Rigour: Was the development of the evaluation tool scientifically rigorous and based on existing evidence on patient and public engagement?
  • Patient and Public Perspective: Does the evaluation tool take into account the views of patients and the public (both in its development and use)?
  • Comprehensiveness: Is the tool comprehensive in evaluating the context, process, outcomes and impacts of patient and public engagement?
  • Usability: Is the evaluation tool easy to use?
Each criterion has 5 question-items. We gave 1 point per item if the answer to the question was YES, 0 points if the answer was NO or CANNOT ANSWER.
  • Is the evaluation tool based on a comprehensive literature review on patient and public engagement research?
  • Is the evaluation tool based on the experience/expertise of key stakeholders?
  • Is the evaluation tool based on a conceptual/theoretical framework of patient and public engagement?
  • Was the evaluation tool tested for validity (i.e., the tool evaluates what it is purported to evaluate)?
  • Was the evaluation tool tested for reliability (i.e., the tool produces stable and consistent results)?
  • Were patients and/or the public involved in the development of the evaluation tool?
  • Is the tool designed to be completed by patients and/or members of the public (self-administered)?
  • Does the tool explicitly state that the evaluation results must be reported back to patients and the public?
  • Was the tool specifically designed to evaluate patient and public engagement activities?
  • Does the tool capture the influence of patients and the public? (e.g., on the engagement process, on the final decisions, etc.)
  • Does the tool document the context of engagement?
  • Does the tool document the process of engagement?
  • Does the tool document the outcome/impact of engagement?
  • Does the tool monitor the engagement process at multiple moments?
  • Does the tool consist of a set of open and closed questions?
  • Is the purpose of the evaluation tool stated?
  • Is the evaluation tool freely available?
  • Is the evaluation tool available in an applicable format?
  • Is the evaluation tool easy to read and understand?
  • Is the tool accompanied by instructions for use?

Co-principal investigators
Antoine Boivin, MD PhD
Canada Research Chair on Patient and Public Partnership, CRCHUM
Co-director, CEPPP

Julia Abelson, PhD
Centre for Health Economics and Policy Analysis (CHEPA)
McMaster University

Research associate
Audrey L’Espérance, PhD
CEPPP

François-Pierre Gauvin, PhD
CEPPP

Research team members
Pascale Lehoux (Université de Montréal)
Vincent Dumez (Université de Montréal)
Ann C. Macaulay (McGill University)

Acknowledgment
This project was made possible by the financial support of the following regional Support for People and Patient-Oriented Research and Trials (SUPPORT) Units within the Strategy for Patient-Oriented Research (SPOR) of the Canadian Institutes of Health Research (CIHR) and partners: British Columbia, Alberta, Saskatchewan, Manitoba, Ontario, Québec, Maritime, and Newfoundland and Labrador. Thanks to Sujane Kandasamy, Benjamin Mathiot, Laura Tripp, and Archie Zhang for their assistance during the systematic review process and the toolkit development. Web design and conception of the online platform was done by Frédéric Tousignant.

Contact Us

Partners

Logo of the CEPPP (pied de page)
Shares