Population & Community Health Unit

Service |  Information |  Research |  Education |  Network  

PCHU Reports Series

Copies of our Research Reports are available upon email request. Please email your request to pchu@uwo.ca

Evaluation of a Cold/ Flu Self-Care Public Education Campaign

Evaluation of the Lawson Foundation Granting Approach and the Development of a Knowledge DIffusion and Utilization Model for Social and Health Programs

Healthy Mothers, Healthy Babies (Argyle): A Formative Evaluation of Year One

Injury & Road Safety Research: The Australian Experience

Smart Start for Babies: A Prenatal Advantage Program: An Intermediate Evaluation

What Factors Predict Adolescent Self-Rated Health and Health Care Utilization?

Process Evaluation of the Hamilton HSO Mental Health and Nutrition Program

Evaluation of a Cold/ Flu Self-Care Public Education Campaign

E. Vingilis, U. Brown, R. Koeppen, B. Hennen, M. Bass, M. Stewart, K. Payton, and J. Downe.
December 1994

The Ontario Ministry of Health (MOH) cold/flu self-care public education campaign to reduce unnecessary patient visits to doctors is evaluated. The MOH campaign consisted of an information booklet delivered to every household in an Ontario city, newspaper ads and radio spots. The program ran Jan. - March 1994. The evaluation consisted of: 1) 2x2 telephone survey in London (experimental area) and Windsor (comparison area), before and during the campaign, and 2) a telephone survey of London family practitioners.

In addition, data on the incidence of cold/flu visits to three hospital emergency departments and to a sample of family physicians' offices were gathered. The data suggest that program rationale may have been questionable because the majority of the surveyed public were knowledgeable and self-reported appropriate doctor visits for cold/flu. Campaign evaluation showed limited impact. Message penetration was low; only one-third of London residents knew of the campaign or read the booklet. Only two of ten questions showed increases in knowledge in London and no changes were found for beliefs, attitudes, acquisition of new health practices or self-reported visits to the doctor. The physician survey, emergency room and family physician office visit data were consistent with the public survey findings.

Top of Page

Evaluation of the Lawson Foundation Granting Approach and the Development of a Knowledge Diffusion and Utilization Model for Social and Health Programs

E. Vingilis and S. Lindsay
Funded by the The Lawson Foundation
April 2001
Also, The Lawson Foundation has "Toward the Development of a "Know-How" Knowledge Diffusion and Utilization Model for Social and Health Program. An Evaluation of The Lawson Foundation Granting Approach"
by E. Vingilis and S. Lindsay available.

In the 1990's a new attitude emerged within both the public and corporate sectors, which ushered in an era of increased accountability, responsibility, and fiscal belt-tightening. This attitude of increased accountability and fiscal responsibility to stakeholders and the public has also been adopted by philanthropic organizations who began asking the difficult question of whether or not they and the programs of the non-profit organizations that they were supporting were really "making a difference." The recent focus on whether philanthropic organizations and programs in general were making a real difference spurred a wave of evaluation publications including philanthropic evaluation publications. The key themes of these publications were on why and how to evaluate. Yet within the theme of "making a difference" was a second element which was summarily mentioned, if at all in these publications. If philanthropic organizations and the programs that they support are to "make a difference," then not only must the programs be found to work in their own local communities, but also the knowledge gathered from these programs must be shared with others and the knowledge used by others. That is, the knowledge obtained on whether a particular program "made a difference" locally needs to be diffused to and utilized by a wide range of communities.

Thus, evaluation of whether philanthropic organizations and the programs they support are "making a difference" should include not only evaluation of program outcomes but also evaluation of diffusion and utilization of knowledge. Unfortunately, despite the importance of diffusion and utilization of knowledge, very little recent research has been conducted on "how" knowledge is diffused and utilized, although increasingly its importance is being acknowledged. Most evaluations are still outcomes based and do not assess if and how knowledge was transferred and used. Thus, researchers, practitioners, public and private funders do not have a full understanding of the actual methods and processes involved in successful knowledge diffusion and utilization. Principles may be known, but the mechanisms are not.

The general goal of this study is to develop an in-depth understanding of the process of knowledge diffusion and utilization and to explore how it is done, using a multiple case study approach.

Top of Page

Healthy Mothers, Healthy Babies (Argyle): A Formative Evaluation of Year One

G. Schmidt
Prepared with the members of the Health Mothers, Healthy Babies Evaluation Committee: U. Brown, C. Francis, J. Richardson and E. Vingilis
October 1996

Healthy Mothers, Healthy Babies (Argyle) is a project for the prevention of low birth weight in London's Argyle community. One of the project's emphases is on the use of a community development approach to build the Argyle community's capacity to plan, implement and evaluate their own solutions to the area's prevalence of low birth weight outcomes and related concerns.

The formative evaluation presented in this report was undertaken to provide the program's stakeholders with a description of the program's delivery, and to identify aspects of the program to target for improvement. The evaluation also addresses program impacts in a very preliminary fashion.

Highlights of the evaluation findings include the following:

  • The program as planned and delivered is based on a clear rationale, supported by research literature and grounded in the identified needs of the Argyle community.
  • Three types of factors (distal/structural, situational/personal, and organizational/administrative) challenged the implementation of the Healthy Mothers, Healthy Babies program within the first year of its operation. The findings of this evaluation included observations of "buy-in" at the community level, and of satisfaction among program participants, despite the challenges identified in both the structural and personal domains. These are good signs, especially given the program's short history within the Argyle community.
  • The program has relatively strong nutritional supplementation/prenatal education, prenatal support, and outreach components.
  • Program records are incomplete and lack comparability on information gathered for earlier and later program participants.
  • These and other findings presented in this report support the following recommendations for the Healthy Mothers, Healthy Babies program:

  • The program should continue to provide support for the basic needs of the target population, and therefore the program should maintain its focus on prenatal nutritional supplementation and education. The program should, however, attempt to cover behavioural issues as far as possible.
  • The program should continue to evolve as necessary to meet the changing/emergent needs of participants and the Argyle community.
  • The program should continue to place a high priority on outreach activity;
  • Program delivery should continue to be tailored to the culture of participants;
  • The program stakeholders should collectively be involved in reconsidering the data items and protocols for data collection to inform routine monitoring of program activities and ongoing evaluation of Healthy Mothers, Healthy Babies program, taking advantage of opportunities for future evaluation described in this report;
  • The program stakeholders should continue to work toward smoothing out the program's functioning at the administrative and organizational level, looking at the issues of communication and mutual expectations;
  • The program stakeholders should plan for continuous growth in the program numbers of program participants; and;
  • Program stakeholders should ensure that evaluation is integral to the design and implementation of modifications to the program.
  • Top of Page

    Injury & Road Safety Research: The Australian Experience

    E. Vingilis
    April 2003

    In Australia both the health and transportation sectors focus on injury prevention and control. Although the federal and state transportation sectors have traditionally focussed on injury prevention and control and developed national and state strategies for injuries and road crashes, in the last two decades the health sector has also developed national and state injury prevention and control priorities. The different injury prevention strategies documents exemplify a strong focus on evidenced-based strategies, forging of continued partnerships and collaborations between researchers and non-researchers and an acknowledgment of the importance of continued research. This strong research focus may be partially attributed to the various commissions established on injury prevention that included researchers and representatives from the National Medical and Health Research Council (NHMRC).

    Based on the number of researchers and projects that have and are being funded within the governmental, university, and commercial sectors, it seems that substantive resources are available to conduct research on injury prevention and control. Importantly, injury seems to be an identifiable research theme in Australia. Within in NHMRC, injury is identified as a distinct and independent research area, although one centre director suggested that the injury focus is more apparent than real. Federal and state governments also fund research as do a number of state-run insurance commissions. Additionally, creative funding exists: for example, revenues from speed camera fines.

    What is perhaps most unique to Australia is the large number of university-based research centres engaged in injury prevention and control research, education and other activities, creating a critical mass of researchers. There are at least 22 research centres/units/institutes in Australia and New Zealand. Additionally there are injury research programs in trauma departments, government departments, public health units, transport and other associations, organizations, etc., and individual 'lone wolf' researchers in universities and as private consultants. An important point in support of research centres is that they are perceived to be "helping reverse the 'brain drain'". Moreover, these centres are involved in undergraduate, graduate and post-graduate education and training, thereby producing the next generation of injury and road safety researchers.

    Knowledge translation is a key feature. These centres not only engage in investigator-driven research but also in many collaborative applied research and program/service/policy development, implementation and evaluation activities. The centres seem to have successfully developed and maintained strong research and consultancy profiles and collaborative research linkages and activities with government, other universities, and commercial organizations. The solitudes between research and practice, between universities and government were somewhat less apparent.

    Is this applied research and other activities accepted within the university culture? Feedback from centre colleagues indicates a certain degree of acceptance of applied research and related activities. As one centre director stated, their universities are "much less ivory tower". However, he also cautioned that it was "a double-edged sword". The provision of a government base funding meant some contractual obligation to the government that consequently caused some "tension between investigator-driven versus client-driven" research. However, there seems to be evidence of university and academic acceptance of applied research as indicated by the fact that the universities provide some resources to the centres, a variety of funding opportunities including an identified funding theme within NHMRC and the centres are well featured and regarded within university documents.

    The quality and quantity of applied research and activities is seen as a point of pride within many university administrations and funding agencies. These centres were perceived to provide very positive public relations value as the public and political sectors can understand and appreciate research, education and other community-based activities that are focussed on a major health and safety problem. In addition, the types of research conducted by these centres have more current and direct impact on injuries. That is, compared to basic research for which applications may be more difficult to determine, some injury prevention and control research can demonstrate their impact over a shorter time period and can be more directly visible in terms of reductions in injuries and their consequences.

    Interestingly, academics play significant roles on government committees in shaping priorities, policies and practices and in public and community education activities. Posters, billboards, newspaper articles, electronic media presentations, etc., are common. Moreover, Australia is renown for a number of road safety and other successes.

    In summary, there is a large and vibrant community that is reasonable well supported and seems integrated with and informing government and other practices and is training the new generation of researcher. Perhaps the large critical mass of researchers has been instrumental in driving the public and political agenda on road safety and injury prevention and control. Time will tell whether their investment in research and training will assist Australia in meeting their national injury and road safety targets. Certainly their investment in research looks promising.

    Top of Page

    Smart Start for Babies: A Prenatal Advantage Program: An Intermediate Evaluation

    J. Sarkella
    Prepared on behalf of the Smart Start for Babies Evaluation Committee: D. Lemaire, E. Pellerin, A Pope and E. Vingilis
    October 2000

    In July 1994, Health Canada announced the creation of a number of "Canada Prenatal Nutrition Programs" (CPNP). These are comprehensive community-based programs that provide support to pregnant women who face conditions of risk that may threaten the health and development of their babies. Smart Start for Babies: A Prenatal Advantage Program (formerly Healthy Mothers, Healthy Babies) is one of these programs, and began operation in September, 1995.

    The Smart Start for Babies program was designed to reduce the incidence of low birth weight babies delivered by women at risk of poor pregnancy outcomes, and also to improve the health and well-being of these women and their infants. This evaluation describes the characteristics of the program clientele and their babies, evaluates pre and post program changes and analyses the correlated and indicators of low birth weight.

    It appears that the program is serving some high-risk clients, with one in three clients aged 18 years or less, the majority smoking during pregnancy, a substantial proportion having fewer than 12 years of education and being of single marital status, and almost half living in a household with a monthly income less than $1,000.

    On average, clients made 19.6 visits to the program, with an average of 9.48 prenatally and 9.85 postnatally. One in ten clients visited the program only once. Some of the main reasons clients gave for not attending the program more frequently were program timing, distance and transportation. Given that the program is currently expanding, this may alleviate some of the difficulties clients have faced in the past with attending.

    The mean birth weight for babies born to clients in the program was 7.26 pounds. Eighteen babies were low birth weight, constituting 8.9% of all births. Most clients indicated prenatally that they were planning to breastfeed their new baby, touting it as best for the baby, inexpensive and convenient. 85% , were in fact breastfeeding their baby upon discharge from the hospital, and almost half continued to breastfeed at program exit. The program was cited as the most common source of breastfeeding advice/encouragement.

    Top of Page

    What Factors Predict Adolescent Self-Rated Health and Health Care Utilization?

    E. Vingilis, T.J. Wade and J. Seeley
    Funded by Health Canada through the National Population Health Research and Development Program
    November 2000

    Adolescence is the period during which lifestyle patterns of behaviour, such as tobacco, alcohol and other drug use, nutrition, physical activities, etc., are being formed. These behaviours set the stage for future health problems and health care utilization, as risk-taking and health compromising lifestyles are major causal factors for many health problems.

    Furthermore, adolescence is the period of rapid physical and psychosocial changes, a period during which youth become more aware of their bodies and become more introspective. It is also a period of optimal physical health as youth have the lowest rates of disease and death in the western world. During this time, health appraisals are being shaped which may more represent youth's overall sense of psychosocial functioning than their physical functioning. As other researchers have indicated, the tendency to relate general life difficulties to health problems may reflect the underlying patterns of expressing life distress in physical or somatic rather than psychological terms and may explain the consistent patterns of international findings on the relationship between self-rated health and socioeconomic status, family functioning, social supports, etc.

    Despite the fact that adolescence seems to be a crucial period for the formation of lifestyles and perceptions of health, virtually no information is available on the determinants of adolescent self-rated health. Yet self-rated health may be one doorway to understanding health care utilization. As other researchers have found, self-rated health is among the best predictors of patient-initiated physician visits.

    The purpose of this study was to examine what factors predict self-rated health and health care utilization among adolescents. The study was based on data from the two-time period (1994 and 1996) longitudinal National Population Health Survey (NPHS), which used random sampling to select 19,600 households in the first time period from across Canada. The response rate at Time 2 was 93.6%. This study included 1,493 adolescents who were between 12 and 19 at Time 1 interview. Based on Social Cognitive Theory which suggests that personal and socio- environmental factors interact leading to health compromising or health enhancing behaviours, which in turn, affect psychological health status, which affects personal perceptions of health and health care utilization, we examined demographics (sex, age, grade), structural environment (family structure and income), physical health status (disability and chronic health problems), social factors (social support and social and school/work involvement), lifestyle behaviours (exercise, body mass weight, smoking and drinking), and psychological health status (self-esteem, stress and depression). These variables have all been found to predict various measures of self-reported health and health care utilization.

    The results of these analyses found that adolescent perceptions of health are framed not only by their physical health status but also by personal, socio-environmental, behavioural and psychological factors. Not surprisingly, physical health status and change in physical health were the strongest predictors of self-rated health. Yet, adolescent self-rated physical health seems to involve components other than physical health problems. For example, although disability at Time 1 was associated with Time 1 self-rated health, changes in disability status did not affect Time 2 self-rated health. This finding would suggest some stability for self-rated health over time.

    The results support previous research that certain personal and socio-environmental factors increase the vulnerability of adolescents by influencing their lifestyles and psychological distress. In addition, these factors influence physical health ratings and health care utilization. Personal factors of age and sex were found to be predictive of self-rated health. Controlling for health status, increases in age became significantly associated with decreases in self-rated health. While this study focussed only on a two-year interval, it does lend support to the idea that some change in perception also occurs as one ages. As found in other studies, females consistently rated their health lower than males. This consistent finding of lower female health self-ratings warrants further investigation to identify whether these findings are due to physical or psychological health problems.

    Socio-environmental factors also have direct effects on self-rated health. Consistent with previous research, income is an important predictor of self-rated health, with lower income related to lower health self-ratings. Moreover, changes in income over time were associated with changes in self-rated health. The analyses also indicate the importance of social supports and involvement on perceptions of personal health.

    Consistent with Social Cognitive Theory, behavioural lifestyle factors, namely smoking, physical activity and body mass weight were found to influence adolescent self-rated health. As expected, adolescents who smoke and have higher weight rated their health lower. Furthermore, the results suggest that lifestyle factors determine health self-ratings, as changes in smoking and weight status at Time 2 significantly affected Time 2 health self-ratings. However, an unusual finding was that highly active adolescents rated their health more poorly. Whether this finding is due to adolescents who commonly receive injuries because of their activities or who are highly concerned about their health and appearance and are active because of their concern remains to be explored.

    Similar patterns of predictors of self-rated health were found for physician and non-physician (nurses, chiropractors, physiotherapists, social workers/counsellors, psychologists, speech, audiology and occupational therapists) utilization. Although physical health status was a strong determinant of physician and nonphysician utilization, lifestyle, psychological distress and self- rated health also predicted utilization. Among the social factors, only school/work involvement was negatively related to utilization which could mean that school or work-involved adolescents have fewer problems or more commitments which limits time to access services. Dental service utilization, however, showed fewer and different predictors. Contrary to other health care utilization, income was a determinant of dental service utilization. The fewer predictors of dental care overall compared to either physician or nonphysician utilization could reflect the fact that dental problems may be less susceptible to a global sense of well-being.

    Overall all sets of factors appear to have implications for both self-rated health and utilization. These findings support the contention that self-assessment of health is an active process involving thoughts and emotions in addition to physical problems. The good news is that income does not prevent access to health care. Clearly those with health problems or disability use the health care system the most. However, income does predict dental care, clearly a point of concern for low income families. Another important finding is that adolescents do access the health care system. Common belief has been that adolescents, especially those engaging in risky lifestyle behaviours such as smoking do not go to physicians and other health care professionals. Yet this study found that females visited physicians on average about four times a year and males about two times a year. Furthermore, adolescents with poor lifestyles are accessing the health care system more. This means that the health care system could be an important source to targeted and age appropriate counselling on lifestyle issues such as smoking and body weight maintenance.

    In summary, this study suggests that although adolescents access the health care system for physical health problems, they also access the system for lifestyle and psychological problems. Addressing personal, socio-environmental and psychological concerns at earlier stages and by different methods may reduce subsequent health care utilization.

    Top of Page

    Process Evaluation of the Hamilton HSO Mental Health and Nutrition Program

    J. Paquette-Warren, E. Vingilis, J. Greenslade, S. Newnam
    Funded by Ministry of Health and Long-Term Care (Primary Health Care Health Fund Program)
    August 2004

    Objectives: Shared care involves the collaboration among health care practitioners with various expertise to deliver the most appropriate patient care. Key elements of shared care have been described; however, its nature and practitioner responsibilities within shared care are unclear. This study aimed to develop a logic model for and conduct a process evaluation of an existing shared care model.
    Design: A mixed-method, multi-measures evaluation design was used. Data were gathered from the program's central patient database, results of internal studies, and by conducting focus groups with practitioners. Setting: The study was conducted at the central office and practices of the Hamilton Health Service Organization (HSO) Mental Health and Nutrition Program located in the Hamilton, Ontario, Canada.
    Respondents: The nine members of the program's central management team as well as 53 practitioners were involved in the evaluation.
    Results: The practitioners work in interdisciplinary teams to provide comprehensive mental health and nutrition care in a primary care setting. Quantitative data showed a wide range of patients assessed, treated, and referred among team members. The qualitative data demonstrated extensive collaboration and education opportunities for practitioners and patients. Also, the program's central management team (CMT) and practitioners collaborate in program quality improvement.
    Conclusions: The evaluation highlights the complexity of this shared care program, describes increased access to expanded health care provision, illustrates the collaboration among multiple practitioners, and outlines the critical role of a CMT in implementing and maintaining fidelity of a program to its intended objectives.

    Process Evaluation of the Hamilton HSO Mental Health & Nutrition Program Reports and Program Logic Models

    Top of Page

    Home | About Us |  CAREMH |  Family Medicine  

    The Population & Community Health Unit, Family Medicine,
    Schulich School of Medicine & Dentistry, 2nd Floor,
    Dr. Don Rix Clinical Skills Building, London, ON, Canada N6A 5C1
    Telephone: (519) 858-5063, Fax: (519) 661-3878,
    The University of Western Ontario