Office of Review and Evaluation

The Office of Review and Evaluation (ORE) has recently been established within the Department of Premier and Cabinet (DPAC) to support evidence-based decision-making and a culture of continuous improvement across the Tasmanian Government.

The ORE is leading the development of an outcomes-focussed, whole-of-government evaluation framework to strengthen evaluation practice across all agencies.

The ORE is responsible for:

Providing a source of centralised, overarching guidance and support to agencies and state service practitioners in relation to program review and evaluation approaches;
Working in close collaboration with agencies to develop a Tasmanian Government Evaluation Framework, including principles, guidelines and tools to ensure a consistent approach to evaluation within the State Service;
Building a culture of continuous improvement and evaluation;
Leading the evaluation of significant whole-of-government programs for government.

What is Evaluation?

Evaluation refers to the systematic assessment of the effectiveness, efficiency, appropriateness and/or overall value of a program, policy or service. Evaluation encompasses a range of practices and approaches that are understood and applied in diverse ways.

Evaluation involves aspects of design, monitoring, research, reporting and outcomes measurement. Throughout the program/policy cycle, evaluation practices and approaches are used to inform decision-making, accountability, learning and continuous improvement.

There are many sources of evaluation related information and resources, including:

Australian Evaluation Society

Better Evaluation

Other Australian jurisdictions have evaluation centres or units:

New South Wales Government Centre for Program Evaluation

Northern Territory Government Program Evaluation Unit

Australian Government – Evaluation in the Commonwealth

Evaluation practice and culture

There are many aspects of evaluation practice and culture, ranging from reflection and learning in day-to-day work, through to the development and implementation of formal evaluation methodologies for government policies, programs and services.

The table below includes different aspects of evaluation practice and culture, along with examples of what this can ‘look like’ in public service design, delivery and decision-making.

Aspect of practice and culture	Examples
Evaluative thinking	Reflective practice Critical thinking (asking ‘why?’) Identifying lessons
Evaluation as part of program design, management and delivery	Using evaluative approaches in program design (theory of change and program logic) Embedding monitoring, evaluation, reporting and improvement in program planning and management Collecting data throughout program delivery
Planning and conducting evaluations	Designing and implementing formal and systematic evaluation processes Triangulating different sorts of evidence to make evaluative judgments (about appropriateness, effectiveness, efficiency and/or overall value) Identifying evaluation findings and recommendations Reporting, communicating and using evaluation findings and recommendations (contributing to decision-making and accountability)

Evaluation – key questions

There are many approaches to evaluation, but it usually involves questions related to three key aspects: outcomes, process and learning. Essentially the key questions and considerations are:

Have we achieved what we set out to do?
Could we have done things better or more efficiently?
Should we continue to do this or try something else?

Evaluations can be undertaken at various levels (such as project, program or organisational levels) and can happen at all stages (such as the design phase, throughout delivery and end-of-program).

Types of evaluation

There are four common types of evaluation:

Process evaluation which considers program design and initial implementation (often undertaken within 18 months of program commencement);
Outcome evaluation considers program implementation (often after more than 2 years) and short- to medium-term outcomes;
Impact evaluation considers the extent to which the program has contributed to medium-to longer-term outcomes (often after more than 3 years);
Economic evaluation considers a program’s value for money through cost benefit analysis or other economic analyses at various stages of program implementation.

In practice, there is often cross-over between the various types of evaluation.

Priorities

The ORE is consulting with Tasmanian Government agencies to develop an Evaluation Framework. Most Australian jurisdictions already have evaluation frameworks that include policies, guidelines and toolkits. Some examples include:

Australian Government Evaluation Policy

Australian Government Evaluation Toolkit

NSW Government Program Evaluation Guidelines

NSW Evaluation Toolkit

Northern Territory Government Program Evaluation Framework

ACT Government Evaluation Policy and Guidelines

Queensland Government Program Evaluation Guidelines

Western Australia Program Evaluation Guide

The New Zealand Government has compiled a plain English handbook to support evaluation practice:

Making sense of evaluation: a handbook for everyone