Understanding ‘The Monitoring and Evaluation Plan’

October 4, 2021

A monitoring and evaluation (M&E) plan is a document that helps to track and assess the results of the interventions throughout the life of a program. It is a living document that should be referred to and updated on a regular basis. While the specifics of each program’s M&E plan will look different, they should all follow the same basic structure and include the same key elements.

An M&E plan will include some documents that may have been created during the program planning process, and some that will need to be created new. For example, elements such as the logic model/logical framework, theory of change, and monitoring indicators may have already been developed with input from key stakeholders and/or the program donor. The M&E plan takes those documents and develops a further plan for their implementation.

 

Why develop a Monitoring and Evaluation Plan?

It is important to develop an M&E plan before beginning any monitoring activities so that there is a clear plan for what questions about the program need to be answered. It will help program staff decide how they are going to collect data to track indicators, how monitoring data will be analyzed, and how the results of data collection will be disseminated both to the donor and internally among staff members for program improvement. Remember, M&E data alone is not useful until someone puts it to use! An M&E plan will help make sure data is being used efficiently to make programs as effective as possible and to be able to report on results at the end of the program.

 

Who should develop a Monitoring and Evaluation Plan?

An M&E plan should be developed by the research team or staff with research experience, with inputs from program staff involved in designing and implementing the program.

 

When should a Monitoring and Evaluation Plan be developed?

Monitoring and Evaluation plan should be developed at the beginning of the program when the interventions are being designed. This will ensure there is a system in place to monitor the program and evaluate success.

Who is this guide for?

This guide is designed primarily for program managers or personnel who are not trained researchers themselves but who need to understand the rationale and process of conducting research. This guide can help managers to support the need for research and ensure that research staff have adequate resources to conduct the research that is needed to be certain that the program is evidence based and that results can be tracked over time and measured at the end of the program.

Steps

Step 1: Identify Program Goals and Objectives

The first step to creating an M&E plan is to identify the program goals and objectives. If the program already has a logic model or theory of change, then the program goals are most likely already defined. However, if not, the M&E plan is a great place to start. Identify the program goals and objectives.

Defining program goals starts with answering three questions:

  1. What problem is the program trying to solve?
  2. What steps are being taken to solve that problem?
  3. How will program staff know when the program has been successful in solving the problem?

​Answering these questions will help identify what the program is expected to do, and how staff will know whether or not it worked.

Example: If the program is starting educational skill uplift for girls, the answers might look like this:

Problem High rates of unskilled Girls
Solution Skill enhancement of girls in respective community centers
Success Lowered rates of unskilled Girls in locality

From these answers, it can be seen that the overall program goal is to reduce the rates of unskilled and unaware girls in the community.

It is also necessary to develop intermediate outputs and objectives for the program to help track successful steps on the way to the overall program goal.

Step 2: Define Indicators

Once the program’s goals and objectives are defined, it is time to define indicators for tracking progress towards achieving those goals. Program indicators should be a mix of those that measure process, or what is being done in the program, and those that measure outcomes.

Process indicators track the progress of the program. They help to answer the question, “Are activities being implemented as planned?” Some examples of process indicators are:

  • Number of training held with Girls
  • Number of outreach activities conducted at girls-friendly locations
  • Number of courses taught at girls-friendly locations
  • Percent of girls reached with program awareness messages through the media or WOM

Outcome indicators track how successful program activities have been at achieving program objectives. They help to answer the question, “Have program activities made a difference?” Some examples of outcome indicators are:

  • Percent of girls turning up for course in first round
  • Number and percent of trained peer leaders from that from local community  providing services to girls
  • Number and percent of new girls enrolling in course and also, those turning up for jobs with latest skills acquired

These are just a few examples of indicators that can be created to track a program’s success.

Evaluating the performance over time:

Usually a Quasi-experimental design of pre and post comparison of the same group is applied i.e. the treatment group. The individual score will determine the status of girls’ like in our example; score on a specific time i.e. the score at pre time or post time. The difference between the two will determine the progress of girls from the pre to the post situation. This difference will be calculated through single difference and will be considered as impact of the program.

Step 3: Define Data Collection Methods and Timeline

After creating monitoring indicators, it is time to decide on methods for gathering data and how often various data will be recorded to track indicators. This should be a conversation between program staff, stakeholders, and donors. These methods will have important implications for what data collection methods will be used and how the results will be reported.

The source of monitoring data depends largely on what each indicator is trying to measure. The program will likely need multiple data sources to answer all of the programming questions. Below is a table that represents some examples of what data can be collected and how.

Information to be collected Data source(s)
Implementation process and progress Program-specific M&E tools
Service statistics Facility logs, referral cards
Reach and success of the program intervention within audience subgroups or communities Small surveys with primary audience(s), such as provider interviews or client exit interviews
The reach of media interventions involved in the program Media ratings data, broadcaster logs, Google analytics, omnibus surveys
Reach and success of the program intervention at the population level Nationally-representative surveys, Omnibus surveys, community data
Qualitative data about the outcomes of the intervention Focus groups, in-depth interviews, listener/viewer group discussions, individual media diaries, case studies

Once it is determined how data will be collected, it is also necessary to decide how often it will be collected. This will be affected by donor requirements, available resources, and the timeline of the intervention. Some data will be continuously gathered by the program (such as the number of training), but these will be recorded every six months or once a year, depending on the M&E plan.

After all of these questions have been answered, a table like the one below can be made to include in the M&E plan. This table can be printed out and all staff working on the program can refer to it so that everyone knows what data is needed and when.

Indicator Data source(s) Timing
Number of training held with Community Girls Training attendance sheets Every 6 months
Number of outreach activities conducted at Girls-friendly locations Activity sheet Every 6 months
Number of course taught at youth-friendly locations Subject Sheet / teachers manual Every 6 months
Percent of girls receiving program messages through the media or WOM Population-based surveys Annually
Percent of prospective girls willing to take skill enhancement program population-based survey Annually
Number and percent of peer leaders providing trainings to Girls Facility logs Every 6 months
Number and percent of new enrollments population-based survey Annually

Step 4: Identify M&E Roles and Responsibilities

The next element of the M&E plan is a section on roles and responsibilities. It is important to decide from the early planning stages who is responsible for collecting the data for each indicator. This will probably be a mix of M&E staff, research staff, and program staff. Everyone will need to work together to get data collected accurately and in a timely fashion.

Data management roles should be decided with input from all team members so everyone is on the same page and knows which indicators they are assigned. This way when it is time for reporting there are no surprises.

An easy way to put this into the Monitoring and Evaluation plan is to expand the indicators table with additional columns for who is responsible for each indicator, as shown below.

Indicator Data source(s) Timing Data manager
Number of training held with Community Girls Training attendance sheets Every 6 months Activity manager
Number of outreach activities conducted at Girls-friendly locations Activity sheet Every 6 months Activity manager
Number of course taught at youth-friendly locations Course sheet Every 6 months Activity manager
Percent of girls receiving program messages through the media or WOM Population-based survey Annually Research assistant
Percent of prospective girls willing to take skill enhancement program population-based survey Annually Research assistant
Number and percent of peer leaders providing training to Girls Facility logs Every 6 months Field M&E officer
Number and percent of new enrollments population-based survey Annually Research assistant

Step 5: Create an Analysis Plan and Reporting Templates

Once all of the data have been collected, someone will need to compile and analyze it to fill in a results table for internal review and external reporting. This is likely to be an in-house M&E manager or research assistant for the program.

The Monitoring and Evaluation plan should include a section with details about what data will be analyzed and how the results will be presented. Do research staff need to perform any statistical tests to get the needed answers? If so, what tests are they and what data will be used in them? What software program will be used to analyze data and make reporting tables? Excel? SPSS? These are important considerations.

Another good thing to include in the plan is a blank table for indicator reporting. These tables should outline the indicators, data, and time period of reporting. They can also include things like the indicator target, and how far the program has progressed towards that target.

Step 6: Plan for Dissemination and Donor Reporting

The last element of the M&E plan describes how and to whom data will be disseminated. Data for data’s sake should not be the ultimate goal of M&E efforts.  Data should always be collected for particular purposes.

Consider the following:

  • How will M&E data be used to inform staff and stakeholders about the success and progress of the program?
  • How will it be used to help staff make modifications and course corrections, as necessary?
  • How will the data be used to move the field forward and make program practices more effective?

The Monitoring and Evaluation plan should include plans for internal dissemination among the program team, as well as wider dissemination among stakeholders and donors. For example, a program team may want to review data on a monthly basis to make programmatic decisions and develop future work plans, while meetings with the donor to review data and program progress might occur quarterly or annually. Dissemination of printed or digital materials might occur at more frequent intervals. These options should be discussed with stakeholders and your team to determine reasonable expectations for data review and to develop plans for dissemination early in the program. If these plans are in place from the beginning and become routine for the project, meetings and other kinds of periodic review have a much better chance of being productive ones that everyone looks forward to.

How to Counter Fraud in Market Research?

June 10, 2021

Fraud and misrepresentation of reports is the common norm, which erodes elements of transparency in research activity. If proper checks are not set in place, such loopholes can be catastrophic in decision making phase because it relies upon data gathered during the research process.

In Pakistan, research ethics and professionalism is the salt sold very cheap. No standards are in place to guide and curb such sheer violation of research basics. Data is infringed with biased, wrong and irrelevant responses. Least efforts are made to ensure data verifiability and authenticity. According to an estimate, 25-30% fraud rate is obvious in all the research activities.

By reviewing data conformity to objectives on each level can mitigate unwanted practices but this idea is not practical where scope is too high.

Some of the common survey frauds

  • Same surveys are performed multiple times by one respondent – Duplicate respondent error

Unique IP address and link to email accounts can minimize this error also, using one single platform to submit responses will help us in getting unique submissions. Also, at times it can be accidental, Accidental “fraud” occurs when survey participants are not aware they are taking the same survey multiple times. This happens when online surveys do not block duplicate entries. Another source of accidental errors can occur when a survey respondent is a member of multiple online panels that are all sending respondents the same survey. But in all cases, these respondents are not actually committing fraud. They are simply good people accidentally participating more than one time in the same survey.

On other hand, there is this group committing Intentional fraud, it occurs when a survey participant deliberately tries to complete a survey multiple times, or provides inaccurate answers to survey questions. The motives are either to alter the results of the survey or to reap extra financial incentives. These individuals routinely join and participate in as many online panels as they can. They have their browsers reset browser cookies and history after each survey attempt, make common use of “incognito modes,” and utilize VPN services (commercial and free) to circumvent any geographic restrictions. Online surveys that offer bigger incentives tend to attract more intentional fraud. These are individuals you definitely do not want in your online surveys.

Minimizing Fraud – Protection mechanism

  • Research targets such as, households or retail branches when not visited, or doubted as not visit for this purpose – Geo Tagging is used. The first component of a fraud defense is geo-location. Geo-location can be provided by the device itself (mobile devices, etc.), by a previous data point created by survey participants when signing up, or via their computer’s IP address. Geo-location or tagging results are generally trusted and acceptable at the country level.
  • Another widely used method that is used to track survey participants is a device/browser fingerprint. This “digital” fingerprint is built up from components and properties of the browser, such as fonts installed, plugins registered, screen size, color depth, and many other variables that uniquely identify a computer, tablet, or smartphone.
  • Another strategy is to put questions (i.e., cheater traps) in an online survey with nonsensical answers that might trap a participant who is rushing through a survey.
  • Cookies are also used to track respondents across and within online surveys. Cookies are widely used to prevent duplicate survey entries and are easy to implement.
  • To reach right audience or diffuse chances of fraud, random calls are made to assess the authenticity of online and offline surveys.
  • At the same time, Ace research has devised one of the most comprehensive Quality check process. This ensures optimally the integrity and accuracy of data at all levels. It minimizes human errors by putting in place review mechanism.
  • All the reports and submission if cross checked with underlined instruction shared with shoppers and surveyors – with zero tolerance policy for any deviation also helps in mitigating fraud.

Other ways to minimize fraudulent responses

Two less common and more difficult to interpret components of a fraud defense are survey metadata and open-end verbatim analysis. Survey metadata includes details like how long survey participants took to complete a particular survey or question, how long they spent writing an answer to an open-end question, or how many times they changed their answers. Consequently, determining what constitutes fraud from metadata requires some human legwork and tends to vary survey by survey.

The same is true of the analysis of open-end questions. While sentiment analysis is commonplace now, it is fraught with misclassifications for anything but the simplest of cases. Going a step further and performing more sophisticated analysis of open-ends to identify fraudulent responses is even more difficult. However, there are some simple things that can be done to help check for fraudulent open-ends. Checking for gibberish, repetitive answers, and off-topic responses can help identify fraud.

How to Make Money as Field Surveyor?

February 12, 2021

Market research allows a company to discover the target market and get feedback from consumers about their interest in the product or service. Some are face to face interactions and may demand traveling (require car or motorbike) whilst others can be held remotely as long as you have a reliable internet connection, laptop and mobile.

Market research involves the gathering and analyzing information of consumers. This helps businesses understand what products/services people want, who will buy them, and for what price.

If you think that you’d enjoy talking to strangers, and getting their opinions about various products and services, read our simple getting on board FAQs.

WHAT DOES A FIELD SURVEYOR JOB INVOLVE?

COULD I BE A FIELD SURVEYOR?

WHERE CAN I FIND FIELD SURVEYOR JOBS?

HOW MUCH DOES FIELD SURVEYOR EARNS?

WHAT TRAINING CAN I TAKE TO BE A FIELD SURVEYOR?

GET YOURSELF A FIELD SURVEYOR JOB NOW

Market Research Activities involve numerous assignments such as mystery shopping, retail audits, field surveys, focus group discussions, in-depth interviews, consumer surveys etc. Collectively, we recognize them as Field Surveys or on-ground research operations.

WHAT DOES A FIELD SURVEYOR JOB INVOLVE?

You might be approaching people in the street, door-to-door, offices, retail shops, public places or on the phone. Once employed, you’ll typically receive a few days’ training before the required activity. You may also be accompanied by an expert on your first task.

If you’re going door to door asking questions, you’ll probably be given a list of streets to visit, each with a set quota of interviews to conduct. Alternatively you may have a pre-selected list of addresses to contact. These may contain the name of a specific person you need to speak to.

Your activity might involve observing a retail outlet for product or service assessment. Observation might require you not to reveal your identity which often is the case in mystery shopping. (You will be required to visit target site and observe, as per the checklist provided but you will remain anonymous and pretend to be regular customer or services seeker.

Similarly, you might be tasked to visit public places to conduct surveys or may be retail offices to observe customer services and product.

WHAT YOU NEED TO KNOW
  • Be polite and approachable. Always explain what the research is about and what it’ll be used for.
  • Make sure to record your answers. Take down the answers as you receive them. Then once the results are collated, they’re passed back to the organization you’re working for.
  • Researches are usually paid after the completion of assignment. Some companies will also reimburse travel expenses and may offer bonuses to those who meet their targets.
  • Read more about different market research assignments and tasks

DIFFERENT TYPES OF MARKET RESEARCH

There are two different types of market research:

QUANTITATIVE RESEARCH

Quantitative research is based on the quantities or numerical statistics collected by surveys and questionnaires. Mostly the questions are multiple choice.

  • Face to Face Interviews: You’ll be asked to attend a short meeting to answer a series of questions about a product or service. Surveys are normally conducted face to face.
  • Online Surveys: You’ll answer a series of questions about a product, brand or service through a website or by phone. These can be of many different variations. Some may want to put some visuals in front of you for feedback and opinions; others may want you to explore a website with a specific task in mind.  In some cases, you’ll need to fit a certain ‘demographic’ to take part in a particular test. This means that you’ll need to be a particular age, sex, or live in a certain place. All you need to do is take a few moments to fill in the qualifier test.
  • Consumer Researches and Surveys: Market researches are basically executed to analyze consumer needs and preferences. Also, needs of end user is important; this can be customer, businesses, capacity building of target audience – as it happens in assessments of surveys and long term community uplift projects. In all of this, surveyors play a role on forefront in for data collection, recording and sharing it in the required format, based on this data reports are made and ultimately decisions too. 

QUALITATIVE RESEARCH

Qualitative research is based on more in-depth questions, determining people’s actions and thoughts by researching their attitudes and opinions. This type of research often takes place in groups or face to face. They may discuss a certain topic or collection of topics in a session. Participant’s opinions and views will be analyzed and recorded as it is.

  • IDI (In-depth Interviews): One to One interaction will take place and questions related to product or services are asked. Also, interviews are conduced online as well. 
  • Focus group discussion: You’ll be asked to discuss your opinions about a brand, product or service in a small group of people.

COULD I BE A FIELD SURVEYOR?

This is a job that involves interacting with different people of different industries and being interested in what they have to say.

Good Field Surveyor use positive, action-focused language, they also make it easy for participants to complete their survey. You need to be able to hold their interest for the whole survey and keep the results honest. This means not influencing the answers in anyway. You’ll also need to be able to deal with rejection from some people.

Most of all, it’s important to enjoy what you do and have fun being out and about. It tends to be a popular job for younger people, those with some relevant employment experience tending to have a better chance of securing work. Otherwise, with some training and experience you can secure paid research assignment/tasks.

  • Flexible Hours

There is no upper age limit to be a Field Surveyor. However it can be a flexible option for those with other commitments. Also, in most of the market research assignments you are not bound to commit yourself 9-5 but you can perform your task as per your convenience. When you get the required training, time duration for the task will be communicated and then you can manage the task as per your convenience.

WHERE CAN I FIND FIELD SURVEYOR JOBS?

The Ace Research is the professional and experienced employer for the Market Research sector and you’ll find the whole activity managed with utmost flexibility and convenience of researcher/surveyor in mind.

The Ace Research also publishes its blog for detailed insights on our research activities and also we post our vacancies across social media and informal job portals.

HOW MUCH DOES FIELD SURVEYOR EARNS?

Market Research

In all of the varying market research activities mentioned above, on average a field surveyor earns PKR 20 – 50k a month. With experience and dedication earning five digits a month from survey activities is doable. There can be 3 categories of field surveyor;

1. Telephonic Interviews and Online Assessment

The joy of having a telephone/online-centric job is that you can be based anywhere. So jobs are available at home or at a call center.

2. Door-to-Door Surveys

Doing field research is certainly more profitable, particularly in big towns. Outside main towns, the rate is different. Your travel cost is sponsored in most of our research activities, in some cases; an expense allowance for each assignment is also awarded.

3. Retail Assessments/ Branch Visits

Such activities involve sometimes a field surveyor to have distinct characteristics such as; to be well groomed, command over certain language, to belong to a specific location/region and often some prior experience (can be acquired with training and pilot visits). Often it is required to have motorbike or car for carrying out an assessment.

WHAT TRAINING CAN I TAKE TO BE A FIELD SURVEYOR?

If you want to improve your earnings as a Field Surveyor, think about taking some training, which may also open the way to more interesting roles.

  • Studying towards qualification

Various institutes offer various online and on campus certification in market research, which gives a comprehensive grounding in the basic principles and practices of effective market and social research. From there you can work towards Advanced Certificates, Diplomas and even Masters Degrees which will obviously put you in a completely different league and earnings bracket.

Besides having desirable certification or diploma – market research companies also offer short trainings free of cost to interested individuals.

GET YOURSELF A FIELD SURVEYOR JOB NOW

Register yourself with Ace Research if you believe you are consistent, hardworking, a good listener and also dedicated to meeting activity timelines. Ace Research provides complete training to all the shortlisted candidates who will pass our short interview and you could get a market research job down your street (literally) right now.