Conducting A Needs Analysis: Case Of The Disappearing Learners

Engaging Learners By Conducting A Needs Analysis

Meet Sadie, an Instructional Design Manager for a large cell phone retailer with 15,000 salespeople. In her annual review, she noticed that learners are not consuming as much training content as she would like. To understand why, she decides to conduct a needs analysis to determine what learners need and what they are missing. She wants to fully engage learners, set them up for success, and help the company meet its business goals.

Let’s follow Sadie as she tries to resolve the case of the disappearing learners!

Preliminary Research

First, Sadie makes a plan.

• State the business objective: Sadie will need to keep the objective front and center throughout her research so she can regularly refer back to it. Stated simply, Sadie needs to increase course completions among her salespeople.
• Choose a research method: She’ll use surveys and interviews. On-the-job observations would also be a good choice, but they take a lot more time, and she wouldn’t get as much employee feedback.
• Write research question(s) that align with the business objective: For example, why is the learner rate of completion lower than the goal metric?
• Write a problem statement: Such as “Learners are not completing enough of the training content.”
• Write the working proposition (or hypothesis): Sadie’s proposition is that learners don’t have enough time.

Once Sadie has identified her hypothesis, she can begin her methods of research starting with Level 1 Surveys.

Surveys

Sadie first conducts a Level 1 Survey to gather high-level information from a large number of participants. Online surveys make this process efficient, plus the raw data is easy to analyze for trends.

Level 1 Survey questions focus on the learners’ reactions to training using a rating system like a Likert scale. For example, “How confident are you in your ability to sell? How likely are you to complete more courses that have been assigned to you? How easy was it for you to access the courses in the learning system? Based on the courses you have completed, how engaged were you with the subject matter?”

Tips:

1. Send the survey with a clear deadline.
2. Send reminder emails.
3. Include incentives!

Sadie knows that timing is important in surveys. Surveys sent on Mondays and Tuesdays tend to get the most responses, so she sends her survey early in the week and informs participants that it will remain open for two weeks. She sends reminder emails one week after the initial invitation and again on the final day. Sadie also mentions that every participant is entered to win a gift card, knowing that this will increase the number of responses.

Sadie’s Level 1 Survey shows some interesting results. Several participants mention having trouble logging into the LMS to complete the courses. Some said the content in the courses was too difficult.

Sadie anticipates discussing these issues during the focus groups, but next Sadie will conduct empathy interviews.

Empathy Interviews

Sadie’s next step in the needs analysis process is empathy interviews, which are helpful in getting to know the learners and creating a learning persona. She’ll conduct these interviews one-on-one with a representative sample. She chooses five participants who represent the typical learner and schedules 30-minute interviews. A transcription tool like Tactiq will help her by automatically transcribing the interviews, leaving Sadie free to pay attention to participants. It also provides textual data to refer back to.

During the interviews, Sadie plans to get to know her learners on a personal level with the following questions:

• What do you like to do outside of work in your free time (hobbies, interests, etc.)?
• What is your greatest strength? Weakness?
• What motivates you?
• What are your career goals?
• What frustrates you?
• How do you define success?
• If you could sell any product in the world, what would you sell and why?

These answers may be challenging to analyze, but they help you build a persona of the learners, which helps when determining a creative learning strategy.

Sadie finds that her learners enjoy listening to podcasts, watching streaming services, playing video games, and competing in sports. With this information, she creates personas for her typical learner, which will help her write and design content with a specific audience in mind.

Focus Groups

Sadie intends to use focus groups to ask job-related questions and gather more qualitative feedback about the existing training and job responsibilities.

Sadie plans to conduct three focus group interviews in small groups of five to eight people. Although she knows that they can be done simultaneously with the empathy interviews, she schedules them a week or two after and with different, randomly selected participants so that she has time to carefully craft the questions based on the personas derived from the empathy interviews.

A note on content validity: The purpose of a needs analysis is to gather useful information; to avoid having extraneous data to analyze, so Sadie drafts her interview questions and then circles back to her business objective and research questions (remember the questions created at the beginning of the needs analysis?) to verify that the questions asked in the interview can be linked to one or more of these research questions.

Sadie’s questions include:

• How would you prefer to receive job training?
• What are the biggest reasons why you do or don’t complete training?
• What do you like/dislike about the current training you receive?
• Did you receive enough relevant practice in the training you completed?
• If you were to receive training in a game format that allowed you to compete against your peers would you be more likely to complete it?
• Think about your most successful customer interaction. Why was it so successful?
• Think about your most difficult customer interaction. Why was it so hard?

The focus groups provide additional interesting and useful feedback. Learners appreciate the aesthetics of the training, but they are often redundant and time-consuming. Their time is limited and they would prefer job aids or microlearnings. This is the reason most cited for not completing the offered training. Their most successful customer interactions occurred when they received training timed with a store event, and they were least successful when the training came after an event.

Now that Sadie has all of this great data, it’s time for her and her team to analyze the information and create a learning strategy.

Analysis

To analyze the data, Sadie looks for trends not only in numbers (quantitative data) but also in what the participants said (qualitative data).

Sadie’s surveys provide a lot of quantitative data reflecting learners’ satisfaction with the current training options. Overall, Sadie’s learners are satisfied, but their answers reflect Sadie’s hypothesis that the eLearning program could be improved. Sadie pays special attention to areas that received lower ratings when she analyzes the qualitative data from the interviews.

Analyzing the interview data might prove a bit more difficult. Sadie follows these steps:

1. Read the transcripts thoroughly.
2. Color-code and highlight recurring words and phrases.
3. Categorize and chart recurring phrases by putting each participant’s number identifier on the vertical axis and the categories on the horizontal axis. If a participant mentions the category, put a mark in the appropriate column.
4. Finally, revise the problem statement (created at the beginning of the needs analysis) based on the quantified data.

Sadie gathers her team and briefs them on the results so far. Use an online whiteboard to conduct a divergent thinking session during which everyone thinks independently by posting virtual sticky notes to the board with ideas to solve the problem. This process is called “blue skying” because no idea is too grand, too expensive, or too outlandish—everything is a good idea!

Some ideas that the team came up with are to overhaul the entire training catalog, to use text messages to push tidbits of information, and to create simulations for practice.

Next, Sadie and her team ask “How might we…” questions to solve their problem. These ideas should begin to consider what is realistic for their situation.

For example, “How might we create content based on experience level without redoing all courses?”

Finally, the team switches to convergent thinking, where they determine which ideas are both the best for solving the problem and the most feasible. Some blue sky rules carry over in this step in that the team members can only say, “yes and…”. Saying “no” is not allowed. They use these ideas to determine the most appropriate suggestions for new creative learning solutions.

When the team is done, they have a strong learning strategy and a plan for creating better training that their learners are more likely to engage in.

Conclusion

This full-spectrum approach to need analysis provides a depth of research and allows Instructional Designers to make decisions on creative learning solutions for both content and delivery. For example, what should the tone of the training be? Do learners prefer social or independent, synchronous or asynchronous learning? What metaphors or game mechanics might be relevant to them based on their hobbies/interests? What topics would they like to explore further based on their career goals, weaknesses, and strengths?

Training is more personalized now than ever before. The more we know about our learners and the more flexibility and customization we can provide, the more engaged they will be.