🧑🔧 Project Trigger & User's Problem
Meraki offers free Python programming education to young girls from low-income families in India. I joined Meraki when it had 100 students and was 6 months old. Meraki combines learning via self study material and online instructor-led classes.
🔬 Research Goals
The research aimed to uncover reasons for the following problems:
- Low engagement with Python course material despite regular live classes.
- Issues with class enrollment and attendance.
- Student's experience and challenges in class attendance, course material access and support.
🔬 Decisions Informed by this Research
- Converted single classes into a batch, resulting in 28 classes for Python with one-time enrollment.
- Integrated classes and course material within the same Python course.
- Investigating ways to improve interaction with code examples. One of them being an embedded code editor.
- Added practice questions to assess learned concepts.
🔮 Research Impact
- Content team improved strategy, adding images/videos and bite-size chunks, engagement increased by 87.5% as of March 2022.
- Batch classes introduced as trial, after research showed its necessity.
- Outreach team successfully pitched Meraki to schools and onboarded Haryana govt as a partner with 5000 students in December 2021.
- Meraki has adopted a user-centric approach and improved UX maturity through incorporating user feedback and research.
✨ My Role
I led research plan, managed end-to-end research process, and conducted 3 out of 7 interviews with two part-time design volunteers.
⏲️ Timeline
Approximately, the research project took about more or less four weeks.
- Discussing the project: 3 days
- Recruitment: 3 to 4 days (We began as soon as we got students to talk to and more recruitment happened in parallel)
- Conducting interviews: 1.5 week
- Analysis: 1-2 weeks
- Report: About 3 days
🛠️ Research Methodology
Methods used:
- User interviews
- Card Sorting
- Google Analytics reports on engagements, new user acquisition, dropouts etc.
User interviews were used because the team wanted to learn student behavior of using Meraki. 7 students were interviewed for 30 minutes each via Google Meets, with a team member present to take notes.
🧔 Recruitment Criteria and Process
The recruitment criteria for the research were:
- Baseline: Current Meraki users in NavGurukul's (Meraki's parent NGO) campuses
- Age: 15 to 19
Campus manager used Whatsapp group for easy coordination, and set up 1:1 sessions with the students.
❓ Sample Questions and Notes
Here is a link to the interview guide and notes. I used a semi-structured format with open ended questions and probed further according to the responses.
🕵️ Analysis and Synthesis Process
Held debrief sessions with design team, combined notes and identified critical pain points. Moved notes to EnjoyHQ and analyzed them with thematic analysis.
The main themes that emerged were:
- Class enrollment
- Manual work
- Practice questions for class topics
- Barriers to learning
- Time spent on Meraki
📜 Understanding the Data and Deliverables
From the themes, I worked on a journey map to label the pain points at various steps of the student's journey.
Major pain points were (These are not all pain points. Please check the detailed journey map):
- Difficulty finding relevant classes, and the need to schedule classes manually.
- Manual enrollment in all 28 Python classes.
- Complex steps to find upcoming classes.
- Text-heavy material and lack of images and videos.
- Inadequate explanations for code examples, constant switching between editor and course screen.
- Manual process for distributing and receiving practice questions via email.
Other deliverables included (Details in hyperlinks):
Student Personas
Personas were created based on two criteria: exposure to technology and learning style (work in progress). Personas evolved to archetypes represent overall group of users, focusing on problems with Python programming learning.
Persona 1: The Beginner in Tech
Person 2: Headstarter in Technical Knowhow
Application Map
Application map via insights from card sorting and interviews. The cards represented the information to be presented in the tabbed menu items from categories.
Insights from Google Analytics reports
While we asked the development team to look into Mixpanel, we created a few short surveys to catch student engagement. They would be given by the teachers during the classes. However, it did not work as expected as there was a lot of manual work involved. Due to several other priorities, Mixpanel implementation has been delayed.
Task Analysis (and their optimized version)
Six important tasks were identified that the student usually perform in their learning journey. The decisions to optimize them came from the insights from synthesis of data above.
⏭️ Next Steps and Recommendations
Monitored batch-based courses.
Iterating on user personas and task flows as we learn more about the students.
Create and maintain a user research repository in EnjoyHQ (the process is complete now).
💡 Reflections
What went well:
- Team support for UX research and desire to improve UX maturity in the org.
- Methodologies were appropriate. I was able to pick them up based on the time and budget constraints.
Challenges:
- Last minute scheduling difficulties, but mitigated by timeline planning.
- Design volunteers were new to UX design and needed help with multiple iterations of question generation but they quickly improved.