User Testing 2nd round - Takeaway
- Artun Gürkan
- Jonas Bergmeier
- Martha Vasquez
Task |
| Test User 4 - Observation | Test User 5 - Observation | Test User 6 Observation | Test User 7 Observation | Action items |
|---|---|---|---|---|---|---|
Intro | What is your name and age? | Arvid, 30 | Jaime, 34 | Sebastian Frank, 28 |
|
|
| Could you please tell us about the tasks of your job and your job title? | Digital embessator for Vietnam and consulting/supporting in regard to digital transformation | UX designer in Gofore | Data Analyst Operational Support, UN, office undersecretary general |
|
|
| How they usually research new technology | He uses different channels, such as news letter for example certain news channels in youtube, LinkedIn and twitter) The biggest difficulty is time. For example ChatGbt is blocked in Vietnam which he needs to use VPN different phone number. Availibility and time is main challenge for the user to find new technology. | Throughout the internet. Google, youtube and sometimes forms. | Twitter, Reddit, just surfing also classical newspapers
|
|
|
| Did you participate in User Tests before? |
| Yes. Conduct and tested. | yes, 2 |
|
|
| Knowledge regarding Govstack | The user know what is Govstack and he is also working in digitalization process. | No prior knowledge | no previous Govstack knowledge |
|
|
Landing and Intro | Understanding GovStack and general intro feedback |
|
|
|
|
|
Onboardingpage feedback | What’s your first impression and what can and would you do here now? | positive opinion regarding Human-centered digital services He does not understand what we mean by Building block. For him he understand the concept, It founds it little bit complex due to building block. For the politicians have no clue regarding BBs it might be problematic. Visual can be Improved. He just jump to the simulation
He went back to onboarding page but after instruction he started to scroll down.
Landing page is very high meta level.
Several use cases would be helpful. Question is if the govstack is useful for single individual or management departments also.
He could not related to USCT use case | Quite a lot information to process. What he understand from the brief: this is mostly focus to build an application to made a building block.
| expects that this is targeted to users that already know such things, steep learning curve? “using “sustainable” is stretching it there” text is floating, not sure to what it is connected is content connected together? he’s insecure standards? what does is mean? cash transfers → he finds it very interesting, worked on social welfare in M.A. use case could be promoted more prominent sees steps → “ok, this is great” steps don’t make it totally clear that it takes both perspectives what’s this simulation for… also likes the section wonders why the alert box content is so prominent content simulation screenshot has a weird position in his opinion |
|
|
| Expectation |
| What BBs are question: Prefabricated modules to design a case, it would be a application of the government. He asummed that the product is designed for the designers. | “Almost like a trial version of the real thing” Screenshot helped building expectation for people who want to try it out and experience the services |
|
|
| Could you please start the simulation? |
|
|
|
|
|
| OTHER |
|
|
|
|
|
Tutorial | How did you feel about the pacing and content and onboarding of the tutorial? | He click the tutorial via restrart accidently. He checks the tutorial. He couldn't find the close tips button. | He checks the tutorial and thips read thoroughly. he understand the tips and purposes. He is too much considering UX aspects instead of the actual product. (such as being blur etc) He understand the difference between civil servant and applicant. Close help part was confusing. |
|
|
|
| How confident do you feel in your ability to use the simulation after completing the tutorial? |
|
| Seeing in background what he expected “would lower the transparency” otherwise really helpful (help) misses icons for the roles in the help “why is there an A?” misses something “behind it” assumes DIAL is a partner |
| close action
|
| OTHER |
|
|
|
|
|
Simulation | Experience it and think out loud what you’re thinking. | Website looks clean, the thought that he was civil servant at the beginning. He easily check the wireframes. He took the civil servant role, he understand via the wireframes. He did not understand the contextual panel (he thought it was tasks again)
He does not have a clue how does simulation refers to Building block (Explain the building block library on the onboarding page)
He want to see how this use case can be applicable for his own use case. What Building Block does he need for his own use case? He does not need use case close to his need he wants to understand how he can use the Building blocks. He sees that the use case can be adjusted to their own work but he does not understand the Building Block. He does not understand how he can adjust for he needs.
He want to see the technical requirements. This can be something that his employees might use. When he goes to landing page he wants to see that if this thing can fulfill the software requirements. Open-source sounds a lot of work. He wants to see what IT team would implamant, (more information) (Managment/ data security/ user software) He did not understand the building blocks again from the contextual panel. Later at the simulation, he saw the building blocks from side panel. How do they configure that? He would be most interested in workflow. How do I define certain workflows, where do I get additional registries. How do I set this up that is the question?
I am not interested in click through the simulation. Youa re asking for a lot of ability for abstraction.
| There are many information for the user especially for someone who has no previous knowledge. He thought that civil servant and applicant was button. He thought it was about the design.
He needs some time to understand and play with it to understand what this buttons and what these elements are inside.
He uses Building Block DIAL catalogue to learn more. He clearly connected the overlay/A B C button to the wireframes. He is thinking that he should edit the page and interact with the building block to change the wireframes. Information structure very clearly. He does not know what does it mean active and inactive. (But he thought that the BB something to interact) He said that If they are in use he would write how many of them are in use
He understand that it has changed the applicant view (but he does not know why)
My information part (citizen portal) confusing.
What payment building block questions; He check the overlat + DIAL (WIN)
After more experiencing the product it became more clear that the role of the Building Blocks. He clearly understand what payment Building Block does.
| “exactly what I expected” was surprised that he can scroll assumes that BBs can be cross used because of the interaction explanation green line “almost looks like progress” “guess they are active” “assuming this is a very versatile BB…” (Digital Reg.), after reading DIAL explanation confused by defintion of security BB (as a civil servant) not sure, if this would be end product that civil servant would see would assume CS is trained and explanation of program could be hidden thinks program overview is great, but CS wouldn’t need that “ahhhh, this is also done by BB” after he clicks on another interaction, “this is wonderful” wonders about review next candidate option because there’s only one, label could be changed if there’s only one sort candidates by priority? waiting time? customize columns? connects again with BBs |
|
@Martha Vasquez please update text as proposal and share with others: https://www.figma.com/file/iJyoDgPbbG6o9FKs2XVNAJ/Simulation-Prototype?node-id=870-147148&t=YjDnj5O09nUnJZvj-4
|
| What is your first impression about this page? |
|
| pretty much what he expected, expected to interact with application system looks modern, buttons are where expected, confused by BB activity a bit (might still think he’s really looking from CS perspective) would like to jump non-chronologically right panel might need more information from his perspective loves that it seems highly customizable would like to play around with eligibility criteria, color would help civil servants
login as applicant: “all of those options, cool!” walks through next steps quite seamlessly expected that phone number and mail need to be validated
pencil icon not needed in validate step
|
|
|
| How would you describe the structure/layout of the simulation? (Let them explain the navigation & Ul elements, etc.) |
|
|
|
|
|
| Please describe in your own words – what is a building block? |
|
| before he thought it’s lines of codes, now thinking about it more visually → recommends to make a picture or dynamic on the landing page, something interactive maybe?!
|
|
|
| How can you find out more about the information mediator and what is its role? |
|
| did as expected |
|
|
| How would you describe the role of the building blocks for this page? |
|
|
|
|
|
| OTHER |
|
|
|
|
|
(Context change) | What changed on the screen? Could you explain it in your own words. |
| He clearly understand it but asked why it changed? | Understood it clearly |
|
|
| What does this change indicate for you? |
|
|
|
|
|
| Where would you expect to see the perspective changes on the screen? |
|
|
|
|
|
End question: | How would you describe this product/simulation in your own words? | “How do you even start?” he want to see how to implement instead of seeing the use case.
Implementation process is big problem. | Design and structure was quite clear
He wanted to interact and change related building block more.
He understand he was civil servant but what was the goal of the civil servant he does not understand goal of civil servant. At the end he still though that it is for building a website. | a “citizen dev. gov. service sandbox”
|
|
|
| If this was a real website, how would you rate the technical maturity of this product? Were there any indications of this? |
|
| 8 of 10 |
|
|
@ | After experiencing the simulation - did you feel well introduced to it by the website and tutorial? |
|
| thinks he’s familiar now with the product not familiar with mission and vision of govstack through the simulation expects people would spend little time on landing page, more on simulation |
|
|
|
|
|
| 4 out of 5 stars he would rate
|
|
|
Offboarding: | Thank you very much for these great insights and of course for taking the time to help us! |
|
|
|
|
|
| Do you have any questions or feedback for us? |
|
|
|
|
|
|
|
|
|
|
|
|
| OTHER |
|
|
|
|
|