Introduction
This Confluence page documents the results of usability testing conducted for the construction permit use case https://www.figma.com/file/Nb5YWT3QTvwTZl3MyJXzi7/Construction-Permit-Use-Case?type=design&node-id=271%3A37957&mode=design&t=eSBvlJYXTcSW8GWD-1 . The purpose of this testing is to evaluate the user experience and gather valuable feedback for improvements.
Test Details
Date: 12th of September to 15th of September
Moderator: Jonas Bergmeier Artun Gürkan
Number of Participants: 5
Duration: 1 hour
Tests includes 3 phases:
Phase One: Warm-Up Questions
Phase Two: Thinking Aloud test
Phase Three: Qualitative interview
Phase One: Warm-Up Questions
Participant 1 (13/09-T.T.) | Participant 2 (14/09-P.G.) | Participant 3 (15/09-I.O.) Artun Gürkan Notes will be added on Wednesday. | Participant 4 (15/09-K.V) Not Matching with user group | Participant 5 | |
---|---|---|---|---|---|
Age and gender: | 29 f | 37 m | 34 | 39 | |
Location: | Italy | Germany | Gokce Ada/Turkey | Estonia | |
Occupency: | Architect (Design Residential Buildings, Interior Design, 3D Modelling, Rendering) | Comm. Service Technician | Architect (Restoration and construction/Mostly technical drawing and design) | CTO for estonia / Responsible for technology strategy | |
User testing experience and Govstack | No and No | Did surveys before just a little bit of knowledge about GovStack | Yes he conducted user research for architecture projects / No knowledge regarding GovStack | on different formats yes. - He does not know govstack. | |
Experience with Digital Government Platforms: | She is using the one for Turkey for paying depth, one or two times a month. Everything went pretty smoothly, sometimes also for receiving demoghraphing document from Turkey for her residence in Italy | maybe once, two times a year not easy, often you have to do stuff analog/on paper and print etc. Often have to go to government, make appointment. | He has Usually for downloading documents, h | above average, Digital heath information and for internal work. (for citizen or as a CTO) | |
Previous Constuction Permit application: | Italy just one time and it was more for the company by the directed from company. It was online construction permit. It was from the system of municipality not through the goverment. | Has knowledge, has to do it later this year. This will need support from architect/engineer. Takes up to 5 months. | No | ||
Device Preferences: | Computer | Mobile prefered | Both devices |
Phase Two: Thinking Aloud Test
Participant 1 (13/09-T.T.) | Participant 2 (14/09-P.G.) | Participant 3 | Participant 4 | Participant 5 | |
---|---|---|---|---|---|
Home Page observation | First thing she noticed was searching service than popular topic and general topic. | “It’s really clear” Expects it to be a starting page | Portal for e-gove (he understood) In order of topic and announcements does not valid for him ( for him announcement is more important) He does not like the help button He looked for all service he thinks topics can be other services he is wondering He would test this things at the beginning | ||
Log In | It was really easy and understandable | Was easy after he recognized he can just go ahead with provided credentials. | He confused with login as citizen or not and if it is related to ID or not but due to his existing login experience. | ||
Architect Registration Process and their interaction | She found it very helpful especially the first line is required documents She want to directly access to support service She also realized the button changes to apply. | Clear and understandable, expected it to be like this. | He read the text Checks the required documents. He aspect to this will be not automatic due to pdf format. He is curious about the work history. Title something confusing him. After log in he confused due to button (I don't like his reasoning) During the registration he felt confused due to different options. | ||
Consent Process Comfort | It was confusing to see consent page little bit confusing. she did not read it. “is it need to be filled or is it will be imported.” She only concern about the address where she did not want to share. She thinks it is her private address because it was not written as work address | Expects it to be a check or review, didn’t understand the concept of consent here. Seeing information overview would have helped him maybe as he seemed a bit lost. | On consent page he first thought he need to add than later he understand. He liked the privacy policy Thank you for your consent page Review Button Confused the user. He was confused with submit button. He felt confident regarding giving the data. When he looked back it kinda made sense for this. | ||
Submit Page | She saw all the important information on the page where to track etc. | ||||
Navigation Feedback | She really like the saving option for later. Because usually these things takes time and wants to take her time. (IMPORTANT) Easly find to home page | Thinks he’s done with the registration process. Would contact them for more information (used to that in Germany). | Notification etc he likes it. E-mail information also he liked. | ||
Congrats page | She liked the see the result so it was good. | he find it useless to see registration number information and where to find it. | |||
Feedback | She was expecting to see feedback after the application not after registration | He gave 4 | |||
Additional | She want to see the number on during the process too She find it very easy and did not have any page, everything was really easy. She also like the recent activity part. | Might have been confused because of the language barrier and the task. | It was not challenging but he was confused about the page for 3 steps and seeing revie (Optional make review page visually similar to task overview) He find it straightforward. | ||
Construction permit application Interactions/ feedback | First page She want to take screenshot for documents (Important) Maybe if there was a task document she will do that (Great Input) | He read it thoroughly regarding the permit Again he felt confusing due to button and task list. Identification part of the task or not (He found it confusing) | |||
Parcel ID | She find the Parcel map very useful based on her experience too. For confirming and writing right. She perfectly understand the map function (Great input) She find the information good. but it would be nice to see also to see the buildable area square meters | Understood it and found it useful Likes the overview page with all the information of the stakeholders. | Parcel ID seems all fine. (Map)He said he will not use it but as to be sure he wanted use the map. Select button is confusing. | ||
Digital Signature Expectations | She assumes if she lives in there and Digital signature is default information for her. She expect the system will stop her if it digitally signed. Because it gives the file types she does not think the envelope format, so she thinks document will be separately signed. | Expects to sign PDFs with Adobe tooling. Doesn’t have the understanding that’s expected here with Estonia eGov example | It should be a signed documents within the containers. He is wondering if should be digitally signed on the system or not. He expect to system should detect if it is digitally sign or not also allow me to sign I digitally (Good idea) REVIEW PART He expect payment fee information will be on to off the button (Good Idea) | ||
Scheduling | She sees the site inspection. She understand the available dates through color (perfection) She aspect and email with the date and time She saw add to calendar part (Really like it) but she still expect the email notification | Understood it. Did it. All good. Likes option to add to cal. | He was expecting the payment link(not perfect candidate so it is okay) No problem with scheduling add to calendar part was positive (Success) | ||
Tracking | No problem. quite smooth | Thinks clicking construction permit again would be another construction permit → IMO (Jonas) we might think of a more visual way with subtitles (this can explain that one can apply for new and track existing applications) Thinks overview page is for architects only. After further explanations, he found it (but kind of pressured towards it) Would expect it to be in the side panel. → we could add a “my cases” or sth. like this | He was perfect with tracking But he check the burger menu (Burger menu is generally where people are checking/ notifications or NY applications part) Add calendar button also should be added on tracking | ||
Payment | She had no problem, it was really clear for her. She want to see the PDF or email regarding proof of email. (IMPORTANT) she missed the invoice. but after she found it. (Maybe add one more link) | (missed what he said expectation-wise) Not sure if he as architect or owner would have to pay it. Payment option wise, he would pick digital wallet. CC not an option for him. | Application number part for confusing maybe it should be automatically filled on the page. Debit/credit card add Download invoice part confusing He is wondering what happens if approval does not happen (Consider) Download invoice was clear. | ||
Recieving Permit | She like two option of the pdf which she liked. | (missed what he said ) | .pdf is what he wondering | ||
Navigation Feedback | For application process she find everything clear. Application number is really important. Everything was quite great. | Sometimes gives feedback. Ok for him and understood. | Quite smooth it was 4 due to three steps. Thank you for your Feedback can be just notification (Good Input) | ||
Additional | She tried to use task overview button and see if it saves to go back the process, she find it very helpful. She want to see the summary on task review not during the process (Important) But She really liked the review page. She want to see the application Number again for there(IMPORTANT) | Task review part very important possible progress bar |
Phase Three: Qualitative Interview
Participant 1 (13/09-T.T.) | Participant 2 (14/09-P.G.) | Participant 3 | Participant 4 | Participant 5 | |
---|---|---|---|---|---|
Overall Experience: | She found it really easy and smooth. Everything was really easy to find. She found every option she could think of. She said it was almost perfect. | Expects more details about the construction site. Wouldn’t know how to contact the government, but expects it to be easy via mail. | |||
Positive - Negative feedback | (same before) | Liked clear design, clear text. | |||
Comparison with previous experiences | She found it much easier compare to previous experience even though she had written explanation before. She found it easiest and smoothest. | Much easier than actual process, in Germany more questions, have to know “right words”. More difficult, takes more time and more knowledge in advance. | It is quite smooth so he was quite happy with the flow. UX design vs was good. He wants to learn about which steps he is on. (Changing progress bar might be helpful) 3 step progress bar within a progress bar can be a good, new but also crazy addition. Also he find it quite good somewhat better than estonia. he found the process was very good. | ||
Additional Feedback | Expected overview in sub menu | The participant show all the documents they shared during the permit application, the waste of paper for one application is quite big therefore online application is much better method for me. | Virtual assistant can be imported and it can be really good addition. (Interesting idea) Notifications were also great It was not filled with information he was quite happy with the design. |
0 Comments