...
Participant 1 (13/09-T.T.) | Participant 2 (14/09-P.G.) | Participant 3 (15/09-I.O.) Artun Gürkan Notes will be added on Wednesday. | Participant 4 (15/09-K.V) Not Matching with user group | Participant 5 | |
---|---|---|---|---|---|
Age and gender: | 29 f | 37 m | 34 | 39 | 27 |
Location: | Italy | Germany | Gokce Ada/Turkey | Estonia | Tallin/Estonia |
Occupency: | Architect (Design Residential Buildings, Interior Design, 3D Modelling, Rendering) | Comm. Service Technician | Architect (Restoration and construction/Mostly technical drawing and design) | CTO for estonia / Responsible for technology strategy | Architect Make plans and models every day Building permits are part of my job |
User testing experience and Govstack | No and No | Did surveys before just a little bit of knowledge about GovStack | Yes he conducted user research for architecture projects / No knowledge regarding GovStack | on different formats yes. - He does not know govstack. | No |
Experience with Digital Government Platforms: | She is using the one for Turkey for paying depth, one or two times a month. Everything went pretty smoothly, sometimes also for receiving demoghraphing document from Turkey for her residence in Italy | maybe once, two times a year not easy, often you have to do stuff analog/on paper and print etc. Often have to go to government, make appointment. | He has Usually for downloading documents Prints most of the time | above average, Digital heath information and for internal work. (for citizen or as a CTO) | No knowledge of GovStack Now doing a project where we got some comments back from the city on building permits and every day I go back and check for comments. In etstonia we have ?? where we have personal details and things which I use once a month Usually I use a computer because these kinds of sites need you to log in with an eID card so its more comfortable on a computer Without eID, it depends, if I need to upload files then I’d still use computer but if I just need to check something I’d use phone. |
Previous Constuction Permit application: | Italy just one time and it was more for the company by the directed from company. It was online construction permit. It was from the system of municipality not through the goverment. | Has knowledge, has to do it later this year. This will need support from architect/engineer. Takes up to 5 months. | in contact with the municipally sometimes online mostly has to share it in person in contact with architecture chamber | No | I’ve used the estonian site, its quite hard to understand, multiple things on the site already and once you find your documents when you update anything there it doesnt save by default, you have to scroll up and save so the info doesnt get lost, again. It’s a maze going through there. It gets easier over time. Right now I’ve only done this online. It takes about an hour to check an upload everything. It depends on the size, an appartent building needs you to enter each appartment seperately it takes time. A tower could have over 100 apartments.How big is the appartment, what koinds of facilities, a bath or a shower, does it have a balcony. |
Device Preferences: | Computer | Mobile prefered | Mobile | Both devices |
Phase Two: Thinking Aloud Test
Participant 1 (13/09-T.T.) | Participant 2 (14/09-P.G.) | Participant 3 | Participant 4 | Participant 5 | Findings & Takeaways | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Home Page observation | First thing she noticed was searching service than popular topic and general topic. | “It’s really clear” Expects it to be a starting page | Too much information Not good enough hierarchy Thinks its also for citizens = full portal | Portal for e-gove (he understood) In order of topic and announcements does not valid for him ( for him announcement is more important) He does not like the help button He looked for all service he thinks topics can be other services he is wondering He would test this things at the beginning | Its understandable, Each section will link to sub subjects If I want to proceed I’ll click registration of engeneers and architects If i didnt see that I’d click login and if I’m not a user it would ast to sign up [reiterate scenario] I’d just search for what I’d need |
| ||||||
Log In | It was really easy and understandable | Was easy after he recognized he can just go ahead with provided credentials. | Understood | He confused with login as citizen or not and if it is related to ID or not but due to his existing login experience. | OK, this is like my account? | OK | ||||||
Architect Registration Process and their interaction | She found it very helpful especially the first line is required documents She want to directly access to support service She also realized the button changes to apply. | Clear and understandable, expected it to be like this. | Understands he can get support Also understands that UI changed after login | He read the text Checks the required documents. He aspect to this will be not automatic due to pdf format. He is curious about the work history. Title something confusing him. After log in he confused due to button (I don't like his reasoning) During the registration he felt confused due to different options. | OK so I have to log in to do that… I’d log in [logs in…] OK so, these are what I need to have to register as an architect TASK LIST this is getting my background info for why I’m registering myself as an architect, its clear but I’d assume for it to be little boxes where you can write stuff on but this is an overview. I’m going to press start now… I expect it will be done if I click submit. [how would you check the information before you submit] I dont know [clicks personal information] I’d like to see a drop down arrow or something to show something is hidden in here [submits] [reads the info] OK | |||||||
Consent Process Comfort | It was confusing to see consent page little bit confusing. she did not read it. “is it need to be filled or is it will be imported.” She only concern about the address where she did not want to share. She thinks it is her private address because it was not written as work address | Expects it to be a check or review, didn’t understand the concept of consent here. Seeing information overview would have helped him maybe as he seemed a bit lost. | Understands that it takes information from other sources Feels confident about sharing his data | On consent page he first thought he need to add than later he understand. He liked the privacy policy Thank you for your consent page Review Button Confused the user. He was confused with submit button. He felt confident regarding giving the data. When he looked back it kinda made sense for this. | OK so you’ll get that from other places, thats pretty nice so I dont have to enter everything. [how comfortable did you feel] Usually when its the government I’m not usually worried about these things I’ll click back… OK its only permitting me to submit [task list], not to see the information that I did share. | |||||||
Submit Page | She saw all the important information on the page where to track etc. | Understood | ||||||||||
Navigation Feedback | She really like the saving option for later. Because usually these things takes time and wants to take her time. (IMPORTANT) Easly find to home page | Thinks he’s done with the registration process. Would contact them for more information (used to that in Germany). | Understood | Notification etc he likes it. E-mail information also he liked. | used breadcrumbs to navigate as well as ‘back’ | |||||||
Congrats page | She liked the see the result so it was good. | he find it useless to see registration number information and where to find it. | ||||||||||
Feedback | She was expecting to see feedback after the application not after registration | (Understood) | He gave 4 | ok… I usually dont write feedback | ||||||||
Additional | She want to see the number on during the process too She find it very easy and did not have any page, everything was really easy. She also like the recent activity part. | Might have been confused because of the language barrier and the task. | It was not challenging but he was confused about the page for 3 steps and seeing revie (Optional make review page visually similar to task overview) He find it straightforward. | The submit button was confusing but otherwise quite normal | ||||||||
Construction permit application Interactions/ feedback | First page She want to take screenshot for documents (Important) Maybe if there was a task document she will do that (Great Input) | Building vs. Construction → wording questionable would like to see cost estimation Would expect drawings of the buildings also be in place and testings of the earth structure (feasiblity doc) and personal contractors documents and contracts Feasibility doc. should be provided by the owner? Estimated fee would be appreciated to see already | He read it thoroughly regarding the permit Again he felt confusing due to button and task list. Identification part of the task or not (He found it confusing) | clicked search then services OK [reads requirements] [would you have those documents ready at this point] yes once they are all signed I’d have them in a folder in my system/computer |
| |||||||
Parcel ID | She find the Parcel map very useful based on her experience too. For confirming and writing right. She perfectly understand the map function (Great input) She find the information good. but it would be nice to see also to see the buildable area square meters | Understood it and found it useful Likes the overview page with all the information of the stakeholders. | Idea: Search by parcel owner Likes that data are added automatically from other sources Likes the map feature, would even prefer it Wouldn’t have recognized map feature | Parcel ID seems all fine. (Map)He said he will not use it but as to be sure he wanted use the map. Select button is confusing. | parcel is the kind of… [building plot] ok just a translating question [what will happen if you click the button next to text field] I’d search for the address, I wouldnt know the ID/we dont have parcel ID we have plot numbers as you see here. No one knows those by heart. We have to put the coordenates for the end points of the building so it will place the end points on the plots so the system knows the size and shape and where to put it on the map. It just needs to be the widest points of the building IDENTIFICATION This is ok… usually I’d enter these myself OVERVIEW Ok so you can also add a person here UPLOAD OK I’d upload |
| ||||||
Digital Signature Expectations | She assumes if she lives in there and Digital signature is default information for her. She expect the system will stop her if it digitally signed. Because it gives the file types she does not think the envelope format, so she thinks document will be separately signed. | Expects to sign PDFs with Adobe tooling. Doesn’t have the understanding that’s expected here with Estonia eGov example | (ID step is fine) First didn’t recognize that signature is asked to be digital Expects it to be hard to collect it | It should be a signed documents within the containers. He is wondering if should be digitally signed on the system or not. He expect to system should detect if it is digitally sign or not also allow me to sign I digitally (Good idea) REVIEW PART He expect payment fee information will be on to off the button (Good Idea) | well we have this tickytock type thing in estonia so you can sign from that ID card and its also a smart ID as well. Usually every document is signed with this application. So i would choose a file to sign, it will add my signature and if notable people need to sign…??? [do you think the docs will be digitally signed in this process] erm yes, [read content] yes There are usually loads more documents CHECK ANSWERS All fine, OK. That was much easier than estonia’s |
| ||||||
Scheduling | She sees the site inspection. She understand the available dates through color (perfection) She aspect and email with the date and time She saw add to calendar part (Really like it) but she still expect the email notification | Understood it. Did it. All good. Likes option to add to cal. | Using datepicker was easy going for him Wouldn’t use add to cal feature | He was expecting the payment link(not perfect candidate so it is okay) No problem with scheduling add to calendar part was positive (Success) | OK I’d pick a date… |
| ||||||
Tracking | No problem. quite smooth | Thinks clicking construction permit again would be another construction permit → IMO (Jonas) we might think of a more visual way with subtitles (this can explain that one can apply for new and track existing applications) Thinks overview page is for architects only. After further explanations, he found it (but kind of pressured towards it) Would expect it to be in the side panel. → we could add a “my cases” or sth. like this | (not done) | He was perfect with tracking But he check the burger menu (Burger menu is generally where people are checking/ notifications or NY applications part) Add calendar button also should be added on tracking | ok this doesnt have… ok lets try again [new figma link sent] I’d choose notifications… I wouldnt think its under my information. Either notification or… |
| ||||||
Payment | She had no problem, it was really clear for her. She want to see the PDF or email regarding proof of email. (IMPORTANT) she missed the invoice. but after she found it. (Maybe add one more link) | (missed what he said expectation-wise) Not sure if he as architect or owner would have to pay it. Payment option wise, he would pick digital wallet. CC not an option for him. | (Paying with card, seemed all fine) | Application number part for confusing maybe it should be automatically filled on the page. Debit/credit card add Download invoice part confusing He is wondering what happens if approval does not happen (Consider) Download invoice was clear. | hmm ok… [reads] [chooses debit card] I dunno, it doesnt really matter.. |
| ||||||
Recieving Permit | She like two option of the pdf which she liked. | (missed what he said ) | Would provide feedback | .pdf is what he wondering | I’d see a signed overview of everyone who has commented. Signed that they agree now saying you have a ??? building permit. Usually I’d download this | |||||||
Navigation Feedback | For application process she find everything clear. Application number is really important. Everything was quite great. | Sometimes gives feedback. Ok for him and understood. | Quite smooth it was 4 due to three steps. Thank you for your Feedback can be just notification (Good Input) | [5/5] | Success | |||||||
Additional | She tried to use task overview button and see if it saves to go back the process, she find it very helpful. She want to see the summary on task review not during the process (Important) But She really liked the review page. She want to see the application Number again for there(IMPORTANT) | Task review part very important possible progress bar | [how would you find the permit again] OK my applications, yeah so I click on that and yeah I can download it again. [anything missing] The size of the building (net area and square meters and the separate apartments and their size) [do you do this alone or collaborate] I’d do this myself. [should this be automated or should you fill this in?] easier if automatic, putting in the numbers could be along process and error prone due to tiredness. Basically autocad files have the info in. City people just look at the PDFs because they dont have the software to open drawings. |
|
Phase Three: Qualitative Interview
Participant 1 (13/09-T.T.) | Participant 2 (14/09-P.G.) | Participant 3 | Participant 4 | Participant 5 | Findings & Takeaways | |
---|---|---|---|---|---|---|
Overall Experience: | She found it really easy and smooth. Everything was really easy to find. She found every option she could think of. She said it was almost perfect. | Expects more details about the construction site. Wouldn’t know how to contact the government, but expects it to be easy via mail. | “Was pretty easy” “Very easy for me” He thinks with all online information it may not be needed to meet locally | I think its much easier than right now in estonia. We dont get permits this quick, normally get comments, wait a month to reply to comments and then key notes and mistakes | ||
Positive - Negative feedback | (same before) | Liked clear design, clear text. | “Application UI can be more colorful and easier” | I dont know, the first submit button was in my mind. bad part of this. Overall quite nice | ||
Comparison with previous experiences | She found it much easier compare to previous experience even though she had written explanation before. She found it easiest and smoothest. | Much easier than actual process, in Germany more questions, have to know “right words”. More difficult, takes more time and more knowledge in advance. | Would prefer this web tool (Shows paperwork he has to share for local permit → it’s a lot) | It is quite smooth so he was quite happy with the flow. UX design vs was good. He wants to learn about which steps he is on. (Changing progress bar might be helpful) 3 step progress bar within a progress bar can be a good, new but also crazy addition. Also he find it quite good somewhat better than estonia. he found the process was very good. | Make the page more fun, it doesnt have to be so depressing | |
Additional Feedback | Expected overview in sub menu |
...
The participant show all the documents they shared during the permit application, the waste of paper for one application is quite big therefore online application is much better method for me. | Virtual assistant can be imported and it can be really good addition. (Interesting idea) Notifications were also great It was not filled with information he was quite happy with the design. |
Takeaways - Action Points
Panel | ||||||
---|---|---|---|---|---|---|
| ||||||
Takeaways and iteration will be conducted for construction permit application flow only due to scope of the demo application. If the scope increase to registration new iteration and takeaway will be created. |
From User Testing:
In general user testing was a success. Participants found the wireframes and flow quite efficient and they easily interacted with the prototype without having any major problems.
- Clearly inform the user regarding the payment fee (possible estimated amount and automatic calculation process)
- Identification person/entity
- Simultaneous application. (cancel application part)
- Task flow pages needs iteration ( Some users found confusing - Similar language is needed)
- Add progress bar or steppers or mix
- Add similar design element on task page within the task
- Change task overview to Application Overview
- Create different versions of the flow and share with the team
- Change the review page similar to task overview page
- Information in the Parcel ID page should communicate with the user regarding automatically filling it.
- The map option can be shown more prominently.
- Wording in general.
- Wording for digital signature and task overview page was where most confusion happened (need to be revisited)
- Digital signature service will affect the uploading documents page. Scope in this regard needs to be decided. this not only will affect the flow and UI but also will affect which device our users are using (For example: Estonian version of digital signature drives users to use desktop)
- Add to calendar option should be added during tracking scheduled field visit.
- Application number should be added when application was started and additional related locations.
- add debit/credit card
- Add additional download invoice option/payment proof
- Toast messages (or alternative for notifications and error can be added to scope (Artun Gürkan design Jonas Bergmeier Priit Puru Feedback) / this may also allow us to replace disabled buttons with error notifications through toast.
- Change burger menu, allow users to track application via burger menu/ notification page can be added as a separate wireframes.
From Stakeholders:
- Progress bar will be added
- Payment flow feedback needs more explanation during the meeting 21/09/2023
- Approval time/process time should be more clear
- The feedback page needs to be discussed with stakeholders. The feedback page was clear for users but we received feedback to use net promoter ( Artun Gürkan Check https://link.springer.com/article/10.1007/s12208-021-00280-9) For demo purpose this might not be relevant and can be highly depending on the country.
- If the user edit existing information we should notify them these changes will be reflected in the source registry as well.
- More explanation is needed regarding end of the service