User Testing - Takeaway

Age and gender:

29 f

37 m

34

39

27

Location:

Italy

Germany

Gokce Ada/Turkey

Estonia

Tallin/Estonia

Occupency:

Architect (Design Residential Buildings, Interior Design, 3D Modelling, Rendering)

Comm. Service Technician

Architect (Restoration and construction/Mostly technical drawing and design)

CTO for estonia / Responsible for technology strategy

Architect

Make plans and models every day

Building permits are part of my job

User testing experience and Govstack

No and No

Did surveys before

just a little bit of knowledge about GovStack

Yes he conducted user research for architecture projects / No knowledge regarding GovStack

on different formats yes. - He does not know govstack.

No

Experience with Digital Government Platforms:

She is using the one for Turkey for paying depth, one or two times a month.

Everything went pretty smoothly, sometimes also for receiving demoghraphing document from Turkey for her residence in Italy

maybe once, two times a year

not easy, often you have to do stuff analog/on paper and print etc.

Often have to go to government, make appointment.

He has

Usually for downloading documents

Prints most of the time

above average, Digital heath information and for internal work. (for citizen or as a CTO)

No knowledge of GovStack

Now doing a project where we got some comments back from the city on building permits and every day I go back and check for comments.

In etstonia we have ?? where we have personal details and things which I use once a month

Usually I use a computer because these kinds of sites need you to log in with an eID card so its more comfortable on a computer

Without eID, it depends, if I need to upload files then I’d still use computer but if I just need to check something I’d use phone.

Previous Constuction Permit application:

Italy just one time and it was more for the company by the directed from company.

It was online construction permit. It was from the system of municipality not through the goverment.

Has knowledge, has to do it later this year.

This will need support from architect/engineer.

Takes up to 5 months.

in contact with the municipally

sometimes online

mostly has to share it in person

in contact with architecture chamber

No

I’ve used the estonian site, its quite hard to understand, multiple things on the site already and once you find your documents when you update anything there it doesnt save by default, you have to scroll up and save so the info doesnt get lost, again.

It’s a maze going through there. It gets easier over time.

Right now I’ve only done this online. It takes about an hour to check an upload everything. It depends on the size, an appartent building needs you to enter each appartment seperately it takes time.

A tower could have over 100 apartments.How big is the appartment, what koinds of facilities, a bath or a shower, does it have a balcony.

Device Preferences:

Computer

Mobile prefered

Mobile

Both devices

Home Page observation

First thing she noticed was searching service than popular topic and general topic.

“It’s really clear”

Expects it to be a starting page

Too much information

Not good enough hierarchy

Thinks its also for citizens = full portal

Portal for e-gove (he understood)

In order of topic and announcements does not valid for him ( for him announcement is more important)

He does not like the help button

He looked for all service he thinks topics can be other services he is wondering

He would test this things at the beginning

Its understandable,

Each section will link to sub subjects

If I want to proceed I’ll click registration of engeneers and architects

If i didnt see that I’d click login and if I’m not a user it would ast to sign up

[reiterate scenario]

I’d just search for what I’d need

• Wording can be changed

  • Same structure can be added (create harmony)MID

Log In

It was really easy and understandable

Was easy after he recognized he can just go ahead with provided credentials.

Understood

He confused with login as citizen or not and if it is related to ID or not but due to his existing login experience.

OK, this is like my account?

OK

Architect Registration Process and their interaction

She found it very helpful especially the first line is required documents

She want to directly access to support service

She also realized the button changes to apply.

Clear and understandable, expected it to be like this.

Understands he can get support

Also understands that UI changed after login

He read the text

Checks the required documents.

He aspect to this will be not automatic due to pdf format.

He is curious about the work history.

Title something confusing him.

After log in he confused due to button (I don't like his reasoning)

During the registration he felt confused due to different options.

OK so I have to log in to do that…

I’d log in

[logs in…]

OK so, these are what I need to have to register as an architect

TASK LIST

this is getting my background info for why I’m registering myself as an architect, its clear but I’d assume for it to be little boxes where you can write stuff on but this is an overview.

I’m going to press start now…

I expect it will be done if I click submit.

[how would you check the information before you submit]

I dont know

[clicks personal information]

I’d like to see a drop down arrow or something to show something is hidden in here

[submits]

[reads the info] OK

Consent Process Comfort

It was confusing to see consent page little bit confusing. she did not read it.

“is it need to be filled or is it will be imported.”

She only concern about the address where she did not want to share.

She thinks it is her private address because it was not written as work address

Expects it to be a check or review, didn’t understand the concept of consent here.

Seeing information overview would have helped him maybe as he seemed a bit lost.

Understands that it takes information from other sources

Feels confident about sharing his data

On consent page he first thought he need to add than later he understand.

He liked the privacy policy

Thank you for your consent page

Review Button Confused the user.

He was confused with submit button.

He felt confident regarding giving the data.

When he looked back it kinda made sense for this.

OK so you’ll get that from other places, thats pretty nice so I dont have to enter everything.

[how comfortable did you feel]

Usually when its the government I’m not usually worried about these things

I’ll click back… OK its only permitting me to submit [task list], not to see the information that I did share.

Submit Page

She saw all the important information on the page where to track etc.

Understood

Navigation Feedback

She really like the saving option for later. Because usually these things takes time and wants to take her time. (IMPORTANT)

Easly find to home page

Thinks he’s done with the registration process.

Would contact them for more information (used to that in Germany).

Understood

Notification etc he likes it.

E-mail information also he liked.

used breadcrumbs to navigate as well as ‘back’

Congrats page

She liked the see the result so it was good.

he find it useless to see registration number information and where to find it.

Feedback

She was expecting to see feedback after the application not after registration

(Understood)

He gave 4

ok… I usually dont write feedback

Additional

She want to see the number on during the process too

She find it very easy and did not have any page, everything was really easy.

She also like the recent activity part.

Might have been confused because of the language barrier and the task.

It was not challenging but he was confused about the page for 3 steps and seeing revie (Optional make review page visually similar to task overview)

He find it straightforward.

The submit button was confusing but otherwise quite normal

Construction permit application Interactions/ feedback

First page

She want to take screenshot for documents (Important) Maybe if there was a task document she will do that (Great Input)

Building vs. Construction → wording questionable

would like to see cost estimation

Would expect drawings of the buildings also be in place and testings of the earth structure (feasiblity doc) and personal contractors documents and contracts

Feasibility doc. should be provided by the owner?

Estimated fee would be appreciated to see already

He read it thoroughly regarding the permit

Again he felt confusing due to button and task list.

Identification part of the task or not (He found it confusing)

clicked search

then services

OK [reads requirements]

[would you have those documents ready at this point] yes once they are all signed I’d have them in a folder in my system/computer

• screenshoting was interesting input- should be considered for later fo design solutions regarding accesibility.

• Payment fee should me more clear. (price list etc can be considered)

  • Accordion pattern can be added

• Task list

  • Change the design create alternative

  • Taks itself and review page should be in similar visual language with task overview

  • Application overview vs Task overview

• Progress bar add

  • Make it more clear

  • Steppers?

Parcel ID

She find the Parcel map very useful based on her experience too.

For confirming and writing right.

She perfectly understand the map function (Great input)

She find the information good. but it would be nice to see also to see the buildable area square meters

Understood it and found it useful

??? missed this partially !!!

Likes the overview page with all the information of the stakeholders.

Idea: Search by parcel owner

Likes that data are added automatically from other sources

Likes the map feature, would even prefer it

Wouldn’t have recognized map feature

Parcel ID seems all fine.

(Map)He said he will not use it but as to be sure he wanted use the map.

Select button is confusing.

parcel is the kind of… [building plot] ok just a translating question

[what will happen if you click the button next to text field]

I’d search for the address, I wouldnt know the ID/we dont have parcel ID we have plot numbers as you see here. No one knows those by heart.

We have to put the coordenates for the end points of the building so it will place the end points on the plots so the system knows the size and shape and where to put it on the map. It just needs to be the widest points of the building

IDENTIFICATION

This is ok… usually I’d enter these myself

OVERVIEW

Ok so you can also add a person here

UPLOAD

OK I’d upload

• In general Success

• Map function was really helpful

  • Can be made more clear (consider adding a bigger button for selecting/searching from the map)

• Autofilled information were thought will be added by person (Can be informed it will be automatically filled clearly)

• Proposed by user: Existing parcels by the owner or searching by the owner can be added. (I think It will be unnececery and might confuse)

• Save for Later was Success

Digital Signature Expectations

She assumes if she lives in there and Digital signature is default information for her.

She expect the system will stop her if it digitally signed.

Because it gives the file types she does not think the envelope format, so she thinks document will be separately signed.

Expects to sign PDFs with Adobe tooling.

Doesn’t have the understanding that’s expected here with Estonia eGov example

(ID step is fine)

First didn’t recognize that signature is asked to be digital

Expects it to be hard to collect it

It should be a signed documents within the containers.

He is wondering if should be digitally signed on the system or not.

He expect to system should detect if it is digitally sign or not also allow me to sign I digitally (Good idea)

REVIEW PART

He expect payment fee information will be on to off the button (Good Idea)

well we have this tickytock type thing in estonia so you can sign from that ID card and its also a smart ID as well. Usually every document is signed with this application.

So i would choose a file to sign, it will add my signature and if notable people need to sign…???

[do you think the docs will be digitally signed in this process] erm yes, [read content] yes

There are usually loads more documents

CHECK ANSWERS

All fine, OK. That was much easier than estonia’s

• This part will be depending on how digital signature service will work. Based on that design will change.

  • It will be great if the system automatically understands and feedbacks the user if the documents signed by each stakeholder.

  • (Optional) Within the portal we can also ask the sign the documents from different users

• Review information part was confusing for the user and some users were expecting the see task overview visual for this part.

• In general flow was very easy.

• Wording should be different (Envlope wording would be better)

  • digital signature would be left out

  • Share with others (decide based on that) if there are no service left it out.

Scheduling

She sees the site inspection.

She understand the available dates through color (perfection)

She aspect and email with the date and time

She saw add to calendar part (Really like it) but she still expect the email notification

Understood it. Did it. All good.

Likes option to add to cal.

Using datepicker was easy going for him

Wouldn’t use add to cal feature

He was expecting the payment link(not perfect candidate so it is okay)

No problem with scheduling

add to calendar part was positive (Success)

OK I’d pick a date…

• Success/was clear

  • add add to calender when the user was tracking.

Tracking

No problem. quite smooth

Thinks clicking construction permit again would be another construction permit

→ IMO (Jonas) we might think of a more visual way with subtitles (this can explain that one can apply for new and track existing applications)

Thinks overview page is for architects only.

After further explanations, he found it (but kind of pressured towards it)

Would expect it to be in the side panel.

→ we could add a “my cases” or sth. like this

(not done)

He was perfect with tracking

But he check the burger menu (Burger menu is generally where people are checking/ notifications or NY applications part)

Add calendar button also should be added on tracking

ok this doesnt have… ok lets try again

[new figma link sent]

I’d choose notifications… I wouldnt think its under my information. Either notification or…

• Add calendar should be added on tracking

• Tracking application can be added to burger menu

  • Many users also used burger menu to see if they can track the application

  • Notifications section was the first field they check for the application

    • Can be added to notification page or create a new tab for your application / My activities

Payment

She had no problem, it was really clear for her.

She want to see the PDF or email regarding proof of email. (IMPORTANT) she missed the invoice. but after she found it. (Maybe add one more link)

(missed what he said expectation-wise)

Not sure if he as architect or owner would have to pay it.

Payment option wise, he would pick digital wallet. CC not an option for him.

(Paying with card, seemed all fine)

Application number part for confusing maybe it should be automatically filled on the page.

Debit/credit card add

Download invoice part confusing

He is wondering what happens if approval does not happen (Consider)

Download invoice was clear.

hmm ok… [reads] [chooses debit card] I dunno, it doesnt really matter..

• add debit/credit card

• Proof should be added also on success page similar to add calendar.

• Payment estimation can be added previously.

  • Payment success email with email.

Recieving Permit

She like two option of the pdf which she liked.

(missed what he said )

Would provide feedback

.pdf is what he wondering

I’d see a signed overview of everyone who has commented. Signed that they agree now saying you have a ??? building permit.

Usually I’d download this

Navigation Feedback

For application process she find everything clear.

Application number is really important.

Everything was quite great.

Sometimes gives feedback. Ok for him and understood.

Quite smooth

it was 4 due to three steps.

Thank you for your Feedback can be just notification (Good Input)

[5/5]

Success

Additional

She tried to use task overview button and see if it saves to go back the process, she find it very helpful.

She want to see the summary on task review not during the process (Important) But She really liked the review page.

She want to see the application Number again for there(IMPORTANT)

Task review part very important possible progress bar

[how would you find the permit again] OK my applications, yeah so I click on that and yeah I can download it again.

[anything missing]

The size of the building (net area and square meters and the separate apartments and their size)

[do you do this alone or collaborate]

I’d do this myself.

[should this be automated or should you fill this in?]

easier if automatic, putting in the numbers could be along process and error prone due to tiredness.

Basically autocad files have the info in. City people just look at the PDFs because they dont have the software to open drawings.

• Application number should be added when it was submitted and all the related location

• Toast messages can be considered.

Overall Experience:

She found it really easy and smooth.

Everything was really easy to find. She found every option she could think of.

She said it was almost perfect.

Expects more details about the construction site.

Wouldn’t know how to contact the government, but expects it to be easy via mail.

“Was pretty easy”

“Very easy for me”

He thinks with all online information it may not be needed to meet locally

I think its much easier than right now in estonia. We dont get permits this quick, normally get comments, wait a month to reply to comments and then key notes and mistakes

Positive - Negative feedback

(same before)

Liked clear design, clear text.

“Application UI can be more colorful and easier”

I dont know, the first submit button was in my mind. bad part of this.

Overall quite nice

Comparison with previous experiences

She found it much easier compare to previous experience even though she had written explanation before. She found it easiest and smoothest.

Much easier than actual process, in Germany more questions, have to know “right words”. More difficult, takes more time and more knowledge in advance.

Would prefer this web tool

(Shows paperwork he has to share for local permit → it’s a lot)

It is quite smooth so he was quite happy with the flow.

UX design vs was good.

He wants to learn about which steps he is on.

(Changing progress bar might be helpful)

3 step progress bar within a progress bar can be a good, new but also crazy addition.

Also he find it quite good somewhat better than estonia. he found the process was very good.

Make the page more fun, it doesnt have to be so depressing

Additional Feedback

Expected overview in sub menu

The participant show all the documents they shared during the permit application, the waste of paper for one application is quite big therefore online application is much better method for me.

Virtual assistant can be imported and it can be really good addition.

(Interesting idea)

Notifications were also great

It was not filled with information he was quite happy with the design.