avantIQ HR Portal

Running Research/ethnographic study

The biggest reservation that stakeholders usually have about research and user experience metholodgies is that it takes too long and costs too much. My first job was to show stakeholders that we can still get enormous value from getting local inputs as early as possible and that it will reduce development time. Bringing local inputs in our projects conceptualization would help us to effectively design for these multicultural audiences. We have been working on integrating research and make it work alongside a rapid development agile cycle. We have been using simple techniques to get the core template model correct so we could test it with locals once the initial versions have been built. Now that we had found a way to effectively integrate local inputs in our development process, it was time to surround ourselves with the culture we are designing for!

1. Elicitation techniques

Designing a new product or to expand it to a new market is not about assumptions but facts: focus on your users first. We needed to understand how they run standard HR processes and what would make them use the application or not. A combination of several different techniques has been necessary to achieve a successful outcome.

Step #1 : Study your market and document your assumptions

I started by running some secondary research about the countries or group of countries culture, history, political situation and legal environment. That really helped a lot to build trust and credibility in my early interactions with local end users.

Step #2 : Talk to the end users

I noticed that 3 days on-site travel was enough to create a connection and get information without bothering users on their daily routine. They knew a project was going on and were mainly happy to be part of it. I always started by spending a day in their office, observing and taking notes on how they worked and used technology for specific processes. I have also been using questionnaires, focus-group and interviews for the next 2 days.

Step #3 : Validating the information and early testing of the global template

Before we kicked off testing on the global application in order to list local deviations and requirements, I shared my assumptions with both users and stakeholders. This way I was able to validate them and show local team we really care about their culture and the way they work. I then gather insights by holding user testing sessions but kept them under control and stay focused on observing workflows for a specific tasks. Outputs were synthesized and shared with global project teams, then requirements were accepted or rejected and listed as agile epic/stories for this country.

Process

I believe that transparent, open design process leads to happier clients and shorter development time. In this case, local team would be signing-off the final acceptance for their project so it made sense to involve them as much as possible. As we were working on the prototype to be handed-over to dev team we decided to empower local team by letting them access Figma. They could comment or request more information on specific point through the slack channel we opened to communicate with them.

Cross cultural UI overall approach

The original approach was to try to get all the application for a country localized and translated. We quickly realized that we needed local team to be pro-active in this process. We needed to have a native cultural expert with enough coporate culture to be able to let us know what needed to be localized and what not. Overall, if we could not get enough information we aimed to make localize the UI as much as possible and keep the content in english.

Our solution was a combination of localized UI & corporate content in English

I noticed that most countries used some corporate-specific words that did not needed to be translated. Some UI elements did not need translation either. I have made an inventory of these and included them as standard values in our global template. Overall, what we aimed to do was to make sure translations were accurate. If we could not find any cultural expert in our local team, we kept the UI standard (english) as much a possible.

Process example: United Arab Emirates app localization

Local project process followed the lean UX framework (Think, Make, Check). This framework and local inputs or decisions were grounded on the global strategic business goals.

Think : market study, elicitation and first round of required iterations

The first step has been for us to consider local legislation and collect information sample that represents the target audience and list our questions so we could be prepared for our first round of meetings and workshops.

We then travelled to the country company location, presented the big pictures and run our elicitation techniques (1 day observation + 2 days of interviews and focus-group). We would also ensure that we had a person who could function as a “cultural expert,” equally familiar with the target personal and corporate culture. His first task was to make sure the project technical terms and abstract concepts could be translated and understood. In the meantime, we tried to have everyone on the design team develop a clear understanding of the target culture, corporate flows, and processes variations. After this workshop we kicked off the wireframes/prototyping phase in Figma, involving local team on regular progress touchpoints.

Make : scope validation and design kick-off

We then shared our testing results and highlighted what would be the requirements and variations. To do so, we traveled again to local users location and brought our Figma screens to a live prototype so user could validate the scope using a tangible product. Cost evaluation, planning and subsequent tasks were then created in Jira.

Make : Keeping design consistent

Before kicking off local project we already had a global template and a standard application to be presented to locals. This standard application was build on a master design system (tokens) having smart, reusable patterns at our disposal. It was all about access point centralization, reusable design resources and local adjustment, reusable development resources. Most of the design system fails as component are just used once but in our case most of the components would actually be re-used ! In our case, the design system was more a starter kits, components and documentation. Things that people could actually understand and use. The company had already a brand guidelines, master iconography, symbols and pictures and a style guide. All we had to do was to create and adapt our component leaving room for local inputs and variation.

Our unified approach included design tokens for wireframing and prototyping (left) and a complete design system for development (right)

We have run UI localization for most of the countries in Asia, South America and Europe. Arabic localization has been a long and hard process. I wrongly thought at first that mirroring the English UI would be enough. It won’t just do the job either to hire an interpreter to translate text into Arabic and consider the job done. Without the support of our local cultural-expert we would not have been able to achieve this local project.

United Arab Emirates localization example

First difficulty : all arab countries use different dialect. After discussing this with locals we realized it was not a big issue for them to deal with a standard arabic UI. We decided to use Modern Standard Arabic (Fusha) so we would save a huge amount of time on translation process since most of the arab countries in scope would be able to use the app. Our initial research show that Arabic user can easily switch between English and Arabic as long as the design is consistent. We were therefore in position to implement our overall UI approach for these countries (localized UI / corporate content in English).

In terms of Design, we involved local team in regular validation touchpoints. Arabic words are longer and require a bigger size and space than used in the Roman alphabet due to the complexity of its characters. Digits are Left-To-Right even if the UI is Right-To-Left. This implied some iteration and many deviations from the standard UI. We also had to change our icon set as some of them would keep their LTR alignment. Sometimes we just changed the location in the UI, keeping the original alignment and icon directions.

Considering all the changes that the characters and words implied in terms of size, layout and copy we ended up creating a specific sub design system for Arabic UI. We used Noto font for the application for all countries so we made sure we did not get any TOFU.

Check : Usability Study for validation

Once we were in position to get the end-user to test the updated App we ran a Usability Study in order to get their acceptance and make sure all their local requests had been included in the final product.

Methodology

We picked up 8 participants from each country or group of countries for the sessions. I learned that 8 participants is enough to manage and get valuable insights. Our approach was to run unmoderated usability study. We first traveled on site but as the pandemic blew globally we quickly decided to switch to online sessions, using tools such as userzoom. We provided participant a list of tasks, asked them to think out loud and recorded the sessions.

Categorizing findings

One spreadsheet listing all tasks and participants was created. We then rated how difficult it was for each user to complete a task and added observations or suggestions from participants.

Themes

Data synthesis is time and energy consuming but I have to say that I love to do this kind of work. I tried to find similarities in the inputs, and group them under themes in an affinity diagram. I like to see trends emerging. Sometimes one or two participants would bring valuable improvements suggestions that no one in the design team had though off. Alongside these inputs, defects were also collected and categorized following a strict pattern :

Critical defect (Show Stopper: A defect blocking testing and go-live critical. As a business impact),

High priority (no workaround exists for a problem where testing of a significant application component or function cannot continue),

Medium priority (a problem that is not significant but impacts the functionality of the component and no workaround exists OR a problem that has a significant impact to the module but has a work around.  Issue identified for an important functionality within the module but workarounds are in place and user can proceed with the next steps.),

Low priority (no functional or Business Impact, nice to have, to be included in future interations).

Insights and change requests for stakeholders validation

Inputs were then gathered in a high-level presentation providing stakeholders data, insights and last minute change requests for approval.

Design project management

Keeping design decisions documented

Every design deviation from the standard template was reflected in the design system and documented. This helped us to keep the rational and context behind design decisions documented for future changes and also provided clear, written indications to the dev team. We also used design documentation when we required decisions to be taken from the governance team.

Clear phases closure

We kept the process transparent with locals but we also made clear that we had to meet deadlines and phases. Making closure of each steps signed-off helped us to move on and avoid endless design iteration rounds. We had clear criteria for each step and they had been presented to the team during our first kick-off meetings.

Providing team high-level progress and milestone

When working remotely on such a huge project, it's important to keep the team aligned and provide ways for all to understand where we stand and where we are going. We needed to cooperate with several remote teams based in different time zone. This made our communication process harder, so we had to make sure we worked as closely as possible. I tried to always over communicate and provide continuous feedback on the project being as transparent and open as possible when conflicts or misunderstanding popped up. One regular, fixed meeting was enough for us to align. I tried to name a facilitator and minute taker for every single meeting so we could keep track of progress from one week to another. The rest of the time we had "informal" catch-up and our slack channels to communicate on daily progress or issues we faced.

Alongside the Jira tickets I kept updated and shared an high-level design project plan on a page

Epic and stories visual checklist help our team plan the workload and provided a sense of tangible accomplishment in such a large project

Acceptance, deployment and fine-tunning

Local team has been deeply involved in the design phase so we did not face critical issues when running the acceptance tests. We did listed a couple of defects as usual but both design and local teams were happy with the result. I can not underline strongly enough that involving locals and being transparent from the very early stage of the process will ease your project day-to-day, change management and final acceptance. We closed this local project presenting the future iterations and releases. We kept our local cultural-expert in the global project information loop and sometimes reached out for validation when dealling with some standard changes in the UI.

UI Testing & QA

When it comes to testing the front end, there are a few different areas that we wanted to cover:

• Unit Testing
• Integration Testing
• Accessibility Testing
• Performance Testing
• Visual Regression Testing
• Browser/device testing
• User Acceptance Testing

Overall process

When a pull request is created, continuous integration tools start to run against the code in the branch. It’s during this step in the workflow that we run our automated tests, and if any of the automated tests fail, we block the branch from being able to be merged into the master branch.

Unit testing

We used Chai and Jest JavaScript testing tool for unit testing. We wanted to check if components are still working in isolation and if the version of the app in a new language did not impact the existing languages.

Integration testing

We used Jest for integration testing. We wanted to make sure all components interactions worked flowlessly.

Accessibility testing

When deploying a new country app deviation we run Axe automated testing runs as part of the development build to let us know what changes are required in terms of accessibility. We specified a couple of individual rules as mandatory for the app to be deployed to end users. These rules applied to all the countries in scope.

Performance testing

Some countries had a very limited broadband internet connection. We analysed the impact of images in loading time and tried to find design alternative for data-heavy pages.

Visual regression testing

Since we were constantly updating and pushing new code, we wanted to ensure that no visual defects were making their way into our apps. We included visual regression tests during continuous integration testing, and stipulated that any detected visual changes must be explicitly approved before the branch can be merged. Since we had all the design decision documented, it was easier for dev team to check and merge the branch. Since we were using React, we decided to use Chromatic set of side-by-side screenshots showing a before-and-after view with any differences noted for visual regression testing.

Browser/device for the web app

We have been running local testing with the good ol' Browserstack when it comes to browser and devices testing. Comprehensive tests were run or a major realease or a key component implementation. These tests included both desktop and mobile browsers.

User Acceptance Testing

We run formal sessions for user acceptance tests. The approach was the same we used for Usability Testing earlier in the process. We usually travel on-site with 1 or 2 devs and run 1,5 testing all the processes. Local countries had to provide us with their final validation so this testing was a clear project milestone we carefully prepared. If possible, defects where fixed and validated directly during these days on site. If some major issue were spotted, we would follow-up and iterate until local team provide their sign-off.

Feedback and iterations

After a country had been using the application for more than 6 months we started to collect data and planned on-site session for a feedback round. The question we needed to address was basically : are people really using our product? The reason is that we the product team can use it to figure how to really help, add value, and benefit the people using our product. We calculated the adoption rate using the following formula : Adoption rate = number of 30-day active users / total number of users in scope. Luckily in most countries people actually used the app. For those reporting bad figures in terms of adoption, before planning any session we tried to understand if there were any external issue preventing people to use the app (change in mobile phone assets, company IT policies...). The metrics we setup for this project were the following : Time on task (more complicated to measure than it seems!), Succes rate, Error rate, Degree of enjoyment, Time to 1st task and Abandonment rate. Before contacting the countries we also made sure their defect backlog would be empty without any issue unaddressed. Once we were confident with the data we planned 1 day to catch-up with local team to run our metrics testing and make sure nothing has been left behind. This feedback session was the official closure of the project in terms of meetings and tasks. We usually nominated a limited group of user we would contact on a regular basis for short follow-up interviews.

Final product and lessons learned

As I wrote it in the first part of this case study, this project has been a milestone in my career. It allowed me to be involved in all the phases and dimensions of the project. From reasearch to delivery and measurement. The project complexity and pace changed my mindset in some way and made me more humble and I would say, wiser as well. If I had to summarize they key things to keep in mind when working in this kind of project, these would be my advices :