Windows 10 Globalization Walkthrough for Bi-Directional Languages - Text Input Experience
In part one of the Windows 10 Globalization Walkthrough for Bi-Directional languages we provided a review of Desktop, Start Menu, Task Bar, and Notification center. We continue the second part our UX walkthrough with the Text Input Experience, please check out our previous articles if have not done so already:
Text Input Experience:
Text input is the remains a primary scenario for multilingual users regardless of UI language preference. Users expect text input to allow seamless swapping between multiple input languages, and fluid interaction in text editing scenarios. BIDI languages make it especially challenging because text directionality changes. We configured our input via the Time and Language settings to include Arabic, Hebrew, and English (United States).
BIDI users relay on a caret indicator positioned at the top of the cursor to indicate input directionality, We found that in certain text fields the cursor failed to refresh appropriatelyafter using the Win+Space shortcut keys to swap input languages, sometimes it would simply stop blinking, display clipped, and or overlap characters. Many of these issues are regressions from previous Windows releases.
BIDI Text selection UX is tricky, unpredictable and buggy, especially under mixed text scenarios where English text might be nested in Arabic or Hebrew sentences and vice versa.
The problems in BIDI text selection stems from the current implementation which relays on the logical character arrangement within the memory. While the current text selection experience might be behaving as designed, it makes for a poor user experience that might have been acceptable when Windows 95 was shipped.
Native Digit Input Support
Arabic input requires native digit support which allows for numerals to be represented in Eastern Arabic format, our review of both physical and virtual keyboards found inconsistent support for native digits. From a user's perspective there is no guarantee that digit input will display as intended. Microsoft implemented something called Contextual Digit Substitution, it allows for the text rendering engine to swap digit representation based on the direction of the parent string. The swapping occurs when the OS UI language and regional settings are set to Arabic. Contextual Digit Substitution is a patch designed to compensate for the keyboard which under certain scenarios fails to send the appropriate Eastern Arabic digit. Upon further investigation it turns out that Eastern Arabic digits are categorized by the OS as characters rather than integers, Microsoft will first need to address this limitation before updating any Arabic keyboard. Besides the obvious UX impact, this problem is likely costing Microsoft additional localization resources as strings with digits may require special tweaking to ensure they are displayed appropriately.
Text Correction which was used in the creation of this article displayed a couple of issues under the multilingual environment, Often English text would get highlighted by the spell check in the event where Arabic or Hebrew text was inserted into the string. Right clicking on the highlighted word would popup the context menu which would offer to delete a repeated word.
Text prediction for BIDI languages was only made available via the On Screen Keyboard, seems odd that Microsoft has limited text predication capabilities given its long history of language support and its investment in productivity. The On Screen Keyboard had issues with text prediction under certain scenarios, often the text suggestion were flashing back and forth while editing in the Edge browser.
On Screen Keyboard
The On Screen Keyboard has a number of issues, The most obvious was the UI layout which was not adjusted for BIDI languages. Both Window controls and text prediction layout remains left to right oriented.
BIDI Directional commanding keys:
These keys are used to adjust the input direction and are available on BIDI keyboards including Arabic and Hebrew but removed from the English (US) keyboard. leavingmultilingual users without the ability to adjust the text direction during the input process while swapping languages. The missing keys are a regression from previous releases.
One more On Screen Keyboard problem that effected multilingual users had to do with the Shift state. Under certain conditions initiating input into a text field with English (US) keyboard selected would leave the Shift key pressed after the input language was swapped for Hebrew. The Hebrew keyboard layout will display English letters while the Shift key is pressed, leaving the user unsure whether or not the input languages has been swapped correctly.
Inking and Handwriting
For users with hardware that supports inking via a dedicated pen, windows 10 also offers the ability to input text via handwriting. While BIDI languages are not currently recognized, users with Arabic or Hebrew UI language settings should still be able to input other languages such as English via inking. Using the Ink Input Panel we were able to input English handwriting which was recognized correctly, we then attempted to select a mixed language string and found that the Ink Input Panel swapped the positioning of the BIDI text within the sentence. We also attempted to complete a Local Search via the Start Menu but The Ink Input Panel was unable to recognize our English handwriting, upon further investigation we discovered that while the UI appeared correctly it was actually mirroring our handwriting. For example, the letter "b" was recognized as "d" and ink commanding was completely nonfunctional as a result of the mirroring bug.
Microsoft has been leading the accessibility effort in recent years, the narrator has been expanded to include support for several languages including Arabic and Hebrew. Narrators core functionality allows users to hear what they are typing. Unfortunately, this feature completely fails to detect input language changes while using modern App's, leaving users confused and unable to complete simple input scenarios in more than one language. Our Hebrew input was spoken and mapped to corresponding English characters on the keyboard.
Walkthrough Conclusion and Key Takeaways: Input Experience
Microsoft has been very conservative in terms of updates to text interaction, in fact the last time they updated text interaction they helped standardize the BIDI Parenthesis Algorithm in 2011. What was surprising is the number of bugs that were discovered in this walkthrough. Seems as if simple integration testing was excluded from the release which ultimately deteriorated quality.
How Microsoft can Improve:
Educate and engage Design, Development, Program Management, and Quality teams on BIDI market requirements.
Create workflows that allow Design and Program management resources to streamline BIDI UX across the products.
Ensure Development and Quality teams perform product testing on Arabic, Hebrew, or Pseudo Mirrored localized builds.
Incorporate BIDI text input integration testing into the release.
Fix some of the long standing design limitations and eliminate digit substitution for Arabic.
Start investing in BIDI text interaction usability testing and examine alternatives to logical text interaction.
Looking to create a more inclusive multilingual experience?
Check out my recent articles on the topic of multilingual UX:
The World Ready Guides plan to engage with the world's leading software producers by publishing product reviews that focus on software globalization, our goal is to raise more awareness within the design community by providing practical real-world examples sourced out of today's most popular software products.