TDP Project Update - April 2022
Since our last update focused on the results of our research around resubmission, we have been researching the navigation hierarchy of the new TANF Data Portal (TDP).
As you maybe aware, TDP is a web-based platform that will—in its first release—allow users to securely upload and download quarterly TANF files. Future releases will introduce a host of other functionality that enhances the data transmission experience for our users (e.g. revamped transmission and questionable case reports in TDP).
The way pages are organized in a website—its Information Architecture (IA)—is an important aspect of the user experience of any application or website; it’s something that can make the difference between a frustrating or enjoyable experience. Our research on this front has been aimed at ensuring that our users will be able to easily navigate TDP and find information that they need on it right from its earliest versions—as well as to make sure that as TDP grows more advanced in its functionality it doesn’t become more confusing to use.
How the test was set up
There are two core methods used to test the IA of a website:
- Card sorting, where users group menu labels into headers that make sense to them.
- Tree testing, where users are asked to find specific functions within a list of nested labels.
To understand the usability and legibility of labels associated with current and future pages of the TDP platform we opted for a type of tree-testing. Using Google Forms, we created a survey that replicated the questions and answers of a typical tree-test and administered the first round to our users in the Office of Family Assistance (OFA). The second round was facilitated with a variety of TANF grantees.
Through this current study, we wanted to understand whether different pages of the new TDP website are easily identifiable and reachable. In other words, we want to know if the navigation patterns of the new website made sense to user and supported their ability to quickly find what they’re looking for on the portal.
How the form worked
Through a carefully prepared Google Form, we asked questions about where users could find something on the website and asked them to choose from a list of options. Each of these options represented a page or tab within a page on the portal. Each question had two parts. The first introduced a set of menu labels one tier above the intended answer in the page hierarchy (e.g., Sign up, Log in). The second introduced the sections of a page where the user could actually find information they were seeking (e.g. Profile information, Request Access to Portal).
Who participated
Most participants correctly identified menu labels and successfully found relevant information by choosing options that led to destinations aligned with our existing plans for TDP’s Information Architecture.
One of the key metrics used to evaluate a tree-test is its success rate, which is the percentage of participants reaching the intended destination for a particular task. Devoid of any visual cues like search bars or hyperlinks, tree-tests by nature are stripped down to bare menu labels that users are asked to make sense of.
We derived the success rate by calculating the total number of participants who arrived at the correct answer to each question. Statistically, this meant calculating the number of correct answers for Part 2 of each question and dividing those by the total number of participants. Results were interpreted based on the importance and relevance of tasks based on essential TDP functionality for different types of users. For example, for an OFA user might need to navigate some of TDP’s administrative pages whereas an STT user will only need to navigate the more data-centric pages.
Below is a table that details the success rate for our study.
| Categories (OFA & STT) | Correct Path | Success Rate (25 Participants; 15 OFA, 10 STT) | Number of participants who arrived at Question Part 2 | Visualized Pathway Results |
|---|---|---|---|---|
| TANF Information | Resources-> TANF Transmission File Layout | 68% | 21 |  |
| Submission History | Log in->Data files->Submission History | 76% | 19 |  |
| Reports | Log in -> Reports -> Feedback Report | 68% | 20 |  |
| Onboarding | Log in -> Help-> Tutorials | 40% | 13 |  |
| Data Dashboard | Log in -> Help-> Tutorials | 76% | 19 |  |
| OFA User Management | Log in->Admin Profile->Manage Users | 87% | 13 |  |
| STT User Access | Log in->Home-> Request Access to Portal | 20% | 2 |  |
| STT File Upload | Log In -> Data Files -> Upload File by Quarter | 70% | 8 |  |
The success rate is depicted in the table above along with other relevant choices for some questions. These choices are useful for further onboarding research and helping users familiarize with the platform.
Interpreting the results
The proposed hierarchy of pages and nested labels for TDP were found to work for both OFA and STT users. Many our users were able to correctly find TANF information, Manage Users and Access Submission History, Feedback Reports and Analytics according to this survey.
However, finding onboarding material like in-app tutorials proved confusing for many users, with only 40% of participants arriving at the correct choices.
Project impact
A significant insight from the first phase of tree-testing is that the proposed Information Architecture is usable and navigable. However, to create a friction-free navigation experience, it is critical to address and control for incorrect pathways and gaps in success rates.
We map project impact by analyzing the level of risk associated with a user not arriving at an intended destination. In other words, if a user cannot perform critical functions within the app because of confusing navigation patterns, it might require changes to the TDP Sitemap, or to other designs.
Risk Categories:
- High Risk - Critical function, low success rate (less than 50%)
- Moderate Risk - Non-critical function, low success rate (less than 50%)
- Low Risk - Critical & non-critical function, medium success rate (between 50-80%)
- Minimal Risk - Critical & non-critical function, high success rate (greater than 80%)
Conclusions and future research
Our findings broadly indicate that the new TANF Data Portal (TDP) will have a usable and navigable structure with two areas of potential improvement—User Access and Onboarding.
User Access
Given elements of how logging into TDP will need to happen for security reasons, User Access is one of the less alterable areas of functionality. To ensure that sign-up and sign-in to TDP is a smooth experience we’re pursuing a step by step guide on Getting Started in TDP as one of the first materials prospective new users will receive for OFA.
Onboarding
The ‘Help’ label leading to Tutorials and FAQs is one of the areas with a low success rate. Though it is an area that can be addressed by onboarding, as a future page, its position in the hierarchy can be further explored and is a likely area for future research. Future research could also explore whether a different name for the page would be more successful.
Reminder: We have office hours!
In addition to many upcoming structured and planned rounds of research we'd love to offer up some unstructured time to hear from you. If you have questions about the project, feedback about elements of the current system you'd like to see TDP address, or have anything else in mind, we'll be offering a number of time slots to hear it each month. If the available times don't work with your schedule or you'd simply like to get it touch via email, please feel free to continue reaching out via mreiter@goraft.tech.
Book Office Hours