Photo Credit: www.shopkeep.com
Testing Protocol
I chose to take an observation-interview hybrid approach when testing out the Confluence wiki documentation portal with my staff. For the initial reactions from my team, I observed their actions based on a set of steps I had asked them to take when interacting with the wiki site. From here, I would ask them key questions about their thoughts and interaction with the site, to gain feedback on suggested enhancements, areas for improvements and the components that they found valuable in processing student submitted forms.
Below are the documents used for the testing session: first is the blank protocol template that helped guide my testing session and second is the completed protocol with my notes (highlighted with key points that I will discuss below in which valuable feedback was provided for improving the Confluence page):
Below are the documents used for the testing session: first is the blank protocol template that helped guide my testing session and second is the completed protocol with my notes (highlighted with key points that I will discuss below in which valuable feedback was provided for improving the Confluence page):
Testing Report
Prototype:
The prototype being tested was a three page wiki-style website that holds all Degree Progress form documentation. The wiki portal included a main landing page and two developed pages for student forms that our office receives (leave of absence and petition for graduation quarter) so that users could interact with the main frame and subsequent documentation pages for specific form processing. The prototype will eventually include more pages, but for testing purposes I wanted to make sure I was on the right track with design, functionality and the user interface before crafting additional pages.
Context:
The goal of the testing session was to reach out to the previous user(s) identified in the empathy stage (my team members) to provide a solution to the current complexity in searching for form documentation. The wiki page was designed to simplify the search for existing documentation while also providing an update to how forms should be processed, all readily available within one link.
Users:
The users for the testing session were two staff members from my Degree Progress team. I manage three full time staff and one seasonal employee who is in the office during high volume time periods. With only two staff in the office this week, I observed both team members interacting with the website and then asked questions related to their interaction with the wiki page. Eventually, once the page is built out further, I would like to reach out to the Student Services Center staff to gather their feedback and additional suggestions. However, initially as this page is used by the Degree Progress team and designed for their identified needs, I focused on this population.
Method(s) used:
A hybrid model of observation-interview was used during the two tests. Initially, I asked staff members to open a browser window and click the link provided via email. I watched them as they browsed the main page and took some notes on my protocol form. I broke up the testing into testing the main landing page and then testing the documentation specific (leave of absence) pages. After they had briefly browsed the wiki site, I would state a set of simple actions and watch them perform these steps, noting any comments or questions they had along the way. Once complete, I would then ask a series of open-ended questions about the main landing page and documentation pages to gather feedback for future enhancements, areas that they found valuable, and portions of the page where edits could be made. I transitioned the testing session from being an observer, to instructor, to sponge – all with the goal to gather feedback via different approaches to improve the design of our Confluence wiki webpages.
Protocol design:
I used an excel spreadsheet to design a list of action items for the team members to take, with a slide scale of easy to understand, average and difficult to understand for me to quantify the task when observing. Depending on their confidence or hesitancy, I would mark a check mark in one of these areas to make sure the site was not too convoluted or confusing to use. I also included an area for comments, so I could make notes after each step they took, and also to document any suggestions/questions they raised during the testing session. Please see the attached photos above for examples of the protocol design.
Feedback:
Overall, the session was informative with interesting suggestions, questions and points that I had never considered. I’ve outlined a few of the comments and further down below I have created a list of the suggested enhancements that I will try to incorporate into the design.
I really found the testing session beneficial on a larger scale as well, as there were areas that I didn’t consider in my original design. As one of the users was new to wiki-style webpages, many of the questions were related to using Confluence and how to add/edit material. While this is not related to the content, it does bring up an excellent point that a brief team training will need to take place to get all users up to speed on how to use the product. I was focused so much on the design and content that I failed to acknowledge the new medium and how users may interact with a lesser known application. So on the Confluence page I will likely also include a brief training link on how to add, edit and link pages so that my team can design additional pages (or modify current ones) as needed.
I also found my knowledge of the Stanford Design model and the common occurrences to recycle back through the steps made the testing session a more positive experience. Often when you present a project, any feedback or suggestions can be viewed as criticism. Knowing that the testing phase was not my final stage allowed me to see the suggestions in a positive manner; a chance to build on a successful prototype to make the Confluence page even that much more valuable.
Next steps:
After gathering some valuable feedback, it’s back to the drawing board and the wiki page design. Per the suggestions below, I will create a few new features in Confluence and go back to these users for additional testing:
The prototype being tested was a three page wiki-style website that holds all Degree Progress form documentation. The wiki portal included a main landing page and two developed pages for student forms that our office receives (leave of absence and petition for graduation quarter) so that users could interact with the main frame and subsequent documentation pages for specific form processing. The prototype will eventually include more pages, but for testing purposes I wanted to make sure I was on the right track with design, functionality and the user interface before crafting additional pages.
Context:
The goal of the testing session was to reach out to the previous user(s) identified in the empathy stage (my team members) to provide a solution to the current complexity in searching for form documentation. The wiki page was designed to simplify the search for existing documentation while also providing an update to how forms should be processed, all readily available within one link.
Users:
The users for the testing session were two staff members from my Degree Progress team. I manage three full time staff and one seasonal employee who is in the office during high volume time periods. With only two staff in the office this week, I observed both team members interacting with the website and then asked questions related to their interaction with the wiki page. Eventually, once the page is built out further, I would like to reach out to the Student Services Center staff to gather their feedback and additional suggestions. However, initially as this page is used by the Degree Progress team and designed for their identified needs, I focused on this population.
Method(s) used:
A hybrid model of observation-interview was used during the two tests. Initially, I asked staff members to open a browser window and click the link provided via email. I watched them as they browsed the main page and took some notes on my protocol form. I broke up the testing into testing the main landing page and then testing the documentation specific (leave of absence) pages. After they had briefly browsed the wiki site, I would state a set of simple actions and watch them perform these steps, noting any comments or questions they had along the way. Once complete, I would then ask a series of open-ended questions about the main landing page and documentation pages to gather feedback for future enhancements, areas that they found valuable, and portions of the page where edits could be made. I transitioned the testing session from being an observer, to instructor, to sponge – all with the goal to gather feedback via different approaches to improve the design of our Confluence wiki webpages.
Protocol design:
I used an excel spreadsheet to design a list of action items for the team members to take, with a slide scale of easy to understand, average and difficult to understand for me to quantify the task when observing. Depending on their confidence or hesitancy, I would mark a check mark in one of these areas to make sure the site was not too convoluted or confusing to use. I also included an area for comments, so I could make notes after each step they took, and also to document any suggestions/questions they raised during the testing session. Please see the attached photos above for examples of the protocol design.
Feedback:
Overall, the session was informative with interesting suggestions, questions and points that I had never considered. I’ve outlined a few of the comments and further down below I have created a list of the suggested enhancements that I will try to incorporate into the design.
- How can I add additional content and edit the pages? Do we have security to update the wiki page ourselves?
- I see this as a neat tool for new temp staff who need to learn the forms processing, with a single location to visit online
- One page layout with scrolling features to view screenshots is very easy to navigate. Upgrade from Word documents, as the screenshot and content are next to each other for view, versus screenshot and then the directions.
I really found the testing session beneficial on a larger scale as well, as there were areas that I didn’t consider in my original design. As one of the users was new to wiki-style webpages, many of the questions were related to using Confluence and how to add/edit material. While this is not related to the content, it does bring up an excellent point that a brief team training will need to take place to get all users up to speed on how to use the product. I was focused so much on the design and content that I failed to acknowledge the new medium and how users may interact with a lesser known application. So on the Confluence page I will likely also include a brief training link on how to add, edit and link pages so that my team can design additional pages (or modify current ones) as needed.
I also found my knowledge of the Stanford Design model and the common occurrences to recycle back through the steps made the testing session a more positive experience. Often when you present a project, any feedback or suggestions can be viewed as criticism. Knowing that the testing phase was not my final stage allowed me to see the suggestions in a positive manner; a chance to build on a successful prototype to make the Confluence page even that much more valuable.
Next steps:
After gathering some valuable feedback, it’s back to the drawing board and the wiki page design. Per the suggestions below, I will create a few new features in Confluence and go back to these users for additional testing:
- Provide a training page on how to use Confluence, and place this on the main portal page (also show staff how to edit/add their own pages)
- Include links to the actual forms students fill out for reference
- Develop additional form pages and create pages for Reporting documentation next
- Integrate PeopleSoft Axess (student information system) links to the Confluence page to create a documentation page that has accessible links to take you directly to the PeopleSoft panel.