← Back ← Back

Outline

In the class “Designing Interaction”, taught by Serveri Uusitalo, we received a task from Microsoft: to design a multimodal interaction experience to support a creative professional’s need and tools.  
 
Our concept is the plug-in “Outline”: an extension for any prototyping tool such as Adobe XD. The extension allows the user to sketch wireframes directly in the prototyping application by using the pen and to digitalize these hand-drawn sketches quickly into geometric shapes and lines. This allows for a smooth transition from sketches to high fidelity prototypes. Our interactions with the Surface make it easy to change and adjust the designs. 
 

Year: 2018
Client: Microsoft
Task Duration: 3 Weeks
Group Members: Xinyue Du, Lin Liu. Niko Aho and Jui-Fan Yang
Main Tasks: User research and analysis, (Interactive) Prototyping

Brainstorming

After some brainstorming we decided with the team to improve designers’ works, especially in the early stages. We had noticed that in OneNote you can draw with the pen, but we couldn’t find a prototype tool that allows this in their application. In our work we often used a whiteboard or notebooks to brainstorm the first designs together with other designers and stakeholders. After the initial designs had been agreed upon we recreated them into higher fidelity designs with the prototyping tools, such as Figma, Adobe XD, etc.

As the recreation takes a lot of time we wanted to come up with a better way for designers to translate their sketches into high fidelity designs. For this, we researched touchscreen and pen-interaction technologies to grasp the possibilities. We also conducted research into the userbase, which consisted of designers and product managers, and used it to figure out how a sketching tool could help their brainstorming sessions.

Research

1.
Discover
2.
Define
3.
Develop
4.
Deliver
Getting an overview of Microsoft Surface

As we were not too familiar with the product, we investigated more about all the features, applications and devices that were designed for and with Microsoft Surface to get a general overview of the product’s possibilities.

Additionally, we did a contextual inquiry into Microsoft Surface users (mostly artist and designers) and asked them about their behaviours with the pen and tools. Through the inquiry we could already identify one common pain point: 

  • Keyboard/mouse inputs are needed to use shortcuts during the users’ work progress, which disturbs the workflow
User Research

In total we interviewed 7 designers and 5 project managers (who are used to collaborating with each other) to figure out their design process and how they collaborate. Based on the research we created personas, scenarios and hierarchical task analysis.

Interaction research

We read some research about interactions with the tablet and stylus as we had no prior experience of these. The research helped us during our design process to create better experiences. 

Researched Interaction

Test and Iterations

Based on the research we had conducted we thought of interactions which could help the user in the scenarios we had created. Especially our academic research inspired us to use the pen and touchscreen in a more efficient way. Our goal was to avoid any mouse- and keyboard interactions, and to keep the flow of using the pen during brainstorming sessions. 

To design the interactions we first created some wireframes, and built them using paper. We noticed early on that the paper prototypes helped only to some extent with testing the interactions. However, some of our idea could not be represented with them. Therefore, we coded a simple interactive prototype with JavaScript to better visualize the interactions. After a few iterations and usability tests we received some feedback on our designs, such as including a feedforward to give the user a better idea of the functions. Additionally, we created further interactions with video editing software to further communicate our ideas. 

Interaction Designs

Multimodal Inputs

After the tests, we finalized the interaction designs and worked out multimodal inputs into the aspects of our scenarios, which can be seen below.

Touchscreen

Gestures

  • Tap
  • Spread for zooming, pinch for zooming out
  • 2 Fingers to hold and move the canvas
  • Tap and hold to initiate the dial menu 
  • Rotate to navigate the dial menu 
  • Hold with thumb to initiate marking menu 

Dial Menu (Tuck the pen and rotate)

  • Adjust brush size
  • Copy
  • Delete 
  • Create new art board

Marking Menu (Hold with thumb and move) 

  • Copy art boards (Hold and drag artboard)
  • Related documents
  • Switch modes
  • Polish designs 
 

Pen

Pen Tip

  • Draw
  • Write
  • Click
  • Move

Pen Button

  • 1 click to undo
  • 2 clicks to redo
  • Hold to select area

Eraser button

  • Erase elements
  • Disconnect artboard

 

 

Speech

  • Speech to text to make notes

Conclusion

This task provided many challenges, first of which was to have to design new interactions, which are multimodal, and to do it with technology we were not too familiar with. Moreover, we needed to define a concept to apply these interactions to in a useful way.

Our difficulties were mostly in the beginning, when we had struggle to define a precise use case, as we were focused on Microsoft’s idea and branding of their Surface series. As we then decided to focus on the scenario of wireframing we could finalize our key path scenarios and research, which led us to the final interactions. 

Scroll up