Last week I set the scene to walk you through my SIGGRAPH2020 labs session focussing on introducing spatial design concepts for beginners.
What follows is a particular take on how to begin thinking about interaction design 1) in the spatial context that XR enables and 2) visualising ideas via sketching - not necessarily in a traditional way, with pen and paper, but also via creating mockups and collages that communicate ideas.
The SIGGRAPH lab session is a result of pivoting an original workshop format into online, remote delivery. My extended abstract in the SIGGRAPH proceedings documents the original approach, where I employed both physical props and prototyping software in instructing participants to come up with simple spatial designs.
Original workshop activities from AR to VR to physical props:
The workshop consisted of a set of exercises leading from physical spatial problem-solving to transferring the same problem to AR and then VR. I submitted the format to SIGGRAPH and was intending to deliver it in Washington in July. Soon after submission things went pandemic.
How to marry remote, hands-on, and spatial
I struggled quite a bit in taking the approach into a remote version because the centrepiece of the original one was the step from physical objects to their virtual counterparts in augmented reality.
I have observed close-by how moving into manipulating assets in an AR prototyping app, after working with them first physically and therefore having a reference point to their physical qualities, can be an eye-opening moment for workshop participants who have not engaged extensively (if at all) with XR.
In the original workshop format, I used Torch with reproductions of the wooden marble course blocks as 3D models to facilitate spatial thinking via AR:
Yet, experience has also shown that asking participants to come into a remote session with any kind of specialist software, let alone hardware, risks a failure. Therefore for the online delivery, I opted for a more conceptual approach and drew from various sketching and paper prototyping approaches mentioned in the earlier post.
Making collages is sketching for XR
After several iterations, I settled on an approach that uses a visual online collaboration space (happens to be Mural in this case, but another similar service would to the trick) with a set of resources that are planned for facilitating the participants to come up with tangible results in less than 60 minutes.
Overview of the entire Mural board for the session - participants are lead through step by step
The question I’m probing with these two posts is around what sketching for spatial designs can be. The workspace suggests an answer to the question - namely, the answer is using a collaborative online space to quickly mock up collages that communicate a spatial interaction solution.
Evoking and provoking spatial thinking
The approach aligns with the characteristics that Bill Buxton has attributed to sketching, i.e. it being evocative, provocative, explorative, etc. However, the collage resources aim to capture the spatial nature of the interactions being depicted, which is something that “traditional” pen-and-paper sketching would not necessarily emphasize unless the sketcher is experienced in the topic.
By drawing a parallel to the evolution in web design, my aim is to provoke participants into letting go of their rectangle-shaped mindset and embrace the spatial potential of XR:
The experience here refers to a tendency towards thinking interactions in 3D activity zones instead of within 2D, screen-based rectangles. Spatial thinking is in-built in all of us because we interact with most things naturally like that, but it needs to be sussed out in design contexts, especially if the designer comes from screen-based conventions (instead of e.g. product or set design).
Exploding the browser as a step into spatial thinking
To achieve that, I wanted to place something instantly accessible as the centrepiece of the class exercises. Something that we might assume will find its spatial and multisensory equivalent in XR - not just as a screen-based antecedent with a curved surface but as a proper spatial reincarnation that unbundles the features into 1) space and/or 2) into multisensory use cases.
My choice was the browser: “Exploding the browser” became the exercise headline:
What would accessing the web in a spatial XR reincarnation look like?
I felt that the browser was a suitable choice also because breaking down its features, and how they can be accessed, felt at once manageable but there were also features - such as the search field - that had already been taken into a multisensory direction, with browsers offering voice input. Thus, engaging with the challenge the “spatial browser” presents would presumably shift the participants’ mindset towards both spatial and multisensory solutions.
Part of the Mural board
To facilitate the class, the Mural board has a bunch of visual aids to give the activity focus - and a slight sense of urgency - to yield tangible outputs. I tested the approach with colleagues before the Siggraph session, discovered a few key omissions and iterated the Mural and my exercise briefs and materials accordingly.
The collages as loose prototyping specifications
For example, the class has a modified “crazy 8s” ideation exercise around browser features and embodied ways of interacting with them in a workspace setting. This rapid ideation is meant to give ammunition to the next phase. With the help of visual aids and templates (inspired by e.g. the great work by Mike Alger and others), the class aims at each participant being able to visualise and present a couple of ideas in a collage form.
A snapshot from the Mural and the second exercise, where I instruct the participants to mock up collages of spatial browser features snd how to interact with them:
The collages are the main output from the 45-min session. Their function is to suggest potential prototyping directions. A collage can work as a spec for a brown-boxing effort to test the idea’s feasibility in spatial form, using physical materials, or as a rough spec for rapid prototyping in AR or VR, to explore feasible layouts, object forms and scales, functionalities and interactions, in digital form.
That’s it for now - I’m planning to post reflections from the SIGGRAPH session in a week or so.
Thanks for reading & stay safe,
Aki