Renault-Nissan HMI Prototypes
As part of my time as an UX Front End Engineer on the UX & HMI teams at the Renault-Nissan-Mitsubishi Alliance Innovation Lab in Silicon Valley, I explored novel use cases and user experiences for autonomous vehicles. These projects looked at different interaction paradigms for both in-vehicle interactions as well as interactions occuring outside of a vehicle, with a pedestrian or first-responder, for example (some projects not documented here).
This prototype was the beginning of voice-interaction exploration. This prototype evolved into a React web app served on AWS that enabled for much more complex voice commands. This more mature prototype got rid of the projectors and simulated environment. The prototype was integrated into an actual vehicle, enabling testing to be done during actual drives. Below are screenshots from the early prototype of TouchDesigner, a visual node-based programming platform, which I used to control particle engines (to simulate snow and rain) and multiple video outputs to a projector and large display screen. TouchDesigner also received OSC Messages from a raspberrypi equipped with a touchscreen. The raspberrypi ran a processing sketch as well as access the Google AIY Voice API via Python script developed by mechatronics engineer Tomoki Eto. By pushing the button on the touchscreen, a user could toggle vehicle functions, such as winshield wipers, which modified the output in TouchDesigner. You can see the example of windshield wipers being turned on/off.
The wipers could also be activated by voice commands, which was prototyped with the raspberry pi Google AIY Voice kit. Interactions and timing was done in python which sent the relevant OSC message to TouchDesigner.
This system was designed to be flexible in order to either output the entirety of generated visuals to a single external display, or to create a feeling of depth by delegating the relevant assets to two external displays -- one acting as the foreground plane, the other as background. In this example, the windshield wipers and weather artifacts (rain or snow) were projected onto a windshield with a projection film applied to it, while the context of the road was displayed on a large display behind the windshield.
