Welcome to my self-directed projects!

I've always seen programming computers as creative expression. As a developer-designer hybrid, I've spent my career in data and engineering-heavy domains at Google, the Obama 2012 campaign, Pivotal, and the U.S. Space Force (linkedin).

Creating the AI-native apps on this site helped me explore new types of user experiences, but also realize how design and engineering are evolving and merging together. That's led me to think that in this unexplored world, design needs to be in two places at the same time: 1) Redefining how users interact with machines, and 2) Creating the next wave of tools that enable this entirely new kind of software.

kgates@gmail.com

1. Designing Serendipity

I created an AI-native app that re-imagines navigating Wikipedia with left and right swipes. With AI subtly working in the background, my app generates narrative arcs that entices users into new, unexplored territory.

  • • REACT NATIVE
  • • CODE-AS-DESIGN
  • • AI-NATIVE
  • • UX SIMULATIONS
Case study

1. Designing Serendipity

Right-swipes create a constantly emerging narrative

Right swipes work something akin to a moving average — users can swipe as long as they like, and the app will act as a subtle guide that finds connections between ideas.

1. Designing Serendipity

Left swipes back out to broad perspectives

What if a right swipe leads to something that is not intriguing? The user can swipe left and go to a broader, related subject, and keep doing that until they decide to swipe-right.

1. Designing Serendipity

Navigation is non-deterministic

Way-finding will be very different in the world we are entering. For this app, left swipes tend to eventually peter out. When that happens, a warning appears explaining the user has hit an epistimological ceiling.

1. Designing Serendipity

Prompts are part of the creative process

Before I wrote any React Native code or sketched out ideas, I created a Python-based simulation of the user experience.

1. Designing Serendipity

Code is design

I worked through most interaction design problems in code. Iterating on a design with a device in-hand is unparalleled.

1. Designing Serendipity

Speed is fundamental to the UX

I wanted swipes to feel fast, simple, and infinite to the user. I created a simulation to help me figure out how to make preemptive LLM API calls asynchronously.

2. Fusing Gestures, Speech, and AI

I'm fascinated by how emerging technology has the potential to transform how people interact with machines. This prototype explores the fusion of gestures and speech to interact with a locally-running LLM (gemma-2b).

  • • REACT
  • • LOCAL LLM
  • • GESTURE RECOGNITION
  • • SPEECH RECOGNITION
Case study

2. Fusing Gestures, Speech, and AI

The space in front of the screen is part of the interface

The touch screen, keyboard and mouse now have viable alternatives that are untethered from physical devices: voice and gesture.

Case study

2. Fusing Gestures, Speech, and AI

The language model is local

The app uses a locally-running version of Google’s gemma2-b, enabling incredibly fast responses.

2. Fusing Gestures, Speech, and AI

Gestures are JavaScript events

I created a JS wrapper around MediaPipe’s open source gesture-capture library. This allowed me to experiment and iterate in code.

2. Fusing Gestures, Speech, and AI

A raised hand conveys intent

A raised hand conveys intent to speak. That turns on the mic to capture speech, which is turned into text and sent to the local LLM, which streams back a response.