Genies
Senior Technical Artist - Genies - 2021-2026
Overview
I started at Genies in 2021 as the second person in an Avatar Tech R&D department of two. We grew into a small, scrappy team ready to take on any challenge. My years at Genies were deeply academic; constantly pushing my limits and learning new things.
Much of my work over the last three years grew from a simple idea: can we take facial animation channels and retarget them as needed? Can brow and mouth animations drive standardized expression channels on animal ears? Can we retarget eye channels to multiple eyes?
This question launched my work at Genies in two directions: generative character rigging, and generative animation via LLM tagging. On the rigging side, I went from defining our face-rigging pipelines to building an AI-driven facial auto-rigger, and eventually owned synthetic character data generation, powering both the auto-rigger and generative character mesh creation. On the animation side, I went from evangelizing the utility of simple, category-based animation channels to building Unity animation systems designed to catch LLM tags and drive procedural animation.
Generative Characters
Face Rig Pipeline
After building a system in Unity that let us parse out facial animation channels as described above, I launched a campaign to expand the team's capacity to rig non-standard faces for ARKit. I adopted Face-It, mastered it, then trained our artists to use it — greatly improving the pace of "Doll" character production (short, cartoony-themed characters).
AI Facial Auto-Rigger
In 2023, generative AI was becoming more than a buzzword. As our ML team expanded, I began building an AI-driven facial auto-rigger around a simple prompt: given OpenCV-style landmarks projected onto a 3D character face, could that face be procedurally rigged to 80–90% fidelity? Fidelity here didn't mean speeding up an artist's work — it meant production-ready blendshape rigs that performed perfectly with zero white-glove tweaks.
The theory was simple: wrap an existing ARKit face topology to the novel head using the landmark data, scale the deltas to fit the new proportions, transfer the blendshapes, done. In practice, that wrap-and-transfer approach only got us to about 60% fidelity — most facial features needed real development to get right, largely through masking systems that prevented feature bleed by correctly splitting and isolating the eyes, lids, lips, brows, teeth, tongue, eyelashes, and any other separate mesh on the face.
Splitting upper and lower lips made opening a mouth fairly easy. Closing an eye convincingly, with no visible seam, was a much harder problem. After several iterations, I landed on a process that averaged landmark positions into a center line, curved that line to match natural eye proportions, skinned joints along the lids using those landmarks, rotated the lids together at the seam, and used the eyeball mesh itself to gently repel the lids and prevent interpenetration.
Landmark Dataset
An OpenCV-style facial landmark model drove the auto-rigger, with its points serving as the foundation for all downstream rigging. I worked closely with the head of ML to identify where that model needed to improve, and with no staff devoted to synthetic data, I built a local image-generation pipeline myself — bulk re-imagining a pool of head renders and turning my own feature requests directly into training data.
The clearest example came as our target cohort expanded toward cartoony faces with larger eyes, which demanded new eye landmarks. I designed a solution that split the eye corner into two landmarks: for humanoid faces, the two points simply overlapped, preserving backward compatibility, while for cartoony eyes they separated to give better surface coverage and higher-fidelity blink shapes. As a key benefit, the gap between those two landmarks became a simple, AI-determined boolean the auto-rigger could use to detect whether a given eye was realistic or cartoony. I then built the datasets to train the model on this new feature.
Synthetic Character Generation
From here, I owned synthetic character asset generation and led the Tech Art department's efforts to supply datasets for the ML team. I refactored and expanded our modular hair system, built a Houdini pipeline for our first wave of face-generation and segmentation datasets, and created a Blender facial variation tool that generated attractive head variants to improve our auto-gen character pipeline.
The variation tool was the standout piece. It took our base Genie head and, through a mix of blendshapes and facial deformation joints, generated variations directly on the timeline, with real-time texture projection and mesh updates so I could scrub and QC each result instantly.
A web GUI drove the system, letting me define variation points, generation targets, and procedural constraints, then save them as reusable profiles to sculpt output toward specific facial cohorts.
Inside the web GUI I added a Value Trace menu that allowed each value to be simulated, with stepwise value readouts. With so many value tweaks happening under the hood, this clear debug menu was essential to tracking down the cause of unexpected results.
A snapshot system let me export or load heads as JSON metadata, which served two purposes: bulk-exporting good and bad examples for an LLM to find correlations and recommend generation guardrails, and exporting standout heads into an "attraction system" that averaged nearby high-performing heads into a weighted delta, nudging new variations toward attractiveness automatically.
Tagging and Buckets
As the project expanded, we began ingesting a large pool of authored head variations. I designed and implemented a batch LLM API prompt to evaluate and tag each face blendshape based on its features. After building a tag framework and running my first tests, the key difficulty I faced was style adherence. The LLM would consistently describe the Genies art style rather than the individual features of the target head. I was able to build a secondary prompt that leveraged face atlas images to anchor the LLM to the Genies art style. This improved tagging greatly.
The purpose of this tagging system was to further improve our ability to bucket facial features within a pool of 600+ heads. The ML team was going feature by feature, expanding capacity and refining quality. For example, the model had difficulty producing thinner heads, and more data fitting that cohort was required. The tagging system allowed us to agentically build buckets of correct facial features with simple written requests. These facial features were then further varied using the synthetic head variation tool.
Generative Animations
Coming Soon!
