How to make a floating head
In a previous post.
I showed how to run BodyPix on a video stream
and access the segmentation from your shader.
In this post,
I demo the
using it to make a floating head.
You can run it on your own webcam;
A call to
net.segmentPerson returns a
where each value is either
1 (part of a person)
0 (not part of a person).
But the API also provides
which returns an
but where the values are between
-1 being not part of a person,
and the other numbers being various parts.
Here I’m just interested in values
which represent parts “left side of face” and “right side of face”.
To access this data in a WebGL shader,
we need to get it into a texture using
When you pass an array to
you tell it which format to interpret it as.
gl.ALPHA, which has one unsigned byte per pixel.
To convert to this format, we can use
-1 values in signed
ints get converted to
255 in unsigned ints.
This byte interpreted as the alpha channel when the texture is accessed by a shader.
Then in the pixel shader, these values between
are squeezed into the
float range from
So we can check for the original values
alpha <= 2./255..
BodyPix seems to include “neck” as part of the face. This is annoying for my purposes; I wanted to crop the face at the chin. I don’t see an easy way to hack around this.
Finally, here’s what I get when I run the demo against my own webcam feed:
More by Jim
- Your syntax highlighter is wrong
- Granddad died today
- The Three Ts of Time, Thought and Typing: measuring cost on the web
- I hate telephones
- The sorry state of OpenSSL usability
- The dots do matter: how to scam a Gmail user
- My parents are Flat-Earthers
- How Hacker News stays interesting
- Project C-43: the lost origins of asymmetric crypto
- The hacker hype cycle
- The inception bar: a new phishing method
- Time is running out to catch COVID-19
- A probabilistic pub quiz for nerds
- Smear phishing: a new Android vulnerability