[Butterfly Swarm]

A living wall of physical, glowing butterflies interacting with the soundscape. The wings showcase generative patterns that are always evolving. This first iteration includes 48 custom-engineered, individually controllable butterflies, projection-mapped in sync with the movement of their wings.

Key Technologies: Projection Mapping, TouchDesigner, Microcontrollers, Servos, 3D Printing, Art-Net

Powered by TouchDesigner, the system bridges the digital and physical worlds, driving both the projection mapping and movement of the servos.

This project was developed primarily as an immersive backdrop for a concert that I hosted in December. I wanted to create something that would push the limits of what is possible with my current resources. Inspired by the butterfly motif of the headliner Animat's Slowwalker EP, I challenged myself by moving away from static projection mapping to create animatronic butterflies.

In the following deep dive, I will break down the components of this installation and the technical challenges I overcame to build it.

[The Electronics]

Servos. ESP32. Art-Net.

How do you control a swarm of Butterflies? The first thing I needed to do was define the scale. To drive the servos, I chose a WT32-ETH01 ESP32 board, which comes with a built-in Ethernet port for wired Art-Net. To power and control the servos, I needed additional driver boards; the PCA9685 can handle 16 servos, and multiple boards can be daisy-chained. To decide how many I would need for a convincing swarm effect, I created a pre-visualization in TouchDesigner:

I decided to go with three servo boards, totaling 48 butterflies. The pre-vis also helped me decide on a 30 cm wingspan to effectively fill the backdrop of the stage.

For the servos I chose the Feetech FT90B. It's a digital servo which reduces latency and noise. The ones I got are as far as I can tell not genuine, however they still worked and were noticeably quieter than a SG90 clone I had laying around.

-wiring diagram

Above is the wiring diagram. The circuit itself is very simple, but the specific ESP32 board gave me a headache. None of the broken-out GPIO pins seemed safe to use for the servo board because they were all reserved for specific functions. However, after a lot of research, I can confirm that the pins confusingly labeled CFG and 485_EN are in fact GPIO 32 and GPIO 33 and are safe to use.

On the firmware side, I relied on the ArtnetETH library to handle the network communication and the Adafruit PWM Servo Driver library to control the motors. The code listens for incoming Art-Net packets and maps the DMX channels directly to the individual servos. Crucially, I implemented a smoothing algorithm directly on the ESP32. Since standard DMX is limited to discrete steps (0-255), raw input would result in jerky, robotic motion. The code interpolates between these steps dynamically, smoothing slow movement, while not delaying fast movements. I also used ElegantOTA for the first time, which is very handy because it allows me to easily push updates to the ESP32 over the network.

[The not so compliant Mechanism]

Making the wings fold

I always liked compliant mechanisms. And they also seemed like a great fit here.

As opposed to traditional mechanisms made up only of rigid-links and movable joints, the movement of a compliant mechanism is instead in some part achieved through elastic body deformation. This means in practice that traditional joints get replaced with flextures and multiple components get joined into one part. Flextures are the sections of a part that are bending in a controlled manner to match the movement und function of the traditional mechanism. ^[1]

While this sounds complicated, compliant mechanisms are already everywhere: Ketchup bottle lids, kitchen sealing clips, backpack buckles, even zip ties. Compliance makes it possible for all these parts to be injection molded as a single part and thereforce to be cheap at scale.

I planned to use compliance to reduce parts and assembly time for the butterflies, aswell as simply to learn more about it. I decided to go with partial compliance for this project, which means still keeping some traditional, moving joints.

I decided on 3 components: Base, left wing, right wing. To start of with I used PETG as a material. While the hinge worked, it broke fast. This is called concentrated compliance, because most of the body stays rigid and only a very short section actually bends.

The next few iterations I added the servo mount and kept on tweaking the flexture. Primarily I kept increasing the flexture length, which also meant adjusting the entire geometry to keep a mostly linear transfer function. But the flexture kept breaking. While longer flextures distribute the bending force better, eventually every flextures kept buckling and then failing. I studied a lot of reference flextures aswell as designs from BYU, but my motion range of 90 degrees was really big.

The stress in the flexture is linearly proportional to its width. And I had few options to reduce it even further. I experimented a lot with different slicer settings. To get the strongest flexture it had to be two extrusion paths wide, or the slicer would split the flexture as a separate extrusion, creating weak points where we want them least. With the classic slicing mode in prusa slicer and 0.4 mm outer perimeter extrusion width, it was possible to get the flexture width down to 0.5 mm and get two mostly overlapping extrusions. Now the obvious question would be why not just use smaller nozzle? While I believe that would work, it would also significantly increase print time, which was already long to start with.

The next iterations showed, even making the flexture as long as it can be wouldn't be enough to make them durable and there was another issue with distributed flextures: they were essentially springs. And I realized to keep the servos silent I had a new objective: Reducing the force needed to move the wings and hold position.

All my prints thus far were in PETG, which just didn't seem like a viable material anymore. But printing flexible materials is generally slow and difficult. And my deadline was approaching. Cornelius, who was helping me out with 3D printing, recommended trying Bambu Lab 68D TPU which was easy enough to print at scale. And at first it seemed like the solution I was looking for. But letting a test print open and close for 24 hours on my test stand showed it would also fail eventually. This was bad. I tried softer TPU which would have worked, but there was no time. Cornelius suggested just going with a snap fit rigid link system printed in PLA, and we decided this was the only option.

So in the end it wasn't a compliant mechanism. It felt wrong. But then again, I chose a compliant mechanism because I believed it was the best option available, which it wasn't in the end. But compliance might return with the next iteration of the butterfly swarm.

[The Wings]

And trying to dry paper

I had decided on the wing shape pretty early on. I specifically looked at reference images of monarch butterflies sitting on flowers, but I didn't want to limit myself to one Species when it comes to texturing later.

I wanted the wings to be made from paper. That seemed like a good fit: Cheap, easy to work with and a good matte projection surface. I also knew I wanted the wings to be white from the front and black from the back. Some real butterflies also use this so they are hidden in the shadows with their wings closed. I wanted to use this effect to maximize the contrast between butterflies that are open and closed.

The wings themselves could have been just paper cutouts, but I decided to add the extra step of embossing them with an interpretation of a butterfly wing structure. To emboss the wings myself I 3D printed a positive and negative mold. But because I don't have access to a press, my option was limited to making the paper very wet, so that it would deform easily.

I also still needed to think about how I was gonna get one side black. I decided to go with thick white paper and spray paint one side. I spray painted the paper before embossing, because I thought that way I would only have to dry the paper flat once. The big issue it turns out with wetting paper is that drying it without having it deform is not easy. especially when I needed 100 of them.

So I went in this order: Cut out the wing shape, spray paint one side, wet and emboss and then drying it with paper towels and heavy books. It kinda worked, but drying took more time than I had and most of the wings ended up a bit curved and deformed in the end. Which wasn't great for projection mapping, but on the upside I got a lot of comments that said it made them feel more organic and natural.

What was I gonna change for the next iteration? I got some help cutting the wings by hand, so it only took a few hours. Yet lasercutting the paper would have been faster and more consistent. Also I think the right order would have been: First Embossing, fully drying, then painting. I believe the paint sealed one side, making it harder for the moisture to escape after embossing. Painting the dried wing in the end would only wet it very slightly.

[The Invite]

creating interactivity

I wanted the invite to be special. I ended up with an invite that was incomplete. It was incomplete, because everyone invited was tasked with coloring it themselves, like a coloring book. And the best part is: the wing of the butterfly prominent on the top right, is the exact shape as the wings of the butterflies that i was building. And while this maybe was not necessary, I added four ArUco markers around the wing to make it easier for me to automate the tracking and perspective correction of the wing.

While the invite was well received, the amount of people who actually painted them was low. This also wasn't my main focus, but there were a few entries that made it in. For example one friend took some printouts with her to Daycare and had a few kids paint them.

[Projection Mapping]

TouchDesigner mostly

My TouchDesigner project is structured into two parts. The first is the scene component, which generates the wing textures and movement vectors. The other is the rendering component, which takes all the scene outputs and instances each butterfly wing in the real-world orientation and renders it from the correct perspective.

Calibration

But how does TouchDesigner know where each butterfly is? Clearly I had to come up with a good system for calibration since I had to repeat it 48 times. Because time was limited I decided to constrain all butterflies to be positioned on a 3D plane. This simplified things a lot, because it meant i only had to find the x, y and rotation for each butterfly.

And of course I still had to find the projector orientation relative to the plane. For this I could however just use camSchnapper, a tool in TouchDesigner, which is build to help you match a virtual camera perspective to a real life projection. As a proxy geometry I used a rectangle and measured out six points with the same proportions on my backdrop.