The electronics of "How to carry a locust" at Manifesta 16 Ruhr

The artwork

The artwork consists of 10'000 locusts folded out of paper. Several thousand children both in the Ruhr area and in Rojava folded and colored them, with support by Havîn. The locusts are attached to nylon strings and fixed to a metal construction, suspended downwards. The plan was to make the strings move up and down to get a visual swarming effect by using stepper motors at the top to spool and unspool the strings on a winch, controlled by some kind of embedded chips.

The electronics: BigTreeTech Manta M8P with TMC2209s

Since we had so little time we used 3D-printer control boards. When we made this decision it was a Saturday and ordering them would have taken too long, so we sent a mail to online shop meltbro who agreed to sell us more than 20 BigTreeTech Manta M8P V2.0 boards and enough TMC2209 stepper motor drivers. Then we drove to Bonn to pick them up and we had hardware that same evening to be able to start working on the software! Big thanks to Philipp Trares from meltbro for making this possible!

Every one of those boards has an ARM Cortex-M7 MCU running at 550MHz. One board can control eight stepper motors. Originally we were hoping to have 150 motors installed and had bought enough step motors (later we had to reduce the number because of time constraints).

The 160 TMC2209s came in a film roll package, I'm still amazed by this. I've since learned that packaging electronic components in reels is common.

The software: Marlin

Since we had so little time we decided against writing our own custom firmware. Instead we decided to also use an open source 3D-printing firmware, Marlin. The plan was to generate the movement of the strings as G-Code, which Marlin would then execute by stepping the eight motors. This would limit us in the kind of movements we could produce, but would give us well-debugged movement planning. Marlin has a build configuration for the Manta, so getting it built with PlatformIO was quite easy. Flashing it to the Manta is also just a matter of putting the firmware binary on a Micro-SD card and turning the board on. However, we didn't manage to talk to Marlin via the UART-USB bridge at first. We wanted to be able to communicate with Marlin via serial for easier debugging (later it turned out we barely needed it, because Marlin just worked).

Getting the UART to work

Eventually we found a reddit post that explains the UART-USB problem. The Manta is typically combined with a Raspberry Pi compute module. In a real 3D printer the CPU on the Manta is responsible for the steppers, and the compute module for high level tasks like a web interface accepting new printing jobs, etc. The UART-USB bridge is set up to talk to the compute module, not the ARM CPU. In the reddit post somebody had described a workaround, but we didn't understand it. So we started reading the Manta documentation to try to recreate/understand the workaround.

The Manta board has a CH340E chip for USB-to-serial communication:

The Pi-TX/Pi-RX pins of that are connected to the 40-pin GPIO on the Manta:

The Arm MCU of the Manta also has UART pins wired to a TFT connector, in order to display information on an optional TFT (which we didn't have, otherwise we would have used that for debugging).

So in order to see serial output from the ARM MCU it's possible to wire the TX/RX pins from that TFT connector to the Pi-TX/Pi-RX pins on the 40-pin GPIO.

Of course when we first tried that it still didn't work, because we held the board upside down. After fixing that we got our debug output!

(all the screenshots in this section are from the Manta user manual and Manta block diagrams.)

Generating the G-Code

Originally we had planned to try to write code that produces swarm-like behaviour (and we still hope to realize this vision at some point). Again, due to time constraints we settled on random movements. At the start, when everything is turned on, all the strings are in their fully extended configuration. Within the 70 cm maximum movement range we pick a movement direction randomly, and a leg size between 10cm and 40cm.

G-Code cannot actually express moving the motors attached to one Manta truly independently. While it's possible to move a single motor to a certain position with a G-Code command, this would leave the other motors motionless (G-Code is always executed sequentially). The G-Code G1 commands to move the eight motors to a fixed target positions look like this:

G1 X143.26 Y812.98 Z649.72 A213.79 B227.60 C388.65 U136.25 V86.60 F36669

For this to work we configured Marlin so that it knows that we have eight step motors with axis names X, Y, Z, A, B, C, U, V. Marlin will then produce step signals to make the motors move in a straight line in eight-dimensional space. It slowly ramps up the speed up to a maximum, then staying at that speed and braking when getting close enough to the target.

We wrote a script to generate random G-Code files with these and other constraints, with a different seed for every Manta board. Marlin has support for starting a G-Code file from the SD-Card directly after startup, which we used to start the movement when the boards get turned on.

Practical Considerations for the Exhibition

We were quite worried about the fact that Manifesta is open for almost four months, many hours each day, six days a week. We weren't quite sure the step motors or the TMC2209s wouldn't get too hot when running for hours and hours. Therefore we programmed 5 minute breaks into the G-Code after every 20 minutes of movement. Before the breaks, the winch unspools fully again. This makes it possible to turn the artwork off in those breaks too. After 10 hours in total, the execution also just stops (again in parked position).

We would really have liked to have actual homing of the winches. This would have been possible with the sensorless homing feature of the TMC2209s. However, it would have required some more mechanical setup to the way the motors are installed, and we didn't have time to change that part.

Flashing the SD-Cards

We wrote a script to write to 20 SD-Cards, each with the same Marlin image and their own random G-Code. Then we flashed all the boards and ran a brief check on every one of them, whether attached step motors would start working.

Installing the boards and motors

When we started a testing run we discovered that most of the cables we had for connecting the Manta boards to the step motors were actually not the correct ones. So the team had to spend a significant amount of time rewiring them by swapping two of the four cables.

Afterwards we installed all of the Manta boards on three large wooden planks, together with their power supply. The boards were mounted on top of the metal structure by the Manifesta team around Nancy Naser Al Deen. The motors are attached with small pieces of wood and screws with the help of a small crane.

Mechanical Problems

Unfortunately after everything was installed we discovered some mechanical problems with the winches. After a while of running, the very thin and slightly elastic nylon thread we used gets a bit tangled and doesn't unspool properly anymore. This means it gets wound up more and more. Eventually the locusts reach the motor and only spin around it, instead of moving up and down. Also, a lot of the threads were simply ripped and a lot of the locusts fell down.

We have a plan to fix that and some ideas how to prevent the tangling problems by using a different spool setup and stronger nylon string. Unfortunately it involves working on a tall ladder in the off-hours of the exhibitions, so we still need to work out the logistics.

Compression Problems

A friend of CF pointed out that videos of the moving installation look kind of bad, particularly when sent through e.g. Signal. Our theory is that it's because the movement is random enough that video compressors have a hard time dealing with it. This made me incredibly happy, because films of swarms of insects and birds have the same problems! (There's an old Tom Scott explanation for why this happens.)

Here's a screenshot from one of the videos we took, zoomed in. Of course part of the problem is the bad lighting in the church when this was filmed.

Things we want to fix next time

We have a long list of things that we would fix if we did this again with more time. As explained above, the mechanics of the threads and the spools should be designed much more carefully. Having a spool with a larger diameter would allow us to move the locusts more quickly (right now, the maximum speed feels too low). Also we would like a larger movement range. Getting homing to work would remove a lot of concerns about long term drift of the step motors over weeks of running and deal more gracefully with power loss, where right now the spools have to be unwound by hand back to their parking position.

We also would like to write custom firmware. Using Marlin was an amazing experience and definitely the right choice in our situation. But it also limits what movements we can perform to things expressible in G-Code. Really independent movement of all the step motors is not really possible, the planner always works with straight lines in 8D space.

Movement-wise we had hoped to actually make the winches move in coordinated ways across the whole installation. Obviously still with random components, but with every motor also influenced by the movements of its neighbours. The plan was to either pre-compute this and store it on the SD-cards, or to have all the boards use the same random number generator seeds, plus knowledge of their relative positions.

Conclusion

This was a super fun (if slightly stressful) project and I'm so happy to have been part of it! Despite some of the compromises I am very proud of what we achieved. If you're near Gelsenkirchen, go look at it! It's open the entire summer, and quite worth a visit.

Of course by far the best part was the incredible teamwork during the project. Thanks to the Manifesta team around Nancy and thanks to all the wonderful people who were directly involved: <list names>

CF & Nico