My 3D Printer Plays Video Games Better Than You

My friends always say I’m trash at Valorant and I play worse than a bot, so I made a robot that plays better than them.

I wanted the robot to literally use a keyboard, mouse, and monitor to play the game so it plays exactly like a human — no memory exploits whatsoever like in most game hacks.

I had no money so I couldn’t buy high-quality servos, power supplies, and other materials — but I did have a 3D printer! If I attach my mouse to the printer nozzle, I can move the mouse by moving the nozzle.

@butt_kisser 3D printer glowup #3dprinting #techtok #coding #arduino #makersoftiktok #programming #computerscience #computersciencemajor #gaming ♬ som original - 𒉭

This project went through quite a few iterations. My first shot at this project was fundamentally flawed and couldn’t shoot a target standing still. My second shot could beat most of my friends… except for one. He was a top .5% player and the person I needed to beat most.

It took the culmination of almost all the skills I had to be able to make something that could beat him. I was really big into robotics during high school, and I spent the latter two years focusing on making our robot’s PID control systems as performant as possible. In my freshman year of college, I did in an internship at an aerospace company where I worked on developing an FPGA-based coprocessor for physics simulators used to test plasma propulsion engines. This final design was the unification of these two ideas. That jerk now brags that it took rocket science to beat him.

High-Level Overview

Enemies in Valorant are red.
Let’s look for large blobs of red pixels on screen to find enemies.
Move the mouse to aim at the location of the enemy.
When you are aiming at the enemy, shoot.

Medium-Level Overview

Take your VGA video cable and cut it open.
Create an analog circuit that filters the RGB video input for the red pixels of enemies.
Send the results to a SoC that uses that information to calculate the location of enemies.
The SoC sends the location of enemies to my 3D printer.
The 3D printer moves the computer mouse to the correct position.
When the FPGA detects we are aiming directly at the enemy, shoot.

Low-level Overview

The VGA Part

VGA uses a very primitive video transmission format and is effectively transmitted as “plain text”. This makes it a lot easier for me to work with than HDMI or DisplayPort, which require a dedicated chip or IP block to decode.

The VGA signal essentially sends you the color of each pixel one at a time. It has 3 pins sending the RGB value of the pixel, and the other pins are a clock that tells when it wants to send the next pixel.

The one problem is that this is transmitted as an analog signal. RGB colors can be represented as 3 numbers, 0-255, that indicate the “strength” of the color. In a VGA cable, the voltage of the R, G, and B signals each represent their strengths.

How can we convert this into a digital signal that can be used by the FPGA?

The Analog Circuit Part

We need to convert the analog VGA signal into binary values.

I could convert each RGB value into 3 8-bit digital signals, but it’s hard for my FPGA to take in that many inputs. Instead, I implemented all the “look for red pixels” logic into this analog circuit part. Now I only have to communicate one bit - 1 for red and 0 for not red.

I used a voltage comparator to compare the voltage levels of the red channel against the green and blue channels, essentially checking if the red signal is significantly higher than the others, indicating a “red” signal.

The FPGA Part

An FPGA is a special circuit that can be transformed into other circuits. It’s literally like magic. You can write code in a “hardware description language” to describe a digital circuit, upload the code to the FPGA, and it will pretend to be that physical circuit. It’s typically used for developing circuits so you don’t have to fabricate a new circuit board or chip every time you want to test something.

I used an Arduino Vidor because I already owned one. Devices like the Arduino Vidor are cool because they have an FPGA and a normal CPU on one board. Just like how your CPU and GPU work together, you can make the CPU and FPGA work together. They can communicate with each other to accomplish tasks.

I made a circuit that calculates the “mean” position of all the red pixels. This is an approximation of the center of mass of enemies. The underlying algorithm looks something like this:

x, y = 0, 0
mx, my = 0, 0
while True:
    is_red, h_sync, v_sync = read_pins()
    if h_sync:
        x = 0
        y += 1
    elif v_sync:
        x, y = 0, 0
    else:
        x += 1

    if is_red:
        m_x, m_y = calculate_new_running_mean()

    emit(mx, my)

Now all we need to do is communicate the estimated position to my 3D printer… except that’s pretty hard to do. Instead, I used the Arduino CPU on the same board. I sent the location to the Arduino via internal pins and I let it take it from there.

The Arduino Part

The board controlling the 3D printer is also Arduin-based. I connected them using two wires so they could communicate over UART. Arduino has out-of-the-box support for UART so this was trivial to do.

Additionally, when the enemy location indicates that the player is aiming directly at them, the Arduino uses a basic servo to press the mouse button.

The 3D Printer Part

I wrote new firmware for the 3D printer. The original firmware wasn’t performant enough.