AC IN goes off (not grounded) if fan speed is above 2. Need help with circuit
 

A while ago I created a thread that went nowhere about this issue. I'm starting a clean new thread because I just want to focus on the circuit part.
I had no idea back then where to tackle the problem, and summer went away, so I left it.. Some other people were having the same issue but we didn't find a solution.

I pulled out my voltmeter and enabled port status in Tunerstudio, and I think I found the issue, but I need some help coming up with a solution.

Technically when the AC switch is ON and the fan speed is 1 or greater, there should be close to 0v between the switch and ground, because the switch is grounded when pressed. But it seems like the voltage between ground and the AC Switch raises with the fan speed position, as if there was a poor ground at the fan and it needs to circle back through the switch (or this is my only explanation).

So checking the voltage when grounded, at speed 1 there is about 1.2v, and raises ~1v per speed change. So on speed 4 I have around 4v.
The proto circuit in the MS is 5v referenced, so the MS gets 5v unless the wire is grounded (by the switch) and then it should see 0. The problem is that when I'm on speed 3 and 4, the voltage is nowhere near 0v, so the MS does NOT see that port grounded, and on speed 2, sometimes is over, sometimes under.
Watching port status, I see that it's solid ground at 1, at 2 I have the status blink on and off, and 3 and 4 are solid ON status.

I have played with the resistors in the proto circuit trying to make it "easy" to ground, but if I'm getting 4v at the input I'm not gonna ground shit.

So, I could try to fix it in the car's wiring, which is valid, if anyone knows what to check or what to ground, I'd be happy to.
The other thing would be to change the proto circuit so if the input is below 5v, it's grounded. I'm thinking of a Zener diode somewhere but I'm not quite sure.

What do you guys think?

My next test (came up in my head while writing this) would be to disconnect the blower fan and see what voltages I'm getting. If I continue having the same issue it might be the wiring in the switch or something.

The other thing I was thinking about is that in the normal wiring with stock ECU, there is an extra pin that is supposedly grounded when the speed is 3 or 4, this pin is not used by the MS, but I could wire it on the ECU plug and see if I can get a better ground when I go past speed 2, but not sure yet if that makes sense.