Figured it out. You must set assumptions of the variables, whether they are real or complex. Matlab doesn't know how to interpret your symbolic variables so it doesn't.
use the syms 'set' command like so:
syms w R C f real
syms H
H = 1/(sqrt(-1)*w*C*R+1)
H = simplify(abs(H))
Should work
If you want to get it in the form sqrt(x^2 + (iy)^2) do:
syms w R C f real
syms H
H = 1/(sqrt(-1)*w*C*R+1)
H = rewrite(abs(H), 'sqrt')