Non-linear doesn’t mean polynomial. There are many non-linear functions polynomials cannot approximate well, such as softmax() or even just sin(). Ultimately, it’s enough to approximate complex functions with simple functions without having to worry about picking the correct complex function.