Deep regression

Have people been using deep learning to do regression? I noticed
that fitting polynomials using least squares leads to much better
accuracy! Is there any rule of thumb to get arbitrary accuracy with
deep regression?