I like doing VFR in the little GA planes, and not all of them even have an autopilot. 
For takeoff, landing, and just sort of poking around it’s a lot of fun to hand-fly, but for longer trips with long straightaways or climbs it’s helpful to use the AP to reach and maintain altitude and keep in a straight line.
For those journeys, I usually stick it on HDG mode and steer based on my charts and what I see out the window. Twisting the HDG knob isn’t as satisfying as the yoke but it gets the job done. 
When I do IFR I’ll run almost everything on autopilot between TO and landing and concentrate on doing the procedures right – but to me that’s not as much fun as freewheeling it.