In this video, a person asks is there a downside to responsive design for SEO? The short answer is, no! It is the best option for mobile SEO. There are no SEO drawbacks.
Is there a Downside to Responsive Design for SEO Video
As you can see in the video, Matt Cutts from Google clearly states it is the best option.