The keyword difficulty that Keyword Planner shows refers more to Adwords Competition. But yeah. It's definitely an indicator of how competitive the organic positions would be.
And yep.. You are so correct. It takes times for organic traffic to pick up.