People Search Result logic in SharePoint 2013

We are wondering what is the logic on how result returned in People Search in SharePoint 2013. For example, Search is running and if I search for my Manager I get his name showing in Page 5 instead of the Page 1. I would think it would be logical to have my Manager's name displayed in the first page instead of flipping through the pages.

Is there a configuration that I need to modify or based on what logic or algorithm does SharePoint determine which name should be displayed first when users search for people.

I thought SharePoint would display the result based on relative distance to the user who is doing the search, so if I search for John (ex. I report to John who is my manager) then I should see my manager name in the first page in the search results instead of later pages.