c++ - opengl camera zoom to cursor, avoiding > 90deg fov -

i'm trying set google maps style zoom-to-cursor control opengl camera. i'm using similar method 1 suggested here. basically, position of cursor, , calculate width/height of perspective view @ depth using trigonometry. change field of view, , calculate how need translate in order keep point under cursor in same apparent position on screen. part works pretty well.

the issue want limit fov less 90 degrees. when ends >90, cut in half , translate away camera resulting scene looks same larger fov. equation find necessary translation isn't working, strange because comes pretty simple algebra. can't find mistake. here's relevant code.

void visual::scroll_callback(glfwwindow* window, double xoffset, double yoffset) {     glm::mat4 modelview = view*model;     glm::vec4 viewport = { 0.0, 0.0, width, height };      float winx = cursorprevx;     float winy = viewport[3] - cursorprevy;     float winz;      glreadpixels(winx, winy, 1, 1, gl_depth_component, gl_float, &winz);     glm::vec3 screencoords = { winx, winy, winz };      glm::vec3 cursorposition = glm::unproject(screencoords, modelview, projection, viewport);      if (isinf(cursorposition[2]) || isnan(cursorposition[2])) {         cursorposition[2] = 0.0;     }      float zoomfactor = 1.1;     // = zooming in      if (yoffset > 0.0)         zoomfactor = 1/1.1;      //the width , height of perspective view, @ depth of cursor position      glm::vec2 fovxy = camera.getfovxy(cursorposition[2] - ztranslate, width / height);     camera.setzoomfromfov(fovxy.y * zoomfactor, cursorposition[2] - ztranslate);      //don't want fov greater 90, cut in half , move world farther away camera compensate     //not working...     if (camera.zoom > 90.0 && ztranslate*2 > max_depth) {         float prevzoom = camera.zoom;         camera.zoom *= .5;          //need increased distance between camera , world origin, view not appear change when fov reduced         ztranslate = cursorposition[2] - tan(glm::radians(prevzoom)) / tan(glm::radians(camera.zoom) * (cursorposition[2] - ztranslate));     }     else if (camera.zoom > 90.0) {         camera.zoom = 90.0;     }      glm::vec2 newfovxy = camera.getfovxy(cursorposition[2] - ztranslate, width / height);      //translate position under cursor not appear move.     xtranslate += (newfovxy.x - fovxy.x) * (winx / width - .5);     ytranslate += (newfovxy.y - fovxy.y) * (winy / height - .5);      updateview = true; } 

the definition of view matrix. called ever iteration of main loop.

void visual::setview() {     view = glm::mat4();      view = glm::translate(view, { xtranslate,ytranslate,ztranslate });      view = glm::rotate(view, glm::radians(camera.inclination), glm::vec3(1.f, 0.f, 0.f));     view = glm::rotate(view, glm::radians(camera.azimuth), glm::vec3(0.f, 0.f, 1.f));      camera.right = glm::column(view, 0).xyz();     camera.up = glm::column(view, 1).xyz();     camera.front = -glm::column(view, 2).xyz(); // minus because opengl camera looks towards negative z.     camera.position = glm::column(view, 3).xyz();      updateview = false; } 

field of view helper functions.

glm::vec2 getfovxy(float depth, float aspectratio) {     float fovy = tan(glm::radians(zoom / 2)) * depth;     float fovx = fovy * aspectratio;      return glm::vec2{ 2*fovx , 2*fovy }; }  //you have desired fov, , want set zoom achieve that. //factor of 1/2 inside atan because need half-fov. keep full-fov input consistency  void setzoomfromfov(float fovy, float depth) {     zoom = glm::degrees(2 * atan(fovy / (2 * depth))); } 

the equations i'm using can found diagram here. since want have same field of view dimensions before , after angle changed, start with

fovy = tan(theta1) * d1 = tan(theta2) * d2 d2 = (tan(theta1) / tan(theta2)) * d1  d1 = distance between camera , cursor position, before fov change = cursorposition[2] - ztranslate d2 = distance after theta1 = fov angle before theta2 = fov angle after = theta1 * .5 

appreciate help.