UE5获取屏幕上的点在三维空间中的位置

方法

使用Deproject Screen To World方法

Deproject Screen To World

• 输入
  • Player: 当前视口所属的Player的Player Controller
  • Screen Position: 需要转换的屏幕坐标
• 输出
  • World Position: 转换后的三维空间坐标
  • World Direction: 转换后，从当前视口摄像机向三维空间坐标方向的方向向量

原理

UE5 实现源码

bool UGameplayStatics::DeprojectScreenToWorld(APlayerController const* Player
					  , const FVector2D& ScreenPosition
					  , FVector& WorldPosition
					  , FVector& WorldDirection)
{  
    ULocalPlayer* const LP = Player ? Player->GetLocalPlayer() : nullptr;  
    if (LP && LP->ViewportClient)  
    {  
       // get the projection data  
       FSceneViewProjectionData ProjectionData;  
       if (LP->GetProjectionData(LP->ViewportClient->Viewport, /*out*/ ProjectionData))  
       {  
	      // 获得投影矩阵的逆矩阵，然后调用下一段代码中的函数
          FMatrix const InvViewProjMatrix = ProjectionData
	          .ComputeViewProjectionMatrix()
	          .InverseFast();  
          FSceneView::DeprojectScreenToWorld(ScreenPosition
	          , ProjectionData.GetConstrainedViewRect()
	          , InvViewProjMatrix
	          , /*out*/ WorldPosition
	          , /*out*/ WorldDirection);  
          return true;  
       }  
    }  
  
    // something went wrong, zero things and return false  
    WorldPosition = FVector::ZeroVector;  
    WorldDirection = FVector::ZeroVector;  
    return false;  
}

void FSceneView::DeprojectScreenToWorld(const FVector2D& ScreenPos
				, const FIntRect& ViewRect
				, const FMatrix& InvViewProjMatrix
				, FVector& out_WorldOrigin
				, FVector& out_WorldDirection)
{  
    float PixelX = FMath::TruncToFloat(ScreenPos.X);  
    float PixelY = FMath::TruncToFloat(ScreenPos.Y);  
  
    // Get the eye position and direction of the mouse cursor in two stages (inverse transform projection, then inverse transform view).  
    // This avoids the numerical instability that occurs when a view matrix with large translation is composed with a projection matrix  
    // Get the pixel coordinates into 0..1 normalized coordinates within the constrained view rectangle
    // 翻译：首先将需要变换的点的坐标转换到(0, 0)到(1, 1)范围的平面空间中
    const float NormalizedX = (PixelX - ViewRect.Min.X) / ((float)ViewRect.Width());  
    const float NormalizedY = (PixelY - ViewRect.Min.Y) / ((float)ViewRect.Height());  
  
    // Get the pixel coordinates into -1..1 projection space 
    // 翻译：通过简单的数学运算将点转换到(-1, -1)到(1, 1)的投影空间中
    const float ScreenSpaceX = (NormalizedX - 0.5f) * 2.0f;  
    const float ScreenSpaceY = ((1.0f - NormalizedY) - 0.5f) * 2.0f;  
  
    // The start of the ray trace is defined to be at mousex,mousey,1 in projection space (z=1 is near, z=0 is far - this gives us better precision)  
    // To get the direction of the ray trace we need to use any z between the near and the far plane, so let's use (mousex, mousey, 0.01)
    // 翻译：这里UE设定的裁剪空间近平面和远平面分别为 z=1 和 z=0
    // 由于输出结果需要一个方向向量，所以首先需要两点确定一条直线，即下面的Start和End两个点
    // 为什么End点的Z不是0？
    const FVector4 RayStartProjectionSpace = FVector4(ScreenSpaceX, ScreenSpaceY, 1.0f, 1.0f);  
    const FVector4 RayEndProjectionSpace = FVector4(ScreenSpaceX, ScreenSpaceY, 0.01f, 1.0f);  
  
    // Projection (changing the W coordinate) is not handled by the FMatrix transforms that work with vectors, so multiplications  
    // by the projection matrix should use homogeneous coordinates (i.e. FPlane).
    // 翻译：这里使用投影矩阵的逆矩阵将裁切空间中的点转换到世界空间中
    // 由于投影矩阵的一些缩放会更改点的W值，这是我们不需要的，所以在下方再除以W才能得到最终结果
    const FVector4 HGRayStartWorldSpace = InvViewProjMatrix.TransformFVector4(RayStartProjectionSpace);  
    const FVector4 HGRayEndWorldSpace = InvViewProjMatrix.TransformFVector4(RayEndProjectionSpace);  
    FVector RayStartWorldSpace(HGRayStartWorldSpace.X
	    , HGRayStartWorldSpace.Y
	    , HGRayStartWorldSpace.Z);  
    FVector RayEndWorldSpace(HGRayEndWorldSpace.X
	    , HGRayEndWorldSpace.Y
	    , HGRayEndWorldSpace.Z);  
    // divide vectors by W to undo any projection and get the 3-space coordinate  
    if (HGRayStartWorldSpace.W != 0.0f)  
    {  
       RayStartWorldSpace /= HGRayStartWorldSpace.W;  
    }  
    if (HGRayEndWorldSpace.W != 0.0f)  
    {  
       RayEndWorldSpace /= HGRayEndWorldSpace.W;  
    }  
    const FVector RayDirWorldSpace = (RayEndWorldSpace - RayStartWorldSpace).GetSafeNormal();  
  
    // Finally, store the results in the outputs  
    out_WorldOrigin = RayStartWorldSpace;  
    out_WorldDirection = RayDirWorldSpace;  
}

要理解上面两段代码需要一些简单的图形学知识，总体来说，整个流程就是基本渲染流程中 Model-View-Projection 变换的逆变换