Integrating OpenPose into Unity

Requirements

• Unity 2019.4 or later
• A 3D scene
• OpenPose SDK (available for Windows, Mac, and Linux)
• A computer with a webcam or a smartphone with a camera

Installing OpenPose SDK

1. Download the OpenPose ZIP from the official website here and extract the contents of the zip file to a folder on your computer.
2. Open Unity and create a new project.
3. In the Unity editor, go to Assets > Import Package > Custom Package and select the OpenPose SDK folder that you extracted earlier.
4. Click Import to import the OpenPose SDK into your Unity project.

Additionally in the ZIP there is a demo project, such that you can better understand how open pose works with unity

Configuring OpenPose

1. In the Unity editor, go to Edit > Project Settings > Player and select the XR Settings tab.
2. Check the Virtual Reality Supported option and add “OpenPose” to the Virtual Reality SDKs list.
3. (Optional) Under Publishing Settings, add “OpenPose” to the list of target platforms and “Windows” to the list of target SDKs.

Adding OpenPose Objects

1. In the Unity editor, create a new “OpenPoseManager” object (menu: OpenPose > OpenPose Manager) and add it to the scene.
2. Create a new “OpenPoseReceiver” object (menu: OpenPose > OpenPose Receiver) and add it to the scene.
3. Position both objects under a manager object in the hierarchy.

Calibrating OpenPose

1. In the Unity editor, select the OpenPoseManager object and click on the Calibrate button in the Inspector window.
2. Follow the on-screen instructions to calibrate the OpenPose camera.

Using OpenPose in Your Script

1. Create a new script (menu: Assets > Create > C# Script) and name it “OpenPoseExample”.
2. Attach the script to the OpenPoseReceiver object.
3. In the script, add the following code:

   csharp
   using UnityEngine;
   using OpenPose;
   
   public class OpenPoseExample : MonoBehaviour
   {
       private OpenPoseReceiver receiver;
   
       void Start()
       {
           receiver = GetComponent<OpenPoseReceiver>();
       }
   
       void Update()
       {
           // Get the current pose data from OpenPose
           var poseData = receiver.GetPoseData();
   
           // Print the pose data to the console
           Debug.Log("Pose Data:");
           Debug.Log($"Body Index: {poseData.BodyIndex}");
           Debug.Log($"Hand Index: {poseData.HandIndex}");
           Debug.Log($"Finger Index: {poseData.FingerIndex}");
           Debug.Log($"Pose Confidence: {poseData.PoseConfidence}");
       }
   }

4. In the Update method, we get the current pose data from the OpenPoseReceiver component and print it to the console.

Testing OpenPose

1. In the Unity editor, select the OpenPoseReceiver object and click on the Test button in the Inspector window.

Follow the on-screen instructions to test the OpenPose camera.

1. That's it! You have now successfully integrated OpenPose into your Unity project and can use it to track the positions and orientations of objects in 3D space.