Day 24 : Accessing and Saving Point cloud Data using Unity and AR Foundation

Introduction

In this article, we’re going to look at Implementing Cloud point visualization and Saving the point cloud data for analysis.

In the previous articles, we learned about different trackables and implemented them. The Point cloud is basically a set of points in 3D space. They grouped together to form a cloud of points.

These points are created based on the features in the world that tell the AR Engine about depth and perception and position trackables within space and time.

Finally, fused with the accelerometer in the phone, this data allows your phone to more accurately track movement within space.

Getting Started

Prerequisites

1. Unity 2019.2.18f1 or above
2. ARFoundation 3.0.1 setup from an earlier article here.

AR Point Cloud Manager

This is a type of trackable manager that creates point clouds, sets of feature points. A feature point is a specific point in the point cloud which the device uses to determine its location in the world.

A point cloud is a set of feature points which can change from frame to frame. Some platforms only produce one point cloud, while others organize their feature points into different point clouds in different areas of space.

Here the point cloud itself is going to be the trackable whereas the individual feature points are not. However, feature points can be uniquely identified between frames as they have unique identifiers.

To create a new AR Point Cloud Manager go to the AR Session Origin gameobject and Add Component -> AR Point Cloud Manager.

It requires a point cloud prefab just like we saw in the plane detection tutorial to draw a point cloud for us to visualize.

AR Point Cloud Prefab

This point cloud prefab will be nothing but a particle system where each particle will be a single point of the point cloud.

Let’s create this prefab by right-click in the Hierarchy -> XR -> AR Default Point Cloud

This is what it’ll look like.

To make this as a prefab, simply drag and drop it into the Project view.

Finally, assign this prefab the AR Point Cloud Manager’s point cloud prefab property.

That’s it for Visualizing the point clouds!

Saving Point Cloud Data

Now, if you want to save this data for analysis, analytics or even to perform some additional math, you can do that by subscribing to the pointCloudsChanged event of the AR Point Cloud Manager.

Let’s create a script that handles this. I’m naming it PointCloudParser.

This is the whole script.

using System.Collections.Generic;
using UnityEngine;
using UnityEngine.XR.ARFoundation;

public class PointCloudParser : MonoBehaviour
{
    public ARPointCloudManager pointCloudManager;

    private void OnEnable()
    {
        pointCloudManager.pointCloudsChanged += PointCloudManager_pointCloudsChanged;
    }

    private void PointCloudManager_pointCloudsChanged(ARPointCloudChangedEventArgs obj)
    {
        List<ARPoint> addedPoints = new List<ARPoint>();
        foreach (var pointCloud in obj.added)
        {
            foreach (var pos in pointCloud.positions)
            {
                ARPoint newPoint = new ARPoint(pos);
                addedPoints.Add(newPoint);
            }
        }

        List<ARPoint> updatedPoints = new List<ARPoint>();
        foreach (var pointCloud in obj.updated)
        {
            foreach (var pos in pointCloud.positions)
            {
                ARPoint newPoint = new ARPoint(pos);
                updatedPoints.Add(newPoint);
            }
        }
    }
}

public class ARPoint
{
    public float x;
    public float y;
    public float z;

    public ARPoint(Vector3 pos)
    {
        x = pos.x;
        y = pos.y;
        z = pos.z;
    }
}

We have an ARPoint class that we can save the points into. This can then be serialized into JSON or other formats. To learn more about serialization and saving such data into a file, check our previous article here.

The addedPoints and updatedPoints will contain all the point cloud data. You can then save each cloud separately.

That’s it! Now you can proceed to build the app on either an Android or iOS and test it for yourself.