Point-wise segmentation

Scenarios where 3d points need to be colored and marked with a particular class.
Those classes can have instances like motorcycle, pedestrian etc
or can also be non-instantiated like building, road etc

• Create the job as mentioned in Creating sensor-fusion annotation Job

Please note that, point-wise segmentation job doesn't support multiple frames and hence there won't be any frame key in payload. Kindly check the payload in right panel

POST https://api.playment.io/v0/projects/<project_id>/jobs
Parameters
project_id: To be passed in the URL
x-api-key: Secret key to be passed as a header

Create job request payload

{
  "reference_id": "001",
  "data": {
    "sensor_data": {
      "sensors": [
        {
          "sensor_id": "lidar",
          "data_url": "https://xyz.s3.com/123.pcd",
          "sensor_pose": {
            "position": {
              "x": 0,
              "y": 0,
              "z": 0
            },
            "heading": {
              "w": 1,
              "x": 0,
              "y": 0,
              "z": 0
            }
          }
        },
        {
          "sensor_id": "18158562",
          "data_url": "https://xyz.s3.com/123.jpg",
          "sensor_pose": {
            "position": {
              "x": -0.8141599005737696,
              "y": 1.6495307329711615,
              "z": -1.5230365881437538
            },
            "heading": {
              "w": 0.6867388282287469,
              "x": 0.667745267519749,
              "y": -0.21162707775631337,
              "z": 0.19421642430111224
            }
          }
        }
      ],
      "ego_pose": {
        "position": {
          "x": 0,
          "y": 0,
          "z": 0
        },
        "heading": {
          "w": 1,
          "x": 0,
          "y": 0,
          "z": 0
        }
      },
      "sensor_meta": [
        {
          "id": "lidar",
          "name": "lidar",
          "state": "editable",
          "modality": "lidar",
          "primary_view": true
        },
        {
          "id": "18158562",
          "name": "18158562",
          "state": "editable",
          "modality": "camera",
          "primary_view": false,
          "intrinsics": {
            "cx": 0,
            "cY": 0,
            "fx": 0,
            "fy": 0,
            "k1": 0,
            "k2": 0,
            "k3": 0,
            "k4": 0,
            "p1": 0,
            "p2": 0,
            "skew": 0,
            "scale_factor": 1
          }
        }
      ]
    }
  },
  "tag": "Sample_pc_seg",
  "batch_id": "sample UUID"
}

Create job response

{
  "data": {
    "job_id": "3f3e8675-ca69-46d7-aa34-96f90fcbb732",
    "reference_id": "001",
    "tag": "Sample-task"
  },
  "success": true
}

Python

import requests
import json

def Upload_jobs(PROJECT_ID, API_KEY, DATA):
    base_url = f"https://api.playment.io/v0/projects/{PROJECT_ID}/jobs"
    response = requests.post(base_url, headers={'x-api-key': API_KEY}, json=DATA)
    print(response.json())
    if response.status_code >= 500:
        raise Exception("Something wrong at Playment's end")
    if 400 <= response.status_code < 500:
        raise Exception("Something wrong!!")
    return response.json()

"""
Details for creating JOBS,
project_id ->> ID of project in which job needed to be created
x_api_key ->> secret api key to create JOBS
tag ->> You can ask this from playment side
batch_id ->> The batch in which JOB needed to be created
"""
project_id = ''
x_api_key = ''
tag = ''
batch_id = ''
"""
Defining Sensor: This will contain detail about sensor's attributes.
:param _id: This is the sensor's id.
:param name: Name of the sensor.
:param primary_view: Only one of the sensor can have primary_view as true.
:param state(optional): If you want this sensor not to be annotated, provide state as non_editable. Default is editable.
:param modality: This is the type of sensor.
:param intrinsics: In case of a camera modality sensor we will need the sensor intrinsics. 
                This field should ideally become part of the sensor configuration, and not be sent as part of each Job.
                "cx": principal point x value
                "cy": principal point y value
                "fx": focal length in x axis
                "fy": focal length in y axis
                "k1": 1st radial distortion coefficient
                "k2": 2nd radial distortion coefficient
                "k3": 3rd radial distortion coefficient
                "k4": 4th radial distortion coefficient
                "p1": 1st tangential distortion coefficient
                "p2": 2nd tangential distortion coefficient
                "skew": camera skew coefficient
                "scale_factor": The factor by which the image has been downscaled (=2 if original image is twice as
                                large as the downscaled image)
"""

#Name the sensors, similarly you can define this for multiple cameras
lidar_sensor_id = 'lidar'

#Preparing Lidar Sensor
lidar_sensor = {"_id": lidar_sensor_id, lidar_sensor_id: "lidar", "primary_view": True, "modality": "lidar"}

#Preparing Camera Sensor for camera_1
camera_1_intrinsics = {
    "cx": 1024.56301417, "cy": 592.004009216, "fx": 1050.21459961, "fy": 1051.06384277,
    "k1": 0, "k2": 0, "k3": 0, "k4": 0, "p1": 0, "p2": 0, "skew": 0, "scale_factor": 1}

camera_1 = {"_id": "camera_1", "name": "camera_1", "primary_view": False, "modality": "camera",
            "intrinsics": camera_1_intrinsics}

#Preparing Camera Sensor for camera_2
camera_2_intrinsics = {
    "cx": 1024.56301417, "cy": 592.004009216, "fx": 1050.21459961, "fy": 1051.06384277,
    "k1": 0, "k2": 0, "k3": 0, "k4": 0, "p1": 0, "p2": 0, "skew": 0, "scale_factor": 1}

camera_2 = {"_id": "camera_2", "name": "camera_2", "primary_view": False, "modality": "camera",
            "intrinsics": camera_2_intrinsics}

#SENSOR META - it contains information about sensors and it is constant across all jobs of same sensor
sensor_meta = [lidar_sensor, camera_1, camera_2]

#Collect frames for every sensor.
lidar_frames = [
    "https://example.com/pcd_url_1",
    "https://example.com/pcd_url_2"
]

camera_1_frames = [
    "https://example.com/image_url_1",
    "https://example.com/image_url_2"
]

camera_2_frames = [
    "https://example.com/image_url_3",
    "https://example.com/image_url_4"
]

#Preparing JOBS_PAYLOADS
jobs_payload = list()
for i in range(len(lidar_frames)):
    #Define the reference_id for the corresponding JOB
    reference_id = i
    
    #Collect ego_pose if the data is in world frame of reference
    ego_pose = {
    "heading": {"w": 1, "x": 0,
                "y": 0, "z": 0},
    "position": {"x": 0, "y": 0, "z": 0}
    }

    sensor_data = {"sensors": list(), "sensor_meta": sensor_meta, "ego_pose": ego_pose}

    lidar_heading = {"w": w, "x": x, "y": y, "z": z}
    lidar_position = {"x": 0, "y": 0, "z": 0}

    lidar_sensor = {"data_url": lidar_frames[i], "sensor_id": lidar_sensor_id,
                    "sensor_pose": {"heading": lidar_heading, "position": lidar_position}}

    camera_1_heading = {"w": w, "x": x, "y": y, "z": z}
    camera_1_position = {"x": 0, "y": 0, "z": 0}

    camera_1_sensor = {"data_url": camera_1_frames[i], "sensor_id": 'camera_1',
                       "sensor_pose": {"heading": camera_1_heading, "position": camera_1_position}}

    camera_2_heading = {"w": w, "x": x, "y": y, "z": z}
    camera_2_position = {"x": 0, "y": 0, "z": 0}

    camera_2_sensor = {"data_url": camera_2_frames[i], "sensor_id": 'camera_1',
                       "sensor_pose": {"heading": camera_2_heading, "position": camera_2_position}}

    sensor_data['sensors'].append(lidar_sensor)
    sensor_data['sensors'].append(camera_1_sensor)
    sensor_data['sensors'].append(camera_2_sensor)

    jobs_payload.append({"data":{"sensor_data":sensor_data}, "reference_id":reference_id})

for job in jobs_payload:

    data = {"data": job['data'], "reference_id": job['reference_id'], 'tag': tag, "batch_id": batch_id}

    response = Upload_jobs(PROJECT_ID=project_id, API_KEY=x_api_key, DATA=data)

Sample PCD file

# .PCD v0.7 - Point Cloud Data file format 
VERSION 0.7
FIELDS x y z
SIZE 4 4 4
TYPE F F F
COUNT 1 1 1
WIDTH 47286
HEIGHT 1
VIEWPOINT 0 0 0 1 0 0 0
POINTS 47286
DATA ascii
5075.773 3756.887 107.923
5076.011 3756.876 107.865
5076.116 3756.826 107.844
5076.860 3756.975 107.648
5077.045 3756.954 107.605
5077.237 3756.937 107.559
5077.441 3756.924 107.511
5077.599 3756.902 107.474
5077.780 3756.885 107.432
5077.955 3756.862 107.391
...

• Fetching result as mentioned in Fetching Job Result

  GET https://api.playment.in/v0/projects/<project_id>/jobs/<job_id>
  Parameters
  project_id: To be passed in the URL
  x-api-key: Secret key to be passed as a header

Fetch result response

{
    "data": {
        "project_id": "",
        "reference_id": "001",
        "job_id": "fde54589-ebty-48lp-677a-03a0428ca836",
        "batch_id": "b99d241a-bb80-ghyi-po90-c37d4fead593",
        "status": "completed",
        "tag": "sample_project",
        "priority_weight": 5,
        "result": "https://playment-data-uploads.s3.ap-south-1.amazonaws.com/sample-result.json"
    },
    "success": true
}

Result json output

{
         "data": {
			  "url": "https://playment-uploads.s3.ap-south-1.amazonaws.com/submissions/default/5a375fc2-fb4f-4d4b-808f-241ed881b74f_data.pcd",
				"legend": [
					    {
					      "id": "70d3d997-ce98-4e4d-97b4-46d97037c312",
					      "text": "crosswalk",
					      "color": "rgb(244, 35, 242)",
					      "label_key": 1,
					      "value": "crosswalk",
								"class_id" : "12345"
					    },
					    {
					      "id": "4a6aca59-944a-475b-8043-66aeb63ae7cf",
					      "text": "MINIVAN # 1",
						    "color": "rgb(71, 91, 111)",
					      "label_key": 2,
					      "value": "MINIVAN # 1",
								"class_id" : "5678"
					    },
					    {
					      "id": "286e0022-9229-4946-99b3-09f8ae5c79d0",
					      "text": "MINIVAN # 2",
					      "color": "rgb(72, 92, 112)",
					      "label_key": 3,
						     "value": "MINIVAN # 2",
								"class_id" : "5678"
					    }
				  ]
			},
      "type": "segmentation3d"
}

Result JSON Structure

Response Key	Description
data	object having url of a pcd file with labels and list of legends
data..url	String URL of the point cloud data file in pcd format. This file consists of x,y,z cordinatate of the points and their corresponding label. Please check the sample below.
data.legends	List of the legends each having id : String UUID of the legend text : String value of the legend i.e. name, separated by # if class is instanciable label_key : Number of legend, which will be present in the pcd file provided via data.url color : String color of the points in RGB(r,g,b) for this class value : String name of the class separated by # if class is instanciable class_id : String id of the class

output PCD file sample

# .PCD v0.7 - Point Cloud Data file format 
VERSION 0.7
FIELDS x y z label_key
SIZE 4 4 4 4
TYPE F F F F
COUNT 1 1 1 1
WIDTH 47286
HEIGHT 1
VIEWPOINT 0 0 0 1 0 0 0
POINTS 47286
DATA ascii
5075.773 3756.887 107.923 0
5076.011 3756.876 107.865 1
5076.116 3756.826 107.844 1
5076.860 3756.975 107.648 2
5077.045 3756.954 107.605 2
5077.237 3756.937 107.559 3
5077.441 3756.924 107.511 4
5077.599 3756.902 107.474 5
5077.780 3756.885 107.432 6
5077.955 3756.862 107.391 7