Repository files navigation

visionOS 2 + Object Tracking + ARKit means: we can create visual highlights of real world objects around us and have those visualizations respond to the proximity of our hands.

This project is largely a quick repurposing and combining of Apple'sScene Reconstruction sample project (which utilizes ARKit'sHandTrackingProvider) andObject Tracking sample project.

The full demo video with sound ishere.

Some details about putting together this demo are overhere.

1. Choose your Apple Developer Account in: Signing & Capabilities
2. Build

I live in Chicago and purchased the cereal and milk at a local Jewel in June 2024 – your local packaging may vary and prevent recognition. The three products used are:

1. Cap'n Crunch (Large Size)
2. Fairlife 2%
3. Lodge Dutch Oven

If you want to strip out the three bundled objects and use your own:

1. You will need to train on a.udsz file to create a.referenceObject, I recommend using Apple'sObject Capture sample project to create a.usdz file of your object
2. You will need to use Create ML (version 6, or higher, which comes bundled with Xcode 16) to train a.referenceObject from your.usdz, for me this process has taken anywhere from 4 - 16 hours per.referenceObject
3. You will need to bundle your new.referenceObject in the Xcode project
4. You will need to coordinate the naming of your new.referenceObject with the demo'sObjectType enum so everything plays nicely together