Artoolkit Unity



We needed some marker tracking for some internal projects, thus we created lightweight ARToolkit Plus Unity Plugins. We decided to make them publicly available through our website, such that everybody can use it for free.

ARToolKit includes stereo and optical see-through support, is integrated with a range of smart glasses, and allows easy calibration for new devices. Unity 3D Engine Unity is a cross-platform game engine developed by Unity Technologies and used to develop video games for PC, consoles, mobile devices and websites. The Unity section of the forum at ARToolkit seems pretty dead, so I thought I'd ask here for anyone experimenting with it. I have a few things on their Unity implementation that seem a bit confusing in terms of interaction possibilities. ARToolKit does not support Universal Windows Platform yet. So the dlls shipped with ARToolKit unity package are not compatible with HoloLens. I built a wrapper of ARToolKit for UWP, and successfully run ARToolKit on HoloLens.

Graz University of Technology and affeliates wrote ARToolkit Plus as a software library around 2005. Since then it has always served as a reference for many other marker tracking solution for mobile devices. This also includes the famous Studierstube Tracker and the ARToolkit framework. We believe it is still one of the most performant software libraries, despite a few flaws in it’s implementation.

ARToolkit Plus went out of maintenance some time ago and was released under GPL. We forked Pavel Rojtberg’s fix to ARTK Plus repository as the starting point and added the respective wrapper functions for Unity. The plugins separate into a desktop version for OSX and Windows, which includes a demo scene and required scripts. As with the VIZARIO.Find plugins, the mobile version for Android and iOS goes on top of the desktop version.

The initial plugins do not contain all the possible parameter and option settings available through Unity. However, it should be fairly obvious to improve on this and add additional parametrization options on request. We use the plugin mainly for initialization purposes on Hololens projects right now. The marker detection and tracking algorithm runs in under 2 ms on a 2 Megapixel image on a single core Macbook Pro Late 2015. That is actually very fast, although we suggest to use images of 720p at max. We did not thoroughly test image resolutions above.

20 Jan 2017

HoloLens is fascinating in terms of holographic display and in-room localization, but the capability of the front-facing camera (locatable camera) is not fully utilized for potential augmented reality applications.

ARToolKit

Since ARToolKit has served AR/VR for more than 10 years, and it is open-source, it would be nice to have it running on HoloLens. The obstacles towards this goal are quite clear as well:

  • Universal Windows has a whole different set of tools and run-time libraries. The dependencies of existing ARToolKit does not exist on UWP, such as pthreads, video libraries, opencv.
  • ARToolKit is written in native C/C++ code, and an additional wrapper is needed for development on Unity, which is recommended for HoloLens. (If you are not good at Direct3D, and Windows App)

Luckily, after spending some time on this, I successfully get some pieces of code written, and actually make ARToolKit runs on HoloLens.

The repository is called HoloLensARToolKit on Github.

Demos

The following videos and screenshots are taken with HoloLensARToolKit v0.1.

Samples of HoloLensARToolKit

Artoolkit Unity

Artoolkit unity3d

There are three samples of HoloLensARToolKit, each representing a certain kind of marker that is supported: single marker, cube marker and multi marker.

Minion on the Cube

Artoolkit For Unity

This video is taken on HoloLens Device Portal, also known as Mixed Reality Capture. You might have noticed that there is still some lag between the video that is captured and update in the rendering.

More Screenshots

  • Single Marker
  • Cube Marker
  • Multi Marker

Finally

You can access more articles describing the implementation details of HoloLensARToolKit in my blog, simply clicking on the tag: hololens-artoolkit.

With ARToolKit integrated with HoloLens, many more augmented reality applications are possible. If you are AR/VR developer, you must see what I mean here.

Artoolkit Unity Github

Thanks for reading.

Artoolkit Unity Tutorial

Related Posts





Comments are closed.