Course Content

Android development with Unity

• The development for Android is one of the more complex problems in using Unity to produce AR applications. Missing drivers and SDKs are a common problem for many new developers trying to gain a foothold in the Android ecosystem. How to approach the setup of the required drivers and interfaces, development environments and Unity modules we will learn in this chapter. Unity is used as a basis for correct scene setup and a complete pipeline from the editor to the player is explained.

AssetStore Requirements and ARCore Plugins

• Unity itself is only one of the key elements for development with ARCore. Some of the magic happens within internal libraries on the Android/Java page, which must be available in Unity first. We explain how to use the AssetStore and Plugin Manager to integrate a correct ARCore integration plugin and use this knowledge to build our first ARCore connections.

Requirements for a good AR experience

• Video inclusion brings with it its own class of problems and limitations. Lighting scenarios for detail recognition, data accuracy of the read environment, and various other requirements must be understood to create a well-functioning AR experience. We discuss the different ways to integrate AR into real situations and show problems that have to be considered in the development.

Environment detection and ARCore backgrounds

• How does ARCore recognize the environment and allow us to insert 3D graphics into the video images? We look at the underlying principles of visually moving odometry and the resulting capabilities of recognition algorithms to transfer our hierarchies into the displayed background. Coordinate systems are a key element for combining the 3D point cloud read by ARCore with the objects of our scene.

Hit point recognition and dynamic objects

• Now that there is an understanding of our environment, it is necessary to identify a point in the environment so that we can create a dynamic object at its position. Instantiation is the way to solve this problem and together with Prefabs we are talking about a cloning setup to place content in our 3D space. This allows us to place "things" in the world of the video image.

Layer detection and anchoring

• ARCore provides us with various complex information that we can evaluate for positioning purposes. Layers allow us to apply orientations to content related to surfaces like tables, walls, etc. Dynamic anchoring then moves the content in relation to the background scene. How to set this up so that there is a connection with the different game objects is a critical step to set things firmly in scene.

Light assessment and concealment

• For a seamless integration of the placed content, an assessment of the displayed light situation in the video image is necessary. Fortunately, ARCore provides some data of the surrounding lighting situation. We discuss ambient light influences on the scene content in Unity and integrate the ARCore results. Hiding is another topic that is necessary for the credible installation of the prefabs, as objects have to "hide" as soon as walls or visual barriers appear. The capabilities of the ARCore for this problem are discussed in detail..

Debugging and visualization of the environment

• It is often easier to build an understanding of the environmental scenario when it is graphically visualized. By displaying what ARCore is really doing, we get a clearer picture of incorrect behavior in the placement and orientation of our objects. This is done by visualizing specific properties of the underlying point cloud and using this information in the displayed environment.

Performance limitations of mobile devices

• Mobile development confronts us with special hurdles in performance, because within AR applications a lot of information has to be processed and displayed on a small device. Correct 3D geometry and performance limitations of logic and content are very important points to guarantee error-free execution with high frame rates on the device. We talk about typical performance problems and how they can be avoided.

More information

Target group

User Interface Developer, Unity Developer, 3D Developer

Prerequisites

Participants need experience in C# development with Unity and an understanding of creating Unity scenes for publishing. Theoretical background on 3D coordinate systems and augmented reality theory in general is necessary; understanding 3D geometry and shaders is a plus. Existing C# development experience is an advantage.

Organizational matters