As an Associate of Unity Asset Store, this website earns from qualifying purchases & contains affiliate links: check the footer for more info.

AR Shadow & Light Estimation in Unity

How to Create AR Shadow? To define what AR Shadow is, you need to define what the simple Shadow is. To produce a Shadow, you need 4 elements in the chain:

  1. Light source.
  2. 3D Object.
  3. Surface for Shadow of 3D Object that is an obstacle on the path of light from Light Source.
  4. Shader (program that calculates shadow).

In this tutorial, you will learn all these elements specifically for Augmented Reality. I’ve also created some Templates with Real-Time AR Shadows using Unity Game Engine & AR Foundation (ARKit, ARCore).
AR Unity Assets you can find in the AR Bundle.

Unity Asset Store — Download Button

AR Shadow Shader

Shadows in Augmented Reality should fall on transparent planes (e.g., for Plane Detection) because in the most cases your AR scene doesn’t imply the presence of virtual Ground or Floor. This rule is agnostic of AR Engine. To implement such AR Shadows that are projected onto the transparent surfaces, you need a specific AR Shader that is assigned to the Material which describes the appearance of the surface:

/*
=============================================================
Unity Assets by MAKAKA GAMES: https://makaka.org/unity-assets
=============================================================
*/

Shader "AR/ARShadowSurface" 
{
	Properties
	{ 
		_Cutoff("Cutout", Range(0,1)) = 0.5 
	}

	SubShader
	{ 
		Pass 
		{ 
			Alphatest Greater[_Cutoff]
		}	

		Pass
		{ 
			Blend DstColor Zero Tags 
			{ 
				"LightMode" = "ForwardBase" 
			}

			CGPROGRAM

			#pragma vertex vert
			#pragma fragment frag
			#include "UnityCG.cginc"
			#pragma multi_compile_fwdbase
			#include "AutoLight.cginc"

			struct v2f 
			{ 
				float4 pos : SV_POSITION; 
				LIGHTING_COORDS(0,1) 
			};

			v2f vert(appdata_base v) 
			{
				v2f o; 
				o.pos = UnityObjectToClipPos(v.vertex); 
				TRANSFER_VERTEX_TO_FRAGMENT(o);
				return o; 
			}

			fixed4 frag(v2f i) : COLOR
			{
				float attenuation = LIGHT_ATTENUATION(i);
				return attenuation;
			} 

			ENDCG 
		} 
	} 

	Fallback "Transparent/Cutout/VertexLit" 
} 

Unity Assets with AR Shadows:

AR Light Estimation in Unity AR Foundation

The Light Estimation mechanism estimates light data in physical space and applies it to game space depending on what is supported by a particular smartphone. All options are enabled by default:

  • Ambient Intensity,
  • Ambient Color,
  • Ambient Spherical Harmonics,
  • Main Light Direction,
  • Main Light Intensity.

This way, you can add more realism to your AR Application. On iOS, for example, when AR Mode is not Face Tracking, then only Ambient Intensity and Ambient Color are supported.

In AR Gallery (docs) you can turn it off in the Hierarchy window:
XR Origin > Camera Offset > AR Camera: AR Camera Manager > Light Estimation.

AR Shadow Gallery - Unity Asset - Light Estimation

You need to apply the Light Estimation data that you can get from AR Camera Manager to the Light Source. Control Script is attached to the Light game object:

/*
=============================================================
Unity Assets by MAKAKA GAMES: https://makaka.org/unity-assets
=============================================================
*/

using UnityEngine;
using UnityEngine.Rendering;
using UnityEngine.XR.ARFoundation;

/// <summary>
/// A component that can be used to access the most
/// recently received light estimation information
/// for the physical environment as observed by an
/// AR device.
/// </summary>
[RequireComponent(typeof(Light))]
[HelpURL("https://makaka.org/unity-assets")]
public class LightEstimation : MonoBehaviour
{
    [SerializeField]
    [Tooltip("The ARCameraManager which will produce frame events containing" +
        " light estimation information.")]
    private ARCameraManager m_CameraManager;

    /// <summary>
    /// Get or set the <c>ARCameraManager</c>.
    /// </summary>
    public ARCameraManager cameraManager
    {
        get { return m_CameraManager; }
        set
        {
            if (m_CameraManager == value)
                return;

            if (m_CameraManager != null)
                m_CameraManager.frameReceived -= FrameChanged;

            m_CameraManager = value;

            if (m_CameraManager != null & enabled)
                m_CameraManager.frameReceived += FrameChanged;
        }
    }

    /// <summary>
    /// Estimated brightness of the physical environment, if available.
    /// </summary>
    public float? brightness { get; private set; }

    /// <summary>
    /// Estimated color temperature of the physical environment, if available.
    /// </summary>
    public float? colorTemperature { get; private set; }

    /// <summary>
    /// Estimated color correction value of the physical environment,
    /// if available.
    /// </summary>
    public Color? colorCorrection { get; private set; }

    /// <summary>
    /// Estimated direction of the main light of the physical environment,
    /// if available.
    /// </summary>
    public Vector3? mainLightDirection { get; private set; }

    /// <summary>
    /// Estimated color of the main light of the physical environment,
    /// if available.
    /// </summary>
    public Color? mainLightColor { get; private set; }

    /// <summary>
    /// Estimated intensity in lumens of main light of the physical environment,
    /// if available.
    /// </summary>
    public float? mainLightIntensityLumens { get; private set; }

    /// <summary>
    /// Estimated spherical harmonics coefficients of the physical environment,
    /// if available.
    /// </summary>
    public SphericalHarmonicsL2? sphericalHarmonics { get; private set; }

    [SerializeField]
    private float m_BrightnessMod = 2.0f;

    private Light m_Light;

    private void Awake ()
    {
        m_Light = GetComponent<Light>();
    }

    private void OnEnable()
    {
        if (m_CameraManager != null)
        {
            m_CameraManager.frameReceived += FrameChanged;
        }
    }

    private void OnDisable()
    {
        if (m_CameraManager != null)
        {
            m_CameraManager.frameReceived -= FrameChanged;
        }
    }

    private void FrameChanged(ARCameraFrameEventArgs args)
    {
        if (args.lightEstimation.averageBrightness.HasValue)
        {
            brightness = args.lightEstimation.averageBrightness.Value;

            m_Light.intensity = brightness.Value * m_BrightnessMod;
        }

        if (args.lightEstimation.averageColorTemperature.HasValue)
        {
            colorTemperature =
                args.lightEstimation.averageColorTemperature.Value;

            m_Light.colorTemperature = colorTemperature.Value;
        }
        
        if (args.lightEstimation.colorCorrection.HasValue)
        {
            colorCorrection = args.lightEstimation.colorCorrection.Value;

            m_Light.color = colorCorrection.Value;
        }

        if (args.lightEstimation.mainLightDirection.HasValue)
        {
            mainLightDirection = args.lightEstimation.mainLightDirection;

            m_Light.transform.rotation =
                Quaternion.LookRotation(mainLightDirection.Value);
        }

        if (args.lightEstimation.mainLightColor.HasValue)
        {
            mainLightColor = args.lightEstimation.mainLightColor;

            m_Light.color = mainLightColor.Value;
        }

        if (args.lightEstimation.mainLightIntensityLumens.HasValue)
        {
            mainLightIntensityLumens =
                args.lightEstimation.mainLightIntensityLumens;

            m_Light.intensity =
                args.lightEstimation.averageMainLightBrightness.Value;
        }

        if (args.lightEstimation.ambientSphericalHarmonics.HasValue)
        {
            sphericalHarmonics = args.lightEstimation.ambientSphericalHarmonics;

            RenderSettings.ambientMode = AmbientMode.Skybox;
            RenderSettings.ambientProbe = sphericalHarmonics.Value;
        }
    }
}


Unity Assets with Light Estimation in AR:

Unity Assets

Support for Unity Assets

I am Andrey Sirota, Founder of Makaka Games and full-time Publisher on the Unity Asset Store. First, read the latest docs online. If it didn’t help, get the support.

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *

Comment moderation is enabled. Your comment may take some time to appear.

Back to top button