All Posts

  • SVGF

    …

    SVGF

    计算光照时忽略brdf中的材质信息,避免引入高频变化,以diffuse为例 diffuse = L* rho * n * l / pi demodulate albedo = L * n * l / pi Temporal Accumulation 和TAA的重投影类似,通过深度/法线/meshid来reject Variance Esitimation 在时间上积累像素亮度的的一阶和二阶矩,mu_1, mu_2 计算方差=mu_2 - mu_1^2 Edge-Avoiding Atrous Wavelets Wavelet filter用于大范围滤波 法线/深度作为edge avoiding算法避免模糊掉几何边缘

    $$\hat{c}_{i+1}(p)

    \frac{ \sum_{q \in \Omega} h(q)\, w(p,q)\, \hat{c}i(q) }{ \sum{q \in \Omega} h(q)\, w(p,q) }$$ vec3 sum = vec3(0); float weight_sum = 0;

    for q in neighborhood: float weight = wavelet_kernel(q) // atrous wavelelt * depth_weight(p, q) // edge stopping * normal_weight(p, q) * luminance_weight(p, q);

    sum += weight * color[q];
    weight_sum += weight;
    

    color_out[p] = sum / weight_sum;

    墙壁的几何边缘不会被模糊掉

    Luminance Edge Stopping Function 避免模糊掉阴影细节,通过上面计算的variance

    $$w(p,q)

    \exp\left( - \frac{ \left| l_i(p) - l_i(q) \right| }{ \sqrt{ g_{3 \times 3}\left( \operatorname{Var}\left(l_i(p)\right) \right) } } \right)$$ Variance大->噪声多-> 增大模糊,降低噪点 Variance小->噪声小-> 减小模糊,保留细节 不过由于filter中对variance提前做了一遍高斯模糊来降低空域的噪声,一些阴影的细节会出现被模糊的情况 阴影区域不会被模糊掉

    ASVGF

    解决拖影和细节丢失问题, 生成一张temporal gradient图,估计当前帧和历史帧的变化梯度

    asvgf_gradient_reproject.comp
        -> 生成 gradient sample positions
    path tracer
        -> 对这些位置产生 gradient samples
    asvgf_gradient_img.comp
        -> 根据当前/历史亮度差生成 gradient image
    asvgf_gradient_atrous.comp
        -> 把稀疏 gradient 扩散成可用的低分辨率 gradient field
    asvgf_temporal.comp
        -> 读取 gradient,做 anti-lag
    

    Temporal Accumulation & AntiLag, 以primary ray hf为例, 权重alpha受这个temporal gradient的变化

        if(temporal_sample_valid_diff)
        {
            // Compute the antilag factors based on the gradients
            float antilag_alpha_hf = clamp(mix(1.0, global_ubo.flt_antilag_hf * grad_hf_spec.x, global_ubo.flt_temporal_hf), 0, 1);
    
            // Adjust the history length, taking the antilag factors into account
            // gradient大,hist_len_hf小,历史帧越容易被丢弃
            float hist_len_hf = min(temporal_moments_histlen_hf.b * pow(1.0 - antilag_alpha_hf, 10) + 1.0, 256.0);
    
            // Compute the blending weights based on history length, so that the filter
            // converges faster. I.e. the first frame has weight of 1.0, the second frame 1/2, third 1/3 and so on.
            float alpha_color_hf = max(global_ubo.flt_min_alpha_color_hf, 1.0 / hist_len_hf);
            float alpha_moments_hf = max(global_ubo.flt_min_alpha_moments_hf, 1.0 / hist_len_hf);
    
            // Adjust the blending factors, taking the antilag factors into account again
            alpha_color_hf = mix(alpha_color_hf, 1.0, antilag_alpha_hf);
            alpha_moments_hf = mix(alpha_moments_hf, 1.0, antilag_alpha_hf);
    
            // Blend!
            out_color_hf.rgb = mix(temporal_color_hf.rgb, color_curr_hf.rgb, alpha_color_hf);
    
            out_moments_histlen_hf.rg = mix(temporal_moments_histlen_hf.rg, spatial_moments_hf.rg, alpha_moments_hf);
            out_moments_histlen_hf.b = hist_len_hf;
        }
    

    q2rtx impl

    在gbuffer生成后

    一个 compute shader pass reproject到上一帧生成
        imageStore(IMG_ASVGF_GRAD_SMPL_POS_A, pos_grad, uvec4(gradient_idx));
    
        imageStore(IMG_ASVGF_GRAD_HF_SPEC_PING, pos_grad, vec4(found_prev_lum, 0, 0));
    
        imageStore(IMG_ASVGF_RNG_SEED_A, ipos, texelFetch(TEX_ASVGF_RNG_SEED_B, found_pos_prev, 0));
        imageStore(IMG_PT_NORMAL_A, ipos, texelFetch(TEX_PT_NORMAL_B, found_pos_prev, 0));
        imageStore(IMG_PT_BASE_COLOR_A, ipos, texelFetch(TEX_PT_BASE_COLOR_B, found_pos_prev, 0));
        imageStore(IMG_PT_METALLIC_A, ipos, texelFetch(TEX_PT_METALLIC_B, found_pos_prev, 0));
    

    Read More
  • Q2RTX codepath

    …

    Q2RTX codepath

    #define MATERIAL_KIND_MASK           0xf0000000
    #define MATERIAL_KIND_INVALID        0x00000000
    #define MATERIAL_KIND_REGULAR        0x10000000
    #define MATERIAL_KIND_CHROME         0x20000000
    #define MATERIAL_KIND_WATER          0x30000000
    #define MATERIAL_KIND_LAVA           0x40000000
    #define MATERIAL_KIND_SLIME          0x50000000
    #define MATERIAL_KIND_GLASS          0x60000000
    #define MATERIAL_KIND_SKY            0x70000000
    #define MATERIAL_KIND_INVISIBLE      0x80000000
    #define MATERIAL_KIND_EXPLOSION      0x90000000
    #define MATERIAL_KIND_TRANSPARENT    0xa0000000 // Transparent walls. Have a distortion effect applied.
    #define MATERIAL_KIND_SCREEN         0xb0000000
    #define MATERIAL_KIND_CAMERA         0xc0000000
    #define MATERIAL_KIND_CHROME_MODEL   0xd0000000
    #define MATERIAL_KIND_TRANSP_MODEL   0xe0000000 // Transparent models. No distortion, just "see through".
    
    
    struct RayPayloadGeometry {
       vec2 barycentric;
       /* two packed 16 bit integers, buffer index in low 16 bits and
        * instance index in high 16 bits */
       int buffer_and_instance_idx;
       uint primitive_id;
       float hit_distance;
    };
    
    struct RayPayloadEffects {
       uvec2 transparency; // half4x16
       uint distances; // half2x16 - min and max
       uvec4 fog1; // half8x16: .xy = color.rgba; .z = t_min, t_max; .w = density: a and b for (a*t + b)
       uvec4 fog2; // same as fog1 but for a fog volume further away
    #ifndef KHR_RAY_QUERY
       // Store TMax in the payload because gl_RayTmaxEXT changes while the ray is being traced.
       // See the GLSL_EXT_ray_tracing spec near "description for gl_RayTminEXT and gl_RayTmaxEXT"
       // https://github.com/KhronosGroup/GLSL/blob/master/extensions/ext/GLSL_EXT_ray_tracing.txt 
       float rayTmax; 
    #endif
    };
    
    layout(location = RT_PAYLOAD_GEOMETRY) rayPayloadEXT RayPayloadGeometry ray_payload_geometry;
    layout(location = RT_PAYLOAD_EFFECTS) rayPayloadEXT RayPayloadEffects ray_payload_effects;
    
    // primary ray
    vkpt_pt_trace_primary_rays(trace_cmd_buf);
    // qvkCmdTraceRaysKHR(cmd_buf,
    //  &raygen,
    //  &miss_and_hit,
    //  &miss_and_hit,
    //  &callable,
    //  width/2, height, depth); // 2 depth for checkerboard rendering, split
    vkpt_submit_command_buffer();
    
    // indirect reflection/refraction
    if (ref_mode.reflect_refract > 0)
        vkpt_pt_trace_reflections(trace_cmd_buf, 0);
        
    // multiple reflection/refraction
    if (ref_mode.reflect_refract > 1)
        for (int pass = 0; pass < ref_mode.reflect_refract - 1; pass++)
            vkpt_pt_trace_reflections(trace_cmd_buf, pass + 1);
    
    // seperated svgf reprojected pass
    vkpt_asvgf_gradient_reproject(trace_cmd_buf);
    
    vkpt_pt_trace_lighting(trace_cmd_buf, ref_mode.num_bounce_rays);
    
    vkpt_submit_command_buffer();
    
    vkpt_asvgf_filter(post_cmd_buf, cvar_pt_num_bounce_rays->value >= 0.5f);
    
    vkpt_interleave(post_cmd_buf);
    
    vkpt_taa(post_cmd_buf);
    
    vkpt_bloom_record_cmd_buffer(post_cmd_buf);
    vkpt_tone_mapping_record_cmd_buffer();
    vkpt_fsr_do(post_cmd_buf);
    vkpt_submit_command_buffer_simple();
    

    primary_rays.rgen(path tracer overview的1)

     // generate primary ray information, including bias offset to get depth of field
    Ray ray = get_primary_ray(inUV);
    
    
    // primary ray -> path_tracer.rchit/path_tracer.rmiss
    traceRayEXT( topLevelAS[TLAS_INDEX_GEOMETRY], rayFlags, instance_mask,
            SBT_RCHIT_GEOMETRY /*sbtRecordOffset*/, 0 /*sbtRecordStride*/, SBT_RMISS_EMPTY /*missIndex*/,
            ray.origin, ray.t_min, ray.direction, ray.t_max, RT_PAYLOAD_GEOMETRY);
    
    Triangle triangle;
    
    if (found_intersection(ray_payload_geometry)) // hit surface
    {
        ray.t_max = ray_payload_geometry.hit_distance;
        triangle = get_hit_triangle(ray_payload_geometry);
    }
    // particle/explosion/sprite/beam
    traceRayEXT( topLevelAS[TLAS_INDEX_EFFECTS], rayFlags, instance_mask,
            SBT_RCHIT_EFFECTS /*sbtRecordOffset*/, 0 /*sbtRecordStride*/, SBT_RMISS_EMPTY /*missIndex*/,
            ray.origin, ray.t_min, ray.direction, ray.t_max, RT_PAYLOAD_EFFECTS);
    
    vec4 effects = get_payload_transparency_with_fog(ray_payload_effects, ray.t_max);
    
    vec3 bary = get_hit_barycentric(ray_payload_geometry);
    
    // store 2 visibility buffer
    {
        uvec2 vis_buf;
        vis_buf.x = triangle.instance_index;
        vis_buf.y = triangle.instance_prim;
        imageStore(IMG_PT_VISBUF_PRIM_A, ipos, uvec4(vis_buf, 0, 0)); // rg32ui / R32G32_UINT
        imageStore(IMG_PT_VISBUF_BARY_A, ipos, vec4(bary.yz, 0, 0)); // rg16f / R16G16_SFLOAT
    }
    
    // get hit point information
    vec3 position = triangle.positions * bary;
    vec2 texcoord = triangle.texcoords * bary;
    vec3 geo_normal = triangle.normals* bary;
    vec3 flat_normal = normalize(cross(
            triangle.positions[1] - triangle.positions[0],
            triangle.positions[2] - triangle.positions[1]));
    
    /* compute view-space derivatives of depth and motion vectors */
    Ray ray_0 = get_primary_ray(inUV);
    Ray ray_x = get_primary_ray(inUV + vec2(1.0 / float(global_ubo.width), 0));
    Ray ray_y = get_primary_ray(inUV + vec2(0, 1.0 / float(global_ubo.height)));
    
    // larger coneangle means high gradient, means higher mips
    float half_cone_angle = sqrt(1.0 - square(min(
    dot(ray_0.direction, ray_x.direction), 
    dot(ray_0.direction, ray_y.direction))));
    
    vec2 tex_coord_x, tex_coord_y;
    float fwidth_depth;
    // get texure lod
    compute_anisotropic_texture_gradients(position, flat_normal, ray.direction, 
        ray_payload_geometry.hit_distance * half_cone_angle, triangle.positions, 
        triangle.tex_coords, tex_coord, tex_coord_x, tex_coord_y, fwidth_depth);
    
    vec3 pos_ws_curr = position;
    vec3 pos_ws_prev = triangle.positions_prev * bary;
    vec2 screen_pos_curr, screen_pos_prev;
    float distance_curr, distance_prev;
    projection_view_to_screen((global_ubo.V * vec4(pos_ws_curr, 1)).xyz, screen_pos_curr, distance_curr, false);
    projection_view_to_screen((global_ubo.V_prev * vec4(pos_ws_prev, 1)).xyz, screen_pos_prev, distance_prev, true);
    
    // motion vector
    vec3 motion;
    motion.xy = screen_pos_prev - screen_pos_curr;
    motion.z = distance_prev - distance_curr;
    
    imageStore(IMG_PT_VIEW_DEPTH_A, ipos, vec4(distance_curr));
    imageStore(IMG_PT_MOTION, ipos, vec4(motion, fwidth_depth));
    
    // Get the primary surface material parameters
    get_material(
        triangle,
        bary,
        tex_coord,
        tex_coord_x,
        tex_coord_y,
        -1,
        geo_normal,
        primary_base_color,
        normal,
        primary_metallic,
        primary_roughness,
        primary_emissive,
        primary_specular_factor);
    
    // get mat id
    uint material_id = triangle.material_id;
    ...
    
    // handle various material one by one
    
    // write to gbuffer
    // shading normal
    imageStore(IMG_PT_NORMAL_A, ipos, uvec4(encode_normal(normal)));
    // geo normal
    imageStore(IMG_PT_GEO_NORMAL_A, ipos, uvec4(encode_normal(geo_normal)));
    //...
    imageStore(IMG_PT_SHADING_POSITION, ipos, vec4(position.xyz, uintBitsToFloat(material_id)));
    //...
    imageStore(IMG_PT_VIEW_DIRECTION, ipos, vec4(direction, float(checkerboard_flags)));
    imageStore(IMG_PT_THROUGHPUT, ipos, vec4(throughput, distance_curr));
    imageStore(IMG_PT_BOUNCE_THROUGHPUT, ipos, vec4(1, 1, 1, half_cone_angle));
    imageStore(IMG_PT_CLUSTER_A, ipos, ivec4(triangle.cluster));
    imageStore(IMG_PT_BASE_COLOR_A, ipos, vec4(primary_base_color, primary_specular_factor));
    imageStore(IMG_PT_METALLIC_A, ipos, vec4(primary_metallic, primary_roughness, 0, 0));
    imageStore(IMG_PT_GODRAYS_THROUGHPUT_DIST, ipos, vec4(1, 1, 1, distance_curr));
    transparent = alpha_blend_premultiplied(effects, transparent);
    
    imageStore(IMG_PT_TRANSPARENT, ipos, transparent);
    

    reflect_refract.rgen

    reflect_refract.rgen 会在 G-buffer 当前 surface 是 water / slime / glass / chrome / screen / camera / transparent 等特殊材质时,每像素沿 reflection 或 refraction 方向继续追一条 ray,并用命中的 secondary surface 更新 G-buffer / throughput / medium / motion。下面的 IOR + checkerboard 代码是抽象伪码;真实代码会按 material kind 分支处理 thin/thick glass、water/slime medium、total internal reflection、chrome/screen 只反射、transparent see-through 等情况。

    // get shading pos
    vec4 position_material = imageLoad(IMG_PT_SHADING_POSITION, ipos);
    
    if(!( primary_is_water || 
          primary_is_slime || 
          primary_is_glass || 
          primary_is_chrome || 
          primary_is_screen || 
          primary_is_camera || 
          primary_is_transparent ))
        return;
        
    index_of_refraction = get_mat_ior();
    vec3 refracted_direction = refract(direction, normal, index_of_refraction);
    float F = pow(1.0 - n_dot_v, 5.0);
    do_refraction = is_odd_checkerboard && F < 1;
    direction = do_refraction ? refract() : reflect();
    correction_factor *= 2; // 补能量
    throughput *= correction_factor;
    primary_medium = new_medium;//更新介质
    
    int reflection_cull_mask = REFLECTION_RAY_CULL_MASK|xxx;
    
    Ray reflection_ray;
    reflection_ray.origin = position;
    reflection_ray.direction = direction;
    
    trace_geometry_ray(reflection_ray, backface_culling, reflection_cull_mask);
    Triangle triangle;
    
    if (found_intersection(ray_payload_geometry))
    {
        reflection_ray.t_max = ray_payload_geometry.hit_distance;
        triangle = get_hit_triangle(ray_payload_geometry);
    }
    
    vec4 effects = trace_effects_ray(reflection_ray, /* skip_procedural = */ false);
    
    // almost same with primary ray
    ...
    // update gbuffer
    

    asvgf_gradient_reproject.comp

    direct_lighting.rgen// direct diffuse + direct sun shadow

    vec3 high_freq, specular;
    direct_lighting(ipos, is_odd_checkerboard, high_freq, specular);
    
    vec4 view_direction = texelFetch(TEX_PT_VIEW_DIRECTION, ipos, 0);
    vec3 normal = decode_normal(texelFetch(TEX_PT_NORMAL_A, ipos, 0).x);
    vec3 geo_normal = decode_normal(texelFetch(TEX_PT_GEO_NORMAL_A, ipos, 0).x);
    vec4 primary_base_color = texelFetch(TEX_PT_BASE_COLOR_A, ipos, 0);
    float primary_specular_factor = primary_base_color.a;
    vec2 metal_rough = texelFetch(TEX_PT_METALLIC_A, ipos, 0).xy;
    float primary_metallic = metal_rough.x;
    float primary_roughness = metal_rough.y;
    uint cluster_idx = texelFetch(TEX_PT_CLUSTER_A, ipos, 0).x;
    
    vec3 primary_albedo, primary_base_reflectivity;
    get_reflectivity(primary_base_color.rgb, primary_metallic, primary_albedo, primary_base_reflectivity);
    
    // precompute phong coeff
    float alpha = square(roughness);
    float phong_exp = RoughnessSquareToSpecPower(alpha);
    float phong_scale = min(100, 1 / (M_PI * square(alpha)));
    float phong_weight = clamp(specular_factor * luminance(base_reflectivity) / (luminance(base_reflectivity) + luminance(albedo)), 0, 0.9);
    
    /*
    get_direct_illumination(
        position, 
        normal, 
        geo_normal, 
        cluster_idx, 
        material_id, 
        shadow_cull_mask, 
        view_direction.xyz, 
        primary_albedo,
        primary_base_reflectivity,
        primary_specular_factor,
        primary_roughness, 
        primary_medium, 
        spec_enable_caustics != 0, 
        direct_specular_weight, 
        global_ubo.pt_direct_polygon_lights > 0,
        global_ubo.pt_direct_dyn_lights > 0,
        is_gradient,
        0,
        direct_diffuse,
        direct_specular);
    */
    
    //  sample a static light
    // 1. 重要性采样:从很多 polygon lights 里选一个 light
    // float m = spherical_tri_area(light.positions, p, n, V, phong_exp, phong_scale, phong_weight);
    // float light_lum = luminance(light.color);
    // m *= abs(light_lum);
    // 2. 几何采样:在这个 light 的三角形上选一个点
    sample_polygonal_lights(
        cluster_idx,
        position, 
        normal, 
        geo_normal, 
        view_direction, 
        phong_exp, 
        phong_scale,
        phong_weight, 
        is_gradient, 
        pos_on_light_polygonal, 
        contrib_polygonal,
        polygonal_light_index,
        polygonal_light_pdfw,
        polygonal_light_is_sky,
        rng);
        // Limit the solid angle of sphere lights for indirect lighting 
        // in order to kill some fireflies in locations with many sphere lights.
        // Example: green wall-lamp corridor in the "train" map.
        float max_solid_angle = (bounce == 0) ? 2 * M_PI : 0.02;
    
    // sample a dynamic light
    // dynamic lights 内部是在 num_dyn_lights 中 uniform 选一个 light,pdf = 1 / num_dyn_lights,
    // sample_dynamic_lights 里会乘 num_dyn_lights 做 1/pdf 补偿;sphere/spot 的能量用 analytic irradiance 近似。
    sample_dynamic_lights(
        position,
        normal,
        geo_normal,
        max_solid_angle,
        pos_on_light_dynamic,
        contrib_dynamic,
        rng);
    
    float spec_polygonal = phong(normal, normalize(pos_on_light_polygonal - position), view_direction, phong_exp) * phong_scale;
    float spec_dynamic = phong(normal, normalize(pos_on_light_dynamic - position), view_direction, phong_exp) * phong_scale;
    
    // compute luminance of two light
    float l_polygonal  = luminance(abs(contrib_polygonal)) * mix(1, spec_polygonal, phong_weight);
    float l_dynamic = luminance(abs(contrib_dynamic)) * mix(1, spec_dynamic, phong_weight);
    float l_sum = l_polygonal + l_dynamic;
    
    bool null_light = (l_sum == 0);
    
    float w = null_light ? 0.5 : l_polygonal / (l_polygonal + l_dynamic);
    
    // random chose one, polygon or dynamic
    float rng2 = get_rng(RNG_NEE_LIGHT_TYPE(bounce));
    is_polygonal = (rng2 < w);
    vis = is_polygonal ? (1 / w) : (1 / (1 - w));
    vec3 pos_on_light = null_light ? position : (is_polygonal ? pos_on_light_polygonal : pos_on_light_dynamic);
    vec3 contrib = is_polygonal ? contrib_polygonal : contrib_dynamic;
    
    Ray shadow_ray = get_shadow_ray(position - view_direction * 0.01, pos_on_light, 0);
    
    vis *= trace_shadow_ray(shadow_ray, null_light ? 0 : shadow_cull_mask);
    
    // TODO, Adaptive Shadow Testing for Ray Tracing, Ward 1994 optimization
    
    vec3 radiance = vis * contrib;
    
    vec3 L = pos_on_light - position;
    L = normalize(L);
    
    // specular lighting
    
    if(is_polygonal && direct_specular_weight > 0 && polygonal_light_is_sky && global_ubo.pt_specular_mis != 0)
    {
        // MIS with direct specular and indirect specular.
        // Only applied to sky lights, for two reasons:
        //  1) Non-sky lights are trimmed to match the light texture, and indirect rays don't see that;
        //  2) Non-sky lights are usually away from walls, so the direct sampling issue is not as pronounced.
    
        direct_specular_weight *= get_specular_sampled_lighting_weight(roughness,
            normal, -view_direction, L, polygonal_light_pdfw);
    }
    
    vec3 F = vec3(0);
    
    if(vis > 0 && direct_specular_weight > 0)
    {
        vec3 specular_brdf = GGX_times_NdotL(view_direction, normalize(pos_on_light - position),
            normal, roughness, base_reflectivity, 0.0, specular_factor, F);
        specular = radiance * specular_brdf * direct_specular_weight;
    }
    
    float NdotL = max(0, dot(normal, L));
    
    // diffuse lighting
    
    float diffuse_brdf = NdotL / M_PI;
    diffuse = radiance * diffuse_brdf * (vec3(1.0) - F);
    
    high_freq += direct_diffuse;
    o_specular += direct_specular;
    /*
    get_sunlight(
        cluster_idx,
        material_id,
        position,
        normal,
        geo_normal,
        view_direction.xyz,
        primary_base_reflectivity,
        primary_specular_factor,
        primary_roughness,
        primary_medium,
        spec_enable_caustics != 0,
        direct_sun_diffuse,
        direct_sun_specular,
        shadow_cull_mask);
    */
    // similar with polygon/dynamic light computation
    
    high_freq += direct_sun_diffuse;
    o_specular += direct_sun_specular;
    
    o_specular = demodulate_specular(primary_base_reflectivity, o_specular);
    
    high_freq = clamp_output(high_freq);
    o_specular = clamp_output(o_specular);
    
    high_freq *= STORAGE_SCALE_HF;//32
    o_specular *= STORAGE_SCALE_SPEC;//32
    imageStore(IMG_PT_COLOR_LF_SH, ipos, vec4(0));
    imageStore(IMG_PT_COLOR_LF_COCG, ipos, vec4(0));
    imageStore(IMG_PT_COLOR_HF, ipos, uvec4(packRGBE(high_freq))); // diffuse
    imageStore(IMG_PT_COLOR_SPEC, ipos, uvec4(packRGBE(o_specular))); // specular
    

    indirect_lighting.rgen

    // already have primary ray hit surface into
    view_direction = texelFetch(TEX_PT_VIEW_DIRECTION, ipos, 0);
    normal = decode_normal(texelFetch(TEX_PT_NORMAL_A, ipos, 0).x);
    geo_normal = decode_normal(texelFetch(TEX_PT_GEO_NORMAL_A, ipos, 0).x);
    primary_base_color = texelFetch(TEX_PT_BASE_COLOR_A, ipos, 0);
    primary_specular_factor = primary_base_color.a;
    vec2 metal_rough = texelFetch(TEX_PT_METALLIC_A, ipos, 0).xy;
    primary_metallic = metal_rough.x;
    primary_roughness = metal_rough.y;
    
    get_reflectivity(primary_base_color.rgb, primary_metallic, primary_albedo, primary_base_reflectivity);
    float NoV = max(0, -dot(normal, view_direction.xyz));
    // 判断光线类型
    bool is_specular_ray;
    
    // compute the indirect ray direction towards ggx reflection lobe
    if(spec_bounce_index == 0)
    {
        specular_pdf = (primary_metallic == 1 && fake_specular_weight == 0) ? 1.0 : 0.5;
        
        // 50%概率进,specular path
        if(rng_frensel < specular_pdf) 
        {
            mat3 basis = construct_ONB_frisvad(normal);
    
            // Sampling of normal distribution function to compute the reflected ray.
            // See the paper "Sampling the GGX Distribution of Visible Normals" by E. Heitz, 
            // Journal of Computer Graphics Techniques Vol. 7, No. 4, 2018.
            // http://jcgt.org/published/0007/04/01/paper.pdf 
    
            vec3 N = normal;
            vec3 V = view_direction.xyz;
            vec3 H = ImportanceSampleGGX_VNDF(rng3, primary_roughness, V, basis);
            vec3 L = reflect(V, H);
    
            float NoL = max(0, dot(N, L));
            float NoH = max(0, dot(N, H));
            float VoH = max(0, -dot(V, H));
    
            if (NoL > 0 && NoV > 0)
            {
                // See the Heitz paper referenced above for the estimator explanation.
                //   (BRDF / PDF) = F * G2(V, L) / G1(V)
                // Assume G2 = G1(V) * G1(L) here and simplify that expression to just G1(L).
                
                float G1_NoL = G1_Smith(primary_roughness, NoL);
                vec3 F = schlick_fresnel(primary_base_reflectivity, VoH, primary_specular_factor);
    
                bounce_throughput *= G1_NoL * F;
    
                bounce_throughput *= 1 / specular_pdf;
                is_specular_ray = true;
                bounce_direction = normalize(L);
            }
        }
    }
    // 发射diffuse光线
    if(!is_specular_ray)
    {
        vec3 basis_normal, dir_sphere;
    #if ENABLE_SH
        if(spec_bounce_index == 0 && global_ubo.flt_enable != 0)
        {
            dir_sphere = sample_cos_hemisphere_multi(0, 1, rng3, HEMISPHERE_UNIFORMISH);
            basis_normal = geo_normal;
        }
        else
    #endif
        {
            dir_sphere = sample_cos_hemisphere(rng3);
            basis_normal = normal;
        }
    
        mat3 basis = construct_ONB_frisvad(basis_normal);
        bounce_direction = normalize(basis * dir_sphere);
        // diffuse 和 specular 是互斥采样;这里除以选择 diffuse 的概率 (1 - specular_pdf) 做 1/pdf 补偿。
        bounce_throughput *= 1 / (1 - specular_pdf);
    
        vec3 L = bounce_direction.xyz;
        vec3 V = -view_direction.xyz;
        vec3 H = normalize(V + L);
        float VoH = max(0, dot(V, H));
    
        vec3 F = schlick_fresnel(primary_base_reflectivity, VoH, primary_specular_factor);
    
        bounce_throughput *= vec3(1.0) - F;
    }
    
    Ray bounce_ray;
    bounce_ray.origin = position;
    bounce_ray.direction = bounce_direction;
    bounce_ray.t_min = 0;
    bounce_ray.t_max = 10000;
    
    trace_geometry_ray(bounce_ray, true, bounce_cull_mask);
    
    // specular ray 再 trace 特效
    if(is_specular_ray)
    {
        if (found_intersection(ray_payload_geometry))
            bounce_ray.t_max = ray_payload_geometry.hit_distance;
    
        vec4 transparency = trace_effects_ray(bounce_ray, /* skip_procedural = */ true);
        bounce_contrib += transparency.rgb * transparency.a * bounce_throughput * (1.0 - direct_specular_weight);
    }
    Triangle triangle = get_hit_triangle(ray_payload_geometry);
    
    // hit surface properties
    vec3 bary         = get_hit_barycentric(ray_payload_geometry);
    vec2 tex_coord    = triangle.tex_coords * bary;
    uint bounce_material_id = triangle.material_id;
    position;
    normal;
    geo_normal;
    basecolor;
    ...
    
     vec3 emissive = sample_emissive_texture(triangle.material_id, bounce_minfo, tex_coord, vec2(0), vec2(0), is_specular_ray ? 2 : 3);
     emissive += get_emissive_shell(triangle.material_id, triangle.shell) * bounce_base_color;
     // other emissive login
     
     // 1 bounce gi
     vec3 bounce_diffuse, bounce_specular;
    get_direct_illumination(
        bounce_position, 
        bounce_geo_normal, 
        bounce_geo_normal, 
        bounce_cluster_idx, 
        bounce_material_id, 
        shadow_cull_mask, 
        bounce_direction, 
        bounce_base_color,
        vec3(0), // base_reflectivity
        0.0, // specular_factor
        1.0, // roughness
        MEDIUM_NONE, 
        false, // enable_caustics
        0.0, // direct_specular_weight
        global_ubo.pt_indirect_polygon_lights > 0,
        global_ubo.pt_indirect_dyn_lights > 0,
        is_gradient,
        1, // bounce
        bounce_diffuse,
        bounce_specular);
    
    bounce_contrib += bounce_throughput * bounce_diffuse;
    
    imageStore(IMG_PT_GEO_NORMAL2, ipos, uvec4(encode_normal(bounce_geo_normal)));
    imageStore(IMG_PT_SHADING_POSITION, ipos, vec4(bounce_position.xyz, uintBitsToFloat(triangle.material_id)));
    imageStore(IMG_PT_VIEW_DIRECTION2, ipos, vec4(bounce_direction, 0));
    imageStore(IMG_PT_BOUNCE_THROUGHPUT, ipos, vec4(bounce_throughput, is_specular_ray ? 1 : 0));
    
    // denoise without f0
    if (is_specular_ray)
        bounce_contrib = demodulate_specular(primary_base_reflectivity, bounce_contrib); 
    
    if(is_specular_ray)
    {
        bounce_contrib *= STORAGE_SCALE_SPEC;
    
        vec3 specular = unpackRGBE(imageLoad(IMG_PT_COLOR_SPEC, ipos).x);
        specular += bounce_contrib;
        imageStore(IMG_PT_COLOR_SPEC, ipos, uvec4(packRGBE(specular)));
    }
    else
    {
        bounce_contrib *= STORAGE_SCALE_LF;
    
        SH low_freq = load_SH(TEX_PT_COLOR_LF_SH, TEX_PT_COLOR_LF_COCG, ipos);
    
    #if ENABLE_SH
        if(global_ubo.flt_enable == 0)
            low_freq.shY.xyz += bounce_contrib;
        else
        {
            accumulate_SH(low_freq, irradiance_to_SH(bounce_contrib, bounce_direction), 1.0);
        }
    #else
        low_freq.shY.xyz += bounce_contrib;
    #endif
    
        STORE_SH(IMG_PT_COLOR_LF_SH, IMG_PT_COLOR_LF_COCG, ipos, low_freq);
    }
    
    

    总结

    Diffuse GI 比较低频,Q2RTX 的 LF 通道不是完整 RGB SH,而是压缩成:

    • PT_COLOR_LF_SH:一阶 SH,也就是 L0 + L1 的 4 个系数,只存 Y/luma 的方向性。
    • PT_COLOR_LF_COCG:存 Co/Cg 色度,不存方向性,近似为平均 tint。
    • RGB 会先转到 YCoCg:Co = R - B,t = B + Co * 0.5,Cg = G - t,Y = t + Cg * 0.5。
    • SH 写入形式是:result.CoCg = vec2(Co, Cg),result.shY = vec4(L11, L1_1, L10, L00) * Y。
    • 后续 ASVGF 滤波的是 SH + CoCg 这个 lighting representation;normal/depth 不被滤波,但会作为 edge-aware guide 参与滤波权重。最终 composite 前再用当前像素 normal 调用 project_SH_irradiance(filtered_lf, normal),把 LF SH 投影回 RGB irradiance。

    Checkerboard field 不是单纯半分辨率优化 Q2RTX 把最终屏幕的 checkerboard pixels de-interleave 到左右半屏:左半是 even field,右半是 odd field。这样每个 field 都是 dense image,便于 denoiser 做空间滤波;同时 reflection/refraction 可以分到不同 field,例如一半 field 走 reflection,一半 field 走 refraction。后续再用 checkerboard_interleave.comp 合回 flat image。

    Reflect/Refract pass 是 specular guide path,不只是视觉反射 reflect_refract.rgen 会把水、玻璃、chrome、screen 等特殊材质后面看到的 secondary surface 写回 G-buffer,包括 position、normal、base color、material、visbuffer、motion/depth 等。这样后续 direct/indirect lighting 和 denoiser 看到的是“镜子/玻璃后面的表面”,而不是只有 primary glass/mirror surface。

    Ray cone / texture LOD 是 secondary ray 质量关键 Primary pass 用相邻像素 primary ray 的夹角估计 half_cone_angle,secondary hit 时用 hit_distance * half_cone_angle 估算 footprint,再调用 compute_anisotropic_texture_gradients 计算纹理 LOD。否则反射/间接命中后的贴图采样会过锐、闪烁或 alias。

    Path throughput 是跨 pass 的能量账本 IMG_PT_THROUGHPUT / IMG_PT_BOUNCE_THROUGHPUT 记录 reflection/refraction、medium extinction、Fresnel sampling correction、BRDF sampling correction 等路径权重。后续 lighting pass 算到的 radiance 都要乘这个 throughput 才是对 camera pixel 的贡献。

    Specular demodulation 提高 denoiser 稳定性 SPEC channel 在写入前会用 demodulate_specular(base_reflectivity, specular) 除掉材质 F0/specular color,滤波后 composite 再用 modulate_specular 乘回去。这样 denoiser 处理的是更像“光照强度”的信号,而不是光照和金属颜色/贴图细节混在一起的结果。

    One-sample NEE + 重要性采样,比遍历所有灯更适合实时 get_direct_illumination 每个 shading point 通常只打一条 shadow ray:先在 polygon lights 中 importance sample 一个候选,再 uniform sample 一个 dynamic light 候选,然后按贡献估计在 polygon/dynamic 之间二选一,并用 1/pdf 补偿。这样一个样本统计上代表整个候选灯集合。

    Light shadow statistics 是低成本的下一帧采样反馈 Q2RTX 会统计每个 cluster / light / surface orientation 上 shadow ray 是 unshadowed 还是 shadowed,下一帧在 sample_polygonal_lights 里降低经常被挡住的灯的 mass,但保留下限避免完全不采。

    Geometry TLAS 和 Effects TLAS 分离 Q2RTX 把真实几何和粒子、爆炸、sprite、beam 等 effects 放到不同 TLAS。Primary/secondary geometry ray 先找 surface,再用 effects ray 累积透明特效。这样普通 visibility 不必总是遍历 effects,也能给 effects 使用不同 hit shader / any-hit 逻辑。

    Half-res GI profile 不是简单降分辨率,而是降低路径复杂度 pt_num_bounce_rays == 0.5 时,indirect pass 只在隔行像素上跑,并且默认不读取真实 metallic/roughness,等价于低质量 diffuse GI profile,避免半分辨率 specular 带来闪烁和错误重建。

    Ray Query / RT Pipeline 双路径值得保留抽象边界 Q2RTX 同一套 rgen 逻辑会编译成 .pipeline.spv 和 .query.spv。RT pipeline 路径使用 SBT/hit shaders;Ray Query 路径则在 compute shader 内手动执行查询和 hit 逻辑。上层 pass 调度基本通过同一组 pipeline index 抽象。

    Read More
  • UE CPU Cooler

    …

    UE CPU Cooler

    Unreal Engine’s frame limiter avoids sleeping all the way to the target frame time.

    It first sleeps for most of the wait:

    SleepNoStats(WaitTime - 0.002f);
    

    This leaves about 2 ms of slack, because OS sleep calls can oversleep due to scheduler granularity and wake-up latency.

    Then UE waits for the exact frame boundary with:

    while (FPlatformTime::Seconds() < WaitEndTime)
    {
        SleepNoStats(0);
    }
    

    On generic platforms, SleepNoStats(0) becomes sched_yield(). So the thread does not request a timed sleep; it simply gives up its current time slice and checks again.

    This improves frame pacing because UE is less likely to wake up after the target time. The cost is that the final ~2 ms keeps the CPU relatively active, causing extra scheduler work and higher power usage.

    In short, UE trades CPU activity for more accurate frame pacing.

    If your platform is power-sensitive, you can replace the final sched_yield loop with SleepNoStats(0.002f) to avoid repeatedly yielding, at the cost of less precise timing.

    Read More
  • DistanceField Generation of Unreal

    …

    DistanceField Generation of Unreal

    // Runtime\Engine\Private\StaticMesh.cpp
    void UStaticMesh::Serialize(FArchive& Ar)
    

    then

    // Runtime\Engine\Private\StaticMesh.cpp
    FStaticMeshRenderData::Cache
    {
        ...
    
    	static const auto CVar = IConsoleManager::Get().FindTConsoleVariableDataInt(TEXT("r.GenerateMeshDistanceFields"));
    
    	if (CVar->GetValueOnAnyThread(true) != 0 || Owner->bGenerateMeshDistanceField)
    	{
    		if (LODResources.IsValidIndex(0))
    		{
    			if (!LODResources[0].DistanceFieldData)
    			{
    				LODResources[0].DistanceFieldData = new FDistanceFieldVolumeData();
    				LODResources[0].DistanceFieldData->AssetName = Owner->GetFName();
    			}
    
    			// Only generate distance fields and card representations for the base render data, not platform render data.
    			if (this == Owner->GetRenderData())
    			{
    				const FMeshBuildSettings& BuildSettings = Owner->GetSourceModel(0).BuildSettings;
    				UStaticMesh* MeshToGenerateFrom = BuildSettings.DistanceFieldReplacementMesh ? ToRawPtr(BuildSettings.DistanceFieldReplacementMesh) : Owner;
    
    				if (BuildSettings.DistanceFieldReplacementMesh)
    				{
    					// Make sure dependency is postloaded
    					BuildSettings.DistanceFieldReplacementMesh->ConditionalPostLoad();
    				}
    
    				LODResources[0].DistanceFieldData->CacheDerivedData(Owner, MeshToGenerateFrom, BuildSettings.DistanceFieldResolutionScale, BuildSettings.bGenerateDistanceFieldAsIfTwoSided);
    			}
                ...
    		}
            ...
    	}
        ...
    }
    

    only build for lod0

    then

    // Runtime\Engine\Private\DistanceFieldAtlas.cpp
    void FDistanceFieldAsyncQueue::Build(FAsyncDistanceFieldTask* Task, FQueuedThreadPool& BuildThreadPool)
    {
        ...
        GenerateSignedDistanceFieldVolumeData()
        ...
    }
    

    then

    https://github.com/RenderKit/embree

    it generates sparse distance field data with mips

    // Developer\MeshUtilities\Private\MeshDistanceFieldUtilities.cpp
    void FMeshUtilities::GenerateSignedDistanceFieldVolumeData()
    {
    ...
    SetupEmbreeScene()
    AddMeshDataToEmbreeScene()
    BuildSignedDistanceField()
    DeleteEmbreeScene()
    ...
    }
    

    then

    static void BuildSignedDistanceField()
    {
    	...
    	for (int32 MipIndex = 0; MipIndex < DistanceField::NumMips; MipIndex++)
    	{
    		...
    		TArray<FSparseMeshDistanceFieldAsyncTask> AsyncTasks;
    		AsyncTasks.Reserve(IndirectionDimensions.X * IndirectionDimensions.Y * IndirectionDimensions.Z);
    		for (int32 ZIndex = 0; ZIndex < IndirectionDimensions.Z; ZIndex++)
    		{
    			for (int32 YIndex = 0; YIndex < IndirectionDimensions.Y; YIndex++)
    			{
    				for (int32 XIndex = 0; XIndex < IndirectionDimensions.X; XIndex++)
    				{
    					AsyncTasks.Emplace(
    						EmbreeScene,
    						&SampleDirections,
    						LocalSpaceTraceDistance,
    						DistanceFieldVolumeBounds,
    						LocalToVolumeScale,
    						DistanceFieldToVolumeScaleBias,
    						FInt32Vector(XIndex, YIndex, ZIndex),
    						IndirectionDimensions,
    						bUsePointQuery);
    				}
    			}
    		}
    		...
    	}
    }
    

    find closets point of each voxel

    void FSparseMeshDistanceFieldAsyncTask::DoWork()
    {
    	...
    	for (int32 ZIndex = 0; ZIndex < DistanceField::BrickSize; ZIndex++)
    	{
    		for (int32 YIndex = 0; YIndex < DistanceField::BrickSize; YIndex++)
    		{
    			for (int32 XIndex = 0; XIndex < DistanceField::BrickSize; XIndex++)
    			{
    				...
    				if (bUsePointQuery)
    				{
    					RTCPointQuery PointQuery;
    					PointQuery.x = VoxelPosition.X;
    					PointQuery.y = VoxelPosition.Y;
    					PointQuery.z = VoxelPosition.Z;
    					PointQuery.time = 0;
    					PointQuery.radius = LocalSpaceTraceDistance;
    
    					FEmbreePointQueryContext QueryContext;
    					rtcInitPointQueryContext(&QueryContext);
    					QueryContext.Scene = &EmbreeScene;
    					float ClosestUnsignedDistanceSq = (LocalSpaceTraceDistance * 2.0f) * (LocalSpaceTraceDistance * 2.0f);
    					rtcPointQuery(EmbreeScene.Scene, &PointQuery, &QueryContext, EmbreePointQueryFunction, &ClosestUnsignedDistanceSq);
    
    					const float ClosestDistance = FMath::Sqrt(ClosestUnsignedDistanceSq);
    					bTraceRays = ClosestDistance <= LocalSpaceTraceDistance;
    					MinLocalSpaceDistance = FMath::Min(MinLocalSpaceDistance, ClosestDistance);
    				}
    				...
    			}
    		}
    	}
    }
    
    
    
    bool EmbreePointQueryFunction(RTCPointQueryFunctionArguments* args)
    {
    	const FEmbreePointQueryContext* Context = (const FEmbreePointQueryContext*)args->context;
    
    	check(args->userPtr);
    	float& ClosestDistanceSq = *(float*)(args->userPtr);
    
    	int32 GeometryIndex = args->geomID;
    
    	if (Context->instID[0] != RTC_INVALID_GEOMETRY_ID)
    	{
    		// when testing against a geometry instance use instID to index into Scene->Geometries
    		GeometryIndex = Context->instID[0];
    	}
    
    	const FEmbreeGeometryAsset* GeometryAsset = Context->Scene->Geometries[GeometryIndex].Asset;
    	const int32 NumTriangles = GeometryAsset->NumTriangles;
    
    	const int32 TriangleIndex = args->primID;
    	check(TriangleIndex < NumTriangles);
    
    	const FVector3f* VertexBuffer = (const FVector3f*)GeometryAsset->VertexArray.GetData();
    	const uint32* IndexBuffer = (const uint32*)GeometryAsset->IndexArray.GetData();
    
    	const uint32 I0 = IndexBuffer[TriangleIndex * 3 + 0];
    	const uint32 I1 = IndexBuffer[TriangleIndex * 3 + 1];
    	const uint32 I2 = IndexBuffer[TriangleIndex * 3 + 2];
    
    	FVector3f V0 = VertexBuffer[I0];
    	FVector3f V1 = VertexBuffer[I1];
    	FVector3f V2 = VertexBuffer[I2];
    
    	if (Context->instID[0] != RTC_INVALID_GEOMETRY_ID)
    	{
    		// when testing against a geometry instance need to transform vertices to world space
    		FMatrix44f* InstToWorld = (FMatrix44f*)Context->inst2world[0];
    
    		V0 = InstToWorld->TransformPosition(V0);
    		V1 = InstToWorld->TransformPosition(V1);
    		V2 = InstToWorld->TransformPosition(V2);
    	}
    
    	const FVector3f QueryPosition(args->query->x, args->query->y, args->query->z);
    
    	const FVector3f ClosestPoint = (FVector3f)FMath::ClosestPointOnTriangleToPoint((FVector)QueryPosition, (FVector)V0, (FVector)V1, (FVector)V2);
    	const float QueryDistanceSq = (ClosestPoint - QueryPosition).SizeSquared();
    
    	if (QueryDistanceSq < ClosestDistanceSq)
    	{
    		ClosestDistanceSq = QueryDistanceSq;
    
    		bool bShrinkQuery = true;
    
    		if (bShrinkQuery)
    		{
    			args->query->radius = FMath::Sqrt(ClosestDistanceSq);
    			// Return true to indicate that the query radius has shrunk
    			return true;
    		}
    	}
    
    	// Return false to indicate that the query radius hasn't changed
    	return false;
    }
    

    Read More
  • UE4 LightMap Directionality

    …

    UE4 LightMap Directionality

    alt text

    From UE4 Lightmap Format Analysis, Unreal encodes a direction channel in the lower half of the lightmap to interact with the pixel normal.

    With directionality:

    alt text

    Without directionality, Unreal uses 0.6 as an empirical value:

    alt text

    Some mobile games discard the lower half to reduce lightmap size, resulting in very flat lighting with a normal map. If your game uses a forward pipeline, you can utilize geometry normal to interact with world normal to achieve better results with the same lightmap size.

    The code is as follows:

    // old directionality
    
    // float4 SH = Lightmap1 * GetLightmapData(LightmapDataIndex).LightMapScale[1] + GetLightmapData(LightmapDataIndex).LightMapAdd[1]; // 1 vmad
    
    // half Directionality = max( 0.0, dot( SH, float4(WorldNormal.yzx, 1) ) ); // 1 dot, 1 smax
    
    // faked directionality
    
    half Directionality = 0.6 * max( 0.0, dot( normalize(VertexNormal), normalize(WorldNormal) ) );
    

    Result:

    alt text

    Our lightmap now has more detail. Even though we discard the directionality, we still use the empirical value of 0.6 to adjust luminance, resulting in slight differences compared to the original.

    Read More
  • (WIP)Unreal Diffuse Indirect Light Explain

    …

    LightMap

    SH

    ILC

    VLM

    IRRADIANCE VOLUME

    Read More
  • penantumbrathegame

    generate by gemini

    Read More
  • Analysis Android Game Crash

    …

    Read More
  • Use Scene to Monitor Android Power Consumption

    …

    Read More
  • (WIP) Compute Shader cheatsheet

    …

    Compute Shader Basics

    GPU are designed to execute paralled works, we can divide paralled graphics or non-graphics works into group to utilize gpu.

    Sync

    Shared Memory

    WorkGroupSize

    CS vs PS

    Mobile

    Read More
  • Shadow Lod/Proxy Trap

    some optimizations techs are not silver bullet

    Read More
  • Graphics Quality Configuration for Android

    …

    Read More
  • Debug Metal GPU Crash

    open Edit Scheme and open Shader Validation

    Read More
  • Compare Shadow Map Atlas and Shadow Map Texture Array

    some notes on shadowmap rt format

    Read More
  • Explain XXXView in DirectX

    what is RenderTargetView/DepthStencilView/ShaderResourceView means

    Read More
  • Shadow Compression

    some method to compress shadow map

    Read More
  • VRS The Invisble Scenes

    Tier 2 VRS provides a way to specify a shading rate texture for rasterization. Content rendered behind the UI can use a lower shading rate to reduce pixel shader workload.

    Read More
  • Reduce unreal shader permutation

    some ways to reduce ue4 shader permutation count

    Read More
  • Shader Optimization Cheatsheet

    my shader programming cheatsheet

    Read More
  • How to correctly add engine feature

    When developing a game engine, adding new features requires careful consideration of how they are enabled, configured, and controlled. This guide outlines several common methods for managing engine features, helping you choose the most appropriate approach for your use case.

    Read More