ngp代码阅读

Implement

具体的实现细节

Performance vs quality

hash table 的大小\(T\) 过大导致GPU内存不足。

经过实验取得 \(F=2,L=16\)

Implicit hash collision resolution

1. 一对点不太可能同时在每一层都发生碰撞,而且 \(N_{min}\) 保证不发生冲突
2. 一对点对梯度的贡献是不同的,表面的点的影响更大
3. 分辨率的分布是几何的, 分辨率越大的较少的选取

Online adaptivity

如果输入 \(x\) 的分布集中在一个小的区域,那么精细网格会更少的发生哈希冲突,能学习的更精确.

Neural Radiance Caching

Model Architecture

由两个 MLP 并联

1. density MLP: maps encoded position \(enc(\bold{x},\theta)\) to 16 outputs values ， \(\theta\) is the encoding parameters

2. color MLP:

   Input

   the 16 output values of the density MLP, and the view direction projected onto the first 16 coefficients of the spherical harmonics basis. This is a natural frequency encoding over unit vectors

   output:

   RGB color triplet.

   sRGB: sigmoid activation

   HDR: exponential activation

   1-hidden-layer density MLP and a 2-hidden-layer color MLP, both 64 neurons wide.

   Accelerated ray marching

   we concentrate samples near surfaces by maintaining an occupancy grid that coarsely marks empty vs. nonempty space.

   Ray Marching Step Size and Stopping

   1. synthetic NeRF scenes: step size $t= /1024 $ in the unit cube \([0,1]^3\)
   2. Others: \(t\) exponential growth, which means that the computation cost grows only logarithmically in scene diameter, with no perceivable loss of quality.
   3. Stop ray marching and set the remaining contribution to zero as soon as the transmittance of the ray drops below a threshold

   Occupancy Grids

   During ray marching, whenever a sample is to be placed according to the step size from the previous section, the sample is skipped if its grid cell’s bit is low. Which one of the 𝐾 grids is queried is determined by both the sample position x and the step size Δ𝑡: among the grids covering x, the finest one with cell side length larger than Δ𝑡 is queried.

   update the occupancy grids. a second set of grids that store full-precision floating density. update every 16 training steps.

   Number of Rays Versus Batch size

   batch size = Samples per ray * Rays per batch

   Training from a larger number of rays, i.e. incorporating more viewpoint variation into the batch. Include as many rays as possible in batches of fixed size rather that building variable-size batches from a fixed ray count

Input

1. load config/base.json: 包括 loss, optimizer, encoding 的超参等 envmap?

2. train

3. training_prep_nerf: 每过16次迭代，更新occupancy grids.

   if (m_training_step < 256) {
   		update_density_grid_nerf(alpha, NERF_GRIDSIZE()*NERF_GRIDSIZE()*NERF_GRIDSIZE()*n_cascades, 0, stream);
   	} else {
   		update_density_grid_nerf(alpha, NERF_GRIDSIZE()*NERF_GRIDSIZE()*NERF_GRIDSIZE()/4*n_cascades, NERF_GRIDSIZE()*NERF_GRIDSIZE()*NERF_GRIDSIZE()/4*n_cascades, stream);
   }

4. train_nerf_step(uint32_t target_batch_size, Testbed::NerfCounters& counters, cudaStream_t stream):

5. generate_training_samples_nerf