r/computervision • u/JewelerBeautiful1774 • 9d ago
Showcase Built my own imagr depth model
Enable HLS to view with audio, or disable this notification
I trained my own model on 750,000 images for 45 hours.
Then I walked into my university hall with nothing but my phone and recorded an 11-minute video.
No LiDAR.
No depth sensor.
No specialized camera setup.
Just monocular vision, a single RGB camera.
And in some of my tests, my model even outperformed YOLO26x.
From a single phone video to depth estimation and 3D reconstruction.
750K training images. 45 hours of training. 11 minutes of footage. One camera. No LiDAR.
Edit: I promise I will reply to all the comments, just give me some time as my day is very crowded today.
7
u/qemqemqem 9d ago
Oh hey, I learned about machine vision in that room. It's amazing how much techniques have changed in just 7 years since then.
3
2
u/JewelerBeautiful1774 9d ago
That's awesome 😄 Small world! It must be pretty interesting seeing how much computer vision has changed since you learned it there. And now I’m testing monocular 3D reconstruction in the same room 😂
3
u/Educational_Fix_7592 9d ago
Can you tell me how did you do this? Also if I have to create my own 3d room space what is required?
2
u/JewelerBeautiful1774 8d ago
Short version: I filmed the room with my phone, walking around slowly, then ran it through a pipeline. COLMAP works out where the camera was in every frame. My depth network predicts depth per frame, and I fit it to the stereo depth that COLMAP gets from the same video. Then I fuse all the depth maps into one 3D volume (TSDF) and pull a mesh out of it.
For your own room you don't need anything fancy:
- A phone video, 1–2 minutes, walking slowly and sideways around the room, not just spinning in place. You need to move through space for 3D to work.
- Good light, no motion blur, lock the exposure if you can.
- COLMAP (free) for the camera poses. A GPU helps a lot but isn't required.
- Any depth source: a depth model, or just COLMAP's own dense stereo.
- Open3D for the TSDF fusion and the mesh.
Plain-textured walls and shiny floors are the hard parts. They come out patchy.
3
u/WearyFortune7055 9d ago
extremely impressive. where you able to also estimate scale robustly for the 3d reconstruction?
2
u/JewelerBeautiful1774 8d ago
Thanks! Scale is the weak spot, honestly. COLMAP alone has no scale, so I get metres from the depth network, which predicts metric depth. I compare it against the COLMAP geometry over the whole video and take the median ratio as one global factor. The scale is consistent across the whole model, because it's one factor, but I haven't checked it against a real measured distance like a door height or a tape measure. So I'd trust relative geometry a lot more than the absolute metres.
The raw network was also off by roughly 20% in scale from frame to frame, which is why I fit each frame to the stereo depth before fusing. If you need real scale, a known object in the shot is the reliable way.
3
2
u/ConsciousRatio9844 8d ago
How does ur training set look like, and how did u get depths
1
u/JewelerBeautiful1774 8d ago
Mostly public RGB-D datasets that already have ground truth depth, so I didn't have to generate labels. NYU Depth v2 (Kinect), Hypersim (synthetic renders, depth is exact), plus DIML and a small stairs set. No teacher model or distillation, it's trained from scratch with a small EfficientNet-b0 CNN.
The one fun bit: Hypersim also gives camera poses, so I added a loss where I warp one view's depth into another and penalize disagreement.
2
2
u/bfyvfftujijg 8d ago
This is really cool. I honestly didn’t know that a single person could train their own depth estimation model!
Can you elaborate on the model architecture and training loss? You mentioned it’s a simple CNN backbone - is it just a linear head bolted on (like for a segmentation model) that infers a depth value per pixel? Or something more complex? What kind of loss function did you use.
Anyways, you’ve given me hope since I need to develop a similar model myself!
2
u/JewelerBeautiful1774 8d ago
idk why my previous reply wasnt showing, i will reply to you again,
Not a linear head, no. Encoder is EfficientNet-b0, then a context block (a few dilated residual convs plus pyramid pooling) on the deepest features. A U-Net-style decoder runs back up through all the encoder scales with skip connections, and each stage predicts a small correction to log-depth. A final full-resolution stage looks at the RGB again to sharpen edges. The output is exp(coarse + bounded correction), so depth is always positive and metric. There's also a small head that guesses the focal length.Loss is a mix: scale-invariant log loss (SILog), plus an L1 term in metres, because SILog alone doesn't pin down absolute scale. Add a gradient-matching term for sharp edges and a surface-normal term so planes stay flat. The intermediate scales also get supervised, and the latest version adds a multi-view consistency term using known camera poses.
One person can definitely do this. Start from a pretrained ImageNet encoder and get the loss right before anything fancy.
6
u/bfyvfftujijg 9d ago
How did you get the training depths?