You can check it out here: SuperSplat
Here’s what it looks like as millions of colored points:
How and why? I’m a 3D expert and ML/AI coder, I wanted to create a FatMap replacement and first ended up making a unique system that took insta360 video of a ride through a mtb course and turned it into a massive 3D env, here’s a sample (best accessed on desktop, WASD to move):
However, I couldn’t figure out a good business plan, it was hugely computationally expensive to do, and hard to host.
So, I thought smaller, this is unrelated to MTB (as being aero matters only a small amount) and made a startup that turns video of a cyclist into a CFD test:
However, I love mountain biking and a trail builder reached out, he’s building a trail for charity in AUS, and I said I could probably make a cool 3D model from a drone flyover, and I did! (first picture/viewer)
I thought you guys might find this interesting so I might as well post about it. If any other MTB trail maintainers want 3D reconstructions, let me know, I do this stuff for fun.
2 Likes
Very cool. Have you tested the results of the AI wind tunnel against a real wind tunnel?
Not saying/implying results should be the same, cost and access difference is enough to make it a no brainer but am curious how those compare.
That’s the first thing we’re working towards, the models are on par with LIDAR (using handheld laser scanners) models and the CFD software is OpenFoam, and industry standard that has been tested at every level.
The process itself takes over an hour of computation with a very powerful workstation and some manual processes here and there (although I’m automating it more and more using custom AI models and other tools).
So, TL:DR, the models get converged/stable results pretty quickly (if the models had bad mesh/polygons, that would be a problematic portion), they’re scaled to exact rim/tire replicas (I built a custom 3D wheel generation application for this), and the CFD software is quite proven.
Once we get enough orders, we’ll be off to a wind tunnel to find the difference, but we expect single digit % differences, as that’s typical for LIDAR + CFD, and this system, again, isn’t terrible different in end result output.
For what reason would you encode the drone flyover or the ride through footage as a bunch of points ? Is it not more useful as raw footage ?
It’s a simple format of the model in 3D, useful in land analysis, hydrology, and many other areas, pretty common with expensive LIDAR equipment, in this case I’m using Photogrametry techniques and AI.
Also, the drone footage is fine, but dull, I found this entertaining to build and fun to zoom around in.
Yeah I understand my background is in Lidar for forestry,
Point I was trying to make was unless you have a system wide bandwidth or size restriction why use these compressed formats in the first place. These high resolution videos from the drone are already great so what problem do you solve by compressing them ?
It’s not compression, in fact these files are a bit larger than the drone footage.
It’s simply a cool 3D interactive experience, a drone shot doesn’t let you “fly around in a 3D scene as you please”, you’re locked only into where the drone flew.
Additionally, since it’s a 3D scene I can add things to it, like more single-track or wood features.
The point is, this can be used by a trailbuilder in order to help plan, measure, and showcase to clients. A video is okay, but a cool interactive view? So fun, I could make a video game out of it.
In this case, I was hoping the drone would fly a bit closer, but nonetheless it was a fun first attempt.
1 Like
Okay cool that is pretty useful. AI has hit every industry like a great flood including GIS wouldn’t suprise me if ARC are already working or have developed something like this to extend lidar into a workable environment.
This pipeline mimics LIDAR to an extent, but only relies on a camera.
I can already take the output and do a lot with it, make into textured mesh (I built custom software for this), keep it as points, all sorts of stuff.
The AI in this one is NeRF, Neurl Radience Fields, it’s some pretty cool tech, very new.
That was pretty cool. For the drone shot I might suggest a cutoff and a fake skybox just to make the edges of the scene a bit less intrusive. I found when I rotated ‘outside’ the edge of the mapped area I ended up looking through a bunch of noise that blocked my vision.
From monetization, I could see you designing and modeling various trails in the users real environment that serve as various proposals before a single shovel gets touched. Helps the developer know which trees to remove and what not and gather feedback. Maybe helpful for building in remote areas where access is half the battle or areas that are typically covered by snow during the planning season of the year.
I was oddly impressed by the accuracy of the cables on the power poles in the drone version. Did you do any manual modeling or correction or is this straight out of the camera?
Straight from the camera, that’s a Gaussian splat, it’s a new way to represent geo, pretty computer intensive, as you have to render overlapping blurred semi transparent collections of strands of points.
The output can be super detailed, and that’s a great suggestion, quite doable too, concerning the cropping and skybox addition.
Also, this AI can get pretty crazy, there are was to use it so you can type “add tree here” and click and it will add one that fits in the scene, or removal of entities with text prompts and selections. Who knows what’s coming with this tech?
For a downhill course you use drone + rider helmet cam to help model the course and share for rider preview. Let them ‘drive’ around the course from home before they even fly in and get a few days to practice. Maybe theres a version of this that turns into some kind of 3d printed trophy of a specific course? i know i can take my strava GPX files and map a 3d object from the gps and elevation profile. 3d printing in a shadow box makes for some cool art but nothing really that useful.
Well if you want to market it to clients in Canada I have a big book full of survey and other for profit forest service companies, mills, etc. You can PM me .
Yeah, that’s quite possible, and would look super cool, intra camera solving is tough though and requires a bunch of tuning.
Also, yes, I have a ton of ways to make cool 3D printed stuff with this tech.
I’m sure you’re familiar with this demo of unreal engine. Is there a near term reality where you can map enough ‘trails’ to help unreal engine ‘add bike trail here’ with tunes for intensity and technicality?
Then all you need is to go 3d scan a downhill section, load it up, let AI suggest a route and tweak based on what you’d like to see.
Granted this is having AI do the fun stuff and humans still do the boring stuff (digging). I’d love to see it opposite where humans get to design the track and then a little roomba with a shovel goes out and builds it.
I may, as there’re a lot of possibilities with this software, I’m looking for some good directions, other than just the CFD startup I have for it.
Lots of yes’s and no’s to this, as you’d likely expect.
Small long corridor shots like what you’d need in the forest tend to struggle to maintain proper overall alignment, that’s why drone shots work so well.
Also, AI routing has it’s own challenges… I also made these sites:
https://sherpa-map.com and GPX Route Speed Estimator for Cyclists: Multi-Surface, Weather, and Nutrition Strategy
The first is a routing engine not being taken over by some large investment firm (that’s a poke at Komoot), the second is a mixed surface cycling specific physics engine.
I’d probably steal some code from them if I wanted to get into the video game area, AI can do some things, but routing in particular is better modeled with physics than guessed at IMO, and the physics sims run super fast anyway, AI stuff can be quite demanding.
1 Like
