UnderwareDESIGN

PlayBASIC => 3D Development => Topic started by: stevmjon on May 03, 2022, 10:25:50 PM

Title: Build a basic 3D Render Engine
Post by: stevmjon on May 03, 2022, 10:25:50 PM
howdy

i wanted to put this up in stages so people can see the progress, and also to see the code when it is small (makes it easier to read).

this is a render engine i built myself. so i am learning different calculations, and understanding how they work, before i use them.
so far i am having fun doing this. and as usual, when it comes to the mathematics, there is choices to make. so i went for the geometry calculations method. another is quadratic method if you choose.

the progress :
> model 'wireframe' geometry    ;    see a wireframe model of the objects in the scene, but these are not used in the render. the spheres are complete circles with no polygons.
> render is diffuse surface    ;    looks at the surface normals and compres the light angle, to give a shade
> render specular highlight    ;    calculate the light reflection on the object surface, and compare to camera view to give a specular highlight (this is scaled down to suit a nice size)

so the main scene wireframe models (camera view) is drawn directly to the screen, using lockbuffer. it seems to work fast. needs checking on other computers.
then the render screen is drawn to an FX image using fastdot, then when complete is drawn to the screen. i have 2 methods of seeing the display.

if this is working correctly, i will then move onto shadow casting, then surface reflections.
suprisingly, the theory and mathmatics is not difficult so far. i guess it gets tricky when adding realistic surfaces...

anyway, i will even add my own editor, so you can select objects and move them around on screen, as well as add lights and position them too.
and kev, i kept the same menu colors because you said you liked them in the other raytracer i ported.

   enjoy, stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: kevin on May 03, 2022, 11:32:09 PM

   Amazing..  just amazing..   I'm in the middle of building BB2PB thing but will have a closer sticky nose later !   
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on May 09, 2022, 01:53:50 AM
try this one out kev.

i worked out how to speed up the 'render each line' render method.
i am still working on shadow casting. i have the spheres working great, just getting the ground plane to work (as it is unlimited size across the ground)

   stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: kevin on May 09, 2022, 06:31:57 AM
 There's really nothing wrong with the speed, it's actually fine.  In particular when you think it's running on the PlayBASIC runtime..

There's some stuff you could use built in math functions for stuff like getting the length of a vector you can get use getDistance3D(0,0,0,VectorX#,VectorY#,VectorZ#)    

I can sneak in some things if you like ? - I know a guy :)


Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on May 10, 2022, 06:37:10 PM
ah cool kev, i used sqrt() a lot and this costs time. thanks for the getdistance3D() tip, that would calc faster. i will use this straight away.
i should really look at built in maths. i have a habit of manually calculating most things.

i am drawing the main scene, wireframe models, to the screen. is this ok as its mainly just lines, and gouraud boxes for the menus.
i did draw the render itself to an FX surface using fastdot.

any other tips to improve this is appreciated.

  stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: kevin on May 10, 2022, 09:51:30 PM
   Steve,

      Sqrt() functions actually map directly to fpu code.  Ideally we're meant to call such functions (from a machine code level) and then have our code wander off and do something else; while it completes..  Thus interleaving floating point and integer operations side by side.   But this only works if we're not dependent upon the answer; so that doesn't help most of the time.   Which is useless to us, but an interesting factoid :)

     
      In most normal programs the operation cost of a refresh is considerably lower than what your doing;  and for every object you add to the scene we're stacking work load upon work load.   So even small operations will stack up.    

       An expression like;

      dot_length# = Sqrt(rayx#*rayx# + rayy#*rayy# + rayz#*rayz#)

      isn't a big deal at all today; even if called 1000's of times.  But if we're calling it millions of times, then anything we can do to shortcut  it the better.   Just swapping that to a GetDistance3D function to compute the length,  removes the 3 mults and pair of additions from the PB runtime..  even though there small operations compared to the sqrt,  they still fall through the PB runtime every time.  It soon adds up.


      previews: (Drawing LInes)

      When rendering to the 'screen'  or any video image as long as lock /  unlock the buffers at the start and end of the process, rather than for each individual job.  We're generally going to be ok..  

       There are things to consider; such as if the drawing is going to take a long time,  like if you drawing millions of line fragments; then you'd be best to structure the loop(s) so they're doing drawing batches of objects ..  refresh the window and continue..  So there's no dead time for the user and windows doesn't think you app is dead.

     What you could do; which sounds counter intuitive.  is use Shapes to draw the batches of connected line segments.   So for each ring you could dump the screen coordinates of each vert to the shape; where the edge list is already defined to close the list.   Then draw the shape and repeat the process.  

[pbcode]



   Count=1000

   RingShape = NewShape(Count , Count)
   
   //  Set the edge links for this shape.
   For lp =0 to Count-2
      SetShapeEdge  RingShape,lp, lp,lp+1
   next
   // link the last one back to the first   
   SetShapeEdge RingShape,lp,lp,0



   StartTime=Timer()
   do
      
      cls

      mx#=mousex()   
      my#=mousey()   
      for lp =0 to Count-1
            ; randomly init the vertex as polar coordinates         

            Angle#=(360.0/Count)*lp
            Radius# =80+ Cos(WobbleAngle#+(Angle#*Scaler#))*50
            setshapevertex RingShape,lp,Cosradius(Angle#,Radius#)   ,Sinradius(Angle#,Radius#)      
      next

      WobbleAngle#-=2.25
      
      lockbuffer
      drawshape RingShape,mx#,my#,1
      unlockbuffer
      

      Scaler#=wrapvalue(scaler#+0.01,1,50)
      ;(Timer()-StartTime)/100000.0
      
      sync      
   loop


[/pbcode]




      Data Statement support evaluation of constant expressions...  


      so this;

      Data 50,100,200,0,0  ;  red,green,blue,specular,reflection

      Could just be

      Data Rgb(50,100,200) ,0,0  ;  red,green,blue,specular,reflection


[pbcode]


      ThisRgb=ReadData()
      print rgbR(ThisRGB)
      print rgbG(ThisRGB)
      print rgbB(ThisRGB)

      print Readdata()
      
      print readdata()
      
      
      sync
      waitkey

      Data Rgb(50,100,200),1111,2222  ;  red,green,blue,specular,reflection


[/pbcode]
       
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on May 11, 2022, 07:51:38 PM
thanks for the tips kev. i didn't know you could do so much with the SetShapeEdge command. handy. i thought it was straight lines from only point to point, lol.

so Sqrt() uses the gpu? i you pre-calculated the maths then used a variable inside this command will that make it faster?
what was interesting with my computer, i swapped out every Sqrt() with GetDistance3D() and it wasn't any faster... could be my computer though.

good tip with the ReadData() statements. didn't know you could do that.

just a question with drawing lines/shapes to the screen, is there any advantage between drawing to video surface or FX surface?

  thanks stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: kevin on May 11, 2022, 08:42:37 PM

Quotei didn't know you could do so much with the SetShapeEdge command. handy. i thought it was straight lines from only point to point, lol.

  The edges are straight,  but as you know;  if you make the steps between vertex smallest enough you can better approximate curves.

Quote
so Sqrt() uses the gpu? i you pre-calculated the maths then used a variable inside this command will that make it faster?
what was interesting with my computer, i swapped out every Sqrt() with GetDistance3D() and it wasn't any faster... could be my computer though.

    nahh...  Sqrt and ALL floating point maths execute on the CPU  but within it's FPU (Floating point unit)  which is just a small subset of math instructions that work purely with floats.   Normal CPU instructions don't;  they're integer only.   You actually can't mix and match them.   One of the nice things about the FPU instructions is they include various functions built right into the chip.   Such as Sqrt / Cos/ Sin and bunch of others.

   

Quotejust a question with drawing lines/shapes to the screen, is there any advantage between drawing to video surface or FX surface?

     It all depends on much writing your doing to the target surface.   If the surface is in video memory then the speed at which the cpu can write to video memory is the bottleneck.   In modern systems this is less of an issue,  but in older systems this can have a significant impact.   

     Generally I tend to use render to an FX image and never use any video surfaces.   So if i'm drawing lots of stuff to the 'screen' then we're bound by the cpu and its bus to memory.    Rather than the Cpu's bus to video memory with is external and variable. 

Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on May 12, 2022, 07:57:24 PM
ah, lets see if i have this correct:
> if i draw lines, shapes, dots etc to the screen 'back buffer', that is cpu.
> if i draw a video image to the screen 'back buffer' that is gpu.

so graphics commands (excluding images) are cpu driven?
also, if you used openGL, are graphics commands gpu driven instead?

sorry for so many questions, i am getting more interested in how things work.
Title: Re: Build a basic 3D Render Engine
Post by: kevin on May 12, 2022, 09:00:23 PM
Quote
so graphics commands (excluding images) are cpu driven?

 Sort of.   The target surface type (Video/ FX) and type of rendering required do.    So rendering a Video Image or drawing a Rectangle/Box onto a Video surface.  Than those are done by the GPU.    It's limited to Images and rectangle fills as that's all direct draw supports.  

 This is fine if you want opaque rendering of only those two primitives, anything else and you have to do it yourself.

  So in terms of FX images & primitives Dots/Lines/Circles/ Shapes etc etc they're all CPU drawn.     In general there's 8 or more versions of every draw mode in the back end of the libraries.     Different draw modes handle different pixel formats (15bit/16bit/24bit/32bit) with blend mode combinations.  


Quote
also, if you used openGL, are graphics commands gpu driven instead?

  Sort of..   Older editions  OpenGL / Direct3D don't include built in support or all render types or primitives.    Such apis normally give you DOT/LINE and POlygon rendering that's it.    
 
  For Blend modes old versions only really include Alpha Addition / Alpha Multiplication (We call it tint in PB) and Alpha Blending.    The upsides is that data stored on GPU can be drawn / processed by the GPU.    So you get bi-linear texture filtering or free..  The down side is that old GPU's have a tiny amount of memory.    

   - [plink] G2D - 2D OpenGL library for PlayBASIC V1.64P  (https://www.underwaredesign.com/forums/index.php?topic=4210.0)[/plink]
   - PBFX 2D Core (https://www.underwaredesign.com/forums/index.php?topic=4580.0)
    - PlayBasicFX (PBFX) Screen Shot Updates from 2007 (https://www.underwaredesign.com/forums/index.php?topic=1807.0)

   - Search for old conversation about building a PlayBASICFX / PBFX back end..   Such fun :)

   
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on May 24, 2022, 10:19:33 PM
another update.
this version adds shadow casting.

i thought i would upload with each part i add to the code, in case people want to compare versions, and can look at minimum code to begin with, and see a bit more code added with each update.
easier to learn from. the code has a lot of comments to explain what is going on.

  enjoy, stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: kevin on May 25, 2022, 08:52:21 PM

  Only got round to testing this last night; but it's all looking good.
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on June 03, 2022, 03:52:25 AM
update time

this is version 0.4 and adds reflections to the ground plane.
i just wanted to get the routine working, as i needed to modify the routines so i could re-use them for the main ray casting and reflections too.

i will add reflections to the spheres next, so every surface has reflections on.
the reflection value is currently 60%.

   enjoy, stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on June 10, 2022, 02:08:38 AM
this time i added reflections on the spheres.
--- only 1 reflection bounce ---

i also turned up the ambient value to 0.15 . this brightens up the shadows a little bit.

next time i will make multiple 'reflection' bounces so you can see a reflection on a reflection. looks more natural.

   have fun, stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on June 15, 2022, 10:32:14 PM
here is v0.6

i have added reflections to every surface in the scene, and allowed multiple bounces until there is no more colllisions.
what do you guys think?

i am considering adding the ability to modify the scene yourself to change size and position of the objects and the light(s).
and also to modify the surface settings.

   have fun, stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: kevin on June 17, 2022, 01:03:19 PM

  Steve,

     Looking good dude..     The bouncing costs a bit more (50%)  but only handful seconds longer than one of the previous versions. 

     For the scan render (pass #1) you can do updates once per second rather than per scan line updates..  Which give the benefit of showing the user it's working but with minimal system impact.    This makes the two methods run in about the same time here.   

[pbcode]

;-----------------
;--- method #1 ---
;-----------------

If render_option = 1  ;  see one line at a time drawn to screen (slower)
   
   start = Timer()
   
   RefreshTime$=""
   ;--- start at the top left of the screen, and go across each scanline to the end, then move down to the next scanline till reach the bottom of the screen ---
   ;--- draw one whole scanline at a time so user can see progress (will be slower though) ---
   For lp = 0 To screen_h-1  ;  go down the screen
     
      RenderToImage 1  ;  focus drawing to a screen size FX image (draws faster for lots of repeated smaller draw commands)
      
      ;*********
      LockBuffer
      ;*********
     
      seed = Point(0,0)  ;  ensures address of the surface is seeded for 'fastdot'
     
      For lp2 = 0 To screen_w-1  ;  go across the screen
         
         Gosub Ray  ;  check ray collision with objects
         
      Next lp2
     
      ;***********
      UnLockBuffer
      ;***********

      if RefreshTime$<>CurrentTime$()      
          ShowRender()
          RefreshTime$=CurrentTime$()      
      endif
      
   Next lp
   
   ShowRender()
   time = Timer()  - start
   
EndIf


Function ShowRender()
         RenderToScreen  ;  focus back to screen
         DrawImage 1,0,0,0  ;  draw FX image to screen to see line by line progress   
         Sync
EndFunction

[/pbcode]
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on June 17, 2022, 07:33:24 PM
i was thinking about displaying the scanlines every 100 lines, but i like the every line method cause it looks cool.
that is why i added the second render method, but this could be updated to draw to screen every second, to make it look better than watching the progress bar.

glad you like it.
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on September 12, 2022, 11:06:56 PM
hey uncle kev

just a few questions i want to ask, as i have progressed well with the editor i have written for the ray tracer. i am nearly ready to post a demo.

> if i have lots of lines, dots, GouraudQuad, and text to draw, should i draw these to the screen or an FX image?

* currently i am drawing these to the screen using lockbuffer. this works well on my newer computer, but will it work on an older one?

when i 'render' the scene, this is drawn to an FX image. this helps the 'one line at a time' method draw smoothly.
i do know that text/print doesn't like lockbuffer, so i can unlock then draw text last. there is not much text so i am not too concerned.

i am not sure about gouraudquad. is this gpu or cpu?
i am considering drawing everything to an FX image, then draw this to the screen. so main scene to FX image, editor to FX image, and render to FX image...

also with text, what i am doing is drawing gouraudquad then text on top for buttons. i am thinking of just making this into a small image, then just drawing this where the button is.

thanks, stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: kevin on September 13, 2022, 07:43:55 AM

Hey Steve,

Quote
just a few questions i want to ask, as i have progressed well with the editor i have written for the ray tracer. i am nearly ready to post a demo.
> if i have lots of lines, dots, GouraudQuad, and text to draw, should i draw these to the screen or an FX image?
   
* currently i am drawing these to the screen using lockbuffer. this works well on my newer computer, but will it work on an older one?

when i 'render' the scene, this is drawn to an FX image. this helps the 'one line at a time' method draw smoothly.
i do know that text/print doesn't like lockbuffer, so i can unlock then draw text last. there is not much text so i am not too concerned.


   For things like the rendering loop,  you could just write to a 1D array across the row.   Then at the end of the row,  copy the row to the buffer / screen (ie. copymemory (https://playbasic.com/help.php?page=MEMORY%20MANAGEMENT.COPYMEMORY)) 
 
   For GUI stuff you can draw the FX image to the screen, then draw the GUI elements to screen over the top.  This does work but your bound to run into a system that doesn't like all the locking / unlocking of the screen. 

   I generally don't draw anything to the screen, in favor of drawing it all to an FX version, then rendering that to the screen.   Which requires one lock/unlock. 

   For TEXT, you should convert your fonts to CRF.   These can blend (FontBendMode (http://www.playbasic.com/help.php?page=FONTS.FONTDRAWMODE)) to the backdrop and respond to Alpha channel which gives greater control and the rendering code is optimized for rendering text, unlike bitmap / fx fonts modes. 

 
Quote
i am not sure about gouraudquad. is this gpu or cpu?
i am considering drawing everything to an FX image, then draw this to the screen. so main scene to FX image, editor to FX image, and render to FX image...

    Direct draw only supports filling rectangles and image blits (none rotated).   So gouraud rect is all cpu driven, same as it's texture cousins.


Quote
also with text, what i am doing is drawing gouraudquad then text on top for buttons. i am thinking of just making this into a small image, then just drawing this where the button is.

   If you're drawing a bunch of stuff to an area, like a button or something then it's most likely a better idea to draw it all to an AFX surface and just draw the cached image in it's place.   



Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on September 13, 2022, 10:30:01 PM
thanks kev

i will draw everything to an FX image. seems a better method.

out of interest which is better / less problematic:
> if i draw a background FX image to the screen, then draw an object as many lines to the screen over the top of this using lockbuffer.
> draw the background in an FX image, then draw the object as many lines also in this FX image using lockbuffer.

the reason i ask is i am not sure by DEFAULT (not using lockbuffer) if drawing to the screen causes locking/unlocking per object line drawn AND drawing to an FX image by DEFAULT (not using lockbuffer) causes locking/unlocking per object line drawn in the same way.
i know using lockbuffer at start, then draw all objects lines, then use unlockbuffer at end helps resolve this by only a single lock. at the moment i am using about 500 small lines to draw each individual object (spheres).
i want to change this wireframe mode as a choice for the user to also show the objects in 'shaded' mode (color without wireframe). so i am experimenting on different techniques, hence so many questions.

i appreciate the knowledge you share with us, even though we ask similar questions one after the other (at least i do, lol).

  thanks stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: kevin on September 13, 2022, 11:18:29 PM
Quote
out of interest which is better / less problematic:
> if i draw a background FX image to the screen, then draw an object as many lines to the screen over the top of this using lockbuffer.
> draw the background in an FX image, then draw the object as many lines also in this FX image using lockbuffer.

 I don't really follow.   But every time you draw .. everything.... the surface is locked and released as part of the command.      The lockbuffer / unlockbuffer commands just allow us to batch these to avoid any potential stalls.   You can think of locking as owning a resource and when we unlock we're giving it back to the system.   For things like video resources (including the screen which is just a pair of of video images) the system only allows us to modify / read the surface when we're the owner of it.    



Quote
the reason i ask is i am not sure by DEFAULT (not using lockbuffer) if drawing to the screen causes locking/unlocking per object line drawn AND drawing to an FX image by DEFAULT (not using lockbuffer) causes locking/unlocking per object line drawn in the same way.
i know using lockbuffer at start, then draw all objects lines, then use unlockbuffer at end helps resolve this by only a single lock. at the moment i am using about 500 small lines to draw each individual object (spheres).
i want to change this wireframe mode as a choice for the user to also show the objects in 'shaded' mode (color without wireframe). so i am experimenting on different techniques, hence so many questions.


  The same rule apply..  whatever the replacement for the lines, the routine would need to manage the lock state of the surface or suffer the penalties when drawing to video (gpu) memory.  

  But if your drawing everything to an FX image and then draw the final to the screen..  It's irrelevant.  


  This is how all my demos / programs work. From Thesius XIII - Forest Blast Tech (https://www.underwaredesign.com/forums/index.php?topic=1896.0)  to Blitz 2 PlayBASIC convertor (https://www.underwaredesign.com/forums/index.php?topic=4625.0)


   
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on September 14, 2022, 08:17:18 PM
ok kev, i think i am getting it.

* a draw command to a video surface (screen) gets locked > draw > unlock
* a draw command to an FX surface just draws

so the advantage of drawing everything to an FX image, is there is no locking with each draw command, and when finished you draw this FX image to the screen as a single large rectangle block (which video surface excels at).
Title: Re: Build a basic 3D Render Engine
Post by: kevin on September 14, 2022, 11:36:05 PM

Quote
* a draw command to a video surface (screen) gets locked > draw > unlock
* a draw command to an FX surface just draws

so the advantage of drawing everything to an FX image, is there is no locking with each draw command, and when finished you draw this FX image to the screen as a single large rectangle block (which video surface excels at).


  FX surfaces also need to be locked / unlocked.   So we always need to own the surface we're drawing to.   The difference is that video anything is passed off to the underlying GFX driver to do that task.   This might be done immediately, or in the background asynchronously..  The later is best as it off loads the rendering time to an external device.   In ters of performance this works bets for big stuff.

  BUT.... When we overlap calls..  we introduce wait states.     


  picture this..

   rendertoscreen   
   Cls rgb(0,0,255)

   box 100,100,200,200,true


  if CLS and BOX are both to be drawn by the driver in the background.   The driver has to wait for the CLS to be done and released (unlocked) before the BOX can even start.

  The system manages video memory for us, we have no control over it.. So when a command is called,  it will request to be owner the of surface(s) it's about to use.   If something else is working on them, we have to wait.. 

  if we do

   Cls  then DOT  .. Dot includes a lock/unlock (a request to own and release it's ownership of the surface).   This process is not a fixed time operation as the system is sandwiched int he middle of this conversation.   So the locking/unlocking is slower than drawing the dot.   Hence if we group the locking we can remove potential stalls.       


   








 
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on September 15, 2022, 08:38:01 PM
thanks very much kev for the video explanation.
i could watch videos like this all day. i love the detail of how things work.

so a lot is going on when doing drawing operations. now i understand more about FX and video surface.
so, the so called 'issue' is when drawing to a video surface the video card driver takes over and we have no control over how it works. meaning different drivers can be efficient, and others slower/less efficient.
now i see that drawing to an FX surface, then when finished, draw this to the screen allows for a more controlled/smoother outcome.

yay i am understanding more, stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on September 16, 2022, 08:33:33 PM
i actually applied a trick to the scene to now have 'shaded mode' drawn to the screen live (this is when the objects are drawn solid).
how i did this is when you select shaded mode it calculates the scene objects to be drawn solid (diffuse only), and this is drawn to a separate FX image, then when done this image is now drawn as the background.
this way all the buttons etc work as normal. when the user moves the camera, the objects are temporarily changed to wireframe mode (this mode is fast and keeps up with the framerate), then when the camera stops the objects are re-calculated as solid again.

the only decision i need to make is if i should have the light source of the diffuse 'solid mode' as the actual lighting in the scene, or have all objects evenly shaded.
the difference in look is in actual lighting it shades the objects depending on where the scene lights are, so if the light is on the other side of the objects from the camera point of view, you are on the shadow side so the objects will look dark.
otherwise i can evenly light each object so the shading looks the same on all objects, regardless of camera angle.

anyway, i am having fun with this and i shall keep coding, stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on September 28, 2022, 02:24:52 AM
hey kev, hope you're feeling better.

i have now added enough to the editor to release a demo, but...it only has spheres.
you can now edit surfaces, clone objects, move the objects around and also move the main scene camera to take a render from a different angle.
it also has load / save panels for the scene, and color panel for surface editing, all using #include "PBDialogs".

most of the buttons now work (the ones that don't work are related to non-sphere objects eg. rotate won't work on the spheres, so i left it not active, for now).

* so, i am wondering if you want to see a demo now, or do you want to wait while i add other object types, like box, cone, disc (cylinder)?

   stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: kevin on September 28, 2022, 05:08:42 AM

Steve,


   
Quoteso, i am wondering if you want to see a demo now

    yeah.. the more the better...

Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on September 28, 2022, 08:07:25 PM
ok kev, just a few adjustments and i will upload a demo.

> could you have a look at 'PBDialogs2' ColorDialog, as i cannot get it to work. so i use the one in 'PBDialogs' ?

> also, i noticed with 'PBDialogs' ColourDialog that if i use a custom color that the red, green, blue values are reversed (see pic).
   i wondered why the colours didn't look right, lol. i can just reverse the red & blue in the meantime.

  thanks stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: kevin on September 28, 2022, 08:35:50 PM
 Your asking me ? :)


Looking at the picture it's return the colour in BGR rather than RGB.  So you can just swap the R and B and life goes on


[pbcode]

   print Hex$( BGR_to_RGB($4080FF) )
   Sync
   waitkey


function BGR_to_RGB(ThisColour)
        NewRGB = rgb(   rgbb(ThisColour) , rgbg(ThisColour) , rgbr(ThisColour)  ) 
EndFunction NewRGB

[/pbcode]


Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on September 29, 2022, 01:58:26 AM
time for a demo.

this demo as v0.20 has most features added into it, but only has spheres objects that can be duplicated and modified.
the main scene camera can also be moved around to find a better render position.

the inactive buttons, that i intend to add features to in the future, have the buttons in dark text. that way i can see what i intend to add later.

anyway, have fun testing this demo, and if there are any bugs please post, and i am also happy to take requests also.
it has been a longer journey to get this far than i expected, but i have enjoyed it.

  stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: kevin on September 30, 2022, 09:03:42 PM

  Damn didn't know this post would push Steve''s off the page..


  Read about the latest Ray Tracer release / pictures / demo source etc   BACK HERE (https://www.underwaredesign.com/forums/index.php?topic=4644.msg30889#msg30889)
   


 Added to the PlayBASIC Gallery (https://www.playbasic.com/gallery.php?ID=87.Building-A-Ray-Tracer)

Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on October 01, 2022, 12:30:44 AM
thanks kev, i am glad you like this and posted it on playbasic gallery.

it is fun making an editor, getting all the buttons to work, and also getting multiple views of the scene drawn to a quad display, with the ability to select objects and use tools.
it was a learning curve to set up. if i was to do this again, i would make a more efficient set-up code wise, now i know what is involved.

to get the selection of objects to work, in both the orthographic view and the perspective view, i converted the 2D screen coordinates to 3D coordinates, then shot a ray into the scene either in a straight line (orthographic) or from the camera to the 'screen' along the ray (perspective). also, the zoom level is different between the orthographic view and perspective view (i found interesting). in other words the selection routine is a mini ray tracer, lol.

well, i am gonna grow this a bit more and post when done.

   stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: kevin on October 02, 2022, 06:18:01 AM
 Steve,

   It's fantastic but..  yes there's a but..  it's hard coded to rely upon the screen size.  Making it unusable on computers what don't have desk top space for the window, such as my laptop..  

   So in order to run it , I have to scale the window roughly within the limits of my systems desk top size..  This works, but all the SetMouse then break the interface as their absolutes.    So every time I click on anything the mouse is re-positioned away from where It was..  

[pbcode]

   //  Scale


   OpenScreen 1600,960,32,1  ;  open screen in 'window' mode

    ScaleWindowToDeskTop()



   Print "Hello World"
   Sync
   waitkey





linkdll "user32.dll"
  SetWindowPos(hwnd,hWndInsertAfter,x,y,cx,cy,wFlags) alias "SetWindowPos" As integer
Endlinkdll



Function ScaleWindowToDeskTop()

//  Get the Desk top width/height
   dtw=GetDesktopWidth()
   dth=GetDesktopHeight()

// Get the PB screens size
   w=GetScreenWidth()
   h=GetScreenHeight()

   // Get the PB screens window Handle
   hwnd=GetScreenHandle()

   ; Stretch the window to the size users display size
  SetWindowPos(hwnd,hWndInsertAfter,0,0,dtw,dth,wFlags)

   ; Resize PB's GFX viewport to screens (the window) new client area
   StretchGFXscreen


Endfunction

[/pbcode]

    I'm putting together a video this morning, but it's pretty long....



   


Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on October 02, 2022, 11:32:15 PM
hey kev

don't worry, i take this as positive criticism. i actually like it when you look at code and give advice to improve it. how else am i going to learn, lol.

i didn't realise i was going to code this much into the editor, so left it in gosubs. i wanted to challenge myself to see how much i could use only gosubs, with minimal functions.
i was going to do this just for only the ray tracing, but ended up using them everywhere. i actually do the opposite of this code, and use mainly functions.

i will definitely implement the screen size check.
i left it this size because i thought your computer was 1680 * 1080. so i thought i left room around the current window.
also, i don't normally exit something using a gosub jump, because in the past you told me it was not recommended to do that. or was it to not exit a nested for - next loop?

here are some tips for using the editor:
> click & hold the small square buttons then move mouse. when you hold down the button it will turn blue, then when release the button it will go default.
    this works for the 'widget' buttons on top right on each window, and also for the color options in the surface editor panel.
> tool buttons that are longer either activate immediately (momentarily turn blue) or you click them and they remain blue so you can then click in the window to activate/use the tool (eg. mirror)
    NOTE: usually to use a tool successfully you need to have an object(s) selected, else there is nothing for the tool to work on.
> select is to click on the object, and there are 2 ways to de-select. 1) hold shift & click object (this allows single object at a time) 2) click anywhere around the windows and this de-selects all of them.
    also check out the 'view' tab, that has select all & de-select all buttons.

* not sure why in the main scene, when you clicked on 'camera' you could only move backwards ???
   it should work forwards / backwards / left / right with the direction of the mouse movement...

anyway, i really need to write the code again to make it more 'functional'. it is long winded at the moment (result of my gosubs only challenge).

i will implement the changes you recommended, inside my current spaghetti code, but may re-write this soon in the future.
to tell you the truth, it is getting to be a pain to add things and also bug fix... i spent half the day coding, then the other half bug fixing to get it to work properly, lol.

   can't wait to post new demo, stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on October 03, 2022, 08:01:52 PM
a quick question, when you click on a 'widget' button, these are any small square buttons, i made it so the mouse pointer turns off, and stays off while moving mouse, then when you release the mouse button it turns mouse pointer back on.
do you think this is a good idea? should i just leave the mouse pointer on?

i made it this way because i thought it looks better than seeing the mouse pointer jitter around while holding down 'widget' button and moving mouse.
i did have the screen position in desktop saved in a variable, so the mouse pointer should appear where it was when it turns back on.

in the video, when you added code so the screen can be re-sized, was the screen re-opened in a smaller size, or was the 1600*900 screen scaled down to fit inside the desktop?
if the screen was scaled down could that account for the mouse pointer jumping in the editor that you had, or the camera movement in the main scene jumping in position so fast?

the reason i mention this is i have added a check for the desktop size (thanks for that advice), and it will open a smaller screen to fit inside the desktop, and all 'widget' buttons with pointer turned off, seem to return to their correct positions, even when i make a much smaller window for testing. the code for the mouse re-position was already there, so no modifying the code was necessary. it is working when i re-size the screen window to any size on my computer. that's why i didn't understand why it wasn't working on your laptop.
Title: Re: Build a basic 3D Render Engine
Post by: kevin on October 03, 2022, 09:13:45 PM
 I don't think it needs a re-write..  but could be more compartmentalized.     Ideally if you can run the GUI separate from the application.  


Quote

a quick question, when you click on a 'widget' button, these are any small square buttons, i made it so the mouse pointer turns off, and stays off while moving mouse, then when you release the mouse button it turns mouse pointer back on.
do you think this is a good idea? should i just leave the mouse pointer on?


  I'm not a huge fan of it..  if the pathway gets missing when the mouse is off or something else breaks in while you've turn the system mouse off.. this can lead to annoying issues for a user.     I know it can look nicer tho.


Quote


in the video, when you added code so the screen can be re-sized, was the screen re-opened in a smaller size, or was the 1600*900 screen scaled down to fit inside the desktop?
if the screen was scaled down could that account for the mouse pointer jumping in the editor that you had, or the camera movement in the main scene jumping in position so fast?

  That setwindowpos function is just a way we can scale a window to size, regardless of what it's internal surface size is.  It's the same really as if we dragged the window corners in manually.  

  But.. from memory get PB computes mouse coords to the internal surfaces to a ratio of the actual window size.  Which I'm pretty sure occurs when reading the position, but i've got my doubts when forcing the mouse position.   Which would break the positioning.   Could prolly account for it by just getting the windows client area / against it's actual size then scaling for the time being..  

   
    Example Code:  GetWindowRect & GetCleintRect (https://www.underwaredesign.com/forums/index.php?topic=4675.0) 

    If client area is the 'pb screen' inside the window, while the WINDOW rect is the windows absolute size on the display.  There some padding applied for themes I guess..     Read more about    Get Client Rect (https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-getclientrect?ranMID=46131&ranEAID=a1LgFw09t88&ranSiteID=a1LgFw09t88-dvs1AG2iCbN9E8XxYZQ1FA&epi=a1LgFw09t88-dvs1AG2iCbN9E8XxYZQ1FA&irgwc=1&OCID=AID2200057_aff_7806_1243925&tduid=(ir__mwtju19p3okftg3nkk0sohzw0e2xqslzmlrpfvws00)(7806)(1243925)(a1LgFw09t88-dvs1AG2iCbN9E8XxYZQ1FA)()&irclickid=_mwtju19p3okftg3nkk0sohzw0e2xqslzmlrpfvws00) in the Windows SDK


   
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on October 15, 2022, 12:18:59 AM
i am working on an updated version of the ray tracer editor. thought i would keep a list here. most are bug fixes, some changes too.

completed:
> check desktop size :  if smaller than my 'recommended' screen size, it will open a window to fit inside the desktop, allowing a small gap around it for taskbar and window title bar. buttons etc will adjust themselves to fit into the screen. you can also manually change code to open a much larger screen size too. it will adjust to fit all buttons into their sections.

> widget buttons  :  the behaviour of the mouse movement when using widgets is now changed, so it will allow smooth adjustment even if you drag the screen around to a new location.

> added shortcut keys  :  the editor now has shortcut keys for some operations, instead of needing to use the widget buttons (thanks kev for this suggestion). i will add some to the main scene also (listed in help screen).

> help/info screen  :  this has been modified with tabs for easy selection of the info you need. also fits into a smaller size to allow for smaller screens.

> surface editor  :  moved the surface editor panel to allow for small screen sizes. this will display inside the view window of the editor without interfering with widget buttons.

> save files into a readable text format, with different data in their own heading. this allows for any changes in the future to load the file correctly (another handy tip from kev).

> add creating spheres with the button press, rather than needing to clone existing ones (in case you clear the scene and no spheres are in it).

> fixed: a bug is if you change the reflection amount of an objects surface, this reflection in other objects will adjust making this object look semi-transparent, depending on your setting amount.

> fixed: have found a bug where it leaves some 'fireflies' (speckles) in the render on some angles. checking for a fix.


i will edit this post as i complete the above list.

  stevmjon

************************
*** Demo Below _ v0.21 ***
************************
Title: Re: Build a basic 3D Render Engine
Post by: kevin on October 18, 2022, 06:29:51 PM
 Steve,

    I only had a quick play last night and it feels really good..   there's a couple of little niggles with the dialogs overlapping but nothing big..   will have another pick through later. 

    All in All it just worked straight outta the box..  The controls like object moving/camera feel pretty comfortable.



  Big thumbs up..   
 
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on October 19, 2022, 05:39:03 PM
cool kev.

let me know what needs to be adjusted and i can modify the code and release an update version.
checkout the save file too, and see if it is more like what should be done. it has headings and data is all readable text. you can even modify the code in notepad. i have even done this, lol.

  stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: kevin on October 20, 2022, 09:41:54 PM
 I will..  :) 

Pretty busy atm..  but two things that came to mind was when adjust the colours from object editor it's possible to move the mouse (while invisible) and for those clicks to register on other gadgets..

  Another thing that i remember happening was  if i used the colour select dialog, I move the mouse and then was in state where the system mouse was turned off couldn't see how to escape the situation.    I'll try and recreate on video.. I don't recall exactly what I was doing .. 


Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on October 21, 2022, 06:30:17 PM
i have changed the way the click and drag mouse works on the widget buttons. it should work without any issues.
the mouse does turn off (because when left 'on' the mouse looks wierd), but as soon as you release the mouse button the pointer turns on again.
also, while the mouse is off, it keeps the mouse inside the screen window.

i was also thinking of allowing the mouse scroll wheel to adjust colors. you mentioned this earlier in the video, but i forgot to add this.
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on October 25, 2022, 07:21:57 AM
NEW START...

well, i have began to write this code again as it is like spaghetti code, plus it didn't help everything being in gosubs (challenge i gave myself).
my progress has been pretty good, and i have now made good old functions again making sure to write stuff once in a function i can multi-access.
this is making code smaller again, and a lot easier to grow. i have also made more types to make more objects.

Progress:
> screen check  :  checks desktop size and adjusts screen accordingly
> scene objects added
> perspective grid  :  changed this routine, which draws faster, and it also follows the camera around
> camera  :  allow multiple cameras in the scene
> lights  :  allow multiple lights in the scene
> shortcuts (main scene)  :  added shortcut keys to control camera
> main scene buttons  :  buttons and menus are in main scene and are now images and treated as objects (types) . all text on screen has been removed.
> mouse-over hints  :  if you hover the mouse over a button then a hint text will appear in a box at the bottom of the screen. this explains what the button does.
> active button hints  :  shows hints on how to use current tool selected
> render engine  :  rays and vectors are now objects. i also fixed a few bugs i found. but... as a result of adding functions, the render takes a few seconds longer.
> editor buttons  :  buttons and menus are in editor and are now images and treated as objects (types) . most text on screen has been removed, as some panels change/update text.
> editor windows  :  all different views are working in the editor, and are all drawn to an FX image using ImageViewPort command (limits draw zone).
> tools  :  editor tools are now added and updated
> update help screen  :  added scrollable text in case window is too small to fit all text in window (use mouse wheel)
> shaded mode for objects

i will update above list regularly. it keeps me motivated too. i am having fun.

  stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: kevin on October 25, 2022, 07:41:29 PM

Awesome..  I'm just glad people are enjoying themselves... making things of this caliber  !

Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on May 18, 2023, 11:29:27 PM
time for another update to v0.37 , as this is a re-write of the code to take it from gosubs to functions.

this should run smoothly, and i have added a few new features in the editor that show on screen a 'preview' of what will happen if a tool button is clicked.
so if you hover over create > ball it will show a preview on the screen in red to show where the ball will appear if the tool button is selected.
also if you hover over the multiply > mirror x/y/z tools it will show a preview in red of where the mirror border is and also show the new objects in their mirrored position (as long as you have at least 1 object selected).

another feature is 2 wireframe modes to choose from (wireframe & color wireframe), as well as the previous 2 shaded modes to choose from already included.
i added the new wireframe mode in case the user doesn't want to use shaded mode but wants to see what color the objects are.
also the light can be selected and moved around too.

i will keep updating this to add more features, but i wanted to at least post this updated version which should be stable.
i am looking forward to adding planes & boxes, as this will allow much better objects to be created (coming soon).

EDIT: just fixed a minor bug where the light can be moved when it is not selected, while trying to move another object

  enjoy, stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on May 20, 2023, 09:02:24 PM
just updated the demo in the above post to fix a minor bug.

updated to "v0.37 Demo 2".
Title: Re: Build a basic 3D Render Engine
Post by: kevin on May 23, 2023, 11:04:48 PM
 Excellent dude!     was playing with it yesterday, then picked through the code a bit more last night..  had some ideas to simplify some bits here and there.    Mostly things like removing code repetition by building helper functions here and there. was going to make a video but ran out steam last night..  


  Edit: that's annoying my post pushed yours off this page  -   to download 3D Ray Tracing Engine_v0.37_Demo 2  back here (https://www.underwaredesign.com/forums/index.php?topic=4644.msg31200#msg31200)


  Code tidbits

   In Menu.pba
[pbcode]

        state = array(lp).state
        
        Select state
           Case 0  :  image = array(lp).image1  ;  image (default)
           Case 1  :  image = array(lp).image2  ;  image (default mouseover)
           Case 2  :  image = array(lp).image3  ;  image (clicked)
           Case 3  :  image = array(lp).image4  ;  image (clicked mouseover)
        EndSelect


[/pbcode]

    This could just be, if the images field was defined as an array .  ie. images(4)
[pbcode]

        state = array(lp).state
        image = array(lp).image(state)

[/pbcode]


    Boolean expressions can help in places also.  

[pbcode]

   ; so this
  If z1# < znear# Then Inc out
  If z2# < znear# Then Inc out

   ; can be
   out= (z1# < znear#) + (z2# < znear#)

[/pbcode]

      Compares return a true(1) or false(0) which is just an integer.    If one is bellow out will be 1 and if they're both it'll be two

      You could also, just make a region coded out of the two compares and then unroll for all possible outcomes.  


[pbcode]
     
     ; something like this
   out= ((z1# < znear#) << 1)  + (z2# < znear#)

      select out
             case 0  //  no clip required  
                   
                  eitfunction stuff


             case 1  //  Z2 needs clipping but Z1 is front of plane

                      clip Z2

             case 2  //  Z1 needs clipping but Z2 is front of plane

                      clip Z1

             case 3  // both are behind the plane so reject them

[/pbcode]

   
      So clip could be more like this.  Notice that when the line is fully in front or behind, we just pass back the original parameters.       Could most likely get rid of the ix1/iy/iz etc versions also 

[pbcode]


Function clip_nearz(x1#,y1#,z1#,x2#,y2#,z2#)
   
   ;--- clip border 'nearz' ONLY for perspective mode ---
   ;--- NOTE: must be calculated before perspective projection ---
   
   ; NOTE: orthographic mode displays all of z buffer depths (no clipping) ---
   
   ; point(1) = x1 , y1 , z1
   ; point(2) = x2 , y2 , z2
   

   select ((z1# < znear#)<<1) + (z2# < znear#)

         case 0,3   ;  both points inside border (keep)
            exitfunction x1#,y1#,z1#,x2#,y2#,z2#,out=3

         case 1
            ix1# = x1#  :  iy1# = y1#  :  iz1# = z1#  ;  a (source point)
            ix2# = x2#  :  iy2# = y2#  :  iz2# = z2#  ;  b (destination point)

         case 2   
            ix1# = x2#  :  iy1# = y2#  :  iz1# = z2#  ;  a (source point)
            ix2# = x1#  :  iy2# = y1#  :  iz2# = z1#  ;  b (destination point)
            swap_value = 1
   
   endselect   
      
      d1# = iz1#-znear#  ;  NOTE: no dot product required as nearz border is perpendicular to z axis
      d2# = iz2#-znear#
      t# = d1#/(d1#-d2#)
      
      ; i = a + t(b-a)
      ix2# = ix1# + t#*(ix2#-ix1#)  ;  x, y, z
      iy2# = iy1# + t#*(iy2#-iy1#)
      iz2# = iz1# + t#*(iz2#-iz1#)
      
      If swap_value = 1  ;  place points back in original order
         exitfunction ix2#,iy2#,iz2#,ix1#,iy1#,iz1#,Skip
      EndIf
   
EndFunction ix1#,iy1#,iz1#,ix2#,iy2#,iz2#,skip


[/pbcode]
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on May 24, 2023, 08:24:09 PM
thanks for the tips kev, it's always handy to optimise.

i am glad you like the program too. i am still working on another update that will have more geometry included.
so far i have already added cylinders. you can so far create them and they will display as wireframe mode. i am currently working on being able to select them, then i will add render ability.

the more i am programming this the more i am enjoying the program, and i am coming up with more idea's too.

  stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: kevin on May 25, 2023, 11:50:28 PM

Quotethanks for the tips kev, it's always handy to optimise.

    Sorta,  This sort of stuff is  refactoring bits to reduce the code foot print.    What tends to happen is our as project gets larger, we end up spending more time maintaining the code base as changes are mode.   So just a good idea to occasional go back and pick through the older code/approaches and try and do some clean up.     
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on August 09, 2023, 02:29:52 AM
just excited to share a screenshot of my progress.
i have just added cylinders in wireframe mode.

i now need to have them visible in the render itself, which is next.
i already know how to calculate this, just need to code it.

at the moment i am keeping all the code to see the objects mathematically, rather than using polygons. this way the objects stay high resolution when rendered.

   stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on August 23, 2023, 02:28:46 AM
i have added the cylinder to the renderer now, yay.

i just need to adjust it so it registers height. the cylinder is currently infinite in height, lol.
getting this routine added took a lot more than i thought to calculate.
now i know why they stick to spheres and planes.

after i get the cylinder height done, i will add it to fast render view, so it shows up in shaded color mode in the editor.

   stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on August 31, 2023, 03:18:11 AM
just added the cylinders to be able to be seen in 'fast shaded color mode' in the editor.

also it can be selected in the editor too.

adding the cylinder was a good learning curve for me to better understand the maths behind 3D.
i like learning new things, it keeps the mind active.

   stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: kevin on September 01, 2023, 08:06:47 AM
 Hey Steve;

 
Quotejust added the cylinders to be able to be seen in 'fast shaded color mode' in the editor.

    That'd be a nice challenge given there's no nice easy mesh to render

     Looking good dude !

Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on September 02, 2023, 08:51:46 PM
yeah, i needed to go through a bit of maths to find the collision.

full method:
> find skew line minimum distance between the camera ray and cylinder direction ray (direction cylinder is standing)
   > this gives the minimum distance between the 2 rays where a line is perpendicular to both rays
   > also gives this perpendicular lines as a vector
> find point on the camera ray where the minimum distance is located
   > turn the cylinder direction ray into a plane so it passes through the 'shared' perpendicular line
   > do camera ray to above plane collision check (gives location)
> find point on cylinder direction ray
   > turn the camera ray into a plane so it passes through the 'shared' perpendicular line
   > do cylinder direction ray to above plane collision check (gives location)
> calc intersection point on cylinder surface
   > find distance the camera ray is from the cylinder center vector, then calc remaining distance along the camera ray to the cylinder edge (gives distance like a shadow cast directly down from the ray)
   > calc angle the ray is from the cylinder center vector to get actual distance 'along' the ray
> check if collision is behind or in front of camera ray
> check if collision point is within the cylinder height
   > if inside height then calc surface point normal at this collision point on cylinder
   > if outside then calc both the upper cylinder face & lower cylinder face for a collision and calc the surface point normal at this collision point

this is why it took a bit to implement this into the code to show up on the renderer and shaded mode.
the good thing is it also detects a collision when i click on the screen to select the object in the editor.

i leart more about vector calculations doing this cylinder.

   stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on December 16, 2023, 07:43:58 AM
time for an upgrade to v0.43

this has had a few bug fixes i found, plus a few modifications too.

* added cylinders
* added rotation of objects (in editor)
* added size of objects with separate width/height (in editor)
* modified rendering of reflections order (last to first)
* added a different extra default scene
* better checking for save & load files

take it for a spin and see what you think.

i will update again with new objects: cones & boxes.
also will modify the surface editor to be more flexible. currently if you select 'multiple' objects then open surface editor it applies current default color to all selected objects (may add ability to see surface settings for each selected object in a list?).

  stevmjon

EDIT: see updated version in reply #58 (this fixes a clipping bug).
Title: Re: Build a basic 3D Render Engine
Post by: kevin on December 16, 2023, 08:50:16 AM
 Steve,

Without wanting to sound like a broken record,  that's fantastic  !!


 
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on December 16, 2023, 04:46:59 PM
thanks kev. glad you like it.
i fixed every bug i could find in this one. seems to run smoothly.

i should be able to update this with new shapes easier now that all the routines are working properly.
when i add boxes that should make better scenes because you will be able to shape the boxes and make more detailed objects with them.

this has been a learning experience as i have applied my mind more deeply into 3D. gotta keep the mind active.

i'm looking forward to making more updates.

  stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: kevin on December 16, 2023, 08:27:41 PM
It's amazing...  Noticed what looks like plane clipping problem when moving the camera forward in the default scene though.


Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on December 16, 2023, 11:16:01 PM
oh, thanks for the input kev.
that should only effect the wireframe model, and not the render itself (hopefully).

i will take a look at it and post a fixed version.
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on December 18, 2023, 05:01:29 PM
here is the fixed version v0.44

i had clipping for the cylinder top & bottom faces, but didn't properly clip the sides. thanks for spotting that kev.

this version should be stable (fingers crossed).

   stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on January 04, 2024, 09:15:38 PM
thought i would share a screenshot of the boxes i have added to the ray tracer.

i am calculating the box collision similar to planes.
i calculate each side of the box by calc collision with a plane, then if a pos value, calc around the plane border to see if collision point is inside/outside.
seems to work quite fast, but in this screenshot pic it goes very slow when i turn on reflections and they are a bit funny too. same with shadows turned on plus it's even slower again. guess there is more work to be done.

maybe if i use line - box collision instead (not planes)?

   stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on January 23, 2024, 03:19:15 AM
well, i got the shadows & reflections to work now.
it is faster than the post above too.

above post = 192 seconds
this post = 114 seconds

i do have a few idea's to get it to render faster.
one idea is instead of testing every object per ray with the shadows/reflection routines (this causes slow down), i can use the 3D transformation stages to calc the scene like the ray (shadow/reflection) is the look direction of a camera then calc min/max values to test which object spans across the screen center? if yes then test this object only. this should save a lot of calculations.

more to do it seems, stevmjon

Title: Re: Build a basic 3D Render Engine
Post by: kevin on January 23, 2024, 08:24:43 AM

Was going to suggest this earlier but it'd be ncie to be able to set the render resolution.   so 1/2, 1/4, 1/8 resolution.     If the res is set lower the render still goes back and fills in successive levels of detail, just at those lower resolutions the initial pixel takes up like 2by2 , 4by4 or even 8by8  so the user can get a sense of the scene quicker and exit out if something needs to be tweaked
 
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on January 23, 2024, 04:41:34 PM
oh good idea kev.

i can set up different render modes like you suggest. i will add another drop down menu selection with these settings.
i could even allow to set a different camera screen size too, that is not locked to your current window size.
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on February 04, 2024, 11:52:13 PM
kev, i just added a tweaked version of what you suggested with the preview render resolution.

instead of drawing the screen in stages as you suggested, i decided to draw a smaller preview screen size with all the detail in it. still draws so much faster too.
100% = 114 seconds
50% = 28 seconds
25% = 7 seconds

great idea for this suggestion kev. now i don't have to sit thought slow render times, lol.

i will post a new demo very soon too. i now have all the routines in for boxes, and i am now just setting up a nice default scene to view. i wanna get a bit creative with the boxes.

   stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: kevin on February 07, 2024, 07:20:12 AM
  Cool!!!  

 Another thing  I was thinking about was possibly trying a dithering style patterns for the filled preview modes too.   So the user can get a sense of it while something is moving without drawing the while thing every refresh


Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on February 09, 2024, 08:03:28 PM
good idea kev.

i can add a new preview mode that is 'solid mode' where all the objects are replaced with a PB shape.
this will stop the delay that the shaded modes have each time when movement stops.

i will delay the next release a bit more to implement these new ideas. if you have any more let me know and i will add them.

i have also added a new array tool too. i will post a pic when i get the panel for it done.

   stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on March 11, 2024, 07:10:01 PM
well, time for another update to v0.54 March 2024.

this update was delayed because i wanted to add more tools & features requested by Kev.
i also fixed several bugs i found too.

Features :
> 'array' tool : this lets you clone an object in a grid pattern (good for walls, decking etc).
> 'clone' tool : this lets you clone an object in a given direction (good for stairs and above also).
> 'save / load' files : all the shape types are now supported, and the name of a loaded file appears in the screen title (top).
> 'panels' : a panel pops up with settings for different tools. it can be moved around the screen by click & hold the title. it also remembers the last settings used.
> 'panel values' : made sure that small movements of the mouse always give at least 1 integar value (it used to give zero)
> 'solid mode' : this is a new draw mode for the objects. the objects are converted to polygons and drawn solid. the draw order is artists render method so is far - near order.
  * NOTE: the advantage is the objects are drawn solid and are drawn fast. some polygons may 'pop' in front/behind others because i am using artist render method.
> 'more shapes' : there is ball , cylinder & box shapes(new). all shapes can be rotated & scaled.
> 'render modes' : there is multiple smaller render sizes for a quicker view. 100% , 50% , 25% .

i have also included a save file with this demo with the walkway object i made with this program.

   enjoy, stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: kevin on March 11, 2024, 09:22:15 PM

  Brilliant mate !
Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on March 12, 2024, 07:20:59 AM
thanks kev. glad you like it.

i didn't originally intend for the program to get this big. been workin on it for about a year now.
not sure if i should add more or start a new project?

i was thinking of re-writing this with a better set up, as i didn't know what data needed to be shared.
a lot is copy/paste with an edit. it is getting harder to add to the code, as i need to adjust it in many area's, so a re-write would make it smaller and easier to grow.

i was thinking of a 3D editor with all polygons, so you can make any shaped object you like?
there is already free stuff that does this, but i am doing this for the challenge and the fun of it.

anyway, if there is any bugs let me know, or any new idea's i can add.

   stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: kevin on March 13, 2024, 10:19:24 PM
Steve,

Quotedidn't originally intend for the program to get this big. been workin on it for about a year now.not sure if i should add more or start a new project?

   Dude, I wish i had some useful advice,  I really don't know what to tell you ! 

Quotei was thinking of re-writing this with a better set up, as i didn't know what data needed to be shared.
a lot is copy/paste with an edit. it is getting harder to add to the code, as i need to adjust it in many area's, so a re-write would make it smaller and easier to grow.

   Yes sadly this such a problem as programs get bigger... Once the surface area of them grows adding new features / changes existing features become more laborious..  I suffer from this situation a lot these days, trying to change anything in a big code base is tedious.   

    If you want I can pick through and try and give some assistance.  Which really is making libraries out of the various underlying elements.   Which allows for the functionality to be shared/exported between projects easier, but as the surface area of the interfaces get bigger,  around we go again! :)   


Quotei was thinking of a 3D editor with all polygons, so you can make any shaped object you like?
there is already free stuff that does this, but i am doing this for the challenge and the fun of it.

    Anything in that space would be cool..  Ideally something that a user can import and just write their code on top of.   So the editor has a set of libraries a user can import and use to load/show objects in their code. 
 
    I dunno...  but something that helps us move in that next direction is needed..  but as you know building big things a hard !

    Enjoy ! :)

Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on March 15, 2024, 02:41:34 AM
just wondering, i know PB uses legacy direct X (direct draw), but is it possible to add an openGL 3D pipeline, so the user can use parallel computing for faster rendering?

the main slowdown is i render brute force (per pixel).

i know legacy direct x had 3D games, so even if the parallel component was just x8 or x16, that would certainly help. did PB FX have any of this? i did see it supported 3D when i read the info about it in the development stage, but i didn't get into it at the time because i wasn't sure how to use it (i didn't see any getting started tutorials). it wasn't beginner friendly i mean. but with the knowledge i have in 3D now, i could certainly take a look at it if that is recommended.

   stevmjon
Title: Re: Build a basic 3D Render Engine
Post by: kevin on March 16, 2024, 09:38:11 AM
Quotei know PB uses legacy direct X (direct draw), but is it possible to add an openGL 3D pipeline, so the user can use parallel computing for faster rendering?

 Sorta, as it's not a matter of few late nights translating dd calls for gl calls, which people seem to think.   There's various projects around the forums already ( [plink]PB GL library (https://www.underwaredesign.com/forums/index.php?topic=4210.0)[/plink] * PBFX 2D CORE (https://www.underwaredesign.com/forums/index.php?topic=4580.0))


 Even if you swapped one for the other; it wouldn't benefit the way you think.  Take a look at this loop. 

 i.e.

  For poly=0 to Number_Of_Polygons

      // read the vertex data

      //  draw this thing
      tri x1,y1,z1,x2,y2,z2,x3,y3,z3 
   
  next 

 
   Imagine the rendering request is be passed off to the GPU to be drawn off the main thread.   So the first time the loop runs the gpu is sitting there doing nothing while we grab some vertex data for the polygon.  When we hit the tri command the call passed this job to the gpu and the loop continues. 


   But what happens on the second loop and it gets to draw the tri ?  It has wait until the last one is drawn before the next one can be processed.. This turns out to be really inefficient


   The point i'm making is these api's effectively want all the data managed on their site of the conversation and as such, a draw request will draw batches of polygons all in one call..


 
QuotePB FX have any of this?

    Yep..way back in 2007... 

   

    it's mostly identical on the surface ( meaning the idea was to have most things run out of the box) , but things like the newer camera modes you'd have to dig through whatever examples still exist for it.. and there's various differences. none of which can recall   


Title: Re: Build a basic 3D Render Engine
Post by: stevmjon on March 16, 2024, 07:47:29 PM
i actually did use the G2D - 2D OpenGL library a few years back.
i converted the first 3D engine i made to use this. it worked, but the framerate was almost the same speed. just a few fps faster.

when i tried this on my newer computer it farted. the program would not even run.
my old graphics card was a GTX 970 (above worked) and the newer graphics card is RTX 3070 (not happy Jan).
my new card won't even display 16 bit or 24 bit fullscreen modes. 32 bit only.

since there are the different camera modes, i could try again using them. maybe they would work?
Title: Re: Build a basic 3D Render Engine
Post by: kevin on March 17, 2024, 10:20:49 PM
Quotei actually did use the G2D - 2D OpenGL library a few years back.
i converted the first 3D engine i made to use this. it worked, but the framerate was almost the same speed. just a few fps faster.

  Yeah.. can't say I'm terribly surprised as swapping calls just shifts burden from one thing to another.  If the total routine takes 20ms say to do a thing and we switch out a call to something that's only costing a two milliseconds of that total, we're not going to win those 18ms back. 

  If we swapped from  dots -> strips -> triangles..  Obviously drawing tri's removes the burden from the cpu side (the cost per pixel plumets), but a loop like the one above isn't ideal for concurrency.  What I mean is a loop of draw triangle calls isn't just building up and being done by the driver in a queue later in the background; rather we're only winning the overlapping render time of the last call..

  Ie.

  TriC x1,y1,z1 ,x2,y2,z2,x3,y3,z3, colour 
  TriC x1,y1,z1 ,x2,y2,z2,x3,y3,z3, colour

  For lp =1 to 100000
      a=b
  next
  end

  So to draw this we setup the call and call the driver for each TRI.  But this creates two stalls (waits)  we loss time setting it up, and once we call TRI we're not going gaining anything as the next call requires the gpu immediately after.  So we wait fro it finish and setup it's state and call the driver again.. 

  The second call gives up a benefit, as the call returns immediately back to the primary thread, but the rendering is taking place whenever the gpu can do it. So provided we don't do any graphics stuff we end up with an asynchronous result.  So it's rendering while the cpu is doing the for/next thing.   

  So there's this balancing point between the two sides..


Quotewhen i tried this on my newer computer it farted. the program would not even run.
my old graphics card was a GTX 970 (above worked) and the newer graphics card is RTX 3070 (not happy Jan).
my new card won't even display 16 bit or 24 bit fullscreen modes. 32 bit only.

  aren't drivers wonderful ! :)   

  I've not run into a system where they don't work (given it's GL1) but that doesn't mean much..
 

Quotesince there are the different camera modes, i could try again using them. maybe they would work?


  maybe, but given there still 2D..  There's no z buffer / perspective in that stuff.  Those old PBFX demos have a real 3D mode in them (z buffered) it's a matter of worker out how to create meshes through the sprites..  Don't ask me.. I don't remember !