UnderwareDESIGN

PlayBASIC => 3D Development => Topic started by: stevmjon on December 28, 2016, 03:43:19 AM

Title: Building a basic 3D Engine
Post by: stevmjon on December 28, 2016, 03:43:19 AM
hello,

it's my turn for a basic 3D Engine.
i have spent ages working out the theory of 3D, then working out all the mathmatics needed.
i am pleased to have a small demo, that works, and has polygon culling.

for the moment only polygons outside the camera view frustum are culled, and also polygons that have any points outside the view frustum are not drawn. so polygons close the the edge of the screen will disappear. this is so i know the routine works, and i can then add polygon clipping to these edge ones later.

i am excited to grow this, and i will post further versions as i add more features.

   happy holidays,  stevmjon

Latest Build V0.8.7 in Reply#45  (page 4)

Title: Re: Building a basic 3D Engine
Post by: kevin on December 28, 2016, 05:49:50 AM
 Hi Steve,  

   Looking good.  One tip that I would seriously encourage you to take would be to use typed pointers rather than 2D arrays.   It seems logical to use a 2d array for a vertex list for example, but vertex won't actually be stored in emory side by side.  Which can be costly, as CPU's don't like fetching memory randomly.  They rather it that memory be accessed as close as possible.  

   You can use a 1D array (actually it could be any layout, as you'd only be using at as a memory buffer, not an array)  as a cache basically and read/write fields via pointers.   Which is quicker than array access and if you convert to machine code, you'll see a massive performance boast.

[pbcode]


      Count=10

      Type tVertex
               x#,y#,Z#
      EndType


      ; Make the array bigg enough that is never needs to resized
      Dim VertBuffer#(Count * (Sizeof(tVertex)/4) )

      ;  Pointer to use to look at read/write vertex structures
      Dim P as tVertex Pointer



      ; get the pointer to vertex 0
      P = GetArrayPtr(VertBuffer#(),true)
      For lp = 0 to count
      
            P.X = lp
            P.Y = 1000+lp
            P.Z = 2000+lp
      
            ; move pointer to the next vertex
            P += Sizeof(tVertex)
      next



      ; get the pointer to vertex 0
      P = GetArrayPtr(VertBuffer#(),true)
      For lp = 0 to count
      
            print str$(lp)+"  "+str$(P.X)+","+str$(P.y)+","+str$(P.Z)
      
            ; move pointer to the next vertex
            P += Sizeof(tVertex)
      next


sync
waitkey

[/pbcode]

Title: Re: Building a basic 3D Engine
Post by: kevin on January 06, 2017, 10:22:30 PM

  Any progress ?

  In this version we're using a static array and indexing the X,Y,Z fields of the vertex.  It's an emulation of the previous code does, onlu difference is the fields are accessed through and array, rather than named offset such as X,Y,Z in the type structure.   
 
[pbcode]

      Count=10

      Type tVertex
               Fields#(3)
      EndType


      ; Make the array bigg enough that is never needs to be resized
      Dim VertBuffer#(Count * (Sizeof(tVertex)/4) )

      ;  Pointer to use,  to read/write vertex structures
      Dim P as tVertex Pointer



      ; get the pointer to vertex 0, which is element 0 in the array
      P = GetArrayPtr(VertBuffer#(),true)
      For lp = 0 to count
     
            P.Fields#(0) = lp
            P.Fields#(1) = 1000+lp
            P.Fields#(2) = 2000+lp
     
            ; move pointer to the next vertex
            P += Sizeof(tVertex)
      next



      ; get the pointer to vertex 0, which is element 0 in the array
      P = GetArrayPtr(VertBuffer#(),true)
      For lp = 0 to count
     
            print str$(lp)+"  "+str$(P.Fields#(0))+","+str$(P.Fields#(1))+","+str$(P.Fields#(2))
     
            ; move pointer to the next vertex
            P += Sizeof(tVertex)
      next


sync
waitkey



[/pbcode]


Title: Re: Building a basic 3D Engine
Post by: stevmjon on January 17, 2017, 04:36:05 AM
hello

another update to version v0.2
this version adds polygon face culling. this means any polygons that are facing away from the camera are culled.

next i will work on zbuffer polygon culling. this means any polygons in the background that are covered by foreground polygons (so not seen) will be culled.
there are a few ways to go about this, so i will need to experiment.

> this version doesn't contain types yet, i am currently going about this.
> can passing array types into a function be the same as passing a regular array into a function?
> i can only get variable pointers to point at the array in the function call.

example:
stuff(type_array(n).vertex)

function stuff(me as vertex pointer)
   ; this only points to the array element n
   ; i need to bring in other data like the array element maximum so i can make a loop in here etc. ???
endfunction

i guess pointing to a type array is different from pointing to a normal array.
i shall keep working on it.

  stevmjon

Title: Re: Building a basic 3D Engine
Post by: kevin on January 17, 2017, 07:01:52 AM
Quotethis version adds polygon face culling. this means any polygons that are facing away from the camera are culled.

  Cool. Looking really good!

   It's worth noting that you can save calculation work by linking / grouping faces within your object structure/meshs.    Imagine a cube with 12 triangle faces.  If the first tri is facing away, then so is it's bother.   Cubes have a lots of little opt's like you can compute the direction one a side is facing, then this tells you if the other face is visible or not also.   No big deal in a scene with a few cubes, but can really make the differences in scenes with lots of them.


Quotenext i will work on zbuffer polygon culling. this means any polygons in the background that are covered by foreground polygons (so not seen) will be culled. there are a few ways to go about this, so i will need to experiment.

   The standard solution this day and age is per pixel Z buffer, but that's not really doable here without using PB2DLL and writing your polygon filling routines.    For the most basic solution you just throw your triangles into a global pool, sort by depth, then render the far ones before the near one's.   Which is painters algorythm.   You'll pops bwteen faces that intersect each other though.   There are ways around it, which pretty much old PlayStation games use..  


 
Quote
> this version doesn't contain types yet, i am currently going about this.
> can passing array types into a function be the same as passing a regular array into a function?
> i can only get variable pointers to point at the array in the function call.

example:
stuff(type_array(n).vertex)

function stuff(me as vertex pointer)
  ; this only points to the array element n
  ; i need to bring in other data like the array element maximum so i can make a loop in here etc. Huh
endfunction

i guess pointing to a type array is different from pointing to a normal array.


   You can't pass an array into a type.  Types are like blades of grass, there might be millions of them, but the don't actually know anything about the others.   If you have a type pointer and just a sgtructure in memory, you can have any crazy structure you want. 


   Above all you do is pass the typed array by handle.  So the function is now looking at a handle of the array. So you can look at all thethings in the  array from inside the function.


[pbcode]
   Type vertex
         x#,y#,z#   
   EndType
   
   
   dim type_array(10) as vertex
   

   type_array(5)   = new Vertex
   
   type_array(5).x =1000
   type_array(5).y =2000
   type_array(5).z =3000
   

   stuff(type_array())

   Sync
   waitkey


function stuff(me().vertex)
  ; this only points to the array element n
  ; i need to bring in other data like the array element maximum so i can make a loop in here etc. Huh
      For lp =0 to getarrayelements(me())
            s$=digits$(lp,2)+"="
            
            if me(lp)=0
                   s$+="Empty"      
            else
            
                  s$+="x["+str$(Me(lp).x)+"],"            
                  s$+="y["+str$(Me(lp).y)+"],"            
                  s$+="z["+str$(Me(lp).z)+"]"         
            endif
            
            print s$      
      next

endfunction

[/pbcode]



Title: Re: Building a basic 3D Engine
Post by: stevmjon on January 17, 2017, 08:04:07 PM
awsome kev. thanks for the sample code. that is what i was looking for.
there was so many mini samples to look at, i was getting a little confused. types are quite flexible.

in regards to polygon culling, i am only calculating the vectors of the objects, not the individual polygons, so this saves calculating the same point multiple times when it is shared by surrounding polygons. i keep this method right to the end until i draw the polygons themselves. here i look at a polygon table and check if the vector is visible or not, and if all vectors for that polygon are 'on', then draw poly.

it is fun coming up with different idea's and getting them to work.

PB2DLL. i will have a look into this. if this can do per pixel z buffer that is good.

well, going about the z buffer culling. i will experiment and see what i can get working. i wonder if shape collision will work also?

this is getting more fun the further i get into it. i had no idea the amount you need to learn though, and the mathmatics involved.
i almost gave up several times, especially getting all the theory and putting it together.

  stevmjon
Title: Re: Building a basic 3D Engine
Post by: stevmjon on February 08, 2017, 07:15:01 PM
hello

another update to v0.3
this version adds polygon clipping when a polygon is part inside & outside camera view.

it is kind of cool to watch the polygons divide when rotating the camera slowly.
so far i have kept the vectors all the way through the transformation process right till the screen pixels. it appears to work, but i may change to a polygon list instead.
i thought vectors will save calc time, but they need to be calced twice anyway(at draw routine), so a polygon list will be bigger, but only need to be calced once.

the next version will have polygon textures added.
i have kept camera rotation to heading (left / right only) because there is not enough objects in the scene, so i don't want view getting 'lost'.

  enjoy, stevmjon
Title: Re: Building a basic 3D Engine
Post by: kevin on February 09, 2017, 08:50:27 AM

Progressing nicely..  The clipping seems to work pretty well,  much like the method used when filling N sided polygons on the Amiga with the blitter.  Generally all i do is compute region codes for the points then, either reject and pass hrough to render routines.   The compares are a drop in the ocean to the rest of the rendering loops..     

Title: Re: Building a basic 3D Engine
Post by: stevmjon on February 21, 2017, 03:14:58 AM
hello,

i am excited to put up another update to v0.4
this version adds textures to non-clipped polygons. i also modified the clipping routine too. added some sky as well.

the further i go, the more interesting the process gets. it was hard getting started and learning the theory, but once you get started it flows from there.
i have added textures to the non-clipped polygons so far, as it was easier to begin with, and will add textures to the clipped polygons next update.
also, the clipping routine needed to be modified from the earlier version, as it wasn't clipping properly on some angles and when the polygons were half behind camera (all fixed now).
i also added partial sky to see how it looks. seems to be o.k. at the moment.
it is nice to see some colour on the screen too.

  enjoy, stevmjon


Title: Re: Building a basic 3D Engine
Post by: kevin on February 21, 2017, 11:43:00 PM

Very cool.. keep it up !
Title: Re: Building a basic 3D Engine
Post by: kevin on March 16, 2017, 01:12:47 AM

was looking at this today and couldn't but think of you :)

Title: Re: Building a basic 3D Engine
Post by: stevmjon on March 17, 2017, 11:25:32 PM
thanks kev, i can't wait to get to that level in the video (where everything works).

i am just posting a pic to let you all know i am still working on the 3D engine. i am trying to figure out a method of getting more accurate textures (see pic).
quad's work well because they can stretch to the shape of the polygon comparing both sides of the U & V.
tri's can't compare like quads, so have limited stretching.

i am not sure the best method to get around this?
whether to add more polygons to each surface when it is close to the camera,
or redraw the tri-texture taking the z into account using CopyStripH or CopyStripV to re-draw the polygon texture with perspective.

i would of had this done faster, but i have been playing "oxygen not included". it is a fun and addictive game. i got the early alpha release from steam.

  stevmjon

Title: Re: Building a basic 3D Engine
Post by: kevin on March 19, 2017, 05:18:32 AM
 well.. both  textured TRI and QUAD are linear mapping.  LInear is fine when all Z's are the same but will appear to warp in projected scene like this.  3D engines  (with 6DOF) don't use linear texture mapping, they use perspective texture mapping approximations.  Old hardware like the PSX for example only have linear mapping, so they chop the faces down to smaller chunks to hold better perspective.

In a scene, one idea would be to subidivide take objects that are closest the camera more than those far away..  The small faces will wrap but it's noticable..  The faces get cut up by some required level of detail..   think there's and only spinning cube demo in the source code board that does something like this..

You could always write a perspective texture mapper. and run it through PB2DLL .. which is what I assumed you were doing to do originally   
Title: Re: Building a basic 3D Engine
Post by: stevmjon on March 21, 2017, 01:37:43 AM
i found the demo code of the spinning cube.
i found it in the projectsv164L learning edition folder. > projects > demos > 3d_textured_cube.

i see you divided the polygon by averaging the outer points giving an extra center point, then averaging the outer with center point giving extra again, then applied this same calculation to the texture UV map. works well.

it is great getting idea's from other people. always helps to expand your knowledge.
just wondering, where did you get your 3D knowledge from? is there some recommended websites or books?

  thanks, stevmjon

Title: Re: Building a basic 3D Engine
Post by: kevin on March 24, 2017, 07:14:38 AM
  
Quotewhere did you get your 3D knowledge from? is there some recommended websites or books?

  we did some stuff way back in high schools days..  but by the time It came time use it in the early 90's that was really useless.   There wasn't really any books that i had access to initially,  just me and my amiga and lots of head banging.   The stuff in the PC scene was driving what people were interested in the Amiga scene at the time.  

  around the end of the Id's Quake release perspective texture mapping was becoming the new big thing and there was series in the a programming mag about it.  I can't recall the name kf the mag, but the guy was Chris Hecker (so similar)..  Today there's lots of tidbits all over the place as people try and get retro systems to do things nobody imagined was even possible let alone on some 20 plus year old system

 A Walk Down Kev's Amiga History Lane (http://www.underwaredesign.com/forums/index.php?topic=3801.0)
Title: Re: Building a basic 3D Engine
Post by: kevin on April 09, 2017, 12:20:36 AM


Steve,

     dragged some super old code out and made a version  for PB that supports Z buffered rendering at least..  if nothing more it might be handy as reference.  Actually runs OK in PBV1.65, but we are talking about plotting pixels here so it'll never be extreme in plain PB.. 

Title: Re: Building a basic 3D Engine
Post by: stevmjon on April 10, 2017, 09:08:52 AM
cool kev, i would like to see the code for reference.

i have looked at a few videos about this, and it seems you need to be able to code in parallel for the graphics card, for this to work effectively, so that several pixels are calculated at the same time. has playbasic got commands to do this?

   thanks, stevmjon
Title: Re: Building a basic 3D Engine
Post by: kevin on April 10, 2017, 09:47:11 AM
Quote
i have looked at a few videos about this, and it seems you need to be able to code in parallel for the graphics card, for this to work effectively, so that several pixels are calculated at the same time. has playbasic got commands to do this?

 They're most likely talkings about shaders..  Old hardware renders are fixed function pipeline,  which is what it sounds like, the old device has a chunk of logic to render a triangle (or a DOT/LINE etc etc or each surface stype).   The issue with those is the more effects they need to jam in the bigger foot print of the GPU..

  So they move to the shaders about DX9..  These are user defined code code fragments that run on the GPU.  They used to be very limited in terms of size and speed..  but today high end GPU's are like like a rats nest of shader units..  So  shader code runs in parallel across as many many shader units at time as possible. So rather doing one texel at time they can do many.

 You could do shader via OpenGL ..    In other words do all your scene and just render meshes...  Which is how GL apps work anyway..  


  [plink] G2D - 2D OpenGL library for PlayBASIC V1.64P (http://www.underwaredesign.com/forums/index.php?topic=4210.0)[/plink]

 
Title: Re: Building a basic 3D Engine
Post by: kevin on April 11, 2017, 09:32:08 PM
 Z buffered gouraud shading in PB V1.65 runtime  :  get Source Code (http://www.underwaredesign.com/forums/index.php?topic=4382.0)

Title: Re: Building a basic 3D Engine
Post by: stevmjon on April 12, 2017, 02:47:50 AM
that looks good kev.

can you post the code?

just a question, i am not too sure about part 3 in my pic. it seems you need to squish the frustum down the z to form a cube. that i understand, but the z values are too high and bunched up(0.8 to 1.0). only the points very close to the camera get lower values. not sure if you are supposed to center part 2 in the 3D world then do part 3?

NOTE: the formulas in the pic are only there as reference. i don't expect you to calculate them (i know you get busy). just wondering if my theory is correct.

thanks,  stevmjon






Title: Re: Building a basic 3D Engine
Post by: kevin on April 12, 2017, 04:50:58 AM

  The code was posted to the source code (http://www.underwaredesign.com/forums/index.php?topic=4382.0) board ..  :)
Title: Re: Building a basic 3D Engine
Post by: kevin on August 30, 2017, 08:11:35 AM

Here's some OLD examples you might find useful,


  - Direct X (ASCII) loader (http://www.underwaredesign.com/forums/index.php?topic=2176.0)
  - ASC II 3D Object Loader / Viewer V0.02 (http://www.underwaredesign.com/forums/index.php?topic=4062.0)


Title: Re: Building a basic 3D Engine
Post by: stevmjon on September 02, 2017, 02:42:00 AM
thanks kev, you read my mind.
i was thinking about loading .obj files in soon. i can make them with 3D software i have(Lightwave 3D).

i am tinkering with the UV maps at the moment. i have re-made my clipping routines with a more efficient one(i am clipping in two locations), and i am just getting the UV maps to match the clip, then i will subdivide the closer polygons for a better looking texture. i am also reading up on perspective correct texture routines too.

i hope to release another update very soon.

stevmjon

Title: Re: Building a basic 3D Engine
Post by: kevin on December 11, 2017, 08:32:29 AM
 Here's a site with some 3D game model rips

https://www.models-resource.com/


something like StarFox64 could be fun

https://www.models-resource.com/nintendo_64/starfox64/


Could prolly do somthing like the SNES version at least.






Title: Re: Building a basic 3D Engine
Post by: stevmjon on January 08, 2018, 04:26:42 AM
hello

   about time i put up another update. this one has all polygons textured, and i have modified the clipping routine to clip in 2 locations in the game engine.

1> clip early in the camera space by clipping everything behind the near border (this gets rid of everything behind the camera).
2> then clip later in the perspective space (frustum transformed into a cube) by clipping on the remaining borders because here is a faster clipping operation.

   the reason for this split gives the advantage of 1 clip done early in the engine with 1 efficient border check, and the remainder of the clips done in the more efficient perspective cube stage, preventing the need to interrupt the matrix calculations at this perspective stage if all borders were checked here.
    this way all the clips are done on frustum borders that are perpendicular to an axis, so eliminating the need for dot product calculations in the border checks. all i need to do is simply check the position of a point/vector and compare this with the border location. eg. is X<-1(left border) , is X>1(right border) etc. even the camera space near border is perpendicular to the z axis so is just a location check too(eg. is Z<0.1(near border).

i have applied 1 formula for all polygon clipping routines regardless of where in the game engine, and also applied this to the clipped UV texture too. the UV's seems to work well with clipped quad polygons, but i don't think the UV is completely accurate with clipped tri polygons(mainly on the inside edge). i may have play around with this a little more.
i want to work out this UV clip before i move on to subdividing poly's etc.

Version 0.5.2

   enjoy this early update, stevmjon

Title: Re: Building a basic 3D Engine
Post by: stevmjon on January 08, 2018, 04:35:02 AM
here is a pic for the above post. it wouldn't let me post both the image and the pic, saying over maximum allowed size. i thought each attachment was up to 3,500k each? it must be a total of 3,500k for all attachments combined size?

pic for above post v0.5.2

Title: Re: Building a basic 3D Engine
Post by: stevmjon on April 07, 2018, 09:00:02 PM
yay, another update.

this update has subdivided polygons and subdivided textures, depending on where the polygon is located from the camera.

> polygons that are clipped, get the texture subdivided first, then clipped to give smoother textures.
> polygons that are whole, but very close to the camera, get the polygon subdivided to give smoother textures.
> polygons that are whole, but a little further away from the camera get the texture subdivided the give smoother textures.

so now when you look around the world, the polygon textures look smooth, especially beneath the camera.
this took me quite some time to experiment with and get right. it is still not perfect, but looks better anyway.

this one can be called the experimental update:
* press " i " key  =  change the texture (total 4)
* press " L " key  =  show sub-divide / clipping mode (total 6)
* press " R Arrow " key  =  turn ON outline all polygons _ press "L Arrow" key = turn OFF outline

there are plenty of other key presses you can do by looking at the text on the bottom of the screen for the letter in brackets next to the name.
well, i hope you like this, it took a while to do with all the experimenting, so the code is pretty big. i will probably reduce this down, now i know the technique i like (subdivide mode = 6).
i did a lot of copy / paste then edit of functions, and spent a lot of time in debug mode. it would probably have been quicker to re-write code instead.

NOTE: the only bug i can obviously see is small gaps between the border of subdivided polygons next to whole polygons.

the next update i will work on a render method.

Version 0.6.1

   enjoy, stevmjon

Title: Re: Building a basic 3D Engine
Post by: kevin on April 08, 2018, 10:46:32 AM

looking really cool Steve.. 
Title: Re: Building a basic 3D Engine
Post by: stevmjon on May 08, 2018, 06:03:51 AM
howdy

in this update i have added a light source (sun) so the textures will change depending on the angle they are from the sun.
so, it's not incidence angle from the camera to sun glancing off the object surface, but it still looks cool anyway.

also added merge sort method for the draw order of polygons (suits artists render method). it is not accurate when polygons are very close, or overlap.
still, i had fun calculating this. i may tweak this a little more though.

NOTE: the only bug i can obviously see is small gaps between the border of subdivided polygons next to whole polygons.

Version 0.6.2

   enjoy, stevmjon
Title: Re: Building a basic 3D Engine
Post by: stevmjon on July 02, 2018, 10:38:08 PM
hello

update time again.
> added 3D textures (manually drawn).

the textures look great, with all lines in the texture nice and straight, finally.
since each pixel is manually calculated for 3D, it runs a lot slower.
i should write new code to make it more optimised, as there is now 12 different draw modes. it's what you get when your learning and experimenting at the same time.

next i would like to work on loading objects into the engine, and also adding collision for the player (so you can't walk through walls etc).

Version 0.7.1

   enjoy, stevmjon
Title: Re: Building a basic 3D Engine
Post by: kevin on July 04, 2018, 11:07:46 AM
 Steve,

 wow, that's a really impressive demo... mind blown !  

 yeah, It could well benefit from some refactoring, like everything else,  but it's interesting picking through it to see how your adventure is unfolding, just wish I had more time to play with it    


  Note: Link to demo on the previous page (http://www.underwaredesign.com/forums/index.php?topic=4365.msg29631#msg29631)

Title: Re: Building a basic 3D Engine
Post by: kevin on August 17, 2018, 12:07:06 PM

ran into this old StarGlider video and couldn't help but throw out this hint !



Title: Re: Building a basic 3D Engine
Post by: stevmjon on August 23, 2018, 08:06:18 PM
questions and explanations about v0.7.1

*** to see mode look at text at bottom of screen > " subdivide= mode/12  (L) "

mode 1:
> standard 2D  :  uses 2D polygon commands

mode 2 - 6:
> experiment with different subdivide methods to improve the look of 2D textures, and drawing using 2D polygon commands
> mode 6 added polygon z sorting using merge sort routine (sorts furthest polygon to closest polygon in order : artists method)
> mode 6 added sun light correctly

mode 7:
> (start preparing for 3D textures by drawing polygons myself)
> manually draw 2D texture only
> use fastdot to draw each pixel of the scanline for each individual polygon
> this just used random color to test routine

mode 8:
> manually draw 2D texture only
> use texturstrip to read from the polygon image to draw whole scanline for each polygon using a command

mode 9:
> manually draw 2D texture only
> use peek & fastdot to manually read the polygon image myself, and manually draw each pixel myself
> this is because i wanted to prepare for the interperlate routine which needs to calculate each individual pixel for a 3D perspective correct texture

mode 10:
> manually draw 2D texture only
> test z buffer method for a more accurate z depth test (this is more accurate than the artists z sort method)

mode 11:
> manually draw 3D texture
> add interperlate 3D method for perspective correct texture, now i have individual texel checking for the polygon texture that was set-up from mode 9
> try with artists z sort method (to compare with next mode)

mode 12:
> manually draw 3D texture
> still interperlated for 3D texture
> try with z buffer sort method (to test speed with max loading, using 3D interperlate calculation combined with z buffer read/write)


QUESTIONS:
i am using type arrays for the polygon points data.
* is it better to use standard array instead?
you mentioned a change in 1.64 to 1.65 with typed arrays in the video you recorded showcasing this demo.

for the z buffer sorting, i am using standard array in memory that is same size as screen in pixels.
* should i use image that is screen sized instead?
i noticed you did this in one of your demo's, UW3D-RENDER-ENGINE using a library #Include "SaveBitMap".
* is writing to or reading from a screen sized image faster?

i am asking these because i am going to re-write the code using just 3D textures, so i want to use any faster methods where possible.
if anybody has tips to improve my code, please comment, and i will look into it.

   thanks stevmjon


p.s. i looked at the video of starglider, looks interesting kev.
Title: Re: Building a basic 3D Engine
Post by: kevin on August 24, 2018, 11:13:11 AM
Quote
i am using type arrays for the polygon points data.
* is it better to use standard array instead?
you mentioned a change in 1.64 to 1.65 with typed arrays in the video you recorded showcasing this demo.

 164 / 165 are very different.    165 is a complete rewrite / re imaging of how the internal instruction sets work.   So your code is broken down into a sequence of instructions (serialized)  that the VM executes.  

 The only main thing that 1.65 doesn't have, is  type caching,  which is a set of instructions that handle short cutting accesses of the same structure from within the same expression or across various lines, which could be stuff like typed arrays or variable / list.    This allows the code to be turned into raw memory reads/writes when the same structure is being accessed,  where normal read/writes have nanny code in them to project you from yourself, without them you wouldn't get runtime errors, just crashes.

 There's a whole section in the manual on this under the optexpressions and in the various development blogs .  PlayBASIC V1.64N2 & V1.64N3  (Work In Progress) Gallery (https://www.underwaredesign.com/forums/index.php?topic=3833.0)
 

 To answer your question, you should be using Typed pointers.    You allocate some  memory and just point at it,  type caching effective turns array accesses into typed pointer accesses..  

   This code uses a typed pointer to access the data within an floating point array, effectively treating it as  buffer..   You could write to anything you can get a porinter from..  Array / Bank / Image whatever
[pbcode]



      Type Vertex
               x#
               Y#
               z#
      EndType
   
      print SizeOf(Vertex)   


      NumberOfVertex = 10

      // Using an array ass a BANK
      
      Dim Buffer#( (SizeOf(Vertex)/4) * NumberOfVertex )

      Dim P as Vertex Pointer
      P  = GetArrayPtr(Buffer#(),true)

      for lp=0 to NumberOfVertex-1

            // Write some data to memory
            P.X = 100 +lp
            P.Y = 200 +lp
            P.Z = 300 +lp

            // Move to the next vertex
            P=int(p)+SizeOf(Vertex)

      next


      for lp =0 to NumberOfVertex-1
            Index = Lp * (SizeOF(Vertex)/4)
            print "XYZ"   
            print Buffer#(Index)
            print Buffer#(Index+1)
            print Buffer#(Index+2)
         
      next


      sync
      waitkey
      
[/pbcode]



Quote
for the z buffer sorting, i am using standard array in memory that is same size as screen in pixels.
* should i use image that is screen sized instead?
i noticed you did this in one of your demo's, UW3D-RENDER-ENGINE using a library #Include "SaveBitMap".

 In save bitmap it creates a strip the size of the source image and that's one pixel high.   This strip is forced to be 32bit, so what its doing is pixel colour depth conversion using the Pb graphic library instead of trying to manually do this stuff in PB code.    once the strip is copied from the source image, it grabs a pointer to the strip image (which is known to be 32bit) and dump that directly to disc.   Avoiding the whole pixel  by pixel thing so it's basically batching it.


Quote
* is writing to or reading from a screen sized image faster?

   Memory access in your computer rewards near or localize accesses, this means the CPU is writing/reading from cached memory and now actual memory.    So having all the data your operating upon in one place is faster than having it spread all over the place.   In particular if you want to convert it to machine code, where you'll run head first into this type of problem.  

   What you could do is store the depths and colours together, so make z buffer image twice as wide, then even align cords are Z's and ODD's a re pixels

  [pbcode]


   Type zPixel
               Depth#
               Colour
         EndType

         Width=GetScreenWidth()
         Height=GetScreenHeight()


         Surface = MakeZscreen(GetScreenWidth(),GetScreenHeight())
         
         RenderToImage Surface
         LockBuffer
         ThisRgb=Point(0,0)
         For Ylp=0 to height-1
            
            Dim P as zPixel Pointer
            
            ; init pointer toi the start of this row   
            P = GetImagePtr(Surface) + Ylp * GetImagePitch(Surface)

            For Xlp=0 to Width-1         
                  P.Depth  = 10000.0/Xlp
                  P.Colour = Xlp*YLP
                  // Move to the next pixel along this row
                  P = int(p) + SizeOf(ZPixel)
            next      
         Next
         unLockBuffer
         rendertoscreen
         drawimage Surface,0,0,false
         sync
         waitkey




Function MakeZscreen(Width,Height)
         
         ThisSurface=GetFreeimage()

         ;compute width the number of 32bit pixels (4 bytes per pixel)
         ActualWidth = Width * (SizeOf(zPixel) / 4)

         CreateFXImageex ThisSurface,ActualWidth,Height,32
   
EndFunction ThisSurface

 [/pbcode]





  Ran into another classic VIRUS on the ARCH



Title: Re: Building a basic 3D Engine
Post by: stevmjon on November 01, 2018, 12:48:32 AM
this update converts/reduces the code to display 3D textures only.

V0.7.3

thanks for the info on pointers in the above post kev. the second code snippet displays an interesting image on screen.
i will impliment this later, as i need to change a fair bit of code.

;--------------------------------------------------------------
i have been experimenting with data for a faster display:

Draw > line 211 > array() = PeekInt
Draw > line 214 > PokeInt dest , PeekInt()
Draw > line 215 > CopyMemory source , dest , size

*only have any 1 above line active at a time, and these all run at the same speed.

;-----------------------------------------------------------------------
i also experimented with drawing the screen for a faster display:

Draw > line 303 > FastDot x , y , col
Draw > line 329 > CopyMemory source , dest , size

*change line 296 set = 1 , or set = 2 to test, no speed difference.

;------------------------------------------------------------------------

i have noticed that if i disable the interperlating calculation , comment out Main > line 175 , the game runs at normal speed (set to be 50 fps) but no polygons displayed.
to interperlate, each pixel gets many floating point calculations as the polygon scanline gets calculated across the polygon surface. i think i may need to calc smaller floats?

any comments are welcome.

  stevmjon
Title: Re: Building a basic 3D Engine
Post by: kevin on December 07, 2018, 09:22:50 AM
 
 Hi Steve,

      Sorry for my late reply..     Just fired this up on my laptop  and it's amazing to me that even classic runtime is quick enough to even get this in motion,  there's so much work to chew through per pixel...  wow...  the code is generally excellent too..  



      In your conversion routine,

[pbcode]

    For lp=0 To max
        col = z_buffer4_rgb(lp)
        If col>=0
           FastDot x,y,col  ;  **slightly slower**
        EndIf
        
        Inc x
        If x>=width Then x=0  :  Inc y
               
     Next lp


[/pbcode]

    Should be better expressed as a nested loop,  that will make the cost per pixel lower.   Lowest cost per pixel ='s faster.



[pbcode]

    For ylp=0 To height-1
        For xlp=0 To Width-1
        col = z_buffer4_rgb(RowIndex+Xlp)
        If col=0
           FastDot xlp,ylp,col  ;  **slightly slower**
        EndIf
      Next xlp

            RowIndex += Width
     Next ylp


[/pbcode]

    This should give you a 6 instruction (at worst) inner loop..   (6 PB VM instructions that is)


     You can loop through the rows which would get rid one add pixel

[pbcode]
        RowStart=0
        RowEnd = Width-1

    For ylp=0 To height-1
        For xlp=RowStart To RowEnd
        col = z_buffer4_rgb(Xlp)
        If col=0
           FastDot xlp,ylp,col  ;  **slightly slower**
        EndIf
      Next xlp
                RowStart += Width
                RowEnd += Width
     Next ylp

[/pbcode]

   
  EDIT:  - That's not going to work with FastPoint as the XLP isn't a coordinate.   I guess you could poke it, you'd still need to add the offset, but the addition would be inside IF/ENDIF.    So pixels that are transparent cost less to draw than solid pixels




  Now your other loop is copying the buffer to the target via copy memory,  which of course is the quickestt option as there's not VM overhead and it's just pure machine code .. So the bugger the chuck the better the reward.


   What you could do, which I think is safer (works on all display depths) and would allow you to mask the src image to the destination...  Would be much like I mentioned above, where I create a temp image, where you copy the rows to the temp buffer which is forced to be 32bit regardless of the current machines display depth,  then we jusr draw that image to the target.   This way you shift the work load from pixel by pixel through the PBVM to machine code routines.  So the cost per pixel is tiny in comparison.



[pbcode]


      Width= 800
      Height=600

      Dim ScreenArray(Width*Height)
   
      Do
         cls 255
         
         WidthByHeightMinus1 =Width*Height-1
         For lp=0 to rnd(50)
               
               Index1 = floor(Rnd#(WidthByHeightMinus1))
               
               Index2 = ClipRange(Index1+100,0 ,WidthByHeightMinus1)
               
               For Index = Index1 To index2
                  ScreenArray(index) = rndrgb()
               next
         next
         
         
         if (frames and $ff)=0
            
               ClearArray ScreenArray(),0
         Endif
         
         RenderScreen(Width,Height)
         print frames
         frames++
         
         sync   
      loop



Function RenderScreen(Width,Height)
   
   // This has no clipping

   local ThisSurface= GetSurface()
   
   
   static  Strip32BitImage
   if Strip32BitImage=0
      
            Strip32BitImage= GetFreeimage()
            CreateFxImageEx Strip32BitImage,Width,Height, 32
   endif
   
   
   local  SrcPtr = GetArrayPtr(ScreenArray(),true)
   local DestPtr = GetImagePtr(Strip32BitImage)
   
   
   WidthBy4 = Width*4
   
   For Ylp=0 to Height-1
   
      CopyMemory SrcPtr,DestPtr,WidthBy4   
   
      SrcPtr += (WidthBy4)   
      DestPtr += (WidthBy4)   

   next

   rendertoimage ThisSurface
   lockbuffer
      drawimage Strip32BitImage,0,0,true
   unlockbuffer
   
EndFunction


[/pbcode]


    I'll post a copy of this in the source board also..  as things tend to get lost inside big threads..






Title: Re: Building a basic 3D Engine
Post by: stevmjon on December 09, 2018, 12:53:48 AM
thanks for the tips kev

my program is running 20% faster now. i used the ex image like a mask layer, to only draw the foreground (visible polygons).
the background image (sky) is seperate, and is drawn to screen first, as a sort of clear old data from backbuffer.

the issue is still the calculating the perspective correct interperlating of each pixel in each polygon. 32 bit floats take time to calculate as there are lots of them.
i need to interperlate the start slope point and end slope point, along polygon edges, then interperlate between these 2 points for perspective correct texel from uv map.
i wonder if you could modify the TextureStripH command to add z component? instead of the linear calc it does to get texel, an extra z component interperlates texel ...?  when i used this command in v0.7.1 it ran fast (no perspective interperlate along polygon edges though, just linear steps along slope).

*have you thought of adding command Float / Double ?
i only need 3 or 6 decimal places after the point for the calculations. this may speed things up too?

anyway, i will keep experimenting. i was thinking of making an .obj loader, so i can make objects with my 3D software (lightwave 3D) and bring these into the game. i need to add collisions too, so player can't just walk through everything. even though playbasic is a 2D language, i still enjoy doing 3D in it.

   stevmjon
Title: Re: Building a basic 3D Engine
Post by: kevin on December 09, 2018, 06:07:48 AM
 Steve,

Quote
my program is running 20% faster now. i used the ex image like a mask layer, to only draw the foreground (visible polygons).
the background image (sky) is seperate, and is drawn to screen first, as a sort of clear old data from backbuffer.

 That's good news..  


Quote
the issue is still the calculating the perspective correct interperlating of each pixel in each polygon. 32 bit floats take time to calculate as there are lots of them.

 I know :)


Quote
i need to interperlate the start slope point and end slope point, along polygon edges, then interperlate between these 2 points for perspective correct texel from uv map.
i wonder if you could modify the TextureStripH command to add z component? instead of the linear calc it does to get texel, an extra z component interperlates texel ...?  when i used this command in v0.7.1 it ran fast (no perspective interperlate along polygon edges though, just linear steps along slope).


  Yeah what's normally done is you only compute perspectively correct UV's at fixed intervals like 8 pixels.   So the divides can be wrapped around the integer inner loops,  which are pure linear interpolations.   To do they means writing the loop in assembly as the floating point divide needs to interleaved with integer operations as to not stall execution.  Clearly this is not something a runtime can do or even modern high level C/C++ compilers struggle with such concepts.   None the less, in term of whats built into PB today you could use draw strip to render blocks of 8 or 16 pixels linearly, but you'd have to step the z yourself I guess.  Which should be more optimal than raw brute force per pixel approach.




[pbcode]

           ;--------------------------------------------------
            ;--- setup array data for interperlate scanline ---
            ;--------------------------------------------------
            For lps=0 To distance
               
               t# = lps/distance#  :  t2# = 1-t#
               
               ;--- 3D texel u & v ---
               topu# = t2#*a1# + t#*b1#
               bottu# = t2#*c1# + t#*d1#
               pixelU# = topu# / bottu#  ;  3D U (texel)
               
               topv# = t2#*a2# + t#*b2#
               bottv# = t2#*c1# + t#*d1#
               pixelV# = topv# / bottv#  ;  3D V (texel)
               
               ;--- calc memory address for texel in array ---
               pixelV = pixelV#  :  pixelU = pixelU#
               pixel = address + pixelV*pitch + pixelU*4  ;  integer = 4 bytes
               ;NOTE: polygon scanline can range between 1 pixel to screen width
               
               If sub=10
                  ;-----------------------------------------
                  ;--- draw directly to screen  (option) ---
                  ;-----------------------------------------
                  
                 ;scanline(lps) = PeekInt(pixel)  ;  faster*
                 col = PeekInt(pixel)  :  x=interpl+lps
                 FastDot x,scanY,col  ;  **slower than copy memory**
                 ; NOTE: get data directly from memory then draw to screen
                 
              Else
                 
                 ;---------------------------------------------------------------------------------
                 ;--- save data in z buffer array  (draw to screen in draw_z_buffer() Function) ---
                 ;---------------------------------------------------------------------------------
                 
                  ;;col = PeekInt(pixel)
                 arraypos = scany * width + interpl+lps
                 z_buffer4_rgb(arraypos) = PeekInt(pixel)  ;  **slightly slower than above set, but more accurate z sort**
                 ; NOTE: get data directly from memory then save in Z buffer array
                 
                 ;PokeInt dest_address3+arraypos*4,PeekInt(pixel)  ;  no faster here*
                 ;CopyMemory pixel,dest_address3+arraypos*4,4  ;  no faster here*
                 ; NOTE: get data directly from memory then write to destination directly to memory
                 
               EndIf
              
            Next lps



[/pbcode]


      There's a huge amount of work per pixel, a lot of Mult + Divides,  even in the VM the cost a mult and divide is high as there's a stall waiting for the FPu to finish the computation.   To make CPU's quicker they like to stack opcodes and execute them in 2 pipes.  So instructions are executed out of order when they're not dependent.   Which is cool in assembly programming where you can manually lay out every opcode to try and get it all to pair and miss as many potential stalls as possible.   But in a runtime we can't really do this, or at least I visualize a way to do this from C

     
      If you want to use test code inside time critical loops, then define some type of MODE constant and use #IF   #ENDIF blocks.   So the code can still be within the function can be removed at compile time depending upon the value of the your mode constant.


[pbcode]

               #If sub=10
                  ;-----------------------------------------
                  ;--- draw directly to screen  (option) ---
                  ;-----------------------------------------
                  
                 ;scanline(lps) = PeekInt(pixel)  ;  faster*
                 col = PeekInt(pixel)  :  x=interpl+lps
                 FastDot x,scanY,col  ;  **slower than copy memory**
                 ; NOTE: get data directly from memory then draw to screen
                 
             #Else
                 
                 ;---------------------------------------------------------------------------------
                 ;--- save data in z buffer array  (draw to screen in draw_z_buffer() Function) ---
                 ;---------------------------------------------------------------------------------
                 
                  ;;col = PeekInt(pixel)
                 arraypos = scany * width + interpl+lps
                 z_buffer4_rgb(arraypos) = PeekInt(pixel)  ;  **slightly slower than above set, but more accurate z sort**
                 ; NOTE: get data directly from memory then save in Z buffer array
                 
                 ;PokeInt dest_address3+arraypos*4,PeekInt(pixel)  ;  no faster here*
                 ;CopyMemory pixel,dest_address3+arraypos*4,4  ;  no faster here*
                 ; NOTE: get data directly from memory then write to destination directly to memory
                 
              #EndIf

[/pbcode]

           Depend upon the version of PB, this saves a couple of instructions per pixel, which really adds up of course.     I've got some optimization ideas on the to do list, that might help your inner loops here, but I suspect your best results will be found by rearranging the expressions and try and get to less instructions per pixel.   

       
           When reading the texture, what you could do is a type pointer that pointing at the top left pixexl of the texture. 

           So this,
[pbcode]
           pixel = address + pixelV*pitch + pixelU*4 

[/pbcode]

          Becomes
[pbcode]
           pixel = Teature.Pixels(pixelV * pitch + pixelU) 

[/pbcode]

           Which is 2 instructions (at least) per pixel quicker.   Pitch would need to be divided by 4 tho.   




Quote
*have you thought of adding command Float / Double ?
i only need 3 or 6 decimal places after the point for the calculations. this may speed things up too?

  PB is 32bit internally,  adding doubles (64bit floats & integers) is on the agenda but I doubt you'd get faster.   The FPU is 80 bit interally, all floats / integers are imported and recast to 80 bit,  then the operation is done and the result it's recast back to the target accuracy.   64 gives a wider accuracy but it'd twice the memory bandwidth..    


Quote
anyway, i will keep experimenting. i was thinking of making an .obj loader, so i can make objects with my 3D software (lightwave 3D) and bring these into the game. i need to add collisions too, so player can't just walk through everything. even though playbasic is a 2D language, i still enjoy doing 3D in it.

  Ahh yeah.. I  wrote an OBJ loader way back in the Amiga days,  but I doubt the formats are the same today.  I think that code was even in Amos ..   Will have a look later on my desk top.  


Title: Re: Building a basic 3D Engine
Post by: stevmjon on December 10, 2018, 08:41:04 PM
thanks kev

after the imageEX tip, and the perspective calc every 8 pixels tip, now i get double the frame rate i was getting. yay.
still, i would like to get it faster.

for this i was looking at pointers next, like the code clip you posted above ( pixel = Texture.Pixels(pixelV * pitch + pixelU) ).
*i was wondering, can you make a video to explain pointers?

an explanation about how pointers work, compared to standard array access would be great. like, how does an array access data, compared to how a pointer access the same data. i assume standard array access must have overhead (costs time), but pointer gets around this???
what is the difference between pointer and type pointer? some info about how these are stored/accessed in memory would be great.
i haven't really used pointers/type pointers as i don't fully understand how to use them properly.

any advice would be great, stevmjon
Title: Re: Building a basic 3D Engine
Post by: kevin on December 11, 2018, 10:39:08 AM

    if you can wrap your head around peek & poke, then pointers are the same thing.   Primitive pointers such as Byte/Word/Integer/Float are for reading/write that data type.   Typed pointers allow us to use the type structure as a series of fixed offsets from the pointer.   Think of the pointer as a base address and the fields are like adding an offset to the address your going to poke into.    Eg.    PokeInt RowAddress+(PixelOffset * 4 ) , ThisColour

   From Poke to Pointers (screen fill examples)  (2018-12-11)  (https://www.underwaredesign.com/forums/index.php?topic=4454.0)
Title: Re: Building a basic 3D Engine
Post by: kevin on December 15, 2018, 09:14:56 AM
   UpScale

     What you could is render in %50 of the resolution and then scale that up.   I've used this in a few demos that do mass pixel writing.   So a lot less drawing overhead, mixed with some bilinear filtering on the up scaling..   Depending on the image it can actually look ok


[pbcode]


   PixelScaleX#= 2
   PixelScaleY#= 2   

   Width=getSCreenwidth()/PixelScaleX#
   Height=GetScreenHeight()/PixelScaleY#
   
  Screen = NewImage(Width, Height,2)


   do
            rendertoimage Screen
            
            inkmode 1+64
            for lp =0 to 500
               x1=rnd(width)
               y1=rnd(height)
               x2=rnd(width)
               y2=rnd(height)
               
               linec x1,y1,x2,y2,rndrgb()
   
            next
   
            circle mousex()/PixelScaleX#,Mousey()/PixelScaleY#,20,1
            inkmode 1
   
         refresh Screen
         
loop spacekey()
end   



Function Refresh(VirtualScreen)
   oldsurface=getsurface()
   
   rendertoscreen
   sw=getscreenwidth()
   sh=getscreenheight()
   
   iw=GetimageWidth(VirtualScreen)
   ih=GetimageHeight(VirtualScreen)
   
   ScaleX#=float(sw)/iw   
   ScaleY#=float(sh)/ih   
      
      
   drawRotatedImage   VirtualScreen,0,0,0,ScaleX#,ScaleY#,0,0,false+8   
   
   blurimage VirtualScreen,12.1

   Sync
   
   rendertoimage oldSurface   
   

EndFunction


[/pbcode]



  Also,  Ran into this today and instantly thought of you!   :)  



  Robotron (PS1 Version)

    This is the exact type of thing that comes to mind when doing retro 3D,  simple worlds / objects and no need for 3d physics since it's an arcade shooter,  the object render could do this out of the box.

   






  Tomb Raider (PS1 Version)

    It's amazing looking back at the complexity of the scenes in TR1,  it was big deal back in day to have full screen 3D even at 320*200 resolution in the DOS versions.  The tool sets for content creators and lack of fill rates really push the designers to a result ...   but it's all linear mapping on Playstation versions.   They really did some interest stuff on that system over the years. 

 

Title: Re: Building a basic 3D Engine
Post by: stevmjon on December 15, 2018, 05:39:37 PM
using TextureStripH to fill in the scanline chunks (meaning faster)

V0.7.6

this version runs at a much better speed now, compared to the older version.
if you launched the game and didn't move the mouse:
> v0.7.3 = 5 fps        (use z buffer array & calc every interporlated polygon pixel)
> v0.7.4 = 10 fps        (use z buffer array & calc every 8th interperlated polygon pixel and linear interperlate between, and draw to EX screensize image) * thanks kev for this tip
> v0.7.6 = 25 fps        (calc every 64 interperlated polygon pixel and use TextureStripH to interperlate between, and draw to EX screensize image)

i am excited about this increase in speed. the only setback is i am no longer using the z buffer array in v0.7.6, so i can't get pixel perfect polygon overlap accuracy, but i do get polygon z sorting with back to front drawing order.

upscale demo kev, this looks good too. i will check it out.

* press i key to change the texture on the polygons, to see grid lines (total 4 textures)

    enjoy stevmjon
Title: Re: Building a basic 3D Engine
Post by: kevin on April 03, 2019, 07:50:54 AM
  ran in this article today...  interesting

 Pushing Polygons on the Mega Drive (https://jix.one/pushing-polygons-on-the-mega-drive/)




 
Title: Re: Building a basic 3D Engine
Post by: stevmjon on April 04, 2019, 12:05:38 AM
hey kev
the above demo is interesting. left to right drawing without overdraw.

how my 3D engine v0.7.6 is currently rendering, i do get overdraw of pixels. i did this because i used TextureStripH to speed up interperlating textures for the polygons. if i don't use this i get a slow framerate.

funny, i was thinking about how to reduce overdraw lately, so now i have more homework...

at the moment i have been writing collision detections in the 3D engine. 2D collisions are easier to setup, but 3D are more work, but i am getting there.
detecting collisions is the easier part, it is what to do next that is more involved.
eg. if a player walked into a wall, but was facing slightly left/right, you need to be able to push the player back away from the wall AND allow the move left/right movement also, so the player will slide along the wall. i came up with an idea for this and am in the final testing at the moment. can't wait to post the next demo.

  stevmjon
Title: Re: Building a basic 3D Engine
Post by: stevmjon on April 11, 2019, 11:08:39 PM
hello

i just worked out the collision routine, and thought i would share how i done it (see pic).
i used Seperated Axis Theorem (SAT) to detect the collision, then i came up with my own routine to respond to the collision.
it took me several attempts to come up with this, to work in the way i wanted. i wanted the player to 'slide' along walls, rather than just stop.

in game, the player can now collide with objects and the ground, and even jump on top of objects and use them to get higher up.
i am currently adding in gravity, so player falls off edges etc, and i am modifying the scene to have fun with this.
i am also adding some more graphics to have some variety. must keep it under 5 meg to upload on the forums though.

i will post a new scene as soon as this is done.

   stevmjon
Title: Re: Building a basic 3D Engine
Post by: stevmjon on April 20, 2019, 02:45:00 AM
howdy and happy easter

time to release a short demo, to test out my collision routines.

V0.8.7

i used the Seperated Axis Therom (SAT) collision detection, and made my own collision routine as the response to any collision (see above post).
i made a small demo world to test this routine, against different sized objects, and different slopes too. the reason the world size is small, is to keep frames playable, as manually calculating 3D textures slows down frames per second.

> goal is to reach the wood block on the other side of the trench....

i made it a challenge, so i hope there is no rage quitting. if you get there "well done".

  enjoy  stevmjon
Title: Re: Building a basic 3D Engine
Post by: kevin on April 20, 2019, 12:33:02 PM
 Hey Steve

    Just waiting for the some compo's at revision 2019 (https://2019.revision-party.net/live#stream_seminar) and taking a look at the new demo.   Visually it's really looking nice and clean and the collisions seem to work well.


 
Quotei made a small demo world to test this routine, against different sized objects, and different slopes too. the reason the world size is small, is to keep frames playable, as manually calculating 3D textures slows down frames per second.

   Yeah the performance is slowing down in this version, so i've been having a look at the rendering there's some wasted calc's in the inner loops,  it might not seem like a big deal but any redundancy per scan line is then multiplied across the polygon and grows almost exponential as the scene gets more dense.   Imagine something as simple as an extra subtraction on a 100 lines high polygon, well this single subtraction is chewing through at least 25 cpu cycles to execute from the PB vm.    Put a scene full of only a few hundred faces and that's a lot of redundancy for one operation.    


   In other places the convenience of an array is just not worth it, for example it's better to grab the matrix as local variables that apply those to all verts in the like than read a 2d array 12 times per vertex.   Reading an array probably costs a hundred or more cpu cycles.   


[pbcode]

Function calc_matrix_4x4(array().tVertex2 , w)
   
   ; general matrix:
   ; NOTE: a-i = vectors  _  l-n = perspective  _  p-r = transform  _  s = scaler
   ;  x  y  z  w
   ; [a, b, c, p]  
Title: Re: Building a basic 3D Engine
Post by: stevmjon on April 21, 2019, 02:01:27 AM
thanks for the tips kev. i have applied what you showed in the code snippet above. already gained a few frames.

the main part that slows the game down is the 3D texture routine (see Draw.pba). i have used as many variables here as i can to break down longer calcs into smaller ones. i even put in TextureStripH into the interperlate polygon line routine, broken into 64 pixel lengths to gain speed. before i added this the game ran much slower.
if you look at the commented out sections, you can see i have done a lot of experimenting in the interperlate routine.

as you say above, maybe i can optimise this still further, by eliminating some calculations, if i can re-use values instead of calc new ones maybe?

i am glad you like this demo, and the collisions are working fine.

i clicked the revision party link in above post, and i have been listening to the music all afternoon while coding. relaxing.

   stevmjon
Title: Re: Building a basic 3D Engine
Post by: kevin on April 27, 2019, 12:55:36 PM
 Steve.

  Here's my tweak of V087  of your 3d engine.   Long story short most of the overhead was in legacy locking code from some other variation of the draw routine..  I tweak the rendering loop so it's a bit simpler.  These does seem like a good case for a render strip that remembers the last cord / U/V and you only supply the next along this row.



Video:

 





 Download:
Title: Re: Building a basic 3D Engine
Post by: kevin on April 28, 2019, 12:30:59 PM
 Made a video gallery of this ..  fantastic progress



Title: Re: Building a basic 3D Engine
Post by: stevmjon on April 29, 2019, 02:14:45 AM
thanks for the tips kev. i got a good speed boost.

* start game and don't move player *
> fps = 22    (v0.8.7)
> fps = 35    (v0.8.8 - modified draw routine using kev's tips)

it seemed the biggest speed boost was from removing GetImagePtr(0) command. as you said in the video that this reads from the video card = slow.
i had this command read for every polygon that gets drawn. i didn't realize i did it this way. i wasn't even using this command in the later versions.

also in the videos you noticed i used the artist render method, so all visible polygons get drawn, even the ones behind other polygons that aren't visible.  i left it this way, because i was using the image_EX trick you told me about. so i stopped using the Z buffer array method i used in earlier versions that saved the z pos data in an array that was checked before the polygon scanline got drawn. i was using a rgb array and z buffer array, and drew the screen from the rgb array data.

i should implement this back in again, maybe by using only the z buffer array, and checking this before i draw the polygon scanline to the image_EX ?

   thanks agin for the tips,  stevmjon
Title: Re: Building a basic 3D Engine
Post by: kevin on May 01, 2019, 12:02:02 PM
 Steve.

Quote
 it seemed the biggest speed boost was from removing GetImagePtr(0) command. as you said in the video that this reads from the video card = slow.
i had this command read for every polygon that gets drawn. i didn't realize i did it this way. i wasn't even using this command in the later versions.

  Yeah it's just a by product of the older methods all hidden within the routine,  hence it's good idea to refactor those things from time to time.   I often just make a new source from the old one (attach a date or version number) so there's like a builtin history of the major changes even if the older source file is no longer in use (not included) it's still within the project if i screw something up..  Which happens!   I Only went looking due to how much time it was taking, which didn't add up.

 
Quote
   i should implement this back in again, maybe by using only the z buffer array, and checking this before i draw the polygon scanline to the image_EX ?

   It seems doable, but the timing (on my machine) is showing a 1/3 of the work is rotation/culling/etc remainder is rendering.  It's just the proportions really, the bigger the screen resolution, the more cycles rendering is going to chew through.   You might win back from speed from using some pre-computed mipmaps of textures.   These are just versions of the texture that 50% of the source size.  So when  the object is proceded, it uses distance from the viewer (z will do) to work out what resolution of the texture to use.   This can have two sided behind, by smoothing out some 'noise' of distance objects and wins back some performance as smaller texture (in powers of 2) are more cache friendly so the render get less cache hits.. Whicih just means on CPUS that can't store the entire textyure on the highest level of cache,  they're constantly fetching chunks from lower levels or even directly from memory..  


  Create mip Maps (https://www.underwaredesign.com/forums/index.php?topic=1924.0)


  Here's version little zoom demo cut'n paste together. The idea is to only use the high quality texture near the camera and the texture is a power of 2 size it's easier for the render to scale.    Here i've got some logic to sub in the texture detail, but i'd just make this a table...  anyway it can be really beneficially 


  [pbcode]
openscreen 1280,840,32,1

; Include the Dialogs library in this program
  #Include "PBDialogs2"


; Title Of the Dialog
  Filename$="Some Cool FIle.txt"
   Filter$ ="(*.Images)|*.JPG;*.bmp;*.PNG;*tga"   ; Mixture of image files
; Call the dialog
  File$=OpenFileDialog("Load Image",Filename$,Filter$)
 
   if fileexist(file$) 


      loadimage file$,1,8
      scaleimage 1,512,512,1
   
   
      Cls 255
   
      Max=4

      Dim MipMaps(Max)
      Dim Scaled(Max)
      MipMaps(1)=1
      Scaled(1)=1
   




      ThisIMage=1
      For lp=2 to Max
         ; Make the mipmap from the Last image
         ThisImage=CreateMipMap(ThisImage)
         MipMaps(lp)=ThisImage

         ; make the scaled version  (scaling from the original image)
         Width      =GetImageWidth(thisImage)
         Height   =GetImageHeight(thisImage)
      
         ScaledIMage=getFreeimage()
         CopyImage 1,ScaledIMage
         Scaleimage ScaledImage,Width,height,1
         Scaled(lp)=Scaledimage


      next


      ; Draw the mip mapped + scaled versions of the images

      Xpos=100
      For lp=1 to Max
         img=MipMaps(lp)      
         drawimage Img,Xpos,20,false

         img=Scaled(lp)      
         drawimage Img,Xpos,300,false


         Xpos=Xpos+getImageWIdth(img)+16
      next
   else
      print "File Not Found"+File$

   endif

   sync
   waitkey
   waitnokey


   size=512
   ProjectionX#=400
   ProjectionY#=400

   ; objects Width and Height and depth in the scene
   ObjectWidth=size
   ObjectHeight=size
   ObjectDepth=3000
   
   SceneMaxZ   =5000
   
            img=MipMaps(1)


   ; use a spritge to represent this object on screen
   Spr=NewSprite(0,0,img)
   SpriteDrawMode Spr,2
   AutoCenterSpriteHandle spr,true

   Do
         ; Clear the screen
         Cls

         ; project the obejcts screenwidth and height at this depth
         ScreenWidth#   =(ObjectWidth*ProjectionX#)/ObjectDepth
         ScreenHeight#   =(ObjectHeight*ProjectionY#)/ObjectDepth

         
         // only use the hires texture when really close to camera
         if Objectdepth<500
            depth=1
            // use 50% version
         elseif Objectdepth<1500
            depth=2

            // use 25% versions
         elseif Objectdepth<2500
            depth=3

         elseif Objectdepth<10000
            depth=4
         endif
            
         img=mipmaps(depth)
         
         ScaleX#=ScreenWidth#/GetImageWidth(img)
         ScaleY#=ScreenHeight#/GetImageHeight(img)
         spriteimage Spr,Img
         positionsprite spr,GetScreenWidth()/2,GetScreenHeight()/2
         ScaleSpriteXY spr,scaleX#,scaley#
         
         // hold space to tint
         if Spacekey()
               SpriteTint Spr, rgbdepthcue($ffffff,0,Objectdepth,500,SceneMaxZ)
         else
               SpriteTint Spr, -1
           endif   
         drawallsprites
         
         ObjectDepth=wrapvalue(ObjectDepth-10,200,SceneMaxZ)
         print "Mip Map Level:"+str$(depth)
         print "        Image:"+str$(img)
         print "  Image Width:"+str$(getimagewidth(img))
         print " Image Height:"+str$(getimagewidth(img))
         

      Sync
   loop
   
   end





Function CreateMipMap(ThisImage)
   oldSurface=getsurface()
      if GetImageStatus(ThisImage)

      W=GetImageWidth(ThisImage)
      H=GetImageHeight(ThisImage)

      w2=w/2
      h2=h/2
      if GetimageType(ThisImage)=1 then   MipImage=NewImage(W2,h2)   
      if GetimageType(ThisImage)=2 then   MipImage=NewFXImage(W2,h2)   


      if GetImageSTatus(MipIMage)

         Dim Buffer(w,h)

         RenderToImage Thisimage
         lockbuffer
            maskcolour=point(0,0)
            For Ylp=0 to H-1
               For Xlp=0 to W-1
                     Buffer(Xlp,ylp)=FastPOint(xlp,ylp)         
               next
            next
         unlockbuffer

         rendertoimage MipImage
         imagemaskcolour mipimage,maskcolour   
         For ylp=0 to (h and $fffe)-1 step 2
            
            lockbuffer
            ThisPixel=point(0,0)
            Xpos=0
            ylp2=ylp+1
            For Xlp=0 to (W and $fffe)-1 step 2
            
               Xlp2=xlp+1
               C1=Buffer(Xlp,ylp)
               C2=Buffer(Xlp2,ylp)
               C3=Buffer(Xlp2,ylp2)
               C4=Buffer(Xlp,ylp2)
   
               a=(rgba(c1)+rgba(c2)+rgba(c3)+rgba(c4))/4
               r=(rgbr(c1)+rgbr(c2)+rgbr(c3)+rgbr(c4))/4
               g=(rgbg(c1)+rgbg(c2)+rgbg(c3)+rgbg(c4))/4
               b=(rgbb(c1)+rgbb(c2)+rgbb(c3)+rgbb(c4))/4
   

               FastDot Xpos,Ypos,aRgb(a,r,g,b)
               xpos=xpos+1
            next

            unlockbuffer
            inc ypos            
         next

         undim Buffer()

      endif   
endif
   rendertoimage oldsurface
EndFunction   MipImage



  [/pbcode]

Title: Re: Building a basic 3D Engine
Post by: kevin on May 09, 2019, 12:11:32 AM
 Steve,

    One of the old ideas for strip commands was to have auto step form, which fits those situation where you need to draw a series of connected strips .   

     
    This means you'd set the initial starting position, the feed it the target coords.  When it's draws from the last saved coord to the one supplied


    So crude mod of the render loop (original)
[pbcode]

         // screen cords of x1->x2
            x1 = interpl
            x2 = x1+64

              pixelu = int(a1#/c1#) * $10000
            pixelv = int(a2#/c1#) * $10000

                           
               //  process chunks from 2 onwards to Chunks            
               For lps=1 To chunks
               

                  t#  = (lps*64)/distance# 
                  t2# = 1-t#
                  ;--- 3D texel u & v ---
                  topu#    = t2#*a1# + t#*b1#
                  bottu#    = t2#*c1# + t#*d1#
                  destU#    = topu# / bottu#  ;  3D U (dest texel)
                  
                  topv#    = t2#*a2# + t#*b2#
                  // buttv# is the same as bottu#
                  ;bottv#    = t2#*c1# + t#*d1#
                  destV#    = topv# / bottU#  ;  3D V (dest texel)
                  
               
                  destU = destU#
                  destU*= $10000
                  destV = destV#
                  destV*= $10000

                  TextureStripH image,x1,pixelu,pixelv,x2,destu,destv,scany


                  pixelu=destu
                  pixelv=destv

                  x1=x2
                  x2+=64
               
                  Next lps


[/pbcode]


       Could become

[pbcode]

         // screen cords of x1->x2
            x1 = interpl
            x2 = x1+64

              pixelu = int(a1#/c1#) * $10000
            pixelv = int(a2#/c1#) * $10000

                // Call a function to seed the texture cords ...

                StartTextureStrip X1, ScanY , PixelU,PixelV
                           
               //  process chunks from 2 onwards to Chunks            
               For lps=1 To chunks
               

                  t#  = (lps*64)/distance# 
                  t2# = 1-t#
                  ;--- 3D texel u & v ---
                  topu#    = t2#*a1# + t#*b1#
                  bottu#    = t2#*c1# + t#*d1#
                  destU#    = topu# / bottu#  ;  3D U (dest texel)
                  
                  topv#    = t2#*a2# + t#*b2#
                  // buttv# is the same as bottu#
                  ;bottv#    = t2#*c1# + t#*d1#
                  destV#    = topv# / bottU#  ;  3D V (dest texel)
                  
               
                  destU = destU#
                  destU*= $10000
                  destV = destV#
                  destV*= $10000

    /// this would draw from the last point to the one provided next
                  RenderTextureStrip image,x2,destu,destv

                  x2+=64
               
                  Next lps


[/pbcode]


    So the cache version drawn from last save point to the new supplied,  then it copies the new cords to the cords,  that saves a bunch of VM lifting in such and inner loop

     The command names could be anything, and it allows for those functions to have additional feature patched into them. 



Title: Re: Building a basic 3D Engine
Post by: stevmjon on May 09, 2019, 09:47:06 PM
are you thinking of a new command kev?  RenderTextureStrip or something.

what would be great is a command that renders the whole polygon scanline, from poly edge to poly edge, using same as TextureStripH , but has the z value added to it, so it can interperlate non-linear (perspective correct). that will save having to break it down to multiple draws across the polygon scanline.

Title: Re: Building a basic 3D Engine
Post by: kevin on May 10, 2019, 06:05:11 AM

 
Quoteare you thinking of a new command kev?  RenderTextureStrip or something.
 

   always,  since the texture strip functions are drawing strips then it really should cache the last point itself, so we're just feeding it the next unknown point since the last is known .   This takes the need to do 'simple' operations in the VM, even if it's only removing a few assignments per loop.   

   On the opt'ing PB back end to-do list,  is some ideas for detection & implemetation of some common operators into packed opcodes.  It's a blurry line tho as too many can make it slower..   


Quote
what would be great is a command that renders the whole polygon scanline, from poly edge to poly edge, using same as TextureStripH , but has the z value added to it, so it can interperlate non-linear (perspective correct). that will save having to break it down to multiple draws across the polygon scanline.

   Yeah,  but You could do this yourself with PlayBASIC2dLL :) 

Title: Re: Building a basic 3D Engine
Post by: stevmjon on October 01, 2021, 12:21:41 AM
V0.9  :  TESTING MIPMAPPING

i thought i would re-write the 3D Engine to make it smaller, and easier to understand by changing comments too. this uses, for now, 2D polygons.

i want to test MipMaps. in the demo i have got several textures that can be changed in game, and also the mipmap distance can be changed in game. seems every 4 meters from camera is a good average, but... i am not sure how to go about calculating when to change to the lower texture. i have every 'set distance' from the camera by default, just to see what is looks like. the texture gets darker with each smaller sampled texture, to make it easier to see on screen.

* i assume there is a calc where you would measure the width/height of the texture in screen coordinates then select the mipmap texture size from this calculation???

thought i would post this for fun, and get any input.
i am thinking of maybe using shapes like polygons, and make a 3D world set up like this. that way no mipmaps needed, or textured polygons. will run faster too. could make 3D objects using software with colored polygons only and load them in.

   stevmjon
Title: Re: Building a basic 3D Engine
Post by: kevin on October 01, 2021, 06:15:39 AM

  looks cool but  is there an escape key ?
Title: Re: Building a basic 3D Engine
Post by: stevmjon on October 01, 2021, 07:47:08 AM
is this reference for a game this reminds you of?
Title: Re: Building a basic 3D Engine
Post by: kevin on October 01, 2021, 07:50:54 AM
nah, I have to CTRL-ALT-DELETE IT
Title: Re: Building a basic 3D Engine
Post by: kevin on October 01, 2021, 12:08:41 PM
 
 I mocked this up for you tonight,   It's a generic version of an object frame work.  So there's one main array of objects and each object has it's own dynamically created vertex / face data etc etc (and all other properties that you need with in it.

 The only tricky thing i guess is that is uses dynamically created array handles,  which you can do by grabbing the return value from the function / psub call that returns an array(), the value coming out of it is the handle.    So we don't need arrays for each object we make.  etc



[pbcode]

;===============================================================================
;=== types =====================================================================
;===============================================================================

;--- the coordinates in 3D ---
Type tVertex1
   x# , y# , z#
EndType

;--- the coordinates in 3D plus the objects unique id no. ---
Type tVertex2
   x# , y# , z#
   id  ;  each object has a unique id number
   typ  ;  the type of object it is (cube, ground etc)
   w#
EndType

;--- texture UV ---
Type tUV1
   u# , v#  ;  location in texture image for polygon
   ;;image
EndType

Type tUV2
   x# , y# , z#
   u# , v#  ;  location in texture image for polygon
   image
   status
   w#
EndType

;--- polygon points vertex & UV ---
Type tVertexUV
   x# , y# , z#
   id  ;  each object has a unique id number
   vertex , vertmax  ;  vertex number , max vertices
   u# , v#
   image
   w#
   typ  ;  obj type (added for image set)
EndType




 //-------------------------------------------------------------------
 //-------------------------------------------------------------------
 //-------------------------------------------------------------------
 //-------------------------------------------------------------------
 //-------------------------------------------------------------------


      TYPE t3D_OBJECT
            Name$            //  Name of the Object

            Position   as tVertex1
            Scale      as tVertex1


            //  the Array Buffers this ojbect is using..
            //  so we don't need globally declared arrays, rather
            // we create them dynamically and them within the object
            VERTEX_ArrayHandle
            FACE_ArrayHANDLE
            UV_ArrayHANDLE
      EndType

      Dim Objects(10000) as t3D_Object




 //-------------------------------------------------------------------
 //-------------------------------------------------------------------


      //  Spawn a bunch of cubes
      
         for lp =0 to 20
            
            index =newCube()
            
            //  Set this object position
            PositionObject index,Rnd(1000),Rnd(1000),Rnd(1000)
            
            //  sedt this objects scale
            ScaleObject index,Rnd#(100),Rnd(100),Rnd(100)
            
         next      
      Sync
      waitkey
      
      
      end



 //-------------------------------------------------------------------
 //-------------------------------------------------------------------
 //-------------------------------------------------------------------
 //-------------------------------------------------------------------
 //-------------------------------------------------------------------
 //-------------------------------------------------------------------

Function NewCube()
   
         //  Create blank object with space forr 8 verts and 6 faces   
         index=_Init_Blank_3D_Object(8,6)
         if index

            // Set the vertex array, so we can just dump to it like a stack   
            _PRIVATE_SET_VERTEX_ARRAY( Objects(Index).Vertex_ArrayHANDLE )

            //  New we can dump to this objects vertex array,
            //  without referencing it directly
            _PRIVATE_ADD_VERTEX( -1, 1,-1 ) ;  top left front _ 0
            _PRIVATE_ADD_VERTEX( -1, 1, 1 ) ;  top left back   _ 1
           _PRIVATE_ADD_VERTEX(  1, 1, 1 ) ;  top right back   _ 2
           _PRIVATE_ADD_VERTEX(  1, 1,-1 ) ;  top right front _ 3
 
           _PRIVATE_ADD_VERTEX( -1,-1,-1 ) ;  bott left front _ 4
             _PRIVATE_ADD_VERTEX( -1,-1, 1 ) ;  bott left back   _ 5
           _PRIVATE_ADD_VERTEX(  1,-1, 1 ) ;  bott right back   _ 6
           _PRIVATE_ADD_VERTEX(  1,-1,-1 ) ;  bott right front _ 7
            
            //  Set up faces            
            _PRIVATE_SET_FACE_ARRAY( Objects(Index).FACE_ArrayHANDLE )   
            _PRIVATE_ADD_QUAD(0,3,7,4)  ;  front polygon face
            _PRIVATE_ADD_QUAD(2,1,5,6)  ;  back polygon face
            _PRIVATE_ADD_QUAD(1,0,4,5)  ;  left polygon face
            _PRIVATE_ADD_QUAD(3,2,6,7)  ;  right polygon face
            _PRIVATE_ADD_QUAD(0,1,2,3)  ;  top polygon face
            _PRIVATE_ADD_QUAD(4,7,6,5)  ;  bottom polygon face
                        
            
         endif   
EndFunction Index





Function PositionObject(index,X#,Y#,Z#)
      if GetObjectStatus(index)
                  Objects(index).Position.x=x#
                  Objects(index).Position.y=y#
                  Objects(index).Position.z=z#
      endif
EndFunction

Function ScaleObject(index,X#,Y#,Z#)
      if GetObjectStatus(index)
                  Objects(index).Scale.x=x#
                  Objects(index).Scale.y=y#
                  Objects(index).Scale.z=z#
      endif
EndFunction


Function GetObjectStatus(index)
   if Index>0
      if Index<=GetArrayElements(Objects())
            Status=Objects(index)<>0
      endif
   endif
   
EndFunction Status


//  Private Function that creates the empty structure of our 3D object
//  So the make CUBE/SPhere etc betc are based upon this..
Function _Init_Blank_3D_Object(VertexCount,faceCount)
   
      index = GetFreeCell(Objects())
      if Index
            // Allocate this structure
            Objects(Index) = new t3D_Object
   
   
            // Fill out the base Structure
            Objects(Index).Position.x# =0
            Objects(Index).Position.y# =0
            Objects(Index).Position.z# =0

            Objects(Index).scale.x# =1
            Objects(Index).Scale.y# =1
            Objects(Index).Scale.z# =1


            //  Allocate the Arrays we need for this object   
            Objects(Index).Vertex_ArrayHANDLE    = _PRIVATE_MAKE_VERTEX_ARRAY(VertexCount)   
            Objects(Index).Face_ArrayHANDLE       = _PRIVATE_MAKE_FACE_ARRAY(FaceCount)   
   
      endif
EndFunction  Index



//   Some simple functions that allow us to READ/WRITE from

   MakeArray _CURRENT_VERTEX_ARRAY().tVertex1
   global _CURRENT_VERTEX_ARRAY_INDEX
   
Psub _PRIVATE_SET_VERTEX_ARRAY(ArrayHANDLE)
      _CURRENT_VERTEX_ARRAY() = ArrayHANDLE
      _CURRENT_VERTEX_ARRAY_INDEX = 0      
EndPsub

Psub _PRIVATE_ADD_VERTEX(X#,Y#,Z#)
      _CURRENT_VERTEX_ARRAY(_CURRENT_VERTEX_ARRAY_INDEX) = New tVertex1
      _CURRENT_VERTEX_ARRAY(_CURRENT_VERTEX_ARRAY_INDEX).x# =x#
      _CURRENT_VERTEX_ARRAY(_CURRENT_VERTEX_ARRAY_INDEX).y# =y#
      _CURRENT_VERTEX_ARRAY(_CURRENT_VERTEX_ARRAY_INDEX).z# =z#
      _CURRENT_VERTEX_ARRAY_INDEX++
EndPsub



   MakeArray _CURRENT_FACE_ARRAY()
//   _CURRENT_FACE_ARRAY(0,0) = 0 //  Mock expression to make PB think this is 2D
   global _CURRENT_FACE_ARRAY_INDEX
Psub _PRIVATE_SET_FACE_ARRAY(ArrayHANDLE)
      _CURRENT_FACE_ARRAY() = ArrayHANDLE
      _CURRENT_FACE_ARRAY_INDEX = 0   
EndPsub


Psub _PRIVATE_ADD_TRI(v1,v2,v3)
      _CURRENT_FACE_ARRAY(_CURRENT_FACE_ARRAY_INDEX,0) =v1
      _CURRENT_FACE_ARRAY(_CURRENT_FACE_ARRAY_INDEX,1) =v2
      _CURRENT_FACE_ARRAY(_CURRENT_FACE_ARRAY_INDEX,2) =v3
      _CURRENT_FACE_ARRAY_INDEX++
EndPsub

Psub _PRIVATE_ADD_QUAD(v1,v2,v3,v4)
      _CURRENT_FACE_ARRAY(_CURRENT_FACE_ARRAY_INDEX,0) =v1
      _CURRENT_FACE_ARRAY(_CURRENT_FACE_ARRAY_INDEX,1) =v2
      _CURRENT_FACE_ARRAY(_CURRENT_FACE_ARRAY_INDEX,2) =v3
      _CURRENT_FACE_ARRAY(_CURRENT_FACE_ARRAY_INDEX,3) =v4
      _CURRENT_FACE_ARRAY_INDEX++
EndPsub


Function _PRIVATE_MAKE_VERTEX_ARRAY(VertexCount)
      Dim VertexArray( VertexCount) as tVertex1
EndFunction VertexArray().tVertex1

Function _PRIVATE_MAKE_FACE_ARRAY(FaceCount)
      Dim FaceArray( FaceCount,4)
EndFunction FaceArray()


/*
Function obj_cube(array().tVertex1,array2())
   
   ;--- create a cube object ---
   
   ; cube_vector(x)  :  vx, vy, vz
   ; cube_face(x,x)  :  face , points
   
   ;--- cube vectors ---
   ;--- read data for array ---
   max=GetArrayElements(array(),1)  ;  total number of points in object
   
   Restore objvectordata
   For lp=0 To max
      array(lp) = New tVertex1
      array(lp).x# = ReadData()  ; vx  :  cube_vector(x)
      array(lp).y# = ReadData()  ; vy
      array(lp).z# = ReadData()  ; vz
   Next lp
   
   ;--- object vectors _ data x, y, z ---
objvectordata:
  Data -1,1,-1  ;  top left front _ 0
  Data -1,1,1  ;  top left back   _ 1
  Data 1,1,1  ;  top right back   _ 2
  Data 1,1,-1  ;  top right front _ 3
 
  Data -1,-1,-1  ;  bott left front _ 4
  Data -1,-1,1  ;  bott left back   _ 5
  Data 1,-1,1  ;  bott right back   _ 6
  Data 1,-1,-1  ;  bott right front _ 7
   
   
   ;--- cube faces ---
   ;--- read data for array ---
   max=GetArrayElements(array2(),1)
   
   Restore objfacedata
   For lp=0 To max
      array2(lp,0) = ReadData()  ; vector order for faces  :  cube_face(x,x)
      array2(lp,1) = ReadData()
      array2(lp,2) = ReadData()
      array2(lp,3) = ReadData()
   Next lp
   
   ;--- object faces set as quads _ using vector table from above _ clockwise order ---
objfacedata:
   Data 0,3,7,4  ;  front polygon face
   Data 2,1,5,6  ;  back polygon face
   Data 1,0,4,5  ;  left polygon face
   Data 3,2,6,7  ;  right polygon face
   Data 0,1,2,3  ;  top polygon face
   Data 4,7,6,5  ;  bottom polygon face
   
   ;--- set u/v coords in image _ clockwise around image ---
   ;--- this array uses 0 to 1 for u&v ---
   uv_cube(0).u# = 0    :  uv_cube(0).v# = 0    ;  u1, v1
   uv_cube(1).u# = 1    :  uv_cube(1).v# = 0    ;  u2, v2
   uv_cube(2).u# = 1    :  uv_cube(2).v# = 1    ;  u3, v3
   uv_cube(3).u# = 0    :  uv_cube(3).v# = 1    ;  u4, v4
   
EndFunction

*/


[/pbcode]




    Example #2


     This version includes a simple display render function so you can visualize it.  There's no camera, but the objects all randomly move left/right so to show it's running.  SPACE / Mouse button to quit


[pbcode]

; PROJECT : 3D Object Structure Example For Steve
; AUTHOR  :
; CREATED : 2/10/2021
; EDITED  : 2/10/2021
; ---------------------------------------------------------------------

;===============================================================================
;=== types =====================================================================
;===============================================================================

;--- the coordinates in 3D ---
Type tVertex1
   x# , y# , z#
EndType

;--- the coordinates in 3D plus the objects unique id no. ---
Type tVertex2
   x# , y# , z#
   id  ;  each object has a unique id number
   typ  ;  the type of object it is (cube, ground etc)
   w#
EndType

;--- texture UV ---
Type tUV1
   u# , v#  ;  location in texture image for polygon
   ;;image
EndType

Type tUV2
   x# , y# , z#
   u# , v#  ;  location in texture image for polygon
   image
   status
   w#
EndType

;--- polygon points vertex & UV ---
Type tVertexUV
   x# , y# , z#
   id  ;  each object has a unique id number
   vertex , vertmax  ;  vertex number , max vertices
   u# , v#
   image
   w#
   typ  ;  obj type (added for image set)
EndType




  //-------------------------------------------------------------------
  //-------------------------------------------------------------------
  //-------------------------------------------------------------------
  //-------------------------------------------------------------------
  //-------------------------------------------------------------------


      TYPE t3D_OBJECT
            Name$            //  Name of the Object

            Position   as tVertex1
            Scale      as tVertex1

            //  the Array Buffers this ojbect is using..
            //  so we don't need globally declared arrays, rather
            // we create them dynamically and them within the object
            VERTEX_ArrayHandle
            FACE_ArrayHANDLE
            UV_ArrayHANDLE
      EndType

      Dim Objects(10000) as t3D_Object




  //-------------------------------------------------------------------
  //-------------------------------------------------------------------


      //  Spawn a bunch of cubes
      
         for lp =0 to 20
            
            index =newCube()
            
            //  Set this object position
            PositionObject index,Rndrange(-1000,1000),Rndrange(-1000,1000),500+Rnd(1000)
            
            //  set this objects scale
            ScaleObject index,Rnd(100),Rnd(100),Rnd(100)

            
         next      
   
   
      
      Do
            cls
            
            movecount--
            if movecount<0
               Dim Move as tVertex1
               move= new tVertex1
               move.x = rndrange(-10,10)
               movecount=rndrange(10,100)               
            endif         
         
         
            for Index=1 to 1000
               if GetObjectStatus(index)
                  moveobject Index,move.x,move.y,move.z
                  renderobject Index
               endif
            next            
   
            sync
   
      Loop Spacekey()=true or mousebutton()=1

      end

  //-------------------------------------------------------------------
  //-------------------------------------------------------------------
  //-------------------------------------------------------------------
  //-------------------------------------------------------------------
  //-------------------------------------------------------------------
  //-------------------------------------------------------------------

Function NewCube()
   
         //  Create blank object with space forr 8 verts and 6 faces   
         index=_Init_Blank_3D_Object(8,6)
         if index

            // Set the vertex array, so we can just dump to it like a stack   
            _PRIVATE_SET_VERTEX_ARRAY( Objects(Index).Vertex_ArrayHANDLE )

            //  New we can dump to this objects vertex array,
            //  without referencing it directly
            _PRIVATE_ADD_VERTEX( -1, 1,-1 ) ;  top left front _ 0
            _PRIVATE_ADD_VERTEX( -1, 1, 1 ) ;  top left back   _ 1
            _PRIVATE_ADD_VERTEX(  1, 1, 1 ) ;  top right back   _ 2
            _PRIVATE_ADD_VERTEX(  1, 1,-1 ) ;  top right front _ 3
   
            _PRIVATE_ADD_VERTEX( -1,-1,-1 ) ;  bott left front _ 4
              _PRIVATE_ADD_VERTEX( -1,-1, 1 ) ;  bott left back   _ 5
            _PRIVATE_ADD_VERTEX(  1,-1, 1 ) ;  bott right back   _ 6
            _PRIVATE_ADD_VERTEX(  1,-1,-1 ) ;  bott right front _ 7
            
            //  Set up faces            
            _PRIVATE_SET_FACE_ARRAY( Objects(Index).FACE_ArrayHANDLE )   
            _PRIVATE_ADD_QUAD(0,3,7,4)  ;  front polygon face
            _PRIVATE_ADD_QUAD(2,1,5,6)  ;  back polygon face
            _PRIVATE_ADD_QUAD(1,0,4,5)  ;  left polygon face
            _PRIVATE_ADD_QUAD(3,2,6,7)  ;  right polygon face
            _PRIVATE_ADD_QUAD(0,1,2,3)  ;  top polygon face
            _PRIVATE_ADD_QUAD(4,7,6,5)  ;  bottom polygon face
                        
            
         endif   
EndFunction Index





Function RenderObject(index)
         //  check if this pobject exists ??
         if GetObjectStatus(index)

               //  Draw the object in wire frame

               // We're just assume
               
               Dim Position as tVertex1 pointer
               //  Assign the pointer of this objects opsition vector
               //  so we can short cut it as Position.x position.y etc               
               Position=Objects(Index).Position


               // Do the same with the scaler                           
               Dim Scaler as tVertex1 pointer
               Scaler=Objects(Index).Scale
   
   
               //  Delare our empty stub array
               MakeArray FaceList()
               
               //  store our handle in the stub, so now it's the 2d array
               //  of faces
               Facelist() = Objects(Index).Face_ArrayHANDLE


               //  Vertex  Buffer
               MakeArray Vertex().tVertex1
               Vertex()   = Objects(Index).Vertex_ArrayHANDLE

               //  ProcessedVertex
               Dim FaceVertex#(256)   
               lockbuffer
               //  Draw faces
               for LP=0 to GetArrayElements(FaceLIst(),1)-1
                     offset=0
                     for vertlp=0 to 3
                        ThisVert=FaceList(LP,vertlp)      
                        x#=(Vertex(ThisVert).x *Scaler.x)+ POsition.X
                        y#=(Vertex(ThisVert).y *Scaler.y)+ POsition.Y
                        z#=(Vertex(ThisVert).z *Scaler.z)+ POsition.Z
                        
                        sx# = (x#*500)/z#
                        sy# = (y#*500)/z#
                        FaceVertex#(offset) = 400+sx# : offset++
                        FaceVertex#(offset) = 300+sy# : offset++
                     next
                     line FaceVertex#(0),FaceVertex#(1),FaceVertex#(2),FaceVertex#(3)
                     line FaceVertex#(2),FaceVertex#(3),FaceVertex#(4),FaceVertex#(5)
                     line FaceVertex#(4),FaceVertex#(5),FaceVertex#(6),FaceVertex#(7)
                     line FaceVertex#(6),FaceVertex#(7),FaceVertex#(0),FaceVertex#(1)
               next LP   
               unlockbuffer

   
         endif   
EndFunction

Function PositionObject(index,X#,Y#,Z#)
      if GetObjectStatus(index)
                  Objects(index).Position.x=x#
                  Objects(index).Position.y=y#
                  Objects(index).Position.z=z#
      endif
EndFunction

Function MoveObject(index,VX#,VY#,VZ#)
      if GetObjectStatus(index)
                  Objects(index).Position.x+=Vx#
                  Objects(index).Position.y+=Vy#
                  Objects(index).Position.z+=Vz#
      endif
EndFunction


Function ScaleObject(index,X#,Y#,Z#)
      if GetObjectStatus(index)
                  Objects(index).Scale.x=x#
                  Objects(index).Scale.y=y#
                  Objects(index).Scale.z=z#
      endif
EndFunction


Function GetObjectStatus(index)
   if Index>0
      if Index<=GetArrayElements(Objects())
            Status=Objects(index)<>0
      endif
   endif
   
EndFunction Status


//  Private Function that creates the empty structure of our 3D object
//  So the make CUBE/SPhere etc betc are based upon this..
Function _Init_Blank_3D_Object(VertexCount,faceCount)
   
      index = GetFreeCell(Objects())
      if Index
            // Allocate this structure
            Objects(Index) = new t3D_Object
   
   
            // Fill out the base Structure
            Objects(Index).Position.x# =0
            Objects(Index).Position.y# =0
            Objects(Index).Position.z# =0

            Objects(Index).scale.x# =1
            Objects(Index).Scale.y# =1
            Objects(Index).Scale.z# =1


            //  Allocate the Arrays we need for this object   
            Objects(Index).Vertex_ArrayHANDLE    = _PRIVATE_MAKE_VERTEX_ARRAY(VertexCount)   
            Objects(Index).Face_ArrayHANDLE       = _PRIVATE_MAKE_FACE_ARRAY(FaceCount)   
   
      endif
EndFunction  Index



//   Some simple functions that allow us to READ/WRITE from

   MakeArray _CURRENT_VERTEX_ARRAY().tVertex1
   global _CURRENT_VERTEX_ARRAY_INDEX
   
Psub _PRIVATE_SET_VERTEX_ARRAY(ArrayHANDLE)
      _CURRENT_VERTEX_ARRAY() = ArrayHANDLE
      _CURRENT_VERTEX_ARRAY_INDEX = 0      
EndPsub

Psub _PRIVATE_ADD_VERTEX(X#,Y#,Z#)
      _CURRENT_VERTEX_ARRAY(_CURRENT_VERTEX_ARRAY_INDEX) = New tVertex1
      _CURRENT_VERTEX_ARRAY(_CURRENT_VERTEX_ARRAY_INDEX).x# =x#
      _CURRENT_VERTEX_ARRAY(_CURRENT_VERTEX_ARRAY_INDEX).y# =y#
      _CURRENT_VERTEX_ARRAY(_CURRENT_VERTEX_ARRAY_INDEX).z# =z#
      _CURRENT_VERTEX_ARRAY_INDEX++
EndPsub


explicit on
   MakeArray _CURRENT_FACE_ARRAY()
//   _CURRENT_FACE_ARRAY(0,0) = 0 //  Mock expression to make PB think this is 2D
   global _CURRENT_FACE_ARRAY_INDEX
Psub _PRIVATE_SET_FACE_ARRAY(ArrayHANDLE)
      _CURRENT_FACE_ARRAY() = ArrayHANDLE
      _CURRENT_FACE_ARRAY_INDEX = 0   
EndPsub


Psub _PRIVATE_ADD_TRI(v1,v2,v3)
      _CURRENT_FACE_ARRAY(_CURRENT_FACE_ARRAY_INDEX,0) =v1
      _CURRENT_FACE_ARRAY(_CURRENT_FACE_ARRAY_INDEX,1) =v2
      _CURRENT_FACE_ARRAY(_CURRENT_FACE_ARRAY_INDEX,2) =v3
      _CURRENT_FACE_ARRAY_INDEX++
EndPsub

Psub _PRIVATE_ADD_QUAD(v1,v2,v3,v4)
      _CURRENT_FACE_ARRAY(_CURRENT_FACE_ARRAY_INDEX,0) =v1
      _CURRENT_FACE_ARRAY(_CURRENT_FACE_ARRAY_INDEX,1) =v2
      _CURRENT_FACE_ARRAY(_CURRENT_FACE_ARRAY_INDEX,2) =v3
      _CURRENT_FACE_ARRAY(_CURRENT_FACE_ARRAY_INDEX,3) =v4
      _CURRENT_FACE_ARRAY_INDEX++
EndPsub


Function _PRIVATE_MAKE_VERTEX_ARRAY(VertexCount)
      Dim VertexArray( VertexCount) as tVertex1
EndFunction VertexArray().tVertex1

Function _PRIVATE_MAKE_FACE_ARRAY(FaceCount)
      Dim FaceArray( FaceCount,4)
EndFunction FaceArray()


explicit off



[/pbcode]
Title: Re: Building a basic 3D Engine
Post by: stevmjon on October 01, 2021, 10:50:45 PM
well kev, thanks for the homework, lol.

interesting setup. i have never used private arrays then referenced their handle, all while referencing this handle inside a typed array.
so it's like pointing to a location in memory and read/write data, rather than using an array itself to read/write data.

just wondering, since all the data is still there, whether using dynamically created array handle method or making an array for the data, i assume that pointing to the data using handles method works faster?

oh, and i will add escape key to exit 3D game demo. on my rig pressing ecs key exits. didn't realise on some computers this doesn't do the same. thanks for the heads up.

  stevmjon
Title: Re: Building a basic 3D Engine
Post by: kevin on October 01, 2021, 11:19:07 PM

  Updated the  example  (https://www.underwaredesign.com/forums/index.php?topic=4365.msg30384#msg30384) on the previous page.


Quoteinteresting setup. i have never used private arrays then referenced their handle, all while referencing this handle inside a typed array.
so it's like pointing to a location in memory and read/write data, rather than using an array itself to read/write data.

   Here i'm using PRIVATE to denote that these functions are for use within the library and not meant to be called by users.  Ill often call such functions names, so they are not easily guessed,  so i'll put some crap at the head of the function name so they wont clash with any other code is the library / include was inserted within another program.      Other than that it has no impact on anything,  we do have a PRIVATE keyword but it's not in use in PB today.      


    eg..

[pbcode]


 /*
 ---------------------------------------------------------------------------------------------------------------
 ---------------------------------------------------------------------------------------------------------------
 ------------------------------->>  PUBLIC FUNCTIONS <<-----------------------------------------------
 ---------------------------------------------------------------------------------------------------------------

 */



function MakeSomething()


           //  in here we call our helper function that does some job we might commonly need across the library, but don't want the user calling directly

           result=zzzzzzz_MakeSomething(Param1,parma2)

           etc etc

endfunction

 /*
 ---------------------------------------------------------------------------------------------------------------
 ---------------------------------------------------------------------------------------------------------------
 ------------------------------->>  PRIVATE FUNCTIONS <<-----------------------------------------------
 ---------------------------------------------------------------------------------------------------------------

 */

function zzzzzzz_MakeSomething(Param1,parma2)

  // do some commonly used stuff for us
endfunction result

[/pbcode]






Quote
just wondering, since all the data is still there, whether using dynamically created array handle method or making an array for the data, i assume that pointing to the data using handles method works faster?


  nah, it's the same.  Just like passing an array into a function,  the so called passed local function is just a copy of the passed arrays handle.   Once inside the function, it's basically identical to accessing the original array.  


Title: Re: Building a basic 3D Engine
Post by: stevmjon on October 02, 2021, 09:42:05 PM
here is the mipmap test updated to calculate the actual polygons on screen, then choose a mipmap image the closest size the polygon. i made the image darker the smaller the image that is selected for easier viewing.
also you can go upwards with the camera to watch the textures change the further you get. looks cool. i may need to adjust the clipped polygons mipmap though, as it calcs the image based on the clipped polygon size.

kev, i noticed that if you get too small an image, especially using the grass texture, the polygon textures don't look as smooth. there must be another calculation on top of mipmaps? anyway, it was fun to impliment in game, as i never done this before.

have fun testing, stevmjon