Project Perfect Mod Forums
:: Home :: Get Hosted :: PPM FAQ :: Forum FAQ :: Privacy Policy :: Search :: Memberlist :: Usergroups :: Register :: Profile :: Log in to check your private messages :: Log in ::


The time now is Fri Mar 29, 2024 10:08 am
All times are UTC + 0
Reading and parsing through vxl files...
Moderators: Global Moderators, Red Alert 2 Moderators
Post new topic   Reply to topic Page 1 of 2 [89 Posts] Mark the topic unread ::  View previous topic :: View next topic
Goto page: 1, 2 Next
Author Message
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Sun Jun 14, 2015 9:23 am    Post subject:  Reading and parsing through vxl files... Reply with quote  Mark this post and the followings unread

I am wondering if anyone can explain how to read and parse through vxl files...

I have been looking at Eol/DMZ VXL_Format... And its really not making since...  Laughing

I do see its little endian but that doesn't seem to help much...

I do not know much about binary or endian file types... And looking on-line can be a mess as there is so many different types that I have no idea were to start.

How would I go about getting the data to ascii?

for example I am trying to retrieve the vxls palette color index,normal value and i guess vxls them-self...

Also how are voxels stored... an 2d array?

[x,x,x,x]
[x,0,0,x]
[x,0,0,x]
[x,x,x,x]

x's values are just empty/none... and 0's are i guess used/on.

_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Back to top
View user's profile Send private message Skype Account
Banshee
Supreme Banshee


Also Known As: banshee_revora (Steam)
Joined: 15 Aug 2002
Location: Brazil

PostPosted: Sun Jun 14, 2015 1:50 pm    Post subject: Reply with quote  Mark this post and the followings unread

That part of VXLSE III is one of these parts that I've hardly paid any attention for, at all.

Anyway, you can find it here:
http://svn.ppmsite.com/filedetails.php?repname=OS+Voxel+Tools&path=%2Fvxlseiii14x%2Fsource%2Fdocument%2FVoxel.pas

It starts at procedure TVoxel.LoadFromFile(FName: string);

It reads the header:

Code:
TVoxelHeader = packed record
      FileType: packed array[1..16] of Char; // always "Voxel Animation"
      NumPalettes, // always 1
      NumSections,
      NumSections2,
      BodySize: LongInt; // as above
      StartPaletteRemap,
      EndPaletteRemap: Byte; // colour indexes
      PaletteData: packed array[1..256*3] of Byte; // never used
   end;



Then it reads all headers, bodies and tailers (as explained in these DMZ/Eol, etc articles. VXLSE III calculates their position to read section per section.

Then, it reads each section (all at the same file Embarassed):

Look for function TVoxelSection.LoadFromFile(var F: File; HeadOfs, BodyOfs, TailOfs : Integer): EError;

It basically reads the header, body and tailer. Then, it reads the data, which is an array data with some kind of compression, since I don't think it stores the blank voxels explicitly. You'll need to adapt that code for your programming language. It's not that hard at all.

Back to top
View user's profile Send private message Visit poster's website Skype Account
Graion Dilach
Defense Minister


Joined: 22 Nov 2010
Location: Iszkaszentgyorgy, Hungary

PostPosted: Sun Jun 14, 2015 5:36 pm    Post subject: Reply with quote  Mark this post and the followings unread


_________________
"If you didn't get angry and mad and frustrated, that means you don't care about the end result, and are doing something wrong." - Greg Kroah-Hartman
=======================
Past C&C projects: Attacque Supérior (2010-2019); Valiant Shades (2019-2021)
=======================
WeiDU mods: Random Graion Tweaks | Graion's Soundsets
Maintainance: Extra Expanded Enhanced Encounters! | BGEESpawn
Contributions: EE Fixpack | Enhanced Edition Trilogy | DSotSC (Trilogy) | UB_IWD | SotSC & a lot more...

Back to top
View user's profile Send private message Visit poster's website ModDB Profile ID
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Fri Jun 19, 2015 11:33 am    Post subject: Reply with quote  Mark this post and the followings unread

So this is really annoying me... I have never had to deal with reading binary files...

I have no idea if PHP can do this... It looks like it can... I hope it can...

So this is what I am looking at so far...

Code:
<?php
   $filename = "cube.vxl";
   $handle = fopen($filename, "rb");

   $contents = fread($handle, 15);//anything more than this and it reads to much
   print_r( $contents );//this will display "Voxel Animation"

   echo "<br>";

 *** This is were I get lost... Should I only take in 1 byte... or 32  bits for a long?
 ** All I know is I should be getting "1"....
   $contents = fread($handle, 1);
   print_r( bin2hex($contents)  ); ** And I think this needs to be packed... I do not understand that stuff either... [url]http://php.net/manual/en/function.pack.php[/url]

 ** if I read 1 byte I get 00

 ** if I read 32 I get 00010000000100000001000000dc080000101f00000000000000000000000000


   fclose($handle);

?>


The vxl header looks some thing like this...

struct vxl_header
 {
   char filetype[16];    /* ASCIIZ string - "Voxel Animation" */
   long unknown;         /* Always 1 - number of animation frames? */
   long n_limbs;         /* Number of limb headers/bodies/tailers */
   long n_limbs2;        /* Always the same as n_limbs */
   long bodysize;        /* Total size in bytes of all limb bodies */
   short int unknown2;   /* Always 0x1f10 - ID or end of header code? */
   char palette[256][3]; /* 256 colour palette for the voxel in RGB format */    
 };

I have also upload the cube vxl I am testing this on if any one can is willing to help me understand this...



cube.vxl
 Description:

Download
 Filename:  cube.vxl
 Filesize:  3.12 KB
 Downloaded:  6 Time(s)


_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Back to top
View user's profile Send private message Skype Account
G-E
Defense Minister


Joined: 09 Feb 2015

PostPosted: Fri Jun 19, 2015 12:12 pm    Post subject: Reply with quote  Mark this post and the followings unread

One thing I've done using TCL to handle binary files (or binary sockets) is to read clumps at a time, something larger than the file's possible header, then parse that in memory... to start. If you know the next piece of info is 8 bytes, process the next 8 bytes from your clump, and of course do some checks to ensure it conforms to what you expect.

Use fread(expected length), or fread _more_ but only parse X bytes, is it not working?

I don't think you can read bits, so 32bit = 4byte...

_________________
http://www.moddb.com/mods/scorched-earth-ra2-mod-with-smart-ai

Back to top
View user's profile Send private message
G-E
Defense Minister


Joined: 09 Feb 2015

PostPosted: Fri Jun 19, 2015 12:33 pm    Post subject: Reply with quote  Mark this post and the followings unread

Actually I just remembered there was a way to read the whole thing: http://php.net/manual/en/function.file-get-contents.php

I was just thinking maybe PHP is deciding you hit EOF when you haven't, I remember having a similar problem when I wrote my TCL wget-like script for sucking jpegs and other files (used in conjunction with another script I wrote that generated the grab list).

_________________
http://www.moddb.com/mods/scorched-earth-ra2-mod-with-smart-ai

Back to top
View user's profile Send private message
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Fri Jun 19, 2015 12:33 pm    Post subject: Reply with quote  Mark this post and the followings unread

So If I try this...
Code:

$contents = fread($handle, 15)
print_r( $contents );//this will display "Voxel Animation"

$contents = fread($handle, 4);
print_r( bin2hex($contents)  );


I get "00010000"...

Now what I should be seeing is a "1"

...

_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Last edited by MadHQ on Fri Jun 19, 2015 12:36 pm; edited 1 time in total

Back to top
View user's profile Send private message Skype Account
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Fri Jun 19, 2015 12:34 pm    Post subject: Reply with quote  Mark this post and the followings unread

G-E wrote:
Actually I just remembered there was a way to read the whole thing: http://php.net/manual/en/function.file-get-contents.php

I was just thinking maybe PHP is deciding you hit EOF when you haven't, I remember having a similar problem when I wrote my TCL wget-like script for sucking jpegs and other files (used in conjunction with another script I wrote that generated the grab list).


I dont think I want to read the whole file... I really just want this file in a human read able form... I know this sounds crazy...

_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Back to top
View user's profile Send private message Skype Account
G-E
Defense Minister


Joined: 09 Feb 2015

PostPosted: Fri Jun 19, 2015 12:42 pm    Post subject: Reply with quote  Mark this post and the followings unread

Looks like a double-word, is 2 "words" put together, I believe the number you are reading is correct, think of it like little-endian vs big-endian but in blocks of 2 bytes. So like 5,6,7,8,1,2,3,4 in significance...

Unless it's padded that is?

_________________
http://www.moddb.com/mods/scorched-earth-ra2-mod-with-smart-ai

Back to top
View user's profile Send private message
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Fri Jun 19, 2015 12:51 pm    Post subject: Reply with quote  Mark this post and the followings unread

Thats what I really do not understand...

The vxl format description by Eol/DZM says it little-endian...

I do not know enough about binary files/data... I have tried googling the hell out of this topic, with no luck... There is just way to much out there to narrow down a search that makes since...

Are you saying I have to swap the bits(or bytes)?

_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Back to top
View user's profile Send private message Skype Account
G-E
Defense Minister


Joined: 09 Feb 2015

PostPosted: Fri Jun 19, 2015 12:57 pm    Post subject: Reply with quote  Mark this post and the followings unread

Which part are you trying to read? This?

Code:

type
   TVoxelHeader = packed record
      FileType: packed array[1..16] of Char; // always "Voxel Animation"
      NumPalettes, // always 1
      NumSections,
      NumSections2,
      BodySize: LongInt; // as above
      StartPaletteRemap,
      EndPaletteRemap: Byte; // colour indexes
      PaletteData: packed array[1..256*3] of Byte; // never used
   end;


For the palette and section counts you can probably discard the high bits and do a hex conversion on the lowers, then for the body size treat them as 2 hex conversion calculations. Multiply the upper bits by 65536 and add to the lowers.

_________________
http://www.moddb.com/mods/scorched-earth-ra2-mod-with-smart-ai

Last edited by G-E on Fri Jun 19, 2015 1:02 pm; edited 1 time in total

Back to top
View user's profile Send private message
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Fri Jun 19, 2015 1:01 pm    Post subject: Reply with quote  Mark this post and the followings unread

Yup, I am starting from the beginning of file.

I guess I have FileType, as that php code I have can read it just fine. (One of the two none binary data in these files... LOL)

Now I am trying to get NumPalettes witch should be "1" I guess


I know I am really far... I cant get past the binary knowledge step...

Quote:
For the palette and section counts you can probably discard the high bits and do a hex conversion on the lowers, then for the body size treat them as 2 hex conversion calculations.


What...

_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Back to top
View user's profile Send private message Skype Account
G-E
Defense Minister


Joined: 09 Feb 2015

PostPosted: Fri Jun 19, 2015 2:13 pm    Post subject: Reply with quote  Mark this post and the followings unread

I made a quick TCL to check how the voxel header works, it is indeed little-endian, and I am parsing it correctly.

Byte4 to Byte1 in significance, so you'll need to convert each byte to hex or integer, then multiply each by it's offset value, then add them up.

(0x$byte4 * 16777216) + (0x$byte3 * 65536) + (0x$byte2 * 256) + 0x$byte1

That gives you the actual number, but like I say for the palettes and section counts, you can ignore bytes 2-4, just use the first one, there's no way anyone is going to have 257 voxel sections hahah Smile

_________________
http://www.moddb.com/mods/scorched-earth-ra2-mod-with-smart-ai

Back to top
View user's profile Send private message
G-E
Defense Minister


Joined: 09 Feb 2015

PostPosted: Fri Jun 19, 2015 2:19 pm    Post subject: Reply with quote  Mark this post and the followings unread

Code:
set voxel "c:/cc-ra2/ecache07/orbc.vxl"   
catch {open $voxel r} of
if ![string match "file*" $of] {puts "voxel not readable ..." ; exit}
fconfigure $of -translation binary
set len [file size $voxel]
puts "reading $len ..."
set rawvxl [read $of 16]
puts "Voxel Animation = $rawvxl"
set rawvxl [read $of 1] ; set dummy [read $of 3]
binary scan $rawvxl H* hex
puts "Palettes: $hex"
set rawvxl [read $of 1] ; set dummy [read $of 3]
binary scan $rawvxl H* hex
puts "Sections: $hex"
set rawvxl [read $of 1] ; set dummy [read $of 3]
binary scan $rawvxl H* hex
puts "Sections: $hex"
set b1 [read $of 1]
set b2 [read $of 1]
set b3 [read $of 1]
set b4 [read $of 1]
binary scan $b1 H* b1
binary scan $b2 H* b2
binary scan $b3 H* b3
binary scan $b4 H* b4
set num [expr (0x$b4 * 16777216) + (0x$b3 * 65536) + (0x$b2 * 256) + 0x$b1]
puts "Size: $num"
close $of


Code:

reading 13981 ...
Voxel Animation = Voxel Animation
Palettes: 01
Sections: 01
Sections: 01
Size: 13059


Arachnid my walking spider thing...

Code:

reading 60234 ...
Voxel Animation = Voxel Animation
Palettes: 01
Sections: 05
Sections: 05
Size: 58832

_________________
http://www.moddb.com/mods/scorched-earth-ra2-mod-with-smart-ai

Back to top
View user's profile Send private message
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Fri Jun 19, 2015 3:37 pm    Post subject: Reply with quote  Mark this post and the followings unread

Interesting...

Quote:
set rawvxl [read $of 1] ; set dummy [read $of 3]


Why are you dumping the last 3 bytes?

Quote:
set num [expr (0x$b4 * 16777216) + (0x$b3 * 65536) + (0x$b2 * 256) + 0x$b1]


What is the point of this? And I what is it doing, Converting base, if so what base is that, and how did you find that out?

_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Back to top
View user's profile Send private message Skype Account
G-E
Defense Minister


Joined: 09 Feb 2015

PostPosted: Fri Jun 19, 2015 4:01 pm    Post subject: Reply with quote  Mark this post and the followings unread

MadHQ wrote:
Interesting...

Quote:
set rawvxl [read $of 1] ; set dummy [read $of 3]


Why are you dumping the last 3 bytes?

Because we know it's likely 1, but for sure it's less than 257, so the number will never exceed the first byte. Remember least significant byte first, 0-255.

MadHQ wrote:
Quote:
set num [expr (0x$b4 * 16777216) + (0x$b3 * 65536) + (0x$b2 * 256) + 0x$b1]


What is the point of this? And I what is it doing, Converting base, if so what base is that, and how did you find that out?

I just broke it apart so you could see how each byte adds up to a total number, there are more compact ways of doing that of course.

In PHP you need to use pack/unpack and bin2hex/hexdec to reformat the data, and you can tell it to unpack little-endian specifically, skipping my reverse assembly thing Smile

Back to top
View user's profile Send private message
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Sat Jun 20, 2015 4:39 am    Post subject: Reply with quote  Mark this post and the followings unread

okay...  Laughing

This is some what annoying... Why cant binary just be plain text... Who ever invented binary should be punished...  Laughing

Quote:
<?php
$filename = "cube.vxl";
$handle = fopen($filename, "rb");

$contents = fread($handle, 16);

print_r( $contents );

echo "<br>";
$contents = fread($handle, 1);
$dump = fread($handle, 3);
print_r( hexdec(bin2hex($contents))  );

echo "<br>";

$contents = fread($handle, 1);
$dump = fread($handle, 3);
print_r( hexdec(bin2hex($contents))  );

echo "<br>";

$contents = fread($handle, 1);
print_r( hexdec(bin2hex($contents))  );

echo "<br>";

$contents1 = hexdec(bin2hex(fread($handle, 1)));
$contents2 = hexdec(bin2hex(fread($handle, 1))) * 256;
$contents3 = hexdec(bin2hex(fread($handle, 1))) * 65536;
$contents4 = hexdec(bin2hex(fread($handle, 1))) * 16777216;
print_r( $contents1 + $contents2 + $contents3 + $contents4 );

fclose($handle);

?>


Thats what I am at as of right now... I am just trying to recreate the size section for right now... I am trying to base it off you code with the whole multiply thing... As I do not understand the pack/unpack thing in PHP... It just doesn't seem like its working...

So whats wrong with the last section I have....?

Quote:
Voxel Animation
1
1
1
3690987520


This is the output I am getting...

The size seems way to big...

_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Back to top
View user's profile Send private message Skype Account
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Sat Jun 20, 2015 4:51 am    Post subject: Reply with quote  Mark this post and the followings unread

woops!! forgot to dump 3 bytes...

Code:
<?php
   $filename = "cube.vxl";
   $handle = fopen($filename, "rb");

   $contents = fread($handle, 16);

   print_r( $contents );

   echo "<br>";
   $contents = fread($handle, 1);
   $dump = fread($handle, 3);
   print_r( hexdec(bin2hex($contents))  );
   
   echo "<br>";
   
   $contents = fread($handle, 1);
   $dump = fread($handle, 3);
   print_r( hexdec(bin2hex($contents))  );
   
   echo "<br>";

   $contents = fread($handle, 1);
   $dump = fread($handle, 3);
   print_r( hexdec(bin2hex($contents))  );

   echo "<br>";

   $contents1 = hexdec(bin2hex(fread($handle, 1)));
   $contents2 = hexdec(bin2hex(fread($handle, 1))) * 256;
   $contents3 = hexdec(bin2hex(fread($handle, 1))) * 65536;
   $contents4 = hexdec(bin2hex(fread($handle, 1))) * 16777216;
   print_r( $contents1 + $contents2 + $contents3 + $contents4 );

   fclose($handle);

?>


This is now my output....
Should be decimal... Does this look right?

Code:
Voxel Animation
1
1
1
2268
[/code]

_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Back to top
View user's profile Send private message Skype Account
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Sat Jun 20, 2015 7:03 am    Post subject: Reply with quote  Mark this post and the followings unread

The triple post!!!

Code:

<?php

   function readLong($handle){
      $contents=[];
      for($x=0;$x!=4;++$x){//reads 4 bytes for long
         array_push($contents, hexdec(bin2hex(fread($handle, 1))));
      }

      $contents[0] = intval($contents[0]);
      $contents[1] = intval($contents[1]) * 256;
      $contents[2] = intval($contents[2]) * 65536;
      $contents[3] = intval($contents[3]) * 16777216;

      return ($contents[0]+$contents[1]+$contents[2]+$contents[3]);
   }

   $filename = "ody.vxl";
   $handle = fopen($filename, "rb");

   echo "struct vxl_header<br>{<br>FileType = ";

   //reads - FileType: packed array[1..16] of Char; always "Voxel Animation"
   $contents = fread($handle, 16);
   print_r( $contents );

   echo "<br>NumPalettes = ";

   //reads - NumPalettes, always 1
   echo readLong($handle);
   
   echo "<br>NumSections = ";
   
   //reads - long n_limbs; Number of limb headers/bodies/tailers
   $numlimbs = readLong($handle);
   echo $numlimbs;
   
   echo "<br>NumSections2 = ";

   //reads - long n_limbs2; Always the same as n_limbs
   echo readLong($handle);

   echo "<br>BodySize = ";

   //reads - long bodysize; Total size in bytes of all limb bodies
   echo readLong($handle);

   echo "<br>StartPaletteRemap = ";

   $contents = fread($handle, 1);
   $dump = fread($handle, 1);
   print_r( hexdec(bin2hex($contents))  );

   echo "<br>PalleteData = Skips 768 bytes (Doesnt seem like theres anything in there...)";

   //Skips 768 bytes for PalleteData (Doesnt seem like theres anything in there...)
   $contents = fread($handle, 768);


   for($x=0;$x!=$numlimbs;++$x){
      echo "<br>}<br><br>struct vxl_limb_header " . $x . "<br>{<br>limb_name = ";
   
      //reads - char limb_name[16]; ASCIIZ string - name of section
      $contents = fread($handle, 16);
      print_r( $contents  );

      echo "<br>limb_number = ";

      //reads - long limb_number; Limb number
      echo readLong($handle);

      echo "<br>unknown = ";

      //reads - long unknown; Always 1
      echo readLong($handle);

      echo " (Should Always be 1)<br>unknown2 = ";

      //reads - long unknown2; Always 0
      echo readLong($handle);

      echo " (Should Always be 0)<br>}";
   }

   fclose($handle);

?>


A little further and I think I am getting the hang of this binary file reading...  Laughing

That code outputs this...

Code:
struct vxl_header
{
FileType = Voxel Animation
NumPalettes = 1
NumSections = 7
NumSections2 = 7
BodySize = 61360
StartPaletteRemap = 16
PalleteData = Skips 768 bytes (Doesnt seem like theres anything in there...)
}

struct vxl_limb_header 0
{
limb_name = DUMMY0
limb_number = 0
unknown = 1 (Should Always be 1)
unknown2 = 0 (Should Always be 0)
}
}

struct vxl_limb_header 1
{
limb_name = DUMMY1
limb_number = 1
unknown = 1 (Should Always be 1)
unknown2 = 0 (Should Always be 0)
}
}

struct vxl_limb_header 2
{
limb_name = DUMMY2
limb_number = 2
unknown = 1 (Should Always be 1)
unknown2 = 0 (Should Always be 0)
}
}

struct vxl_limb_header 3
{
limb_name = DUMMY3
limb_number = 3
unknown = 1 (Should Always be 1)
unknown2 = 0 (Should Always be 0)
}
}

struct vxl_limb_header 4
{
limb_name = DUMMY4
limb_number = 4
unknown = 1 (Should Always be 1)
unknown2 = 0 (Should Always be 0)
}
}

struct vxl_limb_header 5
{
limb_name = DUMMY5
limb_number = 5
unknown = 1 (Should Always be 1)
unknown2 = 0 (Should Always be 0)
}
}

struct vxl_limb_header 6
{
limb_name = DUMMY6
limb_number = 6
unknown = 1 (Should Always be 1)
unknown2 = 0 (Should Always be 0)
}


Current reads:
struct vxl_header
struct vxl_limb_header's

does any one know if the PalleteData section is empty? I tried getting numbers out of it with no luck... always seemed to be "0"... So I just skipped 768 bytes... Seems like its fine as its able to read the limb headers...

And I am still not sure my BodySize is correct... It should be... As I am using the same function as the NumSections... and that looks good.

_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Back to top
View user's profile Send private message Skype Account
G-E
Defense Minister


Joined: 09 Feb 2015

PostPosted: Sat Jun 20, 2015 7:05 am    Post subject: Reply with quote  Mark this post and the followings unread

For your single section 3k voxel, yea...

http://xhp.xwis.net/documents/VXL_Format.txt

Looks like you have to work backwards using the body size, minus the multiple of tailers to find where they start. So if I read that right, a 3 section voxel has:

- main header
- section header
- section header
- section header
- body
- body
- body
- section tailer
- section tailer
- section tailer

_________________
http://www.moddb.com/mods/scorched-earth-ra2-mod-with-smart-ai

Back to top
View user's profile Send private message
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Sat Jun 20, 2015 7:37 am    Post subject: Reply with quote  Mark this post and the followings unread

Okay... So then by what you are saying, I have to skip the body section.

Then go through the trailers...



But I have new problem... This transformation matrix...

float transform[4][4]; /* Inverse(?) transformation matrix */

My question now is how do I handle floats...?

size of a float is 32 bits just like the longs...

I just tried to reuse the long function with no luck...

This is what I see so far...

Code:
span_start_off=0
span_end_off=2552
span_data_off=5104


1034594987
4287365009
4292870099


The first 3 lines seem fine... But the floats do not look right...  Laughing

_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Back to top
View user's profile Send private message Skype Account
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Sat Jun 20, 2015 9:52 am    Post subject: Reply with quote  Mark this post and the followings unread

This is odd....

This is from OS Voxel Tools...
Code:
TVoxelSectionTailer = packed record
      SpanStartOfs,
      SpanEndOfs,
      SpanDataOfs: LongInt;

      ***Det: Single;

      Transform: packed array[1..3,1..4] of Single;
      MinBounds, MaxBounds: packed array[1..3] of Single;
      XSize,
      YSize,
      ZSize,
      NormalsType: Byte; // always 2 (or 4?); possibly normals-encoding scheme selection
   end;


Whats bothering me is this "Det: Single;" This is another floating point...

and it looks like there array is [3][4]....

In DMZ/Eol There is no Det... as showen bellow... They do not even have that extra float... Witch one is correct?

Code:
struct vxl_section_tailer
  {
    long  span_start_off;  /* Offset into body section to span start list */
    long  span_end_off;    /* Offset into body section to span end list */
    long  span_data_off;   /* Offset into body section to span data */
    float transform[4][4]; /* Inverse(?) transformation matrix */
    float scale[3];        /* Scaling vector for the image */
    char  xsize;           /* Width of the voxel limb */
    char  ysize;           /* Breadth of the voxel limb */
    char  zsize;           /* Height of the voxel limb */
    char  unknown;         /* Always 2 - unknown */
  };


And I am still not sure on how to deal with floats...  Laughing I keep trying... and  am haveing no luck...  Laughing

_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Back to top
View user's profile Send private message Skype Account
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Sat Jun 20, 2015 10:27 am    Post subject: Reply with quote  Mark this post and the followings unread

The triple post again!  Laughing

Current code... Razz

Code:

<?php

   function readLong($handle){
      $contents=[];
      for($x=0;$x!=4;++$x){//reads 4 bytes for long
         array_push($contents, hexdec(bin2hex(fread($handle, 1))));
      }

      $contents[0] = $contents[0];
      $contents[1] = $contents[1] * 256;
      $contents[2] = $contents[2] * 65536;
      $contents[3] = $contents[3] * 16777216;

      return ($contents[0]+$contents[1]+$contents[2]+$contents[3]);
   }

   $filename = "ody.vxl";

   echo $filename . ': ' . filesize($filename) . ' bytes';

   $handle = fopen($filename, "rb");

   echo "<br>struct vxl_header<br>{<br>FileType = ";

   //reads - FileType: packed array[1..16] of Char; always "Voxel Animation"
   $contents = fread($handle, 16);
   print_r( $contents );

   echo "<br>NumPalettes = ";

   //reads - NumPalettes, always 1
   echo readLong($handle);
   
   echo "<br>NumSections = ";
   
   //reads - long n_limbs; Number of limb headers/bodies/tailers
   $numlimbs = readLong($handle);
   echo $numlimbs;
   
   echo "<br>NumSections2 = ";

   //reads - long n_limbs2; Always the same as n_limbs
   echo readLong($handle);

   echo "<br>BodySize = ";

   //reads - long bodysize; Total size in bytes of all limb bodies
   $bodysize = readLong($handle);
   echo $bodysize;

   echo "<br>StartPaletteRemap = ";

   $contents = fread($handle, 1);
   $dump = fread($handle, 1);
   print_r( hexdec(bin2hex($contents))  );

   echo "<br>PalleteData = Skips 768 bytes (Doesnt seem like theres anything in there...)";

   //Skips 768 bytes for PalleteData (Doesnt seem like theres anything in there...)
   $contents = fread($handle, 768);


   for($x=0;$x!=$numlimbs;++$x){
      echo "<br>}<br><br>struct vxl_limb_header " . $x . "<br>{<br>limb_name = ";
   
      //reads - char limb_name[16]; ASCIIZ string - name of section
      $contents = fread($handle, 16);
      print_r( $contents  );

      echo "<br>limb_number = ";

      //reads - long limb_number; Limb number
      echo readLong($handle);

      echo "<br>unknown = ";

      //reads - long unknown; Always 1
      echo readLong($handle);

      echo " (Should Always be 1)<br>unknown2 = ";

      //reads - long unknown2; Always 0
      echo readLong($handle);

      echo " (Should Always be 0)<br>}";
   }

   //skips all body data for now...
   $dump = fread($handle, $bodysize);


   echo "<br>";
   echo "<br>";

   function hexTo32Float($strHex) {
      return $strHex;
   }

   for($y=0;$y!=$numlimbs;++$y){
      echo "struct vxl_section_tailer " . $y . "<br>{";

      //reads - span_start_off; Offset into body section to span start list
      echo "<br>SpanStartOfs = " . readLong($handle);

      //reads - span_end_off; Offset into body section to span end list
      echo "<br>SpanEndOfs = " . readLong($handle);

      //reads - span_data_off; Offset into body section to span data
      echo "<br>SpanDataOfs = " . readLong($handle);

      echo "<br>Det = ";

      $con1 = bin2hex(fread($handle, 1));
      $con2 = bin2hex(fread($handle, 1));
      $con3 = bin2hex(fread($handle, 1));
      $con4 = bin2hex(fread($handle, 1));

      echo hexTo32Float( ($con4 . $con3 . $con2 . $con1) );

      echo "<br>Transform = [";

      for($x=0;$x!=12;++$x){
         $con1 = bin2hex(fread($handle, 1));
         $con2 = bin2hex(fread($handle, 1));
         $con3 = bin2hex(fread($handle, 1));
         $con4 = bin2hex(fread($handle, 1));

         echo hexTo32Float( ($con4 . " " . $con3 . " " . $con2 . " " . $con1) );
         if($x!=12){
            echo ",";
         }
      }

      echo "]<br>MinBounds = [";

      for($x=0;$x!=3;++$x){
         $con1 = bin2hex(fread($handle, 1));
         $con2 = bin2hex(fread($handle, 1));
         $con3 = bin2hex(fread($handle, 1));
         $con4 = bin2hex(fread($handle, 1));

         echo hexTo32Float( ($con4 . $con3 . $con2 . $con1) );
         if($x!=2){
            echo ",";
         }
      }   

      echo "]<br>MaxBounds = [";

      for($x=0;$x!=3;++$x){
         $con1 = bin2hex(fread($handle, 1));
         $con2 = bin2hex(fread($handle, 1));
         $con3 = bin2hex(fread($handle, 1));
         $con4 = bin2hex(fread($handle, 1));

         echo hexTo32Float( ($con4 . $con3 . $con2 . $con1) );
         if($x!=2){
            echo ",";
         }
      }

      echo "]<br>XSize = ";

      echo hexdec(bin2hex(fread($handle, 1)));
      echo "<br>YSize = ";
      echo hexdec(bin2hex(fread($handle, 1)));
      echo "<br>ZSize = ";
      echo hexdec(bin2hex(fread($handle, 1)));
      echo "<br>NormalsType = ";
      echo hexdec(bin2hex(fread($handle, 1)));

      echo " (always 2 (or 4?))<br>}";
      echo "<br><br>";
   }

   fclose($handle);

?>


Current output


Code:

ody.vxl: 63002 bytes
struct vxl_header
{
FileType = Voxel Animation
NumPalettes = 1
NumSections = 7
NumSections2 = 7
BodySize = 61360
StartPaletteRemap = 16
PalleteData = Skips 768 bytes (Doesnt seem like theres anything in there...)
}

struct vxl_limb_header 0
{
limb_name = DUMMY0
limb_number = 0
unknown = 1 (Should Always be 1)
unknown2 = 0 (Should Always be 0)
}
}

struct vxl_limb_header 1
{
limb_name = DUMMY1
limb_number = 1
unknown = 1 (Should Always be 1)
unknown2 = 0 (Should Always be 0)
}
}

struct vxl_limb_header 2
{
limb_name = DUMMY2
limb_number = 2
unknown = 1 (Should Always be 1)
unknown2 = 0 (Should Always be 0)
}
}

struct vxl_limb_header 3
{
limb_name = DUMMY3
limb_number = 3
unknown = 1 (Should Always be 1)
unknown2 = 0 (Should Always be 0)
}
}

struct vxl_limb_header 4
{
limb_name = DUMMY4
limb_number = 4
unknown = 1 (Should Always be 1)
unknown2 = 0 (Should Always be 0)
}
}

struct vxl_limb_header 5
{
limb_name = DUMMY5
limb_number = 5
unknown = 1 (Should Always be 1)
unknown2 = 0 (Should Always be 0)
}
}

struct vxl_limb_header 6
{
limb_name = DUMMY6
limb_number = 6
unknown = 1 (Should Always be 1)
unknown2 = 0 (Should Always be 0)
}

struct vxl_section_tailer 0
{
SpanStartOfs = 0
SpanEndOfs = 2552
SpanDataOfs = 5104
Det = 3daaaaab
Transform = [ff 8b ff 91,ff df ff d3,ff 90 ff 86,ff 8d ff 8a,ff 92 ff df,ff 91 ff 9a,ff 9e ff df,ff 9a ff 8d,ff 98 ff df,ff 90 ff 90,ff df ff 9b,ff 97 ff 8c,]
MinBounds = [c0a42c58,c11eb357,bedfc9ab]
MaxBounds = [4124503b,41261975,408ed032]
XSize = 22
YSize = 29
ZSize = 7
NormalsType = 4 (always 2 (or 4?))
}

struct vxl_section_tailer 1
{
SpanStartOfs = 9673
SpanEndOfs = 11389
SpanDataOfs = 13105
Det = 3daaaaab
Transform = [ff 8b ff df,ff 8d ff 9e,ff 9a ff 98,ff 8c ff 8b,ff 95 ff df,ff 8c ff 8a,ff df ff 8b,ff 8a ff 90,ff df ff 8b,ff 99 ff 90,ff 8d ff df,ff 91 ff 9e,]
MinBounds = [c1b36e5f,c072031e,c07a470c]
MaxBounds = [409be01b,407ac9b0,41bb1d85]
XSize = 39
YSize = 11
ZSize = 39
NormalsType = 4 (always 2 (or 4?))
}

struct vxl_section_tailer 2
{
SpanStartOfs = 21544
SpanEndOfs = 22292
SpanDataOfs = 23040
Det = 3daaaaab
Transform = [ff 9c ff df,ff 96 ff 93,ff 94 ff 9c,ff 90 ff df,ff df ff 91,ff 91 ff 90,ff df ff 9a,ff 99 ff 90,ff 86 ff df,ff 8a ff 90,ff df ff 8d,ff 9a ff 92,]
MinBounds = [c10e8ecf,c083732b,c10f20a2]
MaxBounds = [403f5e5f,4065e677,403d1711]
XSize = 17
YSize = 11
ZSize = 17
NormalsType = 4 (always 2 (or 4?))
}

struct vxl_section_tailer 3
{
SpanStartOfs = 25866
SpanEndOfs = 27670
SpanDataOfs = 29474
Det = 3daaaaab
Transform = [ff 8a ff df,ff df ff 8f,ff 91 ff 96,ff 9e ff df,ff 8c ff df,ff 9e ff 92,ff 93 ff 93,ff 9b ff df,ff 99 ff 9a,ff 91 ff 9a,ff 96 ff 8c,ff 9a ff 89,]
MinBounds = [c1c43b2c,c062cc53,c077b8fa]
MaxBounds = [408579b6,4085003e,41aaa27a]
XSize = 41
YSize = 11
ZSize = 36
NormalsType = 4 (always 2 (or 4?))
}

struct vxl_section_tailer 4
{
SpanStartOfs = 37837
SpanEndOfs = 40389
SpanDataOfs = 42941
Det = 3daaaaab
Transform = [ff df ff 9a,ff 9a ff 97,ff 9c ff df,ff 91 ff 9e,ff 99 ff df,ff 8d ff 96,ff df ff 9a,ff 8b ff 9e,ff 9e ff df,ff 98 ff df,ff 9a ff 8d,ff 8b ff 9e,]
MinBounds = [c0a42c58,c12618d8,bf2ad53e]
MaxBounds = [4124503b,411eb3f5,40877225]
XSize = 22
YSize = 29
ZSize = 7
NormalsType = 4 (always 2 (or 4?))
}

struct vxl_section_tailer 5
{
SpanStartOfs = 47428
SpanEndOfs = 48148
SpanDataOfs = 48868
Det = 3daaaaab
Transform = [ff 89 ff 9e,ff df ff 9a,ff 8f ff 8c,ff 9c ff 9a,ff 9e ff 96,ff df ff 93,ff 9d ff 9e,ff 93 ff 96,ff 8b ff 96,ff 9a ff 96,ff df ff 8c,ff 96 ff 93,]
MinBounds = [c119c202,c077af8b,c103ed6f]
MaxBounds = [403f5e5f,40485077,403d1711]
XSize = 18
YSize = 10
ZSize = 16
NormalsType = 4 (always 2 (or 4?))
}

struct vxl_section_tailer 6
{
SpanStartOfs = 51612
SpanEndOfs = 53556
SpanDataOfs = 55500
Det = 3daaaaab
Transform = [ff df ff 92,ff 90 ff 8b,ff 9b ff df,ff 8c ff 9a,ff 8d ff 8b,ff 86 ff 90,ff 8b ff df,ff 9a ff 97,ff 8d ff df,ff 92 ff 9a,ff 96 ff 9e,ff 96 ff 91,]
MinBounds = [c0bfb35d,c11325bf,c05b917a]
MaxBounds = [40d37fd6,411b40a7,40cbd0db]
XSize = 18
YSize = 27
ZSize = 14
NormalsType = 4 (always 2 (or 4?))
}


Long post short... It looks like the DMZ/Eol doc could be wrong... The data structure from OS vxl seems to hold true...

I am not sure what Det could be either...

I just do not know what to do about those floats...

Does any one know how to go from binary to float? Laughing

_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Back to top
View user's profile Send private message Skype Account
Graion Dilach
Defense Minister


Joined: 22 Nov 2010
Location: Iszkaszentgyorgy, Hungary

PostPosted: Sat Jun 20, 2015 10:31 am    Post subject: Reply with quote  Mark this post and the followings unread

Question: why php?

_________________
"If you didn't get angry and mad and frustrated, that means you don't care about the end result, and are doing something wrong." - Greg Kroah-Hartman
=======================
Past C&C projects: Attacque Supérior (2010-2019); Valiant Shades (2019-2021)
=======================
WeiDU mods: Random Graion Tweaks | Graion's Soundsets
Maintainance: Extra Expanded Enhanced Encounters! | BGEESpawn
Contributions: EE Fixpack | Enhanced Edition Trilogy | DSotSC (Trilogy) | UB_IWD | SotSC & a lot more...

Back to top
View user's profile Send private message Visit poster's website ModDB Profile ID
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Sat Jun 20, 2015 10:32 am    Post subject: Reply with quote  Mark this post and the followings unread

My plan is to use data in WebGL

_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Back to top
View user's profile Send private message Skype Account
G-E
Defense Minister


Joined: 09 Feb 2015

PostPosted: Sat Jun 20, 2015 10:48 am    Post subject: Reply with quote  Mark this post and the followings unread

You got me curious haha Smile

Code:
set voxel "c:/cc-ra2/ecache07/arachnid.vxl"   
catch {open $voxel r} of
if ![string match "file*" $of] {puts "voxel not readable ..." ; exit}
fconfigure $of -translation binary
set len [file size $voxel]
puts "Voxel Size: $len ..."
binary scan [read $of 16] a* vxlname
set vxlname [lindex [split $vxlname \x00] 0]
puts "Voxel Descriptor: $vxlname"
binary scan [read $of 4] i* pal
binary scan [read $of 4] i* sects1
binary scan [read $of 4] i* sects2
binary scan [read $of 4] i* num
puts "Palettes: $pal"
puts "Header Sections: $sects1"
puts "Tailer Sections: $sects2"
puts "Body Data Size: $num"
binary scan [read $of 1] H* b1
binary scan [read $of 1] H* b2
puts "Remap Index Range: [expr 0x$b1] to [expr 0x$b2]" ;# palette indexes of remap start/end

# sort out offsets
puts "Current position: [tell $of]"
set dummy [read $of 768] ;# useless RGB palette data
set datastart [tell $of]

# section headers
set loopcnt 0
while {$loopcnt < $sects1} {
   set secname "" ; set rawsec ""
   binary scan [read $of 16] a* secname
   set secname [lindex [split $secname \x00] 0]
   binary scan [read $of 4] i* secnum
   set dummy [read $of 8] ;# discarding useless data
   puts "Voxel Section Header = $secname"
   puts "Section Descriptor: $secnum"
   set limbname($secnum) $secname
   incr loopcnt
}   

# voxel data goes here
# ...except it doesn't!

# section tailers
set dataend [expr $len-(92*$sects2)]
seek $of $dataend
set loopcnt 0
while {$loopcnt < $sects2} {
   binary scan [read $of 4] i* lstart
   binary scan [read $of 4] i* lend
   binary scan [read $of 4] i* loffset
   puts "Voxel Section $loopcnt: $lstart to $lend at offset $loffset"
   
   set dummy [read $of 76] ;# the transform float data
   
   binary scan [read $of 1] H* xsize ;# pixel widths per axis
   binary scan [read $of 1] H* ysize
   binary scan [read $of 1] H* zsize
   set dummy [read $of 1] ;# discard junk
   puts "Voxel Section Dimensions X/Y/Z: [expr 0x$xsize]/[expr 0x$ysize]/[expr 0x$zsize]"
      
   incr loopcnt
}   
close $of


Code:
Voxel Size: 60234 ...
Voxel Descriptor: Voxel Animation
Palettes: 1
Header Sections: 5
Tailer Sections: 5
Body Data Size: 58832
Remap Index Range: 16 to 31
Current position: 34
Voxel Section Header = LF
Section Descriptor: 0
Voxel Section Header = LR
Section Descriptor: 1
Voxel Section Header = RF
Section Descriptor: 2
Voxel Section Header = RR
Section Descriptor: 3
Voxel Section Header = body
Section Descriptor: 4
Voxel Section 0: 0 to 1408 at offset 2816
Voxel Section Dimensions X/Y/Z: 16/22/32
Voxel Section 1: 6634 to 8042 at offset 9450
Voxel Section Dimensions X/Y/Z: 16/22/32
Voxel Section 2: 13268 to 14676 at offset 16084
Voxel Section Dimensions X/Y/Z: 16/22/32
Voxel Section 3: 19910 to 21318 at offset 22726
Voxel Section Dimensions X/Y/Z: 16/22/32
Voxel Section 4: 26552 to 34232 at offset 41912
Voxel Section Dimensions X/Y/Z: 60/32/30

Back to top
View user's profile Send private message
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Sat Jun 20, 2015 2:31 pm    Post subject: Reply with quote  Mark this post and the followings unread

This is weird...

http://zombapro.ppmsite.com/iniedit/vxledit.php

Okay if you go to this link it will parse through the vxl data...

Now what I am totally blown away by is 30 bytes off...

The files SpanDataOfs = 800
but!!!
the real vxl data should start at 830...

header = 802 bytes
limb header = 28 bytes
add those two up and you get 830...

Now I am not sure whats really meant to be in span_start[n] & span_end[n], but I should be able to just read through 100 long(4 bytes each) for both span_start[n] & span_end[n], as my X and Y (10*10)...

But there is a 30 bytes still floating around...

I was able to look in OS vxl edit, and match vxl pal index and normal index with data with from the php script to find out theirs a 30 bytes problem...

code... if you dare to look...
http://zombapro.ppmsite.com/iniedit/vxledit.txt

Any one have any ideas...?

_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Back to top
View user's profile Send private message Skype Account
G-E
Defense Minister


Joined: 09 Feb 2015

PostPosted: Sat Jun 20, 2015 2:58 pm    Post subject: Reply with quote  Mark this post and the followings unread

802 is the fixed header size with the 768 palette bytes, before the section headers, and you know each tailer is 92 bytes. This means your actual section data is between start+830 and end-92 in your case...

This is correct: BodySize = 28336

_________________
http://www.moddb.com/mods/scorched-earth-ra2-mod-with-smart-ai

Back to top
View user's profile Send private message
G-E
Defense Minister


Joined: 09 Feb 2015

PostPosted: Sat Jun 20, 2015 3:15 pm    Post subject: Reply with quote  Mark this post and the followings unread

I added bounds checking to my script... here we can clearly see VXLSE added the wrong scale Smile

Code:
Voxel Size: 60234 ...
Voxel Descriptor: Voxel Animation
Palettes: 1
Header Sections: 5
Tailer Sections: 5
Body Data Size: 58832
Remap Index Range: 16 to 31
Voxel Section Header = LF
Section Descriptor: 0
Voxel Section Header = LR
Section Descriptor: 1
Voxel Section Header = RF
Section Descriptor: 2
Voxel Section Header = RR
Section Descriptor: 3
Voxel Section Header = body
Section Descriptor: 4
Voxel Section 0: 0 to 1408 at offset 2816
Voxel Scale: 0.833329975605011
Section Dimensions L/W/H: 16/22/32
Normals mode: 2
Voxel Bounds:
  Min Length: -7.0
  Min width: 3.5
  Min height: -5.0
  Max Length: 6.0
  Min width: 21.5
  Min height: 22.0
Voxel Section 1: 6634 to 8042 at offset 9450
Voxel Scale: 0.833329975605011
Section Dimensions L/W/H: 16/22/32
Normals mode: 2
Voxel Bounds:
  Min Length: -7.0
  Min width: 3.5
  Min height: -5.0
  Max Length: 6.0
  Min width: 21.5
  Min height: 22.0
Voxel Section 2: 13268 to 14676 at offset 16084
Voxel Scale: 0.833329975605011
Section Dimensions L/W/H: 16/22/32
Normals mode: 2
Voxel Bounds:
  Min Length: -7.0
  Min width: -21.0
  Min height: -5.0
  Max Length: 6.0
  Min width: -3.0
  Min height: 22.0
Voxel Section 3: 19910 to 21318 at offset 22726
Voxel Scale: 0.833329975605011
Section Dimensions L/W/H: 16/22/32
Normals mode: 2
Voxel Bounds:
  Min Length: -7.0
  Min width: -21.0
  Min height: -5.0
  Max Length: 6.0
  Min width: -3.0
  Min height: 22.0
Voxel Section 4: 26552 to 34232 at offset 41912
Voxel Scale: 0.833329975605011
Section Dimensions L/W/H: 60/32/30
Normals mode: 2
Voxel Bounds:
  Min Length: -18.0
  Min width: -9.0
  Min height: 0.5
  Max Length: 18.0
  Min width: 9.0
  Min height: 16.5

_________________
http://www.moddb.com/mods/scorched-earth-ra2-mod-with-smart-ai

Back to top
View user's profile Send private message
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Sun Jun 21, 2015 5:37 am    Post subject: Reply with quote  Mark this post and the followings unread

Almost got it! This is great! it reads multi-sections and the data looks good!

http://zombapro.ppmsite.com/iniedit/vxledit2.php

Now I just have to figure out why more complicated sectioned vxl's do not work 100%... Laughing

G-E how do you do some thing like whats bellow, for floats?

Code:
set num [expr (0x$b4 * 16777216) + (0x$b3 * 65536) + (0x$b2 * 256) + 0x$b1]

_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Back to top
View user's profile Send private message Skype Account
G-E
Defense Minister


Joined: 09 Feb 2015

PostPosted: Sun Jun 21, 2015 10:32 am    Post subject: Reply with quote  Mark this post and the followings unread

http://php.net/manual/en/function.bindec.php

Maybe?

Or use unpack like I suggested earlier, I just looked it up and you can indeed parse 32bit integers and floats with it directly into an array... perhaps you can use it to unpack whole sequences using a larger read?

_________________
http://www.moddb.com/mods/scorched-earth-ra2-mod-with-smart-ai

Back to top
View user's profile Send private message
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Sun Jun 21, 2015 12:29 pm    Post subject: Reply with quote  Mark this post and the followings unread

Ya unpack ended up working.  Razz

I thought it didn't work but it was just that i was not paying attention to the array index...

unpack returns an array, index "0" is used for some thing else... and that's index i was trying to pull the float out of... as it turns out the returning float i wanted was in index "1"...

Silly me...  Laughing

And now I feel as though I have all the data out of a vxl file that I need... as shown in link below! I improved my span_data read code... Got it working right. Razz

http://zombapro.ppmsite.com/iniedit/vxledit3.php




Though the final problem is the Transform section, does not work on some vxls... But I do not think I will need that... Hopefully...

http://zombapro.ppmsite.com/iniedit/vxledit4.php

Transform = [nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan,nan]
MinBounds = [-5.13,-9.92,-0.44]
MaxBounds = [10.27,10.38,4.46]

For some reason they return as "nan" (I guess not a real number...)

Not to sure what to make out of that...  Confused  Any one have any thoughts?





Anyways!!! I can be any more happy I think I have every thing I need to start working on the other part of this little project of mine.

Time to turn read vxl data into point cloud data...  Laughing  This will be fun!  Laughing The good news is that I see the pattern of how the vxl span_dat is stored and how it represents a ... tank I guess.

Wish me luck!

AH and thanks a bunch G-E! you have really helped me allot with this!  Smile

_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Back to top
View user's profile Send private message Skype Account
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Sun Jun 21, 2015 7:30 pm    Post subject: Reply with quote  Mark this post and the followings unread


_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Back to top
View user's profile Send private message Skype Account
G-E
Defense Minister


Joined: 09 Feb 2015

PostPosted: Sun Jun 21, 2015 7:38 pm    Post subject: Reply with quote  Mark this post and the followings unread

Now try compacting your code, read multiple integers/floats at once, pretty sure you can do something like this:

Code:

set voxel "c:/cc-ra2/ecache07/arachnid.vxl"   
catch {open $voxel r} of
if ![string match "file*" $of] {puts "voxel not readable ..." ; exit}
fconfigure $of -translation binary
set len [file size $voxel]
puts "Voxel Size: $len ..."
binary scan [read $of 16] a* vxlname
set vxlname [lindex [split $vxlname \x00] 0]
puts "Voxel Descriptor: $vxlname"
binary scan [read $of 18] {i i i i c c} pal sects1 sects2 num b1 b2
puts "Palettes: $pal"
puts "Header Sections: $sects1"
puts "Tailer Sections: $sects2"
puts "Body Data Size: $num"
puts "Remap Index Range: $b1 to $b2" ;# palette indexes of remap start/end

# sort out offsets, skip palette
seek $of 802

# section headers
set loopcnt 0
while {$loopcnt < $sects1} {
   set secname ""
   binary scan [read $of 16] a* secname
   set secname [lindex [split $secname \x00] 0]
   binary scan [read $of 4] i secnum
   set dummy [read $of 8] ;# discarding useless data
   puts "Voxel Section Header = $secname"
   puts "Section Descriptor: $secnum"
   set limbname($secnum) $secname
   incr loopcnt
}   

# voxel data goes here
# ...except it doesn't!

# section tailers
set dataend [expr $len-(92*$sects2)]
seek $of $dataend
set loopcnt 0
while {$loopcnt < $sects2} {
   binary scan [read $of 16] {i i i f} lstart lend loffset lscale
   puts "Voxel Section $loopcnt: $lstart to $lend at offset $loffset"
   puts "Voxel Scale: $lscale"
   set dummy [read $of 48] ;# the transform float data
   binary scan [read $of 24] {f f f f f f} xmin ymin zmin xmax ymax zmax
   binary scan [read $of 4] {c c c c} xsize ysize zsize normals
   puts "Section Dimensions L/W/H: $xsize/$ysize/$zsize"
   puts "Normals mode: $normals"
   puts "Voxel Bounds:"
   puts "  Min Length: $xmin"
   puts "  Min width: $ymin"
   puts "  Min height: $zmin"
   puts "  Max Length: $xmax"
   puts "  Max width: $ymax"
   puts "  Max height: $zmax"
   incr loopcnt
}   
close $of

_________________
http://www.moddb.com/mods/scorched-earth-ra2-mod-with-smart-ai

Back to top
View user's profile Send private message
G-E
Defense Minister


Joined: 09 Feb 2015

PostPosted: Sun Jun 21, 2015 7:46 pm    Post subject: Reply with quote  Mark this post and the followings unread

MadHQ wrote:
Initial WebGL rendering set-up of a voxel. Can you guess what it is?

Are you cropping any lines off? It looks hollow like a rowboat...

Since we can zoom, is there any reason you're spacing out the dots so much?

Back to top
View user's profile Send private message
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Mon Jun 22, 2015 8:37 am    Post subject: Reply with quote  Mark this post and the followings unread

G-E wrote:
MadHQ wrote:
Initial WebGL rendering set-up of a voxel. Can you guess what it is?

Are you cropping any lines off? It looks hollow like a rowboat...

Since we can zoom, is there any reason you're spacing out the dots so much?


It was an initial set up.  Laughing
My height counter was off... The vxl file has an odd way to store what height its on.. Laughing

As for the point cloud dots, at the time I didn't care about distance or size...

That should be okay for now... As shown below... (You should be able to guess what it now. hopefully...)

http://zombapro.ppmsite.com/iniedit/voxelload/voxel2.php

But now my next problem is that voxel colors do not seem to make since... The pattern of colors does not look right...

And the colors are not right... I am just loading the unittem.pal... That should be okay... Though I did look in the vxl file and it does store rbg color values.... >_< I did try that with no luck...

Unless any one has any ideas on the vxl colors I guess i will just have to keep playing around...

_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Back to top
View user's profile Send private message Skype Account
G-E
Defense Minister


Joined: 09 Feb 2015

PostPosted: Mon Jun 22, 2015 12:12 pm    Post subject: Reply with quote  Mark this post and the followings unread

I'm sure Banshee could answer this all in like 2.3sec, I've had to learn the voxel format along with you to help haha...

Here's a clue about the packed data, VXLSE is doing bit shifts to find the right numbers:
Code:

function TVoxelSection.PackVoxel(Unpacked: TVoxelUnpacked): TVoxelPacked;
begin
   Result := Unpacked.Flags shl 16;
   if Unpacked.Used then
      Result := Result or $100 or Unpacked.Colour
   else
      Result := Result or Unpacked.Colour;
   Result := Result shl 8;
   Result := Result or Unpacked.Normal;
end;

procedure TVoxelSection.UnpackVoxel(PackedVoxel: TVoxelPacked; var dest: TVoxelUnpacked);
begin
   dest.Normal := (PackedVoxel and $000000FF);
   dest.Colour := (PackedVoxel and $0000FF00) shr 8;
   dest.Used :=   (PackedVoxel and $00010000) > 0;
   dest.Flags :=  (PackedVoxel and $FF000000) shr 24;
end;

_________________
http://www.moddb.com/mods/scorched-earth-ra2-mod-with-smart-ai

Back to top
View user's profile Send private message
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Mon Jun 22, 2015 1:25 pm    Post subject: Reply with quote  Mark this post and the followings unread


_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Back to top
View user's profile Send private message Skype Account
G-E
Defense Minister


Joined: 09 Feb 2015

PostPosted: Mon Jun 22, 2015 2:08 pm    Post subject: Reply with quote  Mark this post and the followings unread

Very nice Smile

Now that that's established, if you can make this in an iframe in the context of a larger page that shows some numerical data about the model, that would be cool... still not sure what use it would be to anyone besides curiosity ?

_________________
http://www.moddb.com/mods/scorched-earth-ra2-mod-with-smart-ai

Back to top
View user's profile Send private message
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Mon Jun 22, 2015 6:19 pm    Post subject: Reply with quote  Mark this post and the followings unread

G-E wrote:
Very nice Smile

Now that that's established, if you can make this in an iframe in the context of a larger page that shows some numerical data about the model, that would be cool... still not sure what use it would be to anyone besides curiosity ?


Do you want to see the data?

My plan with this was to try and make some kind of FLH locator.  Razz

Some thing else I need to get sorted through is the hva files, so I can get locations and possibly animations...

I have an attempt in the works right now... No to much luck so far...  Laughing

http://zombapro.ppmsite.com/iniedit/readhva2.php

This is as far as im at... Im just not sure on the file structure at all... Laughing It looks like I got the first parts right... but there's just to much data not used...

Code:
<?php
   $filename = "ody.hva";
   $fileSize = filesize($filename);
   $handle = fopen($filename, "rb");
   echo $filename . ': ' . $fileSize . ' bytes';

   echo "<br>THVA_Main_Header {<br>";

   echo "FilePath = ";
   echo fread($handle, 16);

   echo "<br>N_Frames = ";
   echo hexdec(bin2hex(fread($handle, 1)));

   echo "<br>N_Sections = ";
   $numSections = hexdec(bin2hex(fread($handle, 4)));
   echo $numSections . "<br>}<br>";

   for($y=0; $y!=$numSections; ++$y){
      echo "<br>TSectionName = ";
      echo fread($handle, 16);
   }

   for($y=0; $y!=$numSections; ++$y){
      echo "<br><br>Section Number " . $y . " {<br>";
   
      echo "TTransformMatrix = [";
      for($x=0; $x!=12; ++$x){
         if($x%3==0){
            echo "<br>";
         }
         echo number_format(unpack("f", fread($handle, 4))[1], 2, '.', '');
         echo ",";
      }

      echo "<br>]<br>}";

   }

   echo "<br><br>File pointer at = " . ftell($handle);

   fclose($handle);
?>



I guess this is OS vxl edits load stuff... doesnt seem to be right... >_<

Code:
function THVA.LoadFromFile(const _Filename: String): boolean;
var
   f : file;
   x: integer;
begin
   Result := false;
   if not FileExists(_Filename) then exit;

   AssignFile(F,_Filename);  // Open file
   Reset(F,1); // Goto first byte?

   BlockRead(F,Header,Sizeof(THVA_Main_Header)); // Read Header

   SetLength(Data,Header.N_Sections);

   For x := 0 to High(Data) do
      BlockRead(F,Data[x].SectionName,Sizeof(TSectionName));

   SetLength(TransformMatrices,Header.N_Frames*Header.N_Sections);

   For x := 0 to High(TransformMatrices) do
      BlockRead(F,TransformMatrices[x],Sizeof(TTransformMatrix));

   CloseFile(f);
   If Header.N_Frames < 1 then
      exit;
   WestwoodToOpenGLCoordinates;
   Result := True;
end;

_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Back to top
View user's profile Send private message Skype Account
Banshee
Supreme Banshee


Also Known As: banshee_revora (Steam)
Joined: 15 Aug 2002
Location: Brazil

PostPosted: Mon Jun 22, 2015 6:31 pm    Post subject: Reply with quote  Mark this post and the followings unread

Just to answer a question from a PM from MadHQ here:

The Det from VXLSE III's is the Scale (It originally mistakenly stood for Determinant). In Voxel Section Editor III, you can view it in Section -> Voxel Properties -> Misc. It's the Scale setting there and it is usually expected to be 1/12 {0,0833333358168602}

Back to top
View user's profile Send private message Visit poster's website Skype Account
G-E
Defense Minister


Joined: 09 Feb 2015

PostPosted: Mon Jun 22, 2015 6:52 pm    Post subject: Reply with quote  Mark this post and the followings unread

I could have told him that, you can see mine shows the scale Smile

Quote:

  binary scan [read $of 16] {i i i f} lstart lend loffset lscale
  puts "Voxel Section $loopcnt: $lstart to $lend at offset $loffset"
  puts "Voxel Scale: $lscale"

Back to top
View user's profile Send private message
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Mon Jun 22, 2015 10:14 pm    Post subject: Reply with quote  Mark this post and the followings unread


_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Back to top
View user's profile Send private message Skype Account
G-E
Defense Minister


Joined: 09 Feb 2015

PostPosted: Mon Jun 22, 2015 11:59 pm    Post subject: Reply with quote  Mark this post and the followings unread

Nice work, the ground is a nice addition, but no more zoom?

_________________
http://www.moddb.com/mods/scorched-earth-ra2-mod-with-smart-ai

Back to top
View user's profile Send private message
Mig Eater
Defense Minister


Joined: 13 Nov 2003
Location: Eindhoven

PostPosted: Tue Jun 23, 2015 9:23 am    Post subject: Reply with quote  Mark this post and the followings unread

Any chance I could use this to display units on the D-day wiki once you are done? I'm using Gif anims atm but this would be even better! I'm sure I'm not the only one that could make use of it too.

_________________



Back to top
View user's profile Send private message Visit poster's website ModDB Profile ID YouTube User URL Facebook Profile URL Twitter Channel URL
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Tue Jun 23, 2015 1:25 pm    Post subject: Reply with quote  Mark this post and the followings unread

Quote:
Nice work, the ground is a nice addition, but no more zoom?

Ya I have temporary disabled zoom/pan un-tell I get the scale/sizes fully finished.


Mig Eater wrote:
Any chance I could use this to display units on the D-day wiki once you are done? I'm using Gif anims atm but this would be even better! I'm sure I'm not the only one that could make use of it too.


You know that would be pretty cool! I will see if I can come up with some thing like that.




But I really have a lot more to do and a few obstacles to overcome, before I can make any confirmations on that...

One of the obstacles is the normal's... It seems as though the way I am doing this is not going to allow for them...

IE... I am using a pointcloud to render these, and the points do not interact with lights...

My idea now is to just try and use the normal's as a color multiplayer... Downside with that is once is been generated it says like that... Not very normal-ish... but it could be okay...

BTW is there such a thing as a normal pallet in RA2/YR... I cannot seem to find it... Is is in voxel.vpl (I think that's whats its called...), It just that I remember seeing that grey scale pallet in the old vxl edit... I just cant find it in any mix's...

_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Back to top
View user's profile Send private message Skype Account
G-E
Defense Minister


Joined: 09 Feb 2015

PostPosted: Tue Jun 23, 2015 2:01 pm    Post subject: Reply with quote  Mark this post and the followings unread

Yes voxels.vpl controls what resultant colour lightness "step" is... in your case I would just do it as a base add/subtract to the RGB value... add/sub 10-20 from whatever each byte of the 24bit value is (if they move together, tone stays the same until one hits 0 or 255).

You can also skip much of the work by generating a dot map of which pixels need highlighting at each facing (based on normals), but posterize it do say 4-6 levels only. So say you were looking at it from north, you would lump NNE/N/NNW into one level, NE/NW into the next level, and so on.

If you keep the camera angle the same, and only allow turning and zoom, this should be a fairly efficient way of calculating. There's only 256 possible normals, and you know that any given radial facing would have the same number of normals in your step groups, you'd only need a lookup table for each facing to decide which level it falls into. And while you could rotate it freely, you could set the radial steps for normals colouring to something like 15 degrees to reduce the number of lookup tables.

The lookup tables can be simple searchable arrays, one array for each facing... so in this example, 24 arrays, with 6 lightness/darkness levels, each level being a large bump to the colour.

_________________
http://www.moddb.com/mods/scorched-earth-ra2-mod-with-smart-ai

Back to top
View user's profile Send private message
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Tue Jun 23, 2015 5:49 pm    Post subject: Reply with quote  Mark this post and the followings unread

Well okay... So I am not sure how the voxel.vpl is broken up...

I looked at this... (mostly vks stuff)

http://www.ppmforums.com/viewtopic.php?highlight=voxels+vpl&t=30026

And this is what I end up seeing...

http://zombapro.ppmsite.com/iniedit/voxelload/readnormals.php

I am not really sure how that works...

Shouldn't each section have 256 different colors (with there own RGB value?)  

Or am I looking at this wrong... is each normal color (x,x,x) x = one byte?
or is it ((x),(x+1),(x+2)) x = one byte....

_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Back to top
View user's profile Send private message Skype Account
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Tue Jun 23, 2015 5:56 pm    Post subject: Reply with quote  Mark this post and the followings unread

Arrrggg!!!!

I wish I would play with ideas before posting them...

voxel.vpl format fully read and understood.
http://zombapro.ppmsite.com/iniedit/voxelload/readnormals.php


I think think this is rendering with section 11... could be 12...  Laughing
http://zombapro.ppmsite.com/iniedit/voxelload/muiltivxlnorm.php

Ra3 Giga Fortress (larger vxl...)
http://zombapro.ppmsite.com/iniedit/voxelload/muiltivxl5.php

Now its time to tweek in the normal's values so they do not look so bad...

I had to multiply the normal value by i think 0.15 as they keep bleaching to white...

_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Back to top
View user's profile Send private message Skype Account
Banshee
Supreme Banshee


Also Known As: banshee_revora (Steam)
Joined: 15 Aug 2002
Location: Brazil

PostPosted: Tue Jun 23, 2015 7:21 pm    Post subject: Reply with quote  Mark this post and the followings unread

How are you calculating the normals at the moment?

Back to top
View user's profile Send private message Visit poster's website Skype Account
Display posts from previous:   
Post new topic   Reply to topic Page 1 of 2 [89 Posts] Goto page: 1, 2 Next
Mark the topic unread ::  View previous topic :: View next topic
 
Share on TwitterShare on FacebookShare on Google+Share on DiggShare on RedditShare on PInterestShare on Del.icio.usShare on Stumble Upon
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © phpBB Group

[ Time: 0.2698s ][ Queries: 13 (0.0117s) ][ Debug on ]