Project Perfect Mod Forums
:: Home :: Get Hosted :: PPM FAQ :: Forum FAQ :: Privacy Policy :: Search :: Memberlist :: Usergroups :: Register :: Profile :: Log in to check your private messages :: Log in ::


The time now is Fri Mar 29, 2024 8:02 am
All times are UTC + 0
fclose(), CloseFile() and Close(): I declare war at you!!!
Moderators: stucuk
Post new topic   Reply to topic Page 1 of 1 [9 Posts] Mark the topic unread ::  View previous topic :: View next topic
Author Message
Banshee
Supreme Banshee


Also Known As: banshee_revora (Steam)
Joined: 15 Aug 2002
Location: Brazil

PostPosted: Wed Aug 04, 2004 1:57 am    Post subject:  fclose(), CloseFile() and Close(): I declare war at you!!! Reply with quote  Mark this post and the followings unread

Ok, I'm getting extremely pissed off at these commands. I made a function to save files on compression 3. It used to work fine last month... recently, I adapted the code to make things faster and more objective and now I'm getting access violation with CloseFile() and when I change it to Close(), I get an invalid pointer operation. Does anyone knows what does can cause this?


Anyway, here's the code of the functions have this problems:


The first function detects the best compression to use in the frame. The second one saves only using compression 3. On 2.99.05, you have an option at Preferences to set the best way to load and save files. Fortunatelly, the compression 1 save still works fine

Code:

// Compression 3 and 1.
Procedure SaveSHP(Filename:String; var SHP : TSHP); overload;
var
F : File;
Written,x,c : integer;
Image_Size : cardinal;
Databuffer2 : TDatabuffer;
size : array of integer;
begin

   AssignFile(F,Filename);  // Open file
   Rewrite(F); // Goto first byte?

   BlockWrite(F,SHP.Header,Sizeof(THeader),Written); // Write Header

   setlength(size,SHP.Header.NumImages);
   for x := 1 to SHP.header.NumImages do
   begin
      Encode3(SHP.Data[x].Databuffer,Databuffer2,SHP.Data[x].header_image.cx,SHP.Data[x].header_image.cy,size[x]); // Compression 3
      if size[x] < (SHP.Data[x].header_image.cx*SHP.Data[x].header_image.cy) then
      begin
         SetLength(SHP.Data[x].Databuffer,size[x]+1);
         SHP.Data[x].Databuffer := Databuffer2;
         SHP.Data[x].header_image.compression := 3;
         if x = 1 then
            SHP.Data[x].Header_Image.offset := SizeOf(THeader) + (SHP.Header.NumImages * SizeOf(THeader_Image))
         else
            SHP.Data[x].header_image.offset := SHP.Data[x-1].Header_Image.offset + size[x]; // Works out offset
      end
      else
      begin
         size[x] := SHP.Data[x].header_image.cx*SHP.Data[x].header_image.cy;
         SetLength(SHP.Data[x].Databuffer,size[x]+1);
         SHP.Data[x].Header_Image.compression := 1;
         SHP.Data[x].header_image.offset := WorkOutOffset(SHP,X); // Works out offset
      end;
   end;

   for x := 1 to SHP.header.NumImages do // Save Image Headers
   begin
     BlockWrite(F,SHP.Data[x].header_image,Sizeof(THeader_Image),Written);
   end;

   // Save each image to the file
   for x := 1 to SHP.header.NumImages do
   begin

//      // If it's compression 1 then, it will right databuffer on its way.
//      if shp.data[x].Header_Image.compression <= 1 then
//      begin

         // the ammount of bytes in a image (Width * Height)
//         Image_Size := SHP.Data[x].header_image.cx*SHP.Data[x].header_image.cy;


         // Write byte by byte SHP.Data[x].header_image.cy * SHP.Data[x].header_image.cx times
         if SHP.Data[x].Header_Image.offset <> 0 then
         for c := 1 to size[x] do// Image_Size do
         begin
            BlockWrite(F,SHP.Data[x].Databuffer[c-1],Sizeof(byte),Written);
         end;
   end;

      CloseFile(F);
end;

Procedure SaveSHPCompressed(Filename:String; var SHP : TSHP); overload;
var
F : file;
Written,x,c : integer;
Databuffer2 : TDatabuffer;
size : array of integer;
begin

   AssignFile(F,Filename);  // Open file
   Rewrite(F,1); // Goto first byte?

   BlockWrite(F,SHP.Header,Sizeof(THeader),Written); // Write Header

   setlength(size,SHP.Header.NumImages);
   for x := 1 to SHP.header.NumImages do
   begin
      Encode3(SHP.Data[x].Databuffer,Databuffer2,SHP.Data[x].header_image.cx,SHP.Data[x].header_image.cy,size[x]); // Compression 3
      SetLength(SHP.Data[x].Databuffer,size[x]+1);
      SHP.Data[x].Databuffer := Databuffer2;
      SHP.Data[x].header_image.compression := 3;
      if x = 1 then
         SHP.Data[x].Header_Image.offset := SizeOf(THeader) + (SHP.Header.NumImages * SizeOf(THeader_Image))
      else
         SHP.Data[x].header_image.offset := SHP.Data[x-1].Header_Image.offset + size[x]; // Works out offset
   end;

   for x := 1 to SHP.header.NumImages do // Save Image Headers
   begin
      BlockWrite(F,SHP.Data[x].header_image,Sizeof(THeader_Image),Written);
   end;

 // Save each image to the file
  for x := 1 to SHP.header.NumImages do
  begin
      // Write byte by byte size times
      if SHP.Data[x].Header_Image.offset <> 0 then
      for c := 1 to Size[x] do
      begin
         BlockWrite(F,SHP.Data[x].Databuffer[c-1],Sizeof(byte),Written);
      end;
   end;

   CloseFile(F);
end;



Both functions crash when it reads the command CloseFile(F);

If I remove this command, I have an access violation when I close the window that has the saved SHP file.

Back to top
View user's profile Send private message Visit poster's website Skype Account
Banshee
Supreme Banshee


Also Known As: banshee_revora (Steam)
Joined: 15 Aug 2002
Location: Brazil

PostPosted: Sun Aug 08, 2004 9:38 pm    Post subject: Reply with quote  Mark this post and the followings unread

I still get an invalid pointer operation message, but I've got some progress on this. The file is encoding the stuff perfectly, but offsets are done in a wrong way, as far as I could test the Force Compression 3 save.

My tests with Einstein SHP showed that the first frame was saved correctly, but all other ones got fucked up (image out of order) when I loaded in the old BS SHP Builder way. The OS SHP Builder that uses offsets, got a huge offset value that gave an access violation...

oh well, in short: I must fix the offset. I had this fixed once and I'll have it again.

_________________


Mods, Mods Support, Public Researchs, Map Archives, Tutorials, A Friendly Community and much more. Check it out now!

Back to top
View user's profile Send private message Visit poster's website Skype Account
Banshee
Supreme Banshee


Also Known As: banshee_revora (Steam)
Joined: 15 Aug 2002
Location: Brazil

PostPosted: Mon Aug 09, 2004 4:45 am    Post subject: Progress Report... Reply with quote  Mark this post and the followings unread

I've debugged the code by hand and I've found some failures on the encoding function. I've fixed it, but it's not enough to kill the "invalid pointer operation" error.

But this, and few minor changes in the SaveSHP function, allowed me to open the saved files (even not being closed properly) with the offset way, however, with the offsets fucked up, the image is entirelly distorted... there were some improvements, but it's still distorted since frame 2.

I've renabled the debug system from my new Load SHP function and it logged the saved einstein in its optimized way. About 20 frames were saved with compression 1. From all others, on 30 (of... about 160), it missed (used the wrong compression). It is possible that these misses are caused by fucked up offsets. I've also noticed that I had to change the way it calculates offsets on compression 1 (since the WorksOutOffset(); function ignores the fact that the offset from previous frames are different than the image size.


Anyway, I expect better results with the modifications I made with the latest observations. The next log will have a better format. But, for this one, here's the translation:



Picture: eins.shp re-saved with optimized size by OS SHP Builder 2.99.05

Current Offset: Location of the frame in the file. Note that it never starts at 0 because it saves the header of the SHP file and the header of all frames first. Offset will always be set to zero when the image has no contents (width * height = 0).

ImageSize: Minimum size of the frame image after compression (1 or 3)

TrueSize: Frame width * Frame height. (cx and cy according to XCC)

Compression: If the frame is compressed with compression 1, it will appear, Compression: 1 and ImageSize will be equal to True Size. Otherwise it won't appear.






EDIT: The problem happens AFTER the file is unloaded, when the variables of the function are being loaded.... now which variable does that O.o?.... But that's a good progress already.



openlog.txt
 Description:
Here you can have an idea of the progress of the thing...

Download
 Filename:  openlog.txt
 Filesize:  37.15 KB
 Downloaded:  213 Time(s)


Back to top
View user's profile Send private message Visit poster's website Skype Account
Banshee
Supreme Banshee


Also Known As: banshee_revora (Steam)
Joined: 15 Aug 2002
Location: Brazil

PostPosted: Mon Aug 09, 2004 5:11 am    Post subject: A fast progress report.... Reply with quote  Mark this post and the followings unread

First of all, as you may noticed in the edit above. The problem lies on variable unloading and not on closefile. I guess I should have it fixed soon.

Comparing the variables in the 3 saving functions, we have:


SaveSHPUncompressed (working fine):

var
f : file;
Written,x,c : integer;
Image_Size : cardinal;


SaveSHP (invalid pointer operation)

var
F : File;
Written,x,c : integer;
Image_Size : cardinal;
Databuffer2 : TDatabuffer;
size : array of integer;


SaveSHPCompressed (invalid pointer operation)

var
F : file;
Written,x,c : integer;
Databuffer2 : TDatabuffer;
size : array of integer;



So, the suspect lies on the Databuffer2, used to storage the compressed (3) databuffer (frame image) and on size that stores the size of the frame images used to determine the offset and when it writes the databuffer (frame image) to the file.




In other news, I've made a new test with Einstein and results improved considerably. The first observation is that some of the middle frames loaded perfectly, while some others are still loading wrongly. The ones who loaded perfectly are in its vast majority made with compression 1, although some of the compression 1 frames showed some light distortions. The number of misses are still high, but they are weaker misses. In the other file, there were times where the ImageSize was 1689 while the TrueSize was smaller than 500, reaching above 300% of miss. Now, the highest miss hardly reachs 25% and the file ends being quite smaller. But we still work for a smaller, but not distorted result.



openlog.txt
 Description:
better results, but still a lot to do.

Download
 Filename:  openlog.txt
 Filesize:  37.85 KB
 Downloaded:  213 Time(s)


Back to top
View user's profile Send private message Visit poster's website Skype Account
Banshee
Supreme Banshee


Also Known As: banshee_revora (Steam)
Joined: 15 Aug 2002
Location: Brazil

PostPosted: Mon Aug 09, 2004 12:17 pm    Post subject: W00t! Reply with quote  Mark this post and the followings unread

The invalid pointer operation is still not fixed, but...


The file already saves fully correctly with perfect offsets and no misses. The best of all is:

- eins.shp (Einstein) saved by Westwood and extracted by XCC: 77168 bytes
- eins0908_1.shp (Einstein) from Westwood re-saved by OS SHP Builder: 77139 bytes

It saves: 29 bytes!


Honestly, I wasn't expecting that the SHP Builder would save files smaller than Westwood/XCC, but that they would reach a similar or even a bit higher size.... but anyway, OS SHP Builder's saving efficience is 100,1% Wink Very Happy.

Here's the log of the recent file.... with no misses.



openlog.txt
 Description:
Just... perfect!

Download
 Filename:  openlog.txt
 Filesize:  37.85 KB
 Downloaded:  201 Time(s)


Back to top
View user's profile Send private message Visit poster's website Skype Account
Banshee
Supreme Banshee


Also Known As: banshee_revora (Steam)
Joined: 15 Aug 2002
Location: Brazil

PostPosted: Mon Aug 09, 2004 8:12 pm    Post subject: Reply with quote  Mark this post and the followings unread

Hah! I got it!

Pay attention to the following evil code:

Code:
SHP.Data[x].Databuffer := Databuffer2;



This code looks simply a fast way to clone Databuffer2 into the Databuffer from the SHP. However, every Array is actually a pointer to a memory address.... So, I could also making the Databuffer from the SHP points to the Databuffer2 that gets deleted in the end of the function, making the program deletes with it, a memory space that is not allowed to do: Invalid Pointer Operation! I'll do a manually clone of databuffer, instead of just doing the rushed way. I hope it fixes the problem.

_________________


Mods, Mods Support, Public Researchs, Map Archives, Tutorials, A Friendly Community and much more. Check it out now!

Back to top
View user's profile Send private message Visit poster's website Skype Account
Banshee
Supreme Banshee


Also Known As: banshee_revora (Steam)
Joined: 15 Aug 2002
Location: Brazil

PostPosted: Mon Aug 09, 2004 8:26 pm    Post subject: Reply with quote  Mark this post and the followings unread

W00t!!!! It works!!! I'm locking this topic for now. I hope to have a new beta soon Smile. But I will unlock it if anything goes wrong...

_________________


Mods, Mods Support, Public Researchs, Map Archives, Tutorials, A Friendly Community and much more. Check it out now!

Back to top
View user's profile Send private message Visit poster's website Skype Account
Banshee
Supreme Banshee


Also Known As: banshee_revora (Steam)
Joined: 15 Aug 2002
Location: Brazil

PostPosted: Mon Aug 09, 2004 9:39 pm    Post subject: Reply with quote  Mark this post and the followings unread

Unlocked.... just an ilusion.

_________________


Mods, Mods Support, Public Researchs, Map Archives, Tutorials, A Friendly Community and much more. Check it out now!

Back to top
View user's profile Send private message Visit poster's website Skype Account
Banshee
Supreme Banshee


Also Known As: banshee_revora (Steam)
Joined: 15 Aug 2002
Location: Brazil

PostPosted: Fri Aug 13, 2004 4:14 am    Post subject: Reply with quote  Mark this post and the followings unread

The true problem has been tracked and eliminated. I give a thanks for Saberhawk and Finsta from iDML Mod Launcher for linking me a document with the new features from Delphi 4. In that document, it talked about a command called Finalize which close dynamic variables. That helped me to figure out that the problem was at the variable "size"

Look at the lines below:


Quote:
setlength(size,SHP.Header.NumImages);
for x := 1 to SHP.header.NumImages do
SetLength(SHP.Data[x].Databuffer,size[x]);


If a picture has 1 frame. It would set the array size to have 1 element, the size[0]. But if I change the value of size[1], the program "accepted", but it couldn't dealocate it, because it wasn't alocated.

_________________


Mods, Mods Support, Public Researchs, Map Archives, Tutorials, A Friendly Community and much more. Check it out now!

Back to top
View user's profile Send private message Visit poster's website Skype Account
Display posts from previous:   
Post new topic   Reply to topic Page 1 of 1 [9 Posts] Mark the topic unread ::  View previous topic :: View next topic
 
Share on TwitterShare on FacebookShare on Google+Share on DiggShare on RedditShare on PInterestShare on Del.icio.usShare on Stumble Upon
Quick Reply
Username:


If you are visually impaired or cannot otherwise answer the challenges below please contact the Administrator for help.


Write only two of the following words separated by a sharp: Brotherhood, unity, peace! 

 
You can post new topics in this forum
You can reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © phpBB Group

[ Time: 0.1598s ][ Queries: 13 (0.0085s) ][ Debug on ]