Also Known As: banshee_revora (Steam) Joined: 15 Aug 2002 Location: Brazil
Posted: Wed Aug 04, 2004 1:57 am Post subject:
fclose(), CloseFile() and Close(): I declare war at you!!!
Ok, I'm getting extremely pissed off at these commands. I made a function to save files on compression 3. It used to work fine last month... recently, I adapted the code to make things faster and more objective and now I'm getting access violation with CloseFile() and when I change it to Close(), I get an invalid pointer operation. Does anyone knows what does can cause this?
Anyway, here's the code of the functions have this problems:
The first function detects the best compression to use in the frame. The second one saves only using compression 3. On 2.99.05, you have an option at Preferences to set the best way to load and save files. Fortunatelly, the compression 1 save still works fine
Code:
// Compression 3 and 1.
Procedure SaveSHP(Filename:String; var SHP : TSHP); overload;
var
F : File;
Written,x,c : integer;
Image_Size : cardinal;
Databuffer2 : TDatabuffer;
size : array of integer;
begin
AssignFile(F,Filename); // Open file
Rewrite(F); // Goto first byte?
setlength(size,SHP.Header.NumImages);
for x := 1 to SHP.header.NumImages do
begin
Encode3(SHP.Data[x].Databuffer,Databuffer2,SHP.Data[x].header_image.cx,SHP.Data[x].header_image.cy,size[x]); // Compression 3
if size[x] < (SHP.Data[x].header_image.cx*SHP.Data[x].header_image.cy) then
begin
SetLength(SHP.Data[x].Databuffer,size[x]+1);
SHP.Data[x].Databuffer := Databuffer2;
SHP.Data[x].header_image.compression := 3;
if x = 1 then
SHP.Data[x].Header_Image.offset := SizeOf(THeader) + (SHP.Header.NumImages * SizeOf(THeader_Image))
else
SHP.Data[x].header_image.offset := SHP.Data[x-1].Header_Image.offset + size[x]; // Works out offset
end
else
begin
size[x] := SHP.Data[x].header_image.cx*SHP.Data[x].header_image.cy;
SetLength(SHP.Data[x].Databuffer,size[x]+1);
SHP.Data[x].Header_Image.compression := 1;
SHP.Data[x].header_image.offset := WorkOutOffset(SHP,X); // Works out offset
end;
end;
for x := 1 to SHP.header.NumImages do // Save Image Headers
begin
BlockWrite(F,SHP.Data[x].header_image,Sizeof(THeader_Image),Written);
end;
// Save each image to the file
for x := 1 to SHP.header.NumImages do
begin
// // If it's compression 1 then, it will right databuffer on its way.
// if shp.data[x].Header_Image.compression <= 1 then
// begin
// the ammount of bytes in a image (Width * Height)
// Image_Size := SHP.Data[x].header_image.cx*SHP.Data[x].header_image.cy;
// Write byte by byte SHP.Data[x].header_image.cy * SHP.Data[x].header_image.cx times
if SHP.Data[x].Header_Image.offset <> 0 then
for c := 1 to size[x] do// Image_Size do
begin
BlockWrite(F,SHP.Data[x].Databuffer[c-1],Sizeof(byte),Written);
end;
end;
CloseFile(F);
end;
Procedure SaveSHPCompressed(Filename:String; var SHP : TSHP); overload;
var
F : file;
Written,x,c : integer;
Databuffer2 : TDatabuffer;
size : array of integer;
begin
AssignFile(F,Filename); // Open file
Rewrite(F,1); // Goto first byte?
setlength(size,SHP.Header.NumImages);
for x := 1 to SHP.header.NumImages do
begin
Encode3(SHP.Data[x].Databuffer,Databuffer2,SHP.Data[x].header_image.cx,SHP.Data[x].header_image.cy,size[x]); // Compression 3
SetLength(SHP.Data[x].Databuffer,size[x]+1);
SHP.Data[x].Databuffer := Databuffer2;
SHP.Data[x].header_image.compression := 3;
if x = 1 then
SHP.Data[x].Header_Image.offset := SizeOf(THeader) + (SHP.Header.NumImages * SizeOf(THeader_Image))
else
SHP.Data[x].header_image.offset := SHP.Data[x-1].Header_Image.offset + size[x]; // Works out offset
end;
for x := 1 to SHP.header.NumImages do // Save Image Headers
begin
BlockWrite(F,SHP.Data[x].header_image,Sizeof(THeader_Image),Written);
end;
// Save each image to the file
for x := 1 to SHP.header.NumImages do
begin
// Write byte by byte size times
if SHP.Data[x].Header_Image.offset <> 0 then
for c := 1 to Size[x] do
begin
BlockWrite(F,SHP.Data[x].Databuffer[c-1],Sizeof(byte),Written);
end;
end;
CloseFile(F);
end;
Both functions crash when it reads the command CloseFile(F);
If I remove this command, I have an access violation when I close the window that has the saved SHP file. QUICK_EDIT
Also Known As: banshee_revora (Steam) Joined: 15 Aug 2002 Location: Brazil
Posted: Sun Aug 08, 2004 9:38 pm Post subject:
I still get an invalid pointer operation message, but I've got some progress on this. The file is encoding the stuff perfectly, but offsets are done in a wrong way, as far as I could test the Force Compression 3 save.
My tests with Einstein SHP showed that the first frame was saved correctly, but all other ones got fucked up (image out of order) when I loaded in the old BS SHP Builder way. The OS SHP Builder that uses offsets, got a huge offset value that gave an access violation...
oh well, in short: I must fix the offset. I had this fixed once and I'll have it again. _________________
Mods, Mods Support, Public Researchs, Map Archives, Tutorials, A Friendly Community and much more. Check it out now! QUICK_EDIT
Also Known As: banshee_revora (Steam) Joined: 15 Aug 2002 Location: Brazil
Posted: Mon Aug 09, 2004 4:45 am Post subject:
Progress Report...
I've debugged the code by hand and I've found some failures on the encoding function. I've fixed it, but it's not enough to kill the "invalid pointer operation" error.
But this, and few minor changes in the SaveSHP function, allowed me to open the saved files (even not being closed properly) with the offset way, however, with the offsets fucked up, the image is entirelly distorted... there were some improvements, but it's still distorted since frame 2.
I've renabled the debug system from my new Load SHP function and it logged the saved einstein in its optimized way. About 20 frames were saved with compression 1. From all others, on 30 (of... about 160), it missed (used the wrong compression). It is possible that these misses are caused by fucked up offsets. I've also noticed that I had to change the way it calculates offsets on compression 1 (since the WorksOutOffset(); function ignores the fact that the offset from previous frames are different than the image size.
Anyway, I expect better results with the modifications I made with the latest observations. The next log will have a better format. But, for this one, here's the translation:
Picture: eins.shp re-saved with optimized size by OS SHP Builder 2.99.05
Current Offset: Location of the frame in the file. Note that it never starts at 0 because it saves the header of the SHP file and the header of all frames first. Offset will always be set to zero when the image has no contents (width * height = 0).
ImageSize: Minimum size of the frame image after compression (1 or 3)
TrueSize: Frame width * Frame height. (cx and cy according to XCC)
Compression: If the frame is compressed with compression 1, it will appear, Compression: 1 and ImageSize will be equal to True Size. Otherwise it won't appear.
EDIT: The problem happens AFTER the file is unloaded, when the variables of the function are being loaded.... now which variable does that O.o?.... But that's a good progress already.
openlog.txt
Description:
Here you can have an idea of the progress of the thing...
Also Known As: banshee_revora (Steam) Joined: 15 Aug 2002 Location: Brazil
Posted: Mon Aug 09, 2004 5:11 am Post subject:
A fast progress report....
First of all, as you may noticed in the edit above. The problem lies on variable unloading and not on closefile. I guess I should have it fixed soon.
Comparing the variables in the 3 saving functions, we have:
SaveSHPUncompressed (working fine):
var
f : file;
Written,x,c : integer;
Image_Size : cardinal;
SaveSHP (invalid pointer operation)
var
F : File;
Written,x,c : integer;
Image_Size : cardinal;
Databuffer2 : TDatabuffer;
size : array of integer;
SaveSHPCompressed (invalid pointer operation)
var
F : file;
Written,x,c : integer;
Databuffer2 : TDatabuffer;
size : array of integer;
So, the suspect lies on the Databuffer2, used to storage the compressed (3) databuffer (frame image) and on size that stores the size of the frame images used to determine the offset and when it writes the databuffer (frame image) to the file.
In other news, I've made a new test with Einstein and results improved considerably. The first observation is that some of the middle frames loaded perfectly, while some others are still loading wrongly. The ones who loaded perfectly are in its vast majority made with compression 1, although some of the compression 1 frames showed some light distortions. The number of misses are still high, but they are weaker misses. In the other file, there were times where the ImageSize was 1689 while the TrueSize was smaller than 500, reaching above 300% of miss. Now, the highest miss hardly reachs 25% and the file ends being quite smaller. But we still work for a smaller, but not distorted result.
Also Known As: banshee_revora (Steam) Joined: 15 Aug 2002 Location: Brazil
Posted: Mon Aug 09, 2004 12:17 pm Post subject:
W00t!
The invalid pointer operation is still not fixed, but...
The file already saves fully correctly with perfect offsets and no misses. The best of all is:
- eins.shp (Einstein) saved by Westwood and extracted by XCC: 77168 bytes
- eins0908_1.shp (Einstein) from Westwood re-saved by OS SHP Builder: 77139 bytes
It saves: 29 bytes!
Honestly, I wasn't expecting that the SHP Builder would save files smaller than Westwood/XCC, but that they would reach a similar or even a bit higher size.... but anyway, OS SHP Builder's saving efficience is 100,1% .
Here's the log of the recent file.... with no misses.
Also Known As: banshee_revora (Steam) Joined: 15 Aug 2002 Location: Brazil
Posted: Mon Aug 09, 2004 8:12 pm Post subject:
Hah! I got it!
Pay attention to the following evil code:
Code:
SHP.Data[x].Databuffer := Databuffer2;
This code looks simply a fast way to clone Databuffer2 into the Databuffer from the SHP. However, every Array is actually a pointer to a memory address.... So, I could also making the Databuffer from the SHP points to the Databuffer2 that gets deleted in the end of the function, making the program deletes with it, a memory space that is not allowed to do: Invalid Pointer Operation! I'll do a manually clone of databuffer, instead of just doing the rushed way. I hope it fixes the problem. _________________
Mods, Mods Support, Public Researchs, Map Archives, Tutorials, A Friendly Community and much more. Check it out now! QUICK_EDIT
Also Known As: banshee_revora (Steam) Joined: 15 Aug 2002 Location: Brazil
Posted: Fri Aug 13, 2004 4:14 am Post subject:
The true problem has been tracked and eliminated. I give a thanks for Saberhawk and Finsta from iDML Mod Launcher for linking me a document with the new features from Delphi 4. In that document, it talked about a command called Finalize which close dynamic variables. That helped me to figure out that the problem was at the variable "size"
Look at the lines below:
Quote:
setlength(size,SHP.Header.NumImages);
for x := 1 to SHP.header.NumImages do
SetLength(SHP.Data[x].Databuffer,size[x]);
If a picture has 1 frame. It would set the array size to have 1 element, the size[0]. But if I change the value of size[1], the program "accepted", but it couldn't dealocate it, because it wasn't alocated. _________________
Mods, Mods Support, Public Researchs, Map Archives, Tutorials, A Friendly Community and much more. Check it out now! QUICK_EDIT
You can post new topics in this forum You can reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum You cannot attach files in this forum You can download files in this forum