:: Home :: Get Hosted :: PPM FAQ :: Forum FAQ :: Search :: Memberlist :: Usergroups :: Register :: Profile :: Log in to check your private messages :: Log in ::
Do you want to advertise at Project Perfect Mod. Find out how to do it HERE.

The time now is Mon Dec 11, 2017 2:22 am
All times are UTC + 0
 Forum index » Featured Tools » OS SHP Builder
SHP
Moderators: stucuk
Post new topic   Reply to topic Page 1 of 3 [149 Posts] View previous topic :: View next topic
Goto page: 1, 2, 3 Next
Author Message
stucuk
Geek


Joined: 27 Aug 2002

PostPosted: Wed Aug 17, 2016 5:35 pm    Post subject:   SHP Reply with quote

I read a few threads about people wanting SHP Builder remade. Today i dediced to re-make the basic SHP file loading code. The example app will only view the SHP files (And it only supports TS/RA2 ones). At this stage idk if i will do any more work on the project. But at least there is cleaner code for loading SHP files. I also designed the class to be more friendly to an editor as it stores the Palette Version and RGB Version at the full size (RGB version to reduce rendering overheads).

- SHPToolz.rar



SHP.PAS:
Spoiler (click here to read it):
Code:
unit SHP;
{
 Started: 2016/08/17
 By: Stuart "Stucuk" Carey
}

interface

uses Windows, Graphics, Classes;

 type
 TSHP_Header = record
  Zero, Width, Height, Count : Word;
 end;

 TSHP_Header_Image = record
  X, Y, W, H  : Word;
  Compression : Byte;
  Align       : Array [0..2] of byte;
  RadarColor  : TColor;
  Zero,
  Offset      : Integer;
 end;

 // Code above this line taken from SHP_File.Pas which in turn was taken from XCC.

 PSHP_Header_Image = ^TSHP_Header_Image;

 const
  IMGAREA_X = 0;
  IMGAREA_Y = 1;
  IMGAREA_W = 2;
  IMGAREA_H = 3;

 type
 TImgArea = Array [0..3] of Word;

 TSHP_Palette = Array [0..255] of TRGBTriple;

 TSHP_Frame = Record
  UsedArea : TImgArea;   // X,Y,W,H
  PalData  : PByte;      // Holds Image as Palette colours
  RGBData  : PRGBTriple; // Holds Image as RGB colours
  NeedUpd  : Boolean;    // When true RGBData holds an old image
  RadarCol : TColor;
 end;

 TSHP = Class(TObject)
 protected
  FW,FH     : Integer;
  FFrames   : Array of TSHP_Frame;
  FPalette  : TSHP_Palette;
  FTransCol : TRGBTriple;
  procedure Decode3(Input : TStream; Frame : Integer);
  procedure Decode2(Input : TStream; Frame : Integer);
  procedure Decode1(Input : TStream; Frame : Integer);
  function  GetFrame(Index : Integer)    : TSHP_Frame;
  function  GetRGBFrame(Index : Integer) : PRGBTriple;
  procedure EmptyFrameRGB(Index : Integer);
  procedure BuildFrameRGB(Index : Integer);
  function  GetCount() : Integer;
  procedure SetTransCol(Value : TRGBTriple);
  procedure MarkNeedUpd;
 public
  constructor Create;
  destructor Destroy; override;

  procedure LoadFromFile(Filename : AnsiString);
  procedure SaveToFile(Filename : AnsiString);

  procedure LoadFromStream(Stream : TStream);
  procedure SaveToStream(Stream : TStream);

  procedure SetPalette(Palette : TSHP_Palette);
  procedure LoadPalette(Filename : AnsiString);

  property Frame[Index : Integer]    : TSHP_Frame read GetFrame;
  property RGBFrame[Index : Integer] : PRGBTriple read GetRGBFrame;

  property Width  : Integer read FW;
  property Height : Integer read FH;
  property Count  : Integer read GetCount;

  property TransparentColour : TRGBTriple read FTransCol write SetTransCol;
 end;

implementation

uses SysUtils;

function SetRGBTriple(R,G,B : Byte) : TRGBTriple;
begin
 Result.rgbtRed   := R;
 Result.rgbtGreen := G;
 Result.rgbtBlue  := B;
end;

constructor TSHP.Create;
var
 X : Integer;
 C : Byte;
begin
 Inherited;
 SetLength(FFrames,0);

 // Make Black and White Palette as fallback!
 for X := 0 to 254 do
 begin
  C := Trunc(X/254*255);
  FPalette[X+1] := SetRGBTriple(C,C,C);
 end;

 FTransCol   := SetRGBTriple(0,0,255);
 FPalette[0] := FTransCol;
end;

destructor TSHP.Destroy;
var
 X : Integer;
begin
 Inherited;

 for X := 0 to High(FFrames) do
 begin
  FreeMem(FFrames[X].PalData);
  FreeMem(FFrames[X].RGBData);
 end;

 SetLength(FFrames,0);
end;

procedure TSHP.SetTransCol(Value : TRGBTriple);
begin
 FTransCol   := Value;
 FPalette[0] := Value;

 MarkNeedUpd;
end;

procedure TSHP.LoadFromFile(Filename : AnsiString);
var
 Stream : TStream;
begin
 Stream := TFileStream.Create(Filename,fmOpenRead or fmShareDenyWrite);
  LoadFromStream(Stream);
 Stream.Free;
end;

procedure TSHP.SaveToFile(Filename : AnsiString);
var
 Stream : TStream;
begin
 if FileExists(Filename) then
  Stream := TFileStream.Create(Filename,fmCreate)
 else
  Stream := TFileStream.Create(Filename,fmOpenWrite or fmShareDenyRead);
   SaveToStream(Stream);
  Stream.Free;
end;

function SetArea(X,Y,W,H : Word) : TImgArea;
begin
 Result[0] := X;
 Result[1] := Y;
 Result[2] := W;
 Result[3] := H;
end;

function TSHP.GetFrame(Index : Integer) : TSHP_Frame;
begin
 Result := FFrames[Index];
end;

function TSHP.GetRGBFrame(Index : Integer) : PRGBTriple;
begin
 BuildFrameRGB(Index);

 Result := FFrames[Index].RGBData;
end;

function MakeBlankLine(Color : TRGBTriple; L : Integer) : PRGBTriple;
var
 X : Integer;
 T : Pointer;
begin
 GetMem(Result,L*3);
 T := Result;
 for X := 0 to L-1 do
 begin
  CopyMemory(T,@Color,3);
  Inc(Cardinal(T),3);
 end;
end;

procedure TSHP.EmptyFrameRGB(Index : Integer);
var
 Blank : PRGBTriple;
 T     : Pointer;
 Y     : Integer;
begin
 if not FFrames[Index].NeedUpd then
 Exit;

 Blank := MakeBlankLine(FPalette[0],FW);
 T     := FFrames[Index].RGBData;
 for Y := 0 to FH-1 do
 begin
  CopyMemory(T,Blank,FW*3);
  Inc(Cardinal(T),FW*3);
 end;
 FreeMem(Blank);
end;

function LS(Data : Pointer; Y : Integer; Area : TImgArea; W : Integer; Bytes : Byte = 1) : Pointer;
begin
 Result := Pointer(Cardinal(Data) + ((Y+Area[IMGAREA_Y])*W+Area[IMGAREA_X])*Bytes);
end;

function RGBLineStart(Data : Pointer; Y : Integer; Area : TImgArea; W : Integer) : Pointer;
begin
 Result := LS(Data,Y,Area,W,3);
end;

function PALLineStart(Data : Pointer; Y : Integer; Area : TImgArea; W : Integer) : Pointer;
begin
 Result := LS(Data,Y,Area,W,1);
end;

procedure TSHP.BuildFrameRGB(Index : Integer);
var
 RGB : PRGBTriple;
 PAL : PByte;
 X,Y : Integer;
begin
 if not FFrames[Index].NeedUpd then
 Exit;

 if not Assigned(FFrames[Index].RGBData) then
 GetMem(FFrames[Index].RGBData,FW*FH*3);

 EmptyFrameRGB(Index);

 for Y := 0 to FFrames[Index].UsedArea[IMGAREA_H]-1 do
 begin
  RGB := RGBLineStart(FFrames[Index].RGBData,Y,FFrames[Index].UsedArea,FW);
  PAL := PALLineStart(FFrames[Index].PalData,Y,FFrames[Index].UsedArea,FW);
  for X := 0 to FFrames[Index].UsedArea[IMGAREA_W]-1 do
  begin
   RGB^ := FPalette[PAL^];
   Inc(Cardinal(RGB),3);
   Inc(Cardinal(PAL),1);
  end;
 end;
end;

//Based on Decode3 from SHP_File.Pas
procedure TSHP.Decode3(Input : TStream; Frame : Integer);
var
 Output : Pointer;
 Count  : Word;
 V      : Byte;
 Y      : Integer;
begin
 try
  for Y := 0 to FFrames[Frame].UsedArea[IMGAREA_H]-1 do
  begin
   Output := PALLineStart(FFrames[Frame].PalData,Y,FFrames[Frame].UsedArea,FW);
   Input.Read(Count,2);
   Dec(Count,2);
   while Count > 0 do
   begin
    Dec(Count);
    Input.Read(V,1);
    if V <> 0 then
    begin
     Byte(Output^) := V;
     Inc(Cardinal(Output),1);
    end
    else
    begin
     Dec(Count);
     Input.Read(V,1);
     Inc(Cardinal(Output),V);
    end;
   end;
  end;
 except
 end;
end;

procedure TSHP.Decode2(Input : TStream; Frame : Integer);
var
 Count  : Word;
 Y      : Integer;
begin
 try
  for Y := 0 to FFrames[Frame].UsedArea[IMGAREA_H]-1 do
  begin
   Input.Read(Count,2);
   if Count-2 > 0 then
   Input.Read(PALLineStart(FFrames[Frame].PalData,Y,FFrames[Frame].UsedArea,FW)^,Count-2);
  end;
 except

 end;
end;

procedure TSHP.Decode1(Input : TStream; Frame : Integer);
var
 Y : Integer;
begin
 try
  for Y := 0 to FFrames[Frame].UsedArea[IMGAREA_H]-1 do
   Input.Read(PALLineStart(FFrames[Frame].PalData,Y,FFrames[Frame].UsedArea,FW)^,FFrames[Frame].UsedArea[IMGAREA_W]);
 except

 end;
end;

procedure TSHP.LoadFromStream(Stream : TStream);
var
 Header        : TSHP_Header;
 Image_Header,
 Image_Headers : PSHP_Header_Image;
 X             : Integer;
 Mem           : TMemoryStream;
 MemStart      : Int64;
begin
 Stream.Read(Header,SizeOf(TSHP_Header));
 if (Header.Zero <> 0) or (Header.Width = 0) or (Header.Height = 0) or (Header.Count = 0) then
 Exit;

 FW := Header.Width;
 FH := Header.Height;

 SetLength(FFrames,Header.Count);
 GetMem(Image_Headers,SizeOf(TSHP_Header_Image)*Header.Count);
 Stream.Read(Image_Headers^,SizeOf(TSHP_Header_Image)*Header.Count);

 MemStart := Stream.Position;
 //Load the rest of the file in a MemoryStream to reduce overheads
 Mem := TMemoryStream.Create;
 Mem.CopyFrom(Stream,Stream.Size-Stream.Position);
 Mem.Position := 0;

 Image_Header := Image_Headers;
 for X := 0 to Header.Count-1 do
 begin
  FFrames[X].PalData := Nil;
  FFrames[X].RGBData := Nil;
  FFrames[X].NeedUpd := True;

  if Image_Header^.offset <> 0 then
  begin
   FFrames[X].UsedArea := SetArea(Image_Header^.X,Image_Header^.Y,Image_Header^.W,Image_Header^.H);
   FFrames[X].RadarCol := Image_Header^.RadarColor;
   GetMem(FFrames[X].PalData,FW*FH);
   ZeroMemory(FFrames[X].PalData,FW*FH);

   Mem.Seek(Image_Header^.offset-MemStart,soFromBeginning);
   case Image_Header^.compression of
    3 : Decode3(Mem,X);
    2 : Decode2(Mem,X);
    1,
    0 : Decode1(Mem,X);
    else
    begin
     FFrames[X].UsedArea := SetArea(0,0,0,0);
     asm nop end; //Break Point
    end;
   end;
  end;

  Inc(Cardinal(Image_Header),SizeOf(TSHP_Header_Image));
 end;

 Mem.Free;
end;


procedure TSHP.SaveToStream(Stream : TStream);
begin
 //TODO!!!
end;

procedure TSHP.SetPalette(Palette : TSHP_Palette);
begin
 CopyMemory(@FPalette[0],@Palette[0],SizeOf(TSHP_Palette));

 MarkNeedUpd;
end;

procedure TSHP.MarkNeedUpd;
var
 X : Integer;
begin
 for X := 0 to High(FFrames) do
 FFrames[X].NeedUpd := True;
end;

function TSHP.GetCount() : Integer;
begin
 Result := High(FFrames)+1;
end;

procedure TSHP.LoadPalette(Filename : AnsiString);
var
 Stream : TStream;
 X      : Integer;
 T      : Byte;
begin
 Stream := TFileStream.Create(Filename,fmOpenRead or fmShareDenyWrite);
  Stream.Read(FPalette[0],SizeOf(TSHP_Palette));
 Stream.Free;

 for X := 0 to 255 do
 begin
  T                     := FPalette[X].rgbtBlue;
  FPalette[X].rgbtBlue  := FPalette[X].rgbtRed*4;
  FPalette[X].rgbtGreen := FPalette[X].rgbtGreen*4;
  FPalette[X].rgbtRed   := T*4;
 end;

 FPalette[0] := FTransCol;

 MarkNeedUpd;
end;

end.


NOTE: "FreeMem(Image_Headers);" is missing from the end of LoadFromStream which would cause a small memory leak.
_________________
Free Map Editor - Game Requirements - Stucuk.Net

Back to top
View user's profile Send private message Visit poster's website
stucuk
Geek


Joined: 27 Aug 2002

PostPosted: Wed Aug 17, 2016 7:01 pm    Post subject: Reply with quote

Anyone have any TD/RA1 SHP's that i could use to test TD/RA1 SHP loading code?
_________________
Free Map Editor - Game Requirements - Stucuk.Net

Back to top
View user's profile Send private message Visit poster's website
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Wed Aug 17, 2016 7:17 pm    Post subject: Reply with quote

I guess you could try Lin Kuei Ominae's "Iowa Class Battleship" shp.

Link:
http://www.ppmforums.com/viewtopic.php?t=42444



BTW why did you write this in Pascal?
_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Back to top
View user's profile Send private message Skype Account
Lin Kuei Ominae
Seth


Joined: 16 Aug 2006
Location: Germany

PostPosted: Wed Aug 17, 2016 8:00 pm    Post subject: Reply with quote

no, that is SHP (TS) format too, because SHP Builder corrupted the file when i tried to save as SHP (TD).
MadHQ wrote:
BTW why did you write this in Pascal?
probably because SHP Builder is written in Pascal too


Hello Stucuk, it's great to still see you around and even greater that you still spend some time in improving the tools.
Welcome back. Smile

attached the TD MCV.

Suggestion:
Currently we have no batch processing SHP tool. So how about making SHP Toolz useful beyond a testbed for the new code?
Things that would be nice for batch process are
-palette conversion (using a scheme file like SHP Builder or the auto-color conversion where you only provide the new palette, though scheme file should be far easier to implement)
-radar color editing
-canvas optimization might be also good (so the tool shrinks the canvas to the smallest possible size without removing any pixel of the normal or damage frames, while still making sure the object is well centered)


mcv.shp
 Description:

Download
 Filename:  mcv.shp
 Filesize:  20.67 KB
 Downloaded:  13 Time(s)


_________________
SHP Artist of Twisted Insurrection. Main Nod building SHP creator.

Public SHPs
X-Mech Calendar (28 Mechs for GDI and Nod)
5 GDI, 5 Nod, 1 Mutant unit, 1 Scrin unit, 1 GDI building

Tools
TMP Shop
C&C Executable Modifier

Back to top
View user's profile Send private message
Crimsonum
Seth


Joined: 14 Jul 2005
Location: Fineland

PostPosted: Wed Aug 17, 2016 8:45 pm    Post subject: Reply with quote

This is cool news. All I can really contribute is by giving thumbs up and encouragement for proceeding with this rewrite of the SHP builder!
_________________


Back to top
View user's profile Send private message Visit poster's website
stucuk
Geek


Joined: 27 Aug 2002

PostPosted: Wed Aug 17, 2016 9:05 pm    Post subject: Reply with quote

Delphi is the language i am most proficient in. It also allows for easy RAD (Rapid Application Development) meaning making a front end is simple.

I don't see the point in writing a C++ application for the sake of it being in C++. No one would contribute to the project anyway (C++ has no advantages over Delphi other than their being more people with a copy of C++).

@Lin Kuei Ominae: Thx. What do you mean by the canvas optimization? Just shrinking the overall width/height to the minimum size?
_________________
Free Map Editor - Game Requirements - Stucuk.Net

Back to top
View user's profile Send private message Visit poster's website
Lin Kuei Ominae
Seth


Joined: 16 Aug 2006
Location: Germany

PostPosted: Wed Aug 17, 2016 9:26 pm    Post subject: Reply with quote

Delphi is fine.

yes, making the canvas as small as possible. Many people render with 320x240 or bigger sizes, while the actual object is quite small.
And a small canvas is surely also better for the games engine, which then has to calculate a lot less.

Currently you have to find the outer most pixel of all frames and then shrink the canvas accordingly and finding the smallest bounding box is quite tedious especially on SHPs with many frames.
_________________
SHP Artist of Twisted Insurrection. Main Nod building SHP creator.

Public SHPs
X-Mech Calendar (28 Mechs for GDI and Nod)
5 GDI, 5 Nod, 1 Mutant unit, 1 Scrin unit, 1 GDI building

Tools
TMP Shop
C&C Executable Modifier

Back to top
View user's profile Send private message
Banshee
Supreme Banshee


Joined: 15 Aug 2002
Location: Brazil

PostPosted: Wed Aug 17, 2016 10:08 pm    Post subject: Reply with quote

stucuk wrote:
Delphi is the language i am most proficient in. It also allows for easy RAD (Rapid Application Development) meaning making a front end is simple.

I don't see the point in writing a C++ application for the sake of it being in C++. No one would contribute to the project anyway (C++ has no advantages over Delphi other than their being more people with a copy of C++).


Why don't you convert it to Lazarus? It is also pascal and it should allow more people to work on it, without spending a penny.

Back to top
View user's profile Send private message Visit poster's website Skype Account
stucuk
Geek


Joined: 27 Aug 2002

PostPosted: Wed Aug 17, 2016 10:11 pm    Post subject: Reply with quote



Got $80 decoding working though i cheated (Currently just using the decode80d from SHP Builder's code rather than a re-write). Unfortunatly the MCV doesn't have $20 or $40 compression in any of the frames.

@Lin Kuei Ominae: It should be very easy to achieve. With TS/RA2 SHP files you already have to work out the smallest size for each frame which could then be used to work out the smallest canvas size.

Though as it stands i haven't written any saving code, only loading.

P.S Technically shouldn't be hard to add to SHP Builder but im not touching it. Way too bloated.



@Banshee: True that it could be converted to Laz but i wouldn't personally do it as i don't use Laz myself. Also i doubt it would increase the chance of anyone contributing.
_________________
Free Map Editor - Game Requirements - Stucuk.Net

Back to top
View user's profile Send private message Visit poster's website
tomsons26lv
Scorpion Sniper


Joined: 30 Dec 2009
Location: Latvia

PostPosted: Wed Aug 17, 2016 11:20 pm    Post subject: Reply with quote

stucuk wrote:
Delphi is the language i am most proficient in. It also allows for easy RAD (Rapid Application Development) meaning making a front end is simple.

I don't see the point in writing a C++ application for the sake of it being in C++. No one would contribute to the project anyway (C++ has no advantages over Delphi other than their being more people with a copy of C++).

@Lin Kuei Ominae: Thx. What do you mean by the canvas optimization? Just shrinking the overall width/height to the minimum size?


Doing it in C++ has a lot more point then Delphi.
No one besides you two ever touched SHP Builder because of the language it's written in and no one really will touch this.
From personal experience every single Delphi app i've ever run has speed issues,memory leaks, crashes for no apparent reason, with SHP Builder not being a exception.

From the trends on the web i've seen C# is where the most interest is

On another subject, what people call Format80 is LCW and and 40 is XORDelta
https://forums.cncnet.org/index.php?topic=4403.msg32863#msg32863
_________________
Tiberian Dawn, Red Alert, Tiberian Sun ,Red Alert 2,Renegade, Command & Conquer 3,Tiberium and Tiberium Wars and Westwood related image & video archive
https://picasaweb.google.com/113361105083292812413?noredirect=1

Skype live:tomsons26
Don't forget to state who are you otherwise i'll ignore the invite

Back to top
View user's profile Send private message Visit poster's website
stucuk
Geek


Joined: 27 Aug 2002

PostPosted: Thu Aug 18, 2016 12:01 am    Post subject: Reply with quote

@tomsons26lv: Utter rubbish. Delphi actually has FastMM with modern versions(And it was a free download before it became the standard Memory Manager) which has facilities for tracking memory leaks, etc.

Memory leaks are common in EVERY language(You either have never ran applications or never played any computer games as most leak and aren't made in Delphi) which doesn't automate pointers. Languages like Java and C# have garbage collectors which handle pointers for you (Which can be bad). They can still have memory leaks however (The Irony).

The way to prevent memory leaks is for there to be a competent programmer. SHP Builder never used FastMM and both I and Banshee were not the best in the world at programming at the time.

Speed Issues and Crashes have nothing to do with the Language. Delphi applications are compiled into Machine code, the actual compiler is a C++ compiler modified to work with Pascal Syntax (C++ Builder's Compiler is the same as Delphi's except it can also handle C++ syntax as well as Delphi in the same application).

In re-guards to having it in C++ meaning more contributors, you just need to look at the C++ projects (For CNC), they don't get a massive amount of contributors.

Quote:
No one besides you two ever touched SHP Builder because of the language it's written in and no one really will touch this.


Have you even ran SHP Builder (See credits)? SHP Builder's code being a complete mess is going to be more of a reason why more haven't contributed(I don't even want to touch it and i invented it). What language is used for a real programmer is irrelevant when it comes to contributing.

Quote:
  SHP_BUILDER_BY = 'Banshee & Stucuk';
  SHP_BUILDER_CONTRIBUTORS = 'PaD, VK and Zaaz';


Also VXLSE was programming in Delphi (I never invented it. I only contributed to it with VXLSE 2 after 3 others had worked on it. I re-wrote it for VXLSE 3 and Banshee contributed after a while).


Anyway please refrain from bitching about what language I want to program applications in. All it does is annoy me and i won't have this thread derailed by Bull.

P.S C# is meant to be Delphi and C++ merged as the guy who designed Delphi designed C#. And for the Record when i attach FastMM to SHPToolz as it stands its not leaking.
_________________
Free Map Editor - Game Requirements - Stucuk.Net

Back to top
View user's profile Send private message Visit poster's website
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Thu Aug 18, 2016 1:55 am    Post subject: Reply with quote

I am sorry stucuk, I did not know it would cause an uproar to ask.

I recently just graduate with a B.S. Computer Science degree, and my question was just out of curiosity. I myself only know a few languages and am not familiar with what could be considered "best/better/good", as myself I enjoy scripting languages such as PHP/JavaScript.

I can say it doesn't matter too much what language it is, for the most part I can kind of tell what is going on, and I guess that is all that matters.
_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Back to top
View user's profile Send private message Skype Account
Banshee
Supreme Banshee


Joined: 15 Aug 2002
Location: Brazil

PostPosted: Thu Aug 18, 2016 2:49 am    Post subject: Reply with quote

Stu, do you wanna a SVN repository at PPM for SHP Toolz?

Back to top
View user's profile Send private message Visit poster's website Skype Account
stucuk
Geek


Joined: 27 Aug 2002

PostPosted: Thu Aug 18, 2016 4:52 am    Post subject: Reply with quote

MadHQ wrote:
I am sorry stucuk, I did not know it would cause an uproar to ask.

It doesn't. In the early 2000's we had the C++ Fanboys spreading lies (Either deliberate or through ignorance) and i cba with that bull. Nothing wrong with people suggesting languages (along with positives and negatives) but there is however an issue when people lie and state that Delphi just causes memory leaks, crashes, slowdowns, etc (All 3 have nothing to do with the language used). Iv been programming for over 17 years, so i know Bull when i hear it.

MadHQ wrote:
what could be considered "best/better/good"

It depends on what your doing and what your goals are. For general stuff you can use most languages but for specific stuff like say writing a game for an XBox you would have to use a specific language.

MadHQ wrote:
I can say it doesn't matter too much what language it is, for the most part I can kind of tell what is going on, and I guess that is all that matters.

All languages are basically the same. The Syntax is just slightly different with each.

Banshee wrote:
Stu, do you wanna a SVN repository at PPM for SHP Toolz?

Unless someone actually wants to contribute a SVN is pointless. Though thx for the offer.

----------------------------------

Got TS/RA2 SHP Saving done(Unlike SHP Builder it will compare all 3 compressions and pick the best. Though in reality Decode3 should always win). I haven't got TS or RA2 installed so i can't test it (IIRC TS wouldn't run on Windows 7 when i tried it last year).


_________________
Free Map Editor - Game Requirements - Stucuk.Net

Back to top
View user's profile Send private message Visit poster's website
E1 Elite
Laser Commando


Joined: 28 May 2013

PostPosted: Thu Aug 18, 2016 5:20 am    Post subject: Reply with quote

TS runs on all Win7, 8.1 and 10 too. There has been fixes by means of proxy ddraw.dll. Try TS Client,
it has rendering options, compatible to most of the combination of OS/CPU/GPU. (~207mb)
http://www.ppmforums.com/viewtopic.php?t=40462
http://www.moddb.com/mods/tiberian-sun-client/downloads/tiberian-sun-client-v408
(Set Resources\MainClient.ini -> ModMode=true, to avoid verification for offline changes)

Compression 3 wins but there should be option for not using compression for special cases like
mouse.shp which doesn't work with compression. (i.e. to use only compression 1 for all frames).

Back to top
View user's profile Send private message
stucuk
Geek


Joined: 27 Aug 2002

PostPosted: Thu Aug 18, 2016 8:49 am    Post subject: Reply with quote

E1 Elite wrote:
TS runs on all Win7, 8.1 and 10 too.

Looks like the Issue is MS15-097 disabling secdrv (SecuROM).

E1 Elite wrote:
Compression 3 wins but there should be option for not using compression for special cases like
mouse.shp which doesn't work with compression. (i.e. to use only compression 1 for all frames).


Technically Compression 2 should win if your SHP has alot of holes in it. Comp 2 stores all the pixels in a row from the first pixel to the last non-background. Comp 3 ignores the background (Using 2 bytes, one to indicate a 0 and another for how many pixels to ignore). Since 3 uses 2 bytes each block of background its worse if there is alot of holes.

Adding different save options is as easy as adding new "Filters" to the Save Dialog:


_________________
Free Map Editor - Game Requirements - Stucuk.Net

Back to top
View user's profile Send private message Visit poster's website
E1 Elite
Laser Commando


Joined: 28 May 2013

PostPosted: Thu Aug 18, 2016 9:00 am    Post subject: Reply with quote

TS is freeware, if you want just the game.exe from freeware which doesn't do CD check,
I could share, instead of downloading 1.2gb of it. I didn't have any problem with it on Win7,
though it might be disabling the CD check the same way.

I may be wrong, but I doubt compression 2 is even used at all. SHP Builder also has variable
compression per frame (comp 1 or 3) for normal (not for shadows). Does your save Best option
do the same?

Back to top
View user's profile Send private message
tomsons26lv
Scorpion Sniper


Joined: 30 Dec 2009
Location: Latvia

PostPosted: Thu Aug 18, 2016 12:24 pm    Post subject: Reply with quote

stucuk wrote:

Speed Issues and Crashes have nothing to do with the Language. Delphi applications are compiled into Machine code, the actual compiler is a C++ compiler modified to work with Pascal Syntax (C++ Builder's Compiler is the same as Delphi's except it can also handle C++ syntax as well as Delphi in the same application).

Anyway please refrain from bitching about what language I want to program applications in. All it does is annoy me and i won't have this thread derailed by Bull.

I know that, sorry, shouldn't have made this post while being distracted and tired.

It's not the language, its the components and libraries that come with it, from the experience i have had every time something goes wrong in a Delphi app that's where it mostly happens, and from what i understand people are forced to use as there aren't really much alternatives.

Even if you get VCL's and RTL's and whatnot it has that don't have issues, since you don't see the point in converting to Lazarus, there's still the matter of the price, €392.04 for the Starter(which i'm guessing without the Library sources is useless) and €1,955.36 for the Professional version is enough for people to instantly lose any interest whatsoever on the sight of Delphi in general.
_________________
Tiberian Dawn, Red Alert, Tiberian Sun ,Red Alert 2,Renegade, Command & Conquer 3,Tiberium and Tiberium Wars and Westwood related image & video archive
https://picasaweb.google.com/113361105083292812413?noredirect=1

Skype live:tomsons26
Don't forget to state who are you otherwise i'll ignore the invite

Back to top
View user's profile Send private message Visit poster's website
Banshee
Supreme Banshee


Joined: 15 Aug 2002
Location: Brazil

PostPosted: Thu Aug 18, 2016 12:59 pm    Post subject: Reply with quote

I have programmed with VCL nearly as much as Stucuk did and I didn't had problems of stability, tomsons26lv. The problems that I have with that is that:

- Free versions of Delphi that were released in history only compiles to win32 platform;

- Many libraries were not produced or converted for Delphi;

- Delphi IDE is awfully bugged (at least in the 2006 version, which was the last free Delphi [Turbo Delphi Explorer] released). It crashes after using for some hours due to memory leaks. But that's an issue with the IDE and not the VCL. Other versions of the IDE may not necessarily have this problem.

Back to top
View user's profile Send private message Visit poster's website Skype Account
stucuk
Geek


Joined: 27 Aug 2002

PostPosted: Thu Aug 18, 2016 4:41 pm    Post subject: Reply with quote

E1 Elite wrote:
I may be wrong, but I doubt compression 2 is even used at all. SHP Builder also has variable
compression per frame (comp 1 or 3) for normal (not for shadows). Does your save Best option
do the same?


I know that compression 2 is never saved by SHP Builder though i am not sure if it was simply as compression 2 doesn't often give a better compression or because TS/RA2 don't support it (Way too many years ago).

My "Best" uses each of the 3 compression's (Comp 1 doesn't actually do any compression and is used as input for the other two), gets their size and picks the smallest.

I have added a "Best Half" option which does "Best" for the first half and comp 1 for the rest.


tomsons26lv wrote:
It's not the language, its the components and libraries that come with it

I can't speak for Delphi's after version 6, but i haven't had any real issue in re-guards to memory leaks or crashes (Delphi's IDE has always had issues if you run the IDE for long enough though Version 6/7 has meant to be one of the more stable versions). Standard VCL's only flaw is speed when it comes to things like adding loads of elements to a TreeView/Listbox/etc as it uses the windows API. However there is VirtualTreeView (Free VCL component).

The good thing about VCL is that if you don't like a component(Say you want custom drawing code) you can easily write your own to get around any issues you have (Sometimes just subclassing it and overwriting some stuff is all you need).

tomsons26lv wrote:
since you don't see the point in converting to Lazarus, there's still the matter of the price

Anyone can convert to Lazarus, why would i have to be the one? If someone actually wanted to contribute to it but only if it was Lazarus they would take the code and convert it. The whole concept that only if someone else does the work that people would contribute is a fantasy.

All you need to do is find a copy of Delphi 6/7 Personal Edition (Which was released for free) and a Key for it. Granted you would need to check any files you download with a AV checker since it wouldn't be from the official source (As they removed the download years ago).

Banshee wrote:
Free versions of Delphi that were released in history only compiles to win32 platform;

Only an issue if you need to access more than 3-4GB of memory (Im not sure of the exact limit). Unlikely to be an issue with any CNC project given that TS/RA2 are 32bit apps.

Banshee wrote:
Many libraries were not produced or converted for Delphi;

Most things have a DLL form. Alot of DLL's have delphi headers and for those that don't there are multiple applications designed to convert C/C++ headers to Delphi (Which generally only need minor changes to get them to compile).

Banshee wrote:
Delphi IDE is awfully bugged

Yes. Though Delphi 6 works most of the time with the 2 Patches applied. Occasionally has a fit which either requires the project to be reloaded or the IDE to be closed and restarted. But you can generally run multiple copies for long periods of time before any issues. I haven't noticed any memory leaks with Delphi 6. Bound to have some (Most apps do) but never been an issue for me.
_________________
Free Map Editor - Game Requirements - Stucuk.Net

Back to top
View user's profile Send private message Visit poster's website
Banshee
Supreme Banshee


Joined: 15 Aug 2002
Location: Brazil

PostPosted: Thu Aug 18, 2016 4:55 pm    Post subject: Reply with quote

stucuk wrote:
Anyone can convert to Lazarus, why would i have to be the one?


Maybe... because.... you have the most up to date code and that you've just started the program recently (which makes the conversion procedure be much easier than in later stages)?

Back to top
View user's profile Send private message Visit poster's website Skype Account
stucuk
Geek


Joined: 27 Aug 2002

PostPosted: Thu Aug 18, 2016 5:37 pm    Post subject: Reply with quote

Banshee wrote:
Maybe... because.... you have the most up to date code and that you've just started the program recently (which makes the conversion procedure be much easier than in later stages)?


No one has stated they would help if it was in Lazarus. The whole "If you make it they will come" is a load of rubbish.

I know from personal experience and by looking at other CNC tools (Which aren't made in Delphi) that people generally don't contribute to a project. Most want to make their own projects if they do anything.

Lazarus can convert projects doing the majority of the work for you. Most of the main code (Such as the SHP file stuff) would not be tied to Delphi in any way and should have 0 issues being used in Laz/FreePascal.

People have converted Quake 3 to Delphi, etc. Which is a far greater challenge than running Laz and fixing some minor code differences. Both are pascal based and Laz is designed to be a Delphi replacement (Unlike C/C++ which are completely different languages). If people are too lazy to run Laz or don't have the skilz to fix some minor issues (If any would even exist) then they likely wouldn't be of any use contributing. Its not rocket science, i just don't want to waste my time on a laz port that noone is even going to contribute to.

Bottom line is this. I am going to program for as long as its fun (The BS about how i should do it in Laz/C++/Etc when no one is going to help is just annoying. "Do it in XYZ Stu!!!!! But i won't bother to help you!") and then i will go and crawl back to the rock i have been living under. I'm spending more time writing replies to the BS than i am coding.
_________________
Free Map Editor - Game Requirements - Stucuk.Net

Back to top
View user's profile Send private message Visit poster's website
MadHQ
Commander


Joined: 07 Nov 2003

PostPosted: Thu Aug 18, 2016 8:26 pm    Post subject: Reply with quote

Damn this really has turned into a type-of-code shit fest.... >_<

I am sorry.
_________________
MadHQ's Graveyard - Click here!
(Permissions) - (F.A.Q.)

Back to top
View user's profile Send private message Skype Account
stucuk
Geek


Joined: 27 Aug 2002

PostPosted: Thu Aug 18, 2016 9:26 pm    Post subject: Reply with quote

MadHQ wrote:
Damn this really has turned into a type-of-code shit fest.... >_<

I am sorry.


You have done nothing wrong. The issue is people telling me what language i should code in who are not going to bother to help if i did what they said.

The whole concept of what im doing is really to make a unit that can load/save SHP files which is clean and simple (Not a complete mess spewed over multiple files like SHP Builder). Also one where its coded in a better and more fast way (Like BuildFrameRGB's code). That unit could then be used in any application (Even a simple console application, doesn't need to be a full fledged gui) to do whatever people want to SHP's. Could even make a DLL project giving easy access to SHP's for any language which supported C style DLL's.



Currently does no editing, but that doesn't mean it can't have a nicer interface #Tongue
_________________
Free Map Editor - Game Requirements - Stucuk.Net

Back to top
View user's profile Send private message Visit poster's website
Lin Kuei Ominae
Seth


Joined: 16 Aug 2006
Location: Germany

PostPosted: Thu Aug 18, 2016 9:51 pm    Post subject: Reply with quote

A Zoom ComboBox/NumericUpDown would be nice, since in my test i could get the preview smaller, but not bigger again. (it wasn't intuitive enough how to zoom in the tool)
_________________
SHP Artist of Twisted Insurrection. Main Nod building SHP creator.

Public SHPs
X-Mech Calendar (28 Mechs for GDI and Nod)
5 GDI, 5 Nod, 1 Mutant unit, 1 Scrin unit, 1 GDI building

Tools
TMP Shop
C&C Executable Modifier

Back to top
View user's profile Send private message
stucuk
Geek


Joined: 27 Aug 2002

PostPosted: Fri Aug 19, 2016 12:27 am    Post subject: Reply with quote

Lin Kuei Ominae wrote:
A Zoom ComboBox/NumericUpDown would be nice, since in my test i could get the preview smaller, but not bigger again. (it wasn't intuitive enough how to zoom in the tool)


In the Original you click the window and it will use that as the maximum size (It scales it proportionately). In the latest it has Zoom In/Out buttons with 1x-20x zoom (1x, 2x, 4x, 6x, 8x, 10x, 12x, 14x, 16x, 18x, 20x).
_________________
Free Map Editor - Game Requirements - Stucuk.Net

Back to top
View user's profile Send private message Visit poster's website
stucuk
Geek


Joined: 27 Aug 2002

PostPosted: Fri Aug 19, 2016 6:47 pm    Post subject: Reply with quote

- SHPToolz_2016-08-19.rar

As it stands it still has no editing capabilities.

Included is Both the Release version and Debug. Debug has the memory manager report memory leaks. It will always state their are some leaks though in reality they aren't. They are things which are only ever created one time and are designed to be "Alive" for the entire run of the application (Thus freeing them isn't important as they are only created once and all resources are free'd by windows when the application closes).

Memory Leaks are only where your losing memory each time you perform an operation. The Memory Manager has no way to tell if they are true leaks or not. They are False Positives.

Normal on exit:


I have implemented a Extra Header in the SHP.Pas which will save the Palette, Game and SHP Type to the end of the SHP file. The idea being that published SHP's could be downloaded by people and the editor would not need the palette the SHP was designed for and would know how to handle the SHP. I havn't tested but it shouldn't be an issue for the games since its just added at the end of the file. Its adds 778 bytes to the filesize.

Currently there is no way to set the Game/Type.


_________________
Free Map Editor - Game Requirements - Stucuk.Net

Back to top
View user's profile Send private message Visit poster's website
stucuk
Geek


Joined: 27 Aug 2002

PostPosted: Fri Aug 19, 2016 9:45 pm    Post subject: Reply with quote



Implemented the Side Colours. The way the system works is that the actual palette is untouched. When anything requests the palette(Such as drawing a frame) it returns a new palette which has the Side Colours updated (Unless None is set).

Also added a "One Frame Load" option to the SHP class which will be of use with an "Open Dialog" idea i had.
_________________
Free Map Editor - Game Requirements - Stucuk.Net

Back to top
View user's profile Send private message Visit poster's website
tomsons26lv
Scorpion Sniper


Joined: 30 Dec 2009
Location: Latvia

PostPosted: Fri Aug 19, 2016 10:11 pm    Post subject: Reply with quote

Heh, well it might present a issue ingame actually, it needs to be tested, you might be able to get away with it with TS/RA2, TD to RA1 is a lot less probable.

The Shape formats can actually have a palette in them but Westwood never used this in any game files, probably just a conversion tool thing.
For TD/RA See http://www.ppmforums.com/viewtopic.php?p=544948#544948
Dunno if the game handles it or ignores it.

Not 100% sure but iirc TS had that too.

As for TMP since i know TMPs are supported by SHP Builder to some extent, that format is actually called Icon(however TS TMP format is called IsoTile), if you ever plan to add proper editing support here's the original header from debug builds of older WW games i have
struct IconControlType {
   sint16 Width; //always 24 (ICON_WIDTH)
   sint16 Height; //always 24 (ICON_HEIGHT)
   sint16 Count; //count of cells in set, not same as images
   sint16 Allocated; //is treated like a bool.
   sint16 MapWidth; //tile width in cells
   sint16 MapHeight; //tile height in cells
   sint32 Size; //filesize
   sint32 Icons; //always 0x00000028
   sint32 Palettes; //seems to always be 0x00000000
   sint32 Remaps; //unknown, bitfield?
   sint32 TransFlag; //array of images length, unknown
   sint32 ColorMap; //terrain type index, ra only
   sint32 Map; //image index for each cell

Dunno where my notes on the TS format are all i found right now is my headers collection txt.

SHP_TS:
2 A
2 Width
2 Height
2 Count
2 OffsetX
2 OffsetY
2 CX // Same as in RA used by Westwood's ShapeSet to crop the SHP, should be 0
2 CY // Same as above, should be 0
4 Compression // Might actually be Flags like the earlier games,
// meaning not just compression can be defined but other variables that affect how the SHP is drawn ingame.
; RadarColor
1 Red
1 Green
1 Blue
1 Alpha
4 Zero?
4 FileOffset

Fun random fact, Blade Runner actually introduced another version of Shape format, which was the last one to my knowledge, the SHPs there are 24-bit or even 32-bit, dunno about that.

I actually intend to collect a summation of what is known about the shp formats and the compressions they use and what WW had in general
EOB to Lands of Lore had these
CompressionType{
   NOCOMPRESS = 0,
   LZW12 = 1,              // On eob.wikispaces.com refered to as crunch1
   LZW14 = 2,              // On eob.wikispaces.com refered to as format2, not the same as format2 in Dune2 however
   HORIZONTAL = 3,     // HORIZONTAL RLE, On eob.wikispaces.com refered to as format3, Westwood's Monopoly however has TOPDOWN and BOTTOMUP, not sure what uses those if anything even
   LCW = 4                   // Lempel–Castle–Welch, commonly called Format80 in the community

There are others mentioned throughout the web like Format20, Format5, Format64, Zero-Compression and whatnot and others, gonna try to connect the dots to the debug build symbols i have.
Format20 is just XORDelta, the difference is just how it's applied.
_________________
Tiberian Dawn, Red Alert, Tiberian Sun ,Red Alert 2,Renegade, Command & Conquer 3,Tiberium and Tiberium Wars and Westwood related image & video archive
https://picasaweb.google.com/113361105083292812413?noredirect=1

Skype live:tomsons26
Don't forget to state who are you otherwise i'll ignore the invite

Back to top
View user's profile Send private message Visit poster's website
stucuk
Geek


Joined: 27 Aug 2002

PostPosted: Sat Aug 20, 2016 12:51 am    Post subject: Reply with quote

tomsons26lv wrote:
Heh, well it might present a issue ingame actually, it needs to be tested, you might be able to get away with it with TS/RA2, TD to RA1 is a lot less probable.


Unless it checks the file size (Which there is no reason to do so) it won't be an issue. SHP's don't have a CHUNK style system.

tomsons26lv wrote:
4 Compression // Might actually be Flags like the earlier games,
// meaning not just compression can be defined but other variables that affect how the SHP is drawn ingame.


Compression is meant to be a single byte with the other 3 being "Align". At least according to SHP Builder. Its unlikely compression is a flag since it uses 3 to specify a compression (Bit flags are just true/false for each bit, 1+2=3).


tomsons26lv wrote:
; RadarColor
1 Red
1 Green
1 Blue
1 Alpha


Is it confirmed that RadarColor is RGBA? Why would it need Alpha for a radar color?

tomsons26lv wrote:
As for TMP

TMP's look odd. At least how SHP Builder loads them. The way to work out the offsets just looks horrible (Well to know how many there is).
_________________
Free Map Editor - Game Requirements - Stucuk.Net

Back to top
View user's profile Send private message Visit poster's website
tomsons26lv
Scorpion Sniper


Joined: 30 Dec 2009
Location: Latvia

PostPosted: Sat Aug 20, 2016 1:34 am    Post subject: Reply with quote

stucuk wrote:
Unless it checks the file size (Which there is no reason to do so) it won't be an issue. SHP's don't have a CHUNK style system.

Oh it does, in the first gen C&C Engine(which is what its called btw Very Happy) the Delta(what SHP Builder has as MaxFrameSize) specified in the header is used for memory preallocation for the shp for decompression, if it's horribly wrong the game might crash actually, seems the radar.shp because of the frame amount and resolution of them is prone to this, we discovered this while we were working on getting proper compressions working for RApp.


stucuk wrote:

Compression is meant to be a single byte with the other 3 being "Align". At least according to SHP Builder. It's unlikely compression is a flag since it uses 3 to specify a compression (Bit flags are just true/false for each bit, 1+2=3).

If it is Flags, it's not just a byte, it's all 4 bytes.

From RA debug build
enum ShapeFlags_Type{
   SHAPE_NORMAL = 0x00,
   SHAPE_HORZ_REV =  0x01,
   SHAPE_VERT_REV =  0x02,
   SHAPE_SCALING =  0x04,
   SHAPE_VIEWPORT_REL = 0x10,
   SHAPE_WIN_REL = 0x10,
   SHAPE_CENTER = 0x20,
// Not used in RA anymore, just were in symbols
   SHAPE_FADING = 0x0000,
   SHAPE_PREDATOR = 0x000
   SHAPE_COMPACT = 0x0000
   SHAPE_PRIORITY = 0x0000
   SHAPE_GHOST = 0x0000
   SHAPE_SHADOW = 0x0000
   SHAPE_PARTIAL = 0x0000
   SHAPE_COLOR = 0x00000000

};
Well turns out it's not just me that has a suspicion of that that in TS SHP format are Flags, Blade thinks it could be too.

stucuk wrote:

Is it confirmed that RadarColor is RGBA? Why would it need Alpha for a radar color?

Irrc some TS/RA2 files have 255 there, think it were the rocks. No clue, seems it's obsolete.

Still updating it when i find more but
https://dl.dropboxusercontent.com/u/58357192/RApp/compressionsandstuff.txt

I just officially confirmed one of  the TS compressions is RLE, it's funny how, i checked WWs TS/RA2 ShapeSet which i have and keep forgetting about lol.



So yea TS uses a compression they made way back before the 1990s Very Happy
_________________
Tiberian Dawn, Red Alert, Tiberian Sun ,Red Alert 2,Renegade, Command & Conquer 3,Tiberium and Tiberium Wars and Westwood related image & video archive
https://picasaweb.google.com/113361105083292812413?noredirect=1

Skype live:tomsons26
Don't forget to state who are you otherwise i'll ignore the invite

Back to top
View user's profile Send private message Visit poster's website
stucuk
Geek


Joined: 27 Aug 2002

PostPosted: Sat Aug 20, 2016 2:21 am    Post subject: Reply with quote

tomsons26lv wrote:
Oh it does, in the first gen C&C Engine(which is what its called btw Very Happy) the Delta(what SHP Builder has as MaxFrameSize) specified in the header is used for memory preallocation for the shp for decompression, if it's horribly wrong the game might crash actually, seems the radar.shp because of the frame amount and resolution of them is prone to this, we discovered this while we were working on getting proper compressions working for RApp.


But that should have nothing to do with adding extra data to the end. My understanding is that "Delta" its just the buffer size used to decompress the frames (As in the output buffer).


stucuk wrote:
If it is Flags, it's not just a byte, it's all 4 bytes.

The other 3 bytes could be flags. It just doesn't make sense to have 1,2,3 as values if the first byte was flags since Bit 1 + Bit 2 = 3. With flags you have each flag being a separate bit.

tomsons26lv wrote:
I just officially confirmed one of  the TS compressions is RLE, it's funny how, i checked WWs TS/RA2 ShapeSet which i have and keep forgetting about lol.

Wouldn't that be compression 3? It is kinda RLE except it only does it with pixels that are the background colour ( 0 ).
_________________
Free Map Editor - Game Requirements - Stucuk.Net

Back to top
View user's profile Send private message Visit poster's website
tomsons26lv
Scorpion Sniper


Joined: 30 Dec 2009
Location: Latvia

PostPosted: Sat Aug 20, 2016 2:31 am    Post subject: Reply with quote

stucuk wrote:
tomsons26lv wrote:
Oh it does, in the first gen C&C Engine(which is what its called btw Very Happy) the Delta(what SHP Builder has as MaxFrameSize) specified in the header is used for memory preallocation for the shp for decompression, if it's horribly wrong the game might crash actually, seems the radar.shp because of the frame amount and resolution of them is prone to this, we discovered this while we were working on getting proper compressions working for RApp.


But that should have nothing to do with adding extra data to the end. My understanding is that "Delta" its just the buffer size used to decompress the frames (As in the output buffer).


stucuk wrote:
If it is Flags, it's not just a byte, it's all 4 bytes.

The other 3 bytes could be flags. It just doesn't make sense to have 1,2,3 as values if the first byte was flags since Bit 1 + Bit 2 = 3. With flags you have each flag being a separate bit.

tomsons26lv wrote:
I just officially confirmed one of  the TS compressions is RLE, it's funny how, i checked WWs TS/RA2 ShapeSet which i have and keep forgetting about lol.

Wouldn't that be compression 3? It is kinda RLE except it only does it with pixels that are the background colour ( 0 ).


eh bit sleepy, nvm then i guess.

Guess i need Blade to explain how WW did the Flags part then.

Yea seems that is it, its RLE Zero.

Edit:
Well found something interesting in ShapeSet. There's also a -X command line, the only changes are the first byte of Flags is set to 01(the default is 0, even with RLE, i think its forced on or off, not sure) and after every 310 bytes there's 00 00 and the input file resolution was 310x224 but with this cmd the shp header says 312x224

The ShapeSet i have seems is dumbed down, the main function has 26 switch cases but the main function has actual code for just 9.

Edit2: The file produced by the -X command line is valid. RA2 reads it properly. Also seems i discovered what happens if the frame Flags are 00 00

See how when units pass though it renders in scanlines Very Happy

The plot thickens, with the frame flags the SHPs SHP Builder adds it has a diamond square cutoff

The file isn't corrupt, donno whats that its drawing over it, ib4 its some compression stats or something + from the looks of it  zbuilding shp is at the bottom from that diamond shape it seems

Edit3:
Yea confirmed, what was referred to as Compression and assumed is one byte is in fact 4 bytes and its Flags like the older games, see harvestr.shp in RA2 cache.mix for proof, it has 0x0300CCCC there

So far i've found:
SHP Flags
Most Shps 01 00 00 00
Most Shps 03 00 00 00
50cal 00 00 CC CC
warpin 03 00 CC CC
ucelec 01 00 CC CC
tstlexp 04 00 00 00
progbarm 00 00 00 00
null CC CC CC CC

These are noted down as i encountered them for the first time, after that i didn't make note on the same ones. Only checked conquer.mix and generic.mix so far, but they aren't unique to the files above, a lot of them are used in quite a few SHPs.

null and c_shadow have CC CC CC CC, i think when set to that it affects how its cached/stored in memory given the nature of both files.
Zero? isn't zeros either, extra flags maybe? c_shadow has CC CC CC CC there.
Alpha is CC in c_shadow as well.

Edit4:
SHP Flags
drop000# 01 00 6C 00
tg07a2 03 00 56 00

infshdw and gclock2 Zero? is 00 FB 12 00

Menu UI elements and shps in RA2's setup.mix pretty much all have flags as 00 00 00 00
_________________
Tiberian Dawn, Red Alert, Tiberian Sun ,Red Alert 2,Renegade, Command & Conquer 3,Tiberium and Tiberium Wars and Westwood related image & video archive
https://picasaweb.google.com/113361105083292812413?noredirect=1

Skype live:tomsons26
Don't forget to state who are you otherwise i'll ignore the invite

Last edited by tomsons26lv on Sat Aug 20, 2016 4:52 pm; edited 16 times in total

Back to top
View user's profile Send private message Visit poster's website
E1 Elite
Laser Commando


Joined: 28 May 2013

PostPosted: Sat Aug 20, 2016 6:16 am    Post subject: Reply with quote

Distorted save cases with SHPToolz_2016-08-19:
- Distorted frames in both normal and shadows. Probably because of bigger SHP dimension.
- Compression 2 frames are distorted.

Tested with TS, game shows the same distorted image for compression 2 frames. But it is able
to run them. If comp2 is saved properly then it should work in game. XCC Mixer doesn't seem
to handle compression 2 and it crashes.

Extra trailing bytes doesn't seem to have any problems either with game or other tools. Tested
in TS only, SHP Builder and XCC Mixer.

Attached SHP files to test both distorted save cases. Just save with the tool and view them.


DistortedSaves.rar
 Description:

Download
 Filename:  DistortedSaves.rar
 Filesize:  35.56 KB
 Downloaded:  7 Time(s)


Back to top
View user's profile Send private message
stucuk
Geek


Joined: 27 Aug 2002

PostPosted: Sat Aug 20, 2016 6:14 pm    Post subject: Reply with quote

- SHPToolz_2016-08-20.rar

Fixed Encoding of 2 and 3. 2's issue was that it was using a 0 based index as if it was a count of the pixels (So it was always 1 off). 3's issue was that it wasn't handling when there were more than 255 zero's together in a line (So the byte just looped around back to 0 when it hit 256).

@tomsons26lv: I havn't extracted RA2 files yet. I will need to add a debug mode to the loading so it would store the frame headers if the app requested it (Currently headers are thrown away after they are used in the loading).
_________________
Free Map Editor - Game Requirements - Stucuk.Net

Back to top
View user's profile Send private message Visit poster's website
E1 Elite
Laser Commando


Joined: 28 May 2013

PostPosted: Sat Aug 20, 2016 7:02 pm    Post subject: Reply with quote

With SHPToolz_2016-08-20:

Compression 2 is not fixed yet, it is still distorted. You just made the SHP that I provided
to save it in compression 3. Make any SHP save in comp 2 and then test.

Another SHP attached, this time it gets cropped and image is changed.


Check.rar
 Description:

Download
 Filename:  Check.rar
 Filesize:  731 Bytes
 Downloaded:  6 Time(s)


Back to top
View user's profile Send private message
stucuk
Geek


Joined: 27 Aug 2002

PostPosted: Sat Aug 20, 2016 7:50 pm    Post subject: Reply with quote

@E1 Elite: If i force it to save in Comp2 it does it perfectly fine as i fixed the issue...

--------------------------------------------------

Attached is a txt file displaying the header information of the SHP's i have (768 or something, most from TS).

Format is: X,Y WxH Compression(As a Byte) CompressionHex(As 4 Bytes) RadarColour(RGBA)


SHPs_debug.txt
 Description:

Download
 Filename:  SHPs_debug.txt
 Filesize:  2.04 MB
 Downloaded:  6 Time(s)


_________________
Free Map Editor - Game Requirements - Stucuk.Net

Back to top
View user's profile Send private message Visit poster's website
tomsons26lv
Scorpion Sniper


Joined: 30 Dec 2009
Location: Latvia

PostPosted: Sat Aug 20, 2016 8:04 pm    Post subject: Reply with quote

Hmm interesting. Flags are less set for TS files
some of of the interesting ones from your list

dirtexpl.shp 4D000000
fsair.shp 9B000000
fsgrnd.shp 63000000
fsidle.shp 2A000000
Inviso.shp 48000000
SideNC0#\* 03005600
nahand_b.shp 30000000

and null.shp flag is C8000000, interestingly it only has a header and that's it, no image data guess next step is to see what happens with C8 on a shp with frames

Edit:
SHP shows, but a weird thing happens if i move away and back to it


Hmm i think the first byte tells the game what Blitter to use.. Rest are some other flags
Of the known ones the game has these Blitters (the names are still in the exe)
Spoiler (click here to read it):
RLEBlitTransLucent25AlphaZReadWrite
RLEBlitTransLucent50AlphaZReadWrite
RLEBlitTransLucent75AlphaZReadWrite
RLEBlitTransZRemapXlatAlphaZReadWrite
RLEBlitTransXlatAlphaZReadWrite
RLEBlitTransLucent25AlphaZReadWarp
RLEBlitTransLucent50AlphaZReadWarp
RLEBlitTransLucent75AlphaZReadWarp
RLEBlitTransLucent25AlphaZRead
RLEBlitTransLucent50AlphaZRead
RLEBlitTransLucent75AlphaZRead
RLEBlitTransZRemapXlatAlphaZRead
RLEBlitTransXlatAlphaZRead
RLEBlitTransLucent25Alpha
RLEBlitTransLucent50Alpha
RLEBlitTransLucent75Alpha
RLEBlitTransZRemapXlatAlpha
RLEBlitTransXlatAlpha
RLEBlitTransLucent25ZReadWrite
RLEBlitTransLucent50ZReadWrite
RLEBlitTransLucent75ZReadWrite
RLEBlitTransDarkenZReadWrite
RLEBlitTransZRemapXlatZReadWrite
RLEBlitTransXlatZReadWrite
RLEBlitTransLucent25ZReadWarp
RLEBlitTransLucent50ZReadWarp
RLEBlitTransLucent75ZReadWarp
RLEBlitTransLucent25ZRead
RLEBlitTransLucent50ZRead
RLEBlitTransLucent75ZRead
RLEBlitTransDarkenZRead
RLEBlitTransZRemapXlatZRead
RLEBlitTransXlatZRead
RLEBlitTransLucent25
RLEBlitTransLucent50
RLEBlitTransLucent75
RLEBlitTransDarken
RLEBlitTransZRemapXlat
RLEBlitTransXlat
BlitTransLucent25AlphaZReadWrite
BlitTransLucent50AlphaZReadWrite
BlitTransLucent75AlphaZReadWrite
BlitTransZRemapXlatAlphaZReadWrite
BlitTransXlatAlphaZReadWrite
BlitTransLucent25AlphaZReadWarp
BlitTransLucent50AlphaZReadWarp
BlitTransLucent75AlphaZReadWarp
BlitTransLucent25AlphaZRead
BlitTransLucent50AlphaZRead
BlitTransLucent75AlphaZRead
BlitTransZRemapXlatAlphaZRead
BlitTransXlatAlphaZRead
BlitTranslucent50ZeroAlpha
BlitTranslucent50NonzeroAlpha
BlitTranslucentWriteAlpha
BlitTransXlatMultWriteAlpha
BlitTransXlatWriteAlpha
BlitTransLucent25Alpha
BlitTransLucent50Alpha
BlitTransLucent75Alpha
BlitTransZRemapXlatAlpha
BlitTransXlatAlpha
BlitPlainXlatAlpha
BlitTransLucent25ZReadWrite
BlitTransLucent50ZReadWrite
BlitTransLucent75ZReadWrite
BlitTransDarkenZReadWrite
BlitTransZRemapXlatZReadWrite
BlitTransXlatZReadWrite
BlitPlainXlatZReadWrite
BlitTransLucent25ZReadWarp
BlitTransLucent50ZReadWarp
BlitTransLucent75ZReadWarp
BlitTransLucent25ZRead
BlitTransLucent50ZRead
BlitTransLucent75ZRead
BlitTransDarkenZRead
BlitTransZRemapXlatZRead
BlitTransXlatZRead
BlitPlainXlatZRead
BlitTransLucent25
BlitTransLucent50
BlitTransLucent75
BlitTransDarken
BlitTransZRemapXlat
BlitTransXlat
BlitPlainXlat
RLEBlitTransRemapXlatZReadWrite
RLEBlitTransRemapDestZReadWrite
RLEBlitTransRemapXlatZRead
RLEBlitTransRemapDestZRead
RLEBlitTransRemapXlat
RLEBlitTransRemapDest
BlitTransRemapXlat
BlitTransRemapDest
Blitter
RLEBlitter
BlitPlain
BlitTrans

When i was messing with the header to try to get it to crash so i could find what function checks what i only managed to crash it in the blitter code at 0049919D which is RLEBlitTransXlatAlphaZRead
_________________
Tiberian Dawn, Red Alert, Tiberian Sun ,Red Alert 2,Renegade, Command & Conquer 3,Tiberium and Tiberium Wars and Westwood related image & video archive
https://picasaweb.google.com/113361105083292812413?noredirect=1

Skype live:tomsons26
Don't forget to state who are you otherwise i'll ignore the invite

Last edited by tomsons26lv on Sat Aug 20, 2016 8:29 pm; edited 3 times in total

Back to top
View user's profile Send private message Visit poster's website
E1 Elite
Laser Commando


Joined: 28 May 2013

PostPosted: Sat Aug 20, 2016 8:11 pm    Post subject: Reply with quote

View the compression 2 file in SHP Builder or in game. If you say it is fixed, then game doesn't
display as it should.

Back to top
View user's profile Send private message
stucuk
Geek


Joined: 27 Aug 2002

PostPosted: Sat Aug 20, 2016 9:15 pm    Post subject: Reply with quote

@tomsons26lv: I still don't see how Compression(Byte) is part of a flag. Each frame which has Compression as a value other than 1, 2 or 3 has its width/height as zero. I haven't seen any frames which use any other values and which also have a width and height.

I am also doubting that Radar Color is RGBA. RGB is used but A isn't except on some odd ones. It may be the 4th value is actually something else.


E1 Elite wrote:
View the compression 2 file in SHP Builder or in game. If you say it is fixed, then game doesn't
display as it should.


Well its pointless if the game doesn't work how i thought it did (How SHP Builder works is irrelevant) as the way they seem to be stored in official files is by using the same width for each line.... Which is the same as just storing the raw image except you have an additional 2 bytes per row..........

Whats the point in it...
_________________
Free Map Editor - Game Requirements - Stucuk.Net

Back to top
View user's profile Send private message Visit poster's website
tomsons26lv
Scorpion Sniper


Joined: 30 Dec 2009
Location: Latvia

PostPosted: Sat Aug 20, 2016 10:13 pm    Post subject: Reply with quote

Right so figured out more pieces to the puzzle.
The first byte of Flags is checked during runtime by a function that decides what Blitter to use.
From what i understand from the names and whats going on the Blitters have the decompression code, RLE and some other compression,
or maybe it's even just raw image data being blitted not sure. Might be the only two that exist. RLE might have variations.
If i change the first byte in the game memory it doesn't crash but it renders a corrupt shp on some values,
i think the game decides what blitter type(RLE or orther) its gonna use on the first read for each frame and never re-decides again.

Edit:
Yea the Flags are 4 bytes, traced the header in memory its read at 0069E920  - mov al,[eax+08] by the blitter.
_________________
Tiberian Dawn, Red Alert, Tiberian Sun ,Red Alert 2,Renegade, Command & Conquer 3,Tiberium and Tiberium Wars and Westwood related image & video archive
https://picasaweb.google.com/113361105083292812413?noredirect=1

Skype live:tomsons26
Don't forget to state who are you otherwise i'll ignore the invite

Last edited by tomsons26lv on Sun Aug 21, 2016 5:04 pm; edited 2 times in total

Back to top
View user's profile Send private message Visit poster's website
stucuk
Geek


Joined: 27 Aug 2002

PostPosted: Sat Aug 20, 2016 10:41 pm    Post subject: Reply with quote

@tomsons26lv: That would suggest it has nothing to do with the other 3 Bytes.

tomsons26lv wrote:
If i change the first byte in the game memory it doesn't crash but it renders a corrupt shp on some values, i think the game decides what blitter type(RLE or orther) its gonna use on the first read for each frame and never redecides again.


Sounds like it stores the compressed image in memory and just renders directly from it (as in never storing an uncompressed image). It would make sense as thats what you would expect from the game so that rendering is faster and memory usage is lower. Compression 3 for example would allow the render to completely skip drawing some pixels.
_________________
Free Map Editor - Game Requirements - Stucuk.Net

Back to top
View user's profile Send private message Visit poster's website
tomsons26lv
Scorpion Sniper


Joined: 30 Dec 2009
Location: Latvia

PostPosted: Sat Aug 20, 2016 11:40 pm    Post subject: Reply with quote

stucuk wrote:

Well its pointless if the game doesn't work how i thought it did (How SHP Builder works is irrelevant) as the way they seem to be stored in official files is by using the same width for each line.... Which is the same as just storing the raw image except you have an additional 2 bytes per row..........

Whats the point in it...

There isn't any compression 2 Very Happy
Seems indeed the game just blits raw data to screen for that

stucuk wrote:
Sounds like it stores the compressed image in memory and just renders directly from it (as in never storing an uncompressed image). It would make sense as that's what you would expect from the game so that rendering is faster and memory usage is lower. Compression 3 for example would allow the render to completely skip drawing some pixels.

Yea seems the blit code processes it as needed in the process if RLE is present handling that.
_________________
Tiberian Dawn, Red Alert, Tiberian Sun ,Red Alert 2,Renegade, Command & Conquer 3,Tiberium and Tiberium Wars and Westwood related image & video archive
https://picasaweb.google.com/113361105083292812413?noredirect=1

Skype live:tomsons26
Don't forget to state who are you otherwise i'll ignore the invite

Back to top
View user's profile Send private message Visit poster's website
stucuk
Geek


Joined: 27 Aug 2002

PostPosted: Sun Aug 21, 2016 6:20 am    Post subject: Reply with quote

tomsons26lv wrote:
There isn't any compression 2 Very Happy
Seems indeed the game just blits raw data to screen for that


There is some SHP's which use it. Though its ones where the width is like 1, 2 or 4 (For that frame's area not the SHP Width).

------------------------------

The following is my idea for an Open Dialog (Click it for larger image). Unfortunatly i couldn't get the Modern Shell ComboBox thing to work so currently using horrible Windows 3.1 style directory listbox. The SHP's are loaded (Only first frame and only once) when the EasyListView component needs to render a image so it will only load whatever is within view.


_________________
Free Map Editor - Game Requirements - Stucuk.Net

Back to top
View user's profile Send private message Visit poster's website
tomsons26lv
Scorpion Sniper


Joined: 30 Dec 2009
Location: Latvia

PostPosted: Sun Aug 21, 2016 1:29 pm    Post subject: Reply with quote

lolok
If possible add a path box, it will be actually usable then and will work as a temp until you get the proper one working

I have a idea, can you make a small app with drag and drop support that should show files as raw 8 bpp image? There aren't that many apps out there, 7yuv infact is the only decent one i've found but it isn't freeware.
Then we can indefinately see if something is compressed or not and even see hints of how if it is, case point TS fog.shp

_________________
Tiberian Dawn, Red Alert, Tiberian Sun ,Red Alert 2,Renegade, Command & Conquer 3,Tiberium and Tiberium Wars and Westwood related image & video archive
https://picasaweb.google.com/113361105083292812413?noredirect=1

Skype live:tomsons26
Don't forget to state who are you otherwise i'll ignore the invite

Back to top
View user's profile Send private message Visit poster's website
stucuk
Geek


Joined: 27 Aug 2002

PostPosted: Sun Aug 21, 2016 5:39 pm    Post subject: Reply with quote

tomsons26lv wrote:
If possible add a path box, it will be actually usable then and will work as a temp until you get the proper one working


It is actually usable, you just have to double click the folders to change (And it then updates).

tomsons26lv wrote:
Then we can indefinately see if something is compressed or not and even see hints of how if it is, case point TS fog.shp


fog.shp is not compressed. 0 and 1 are the same. So far there are only the following values: 0, 1, 2, 3. No SHP i have tried loading from TS/RA2 is triggering the breakpoint i have in the code if another type is used (Ones with other values have no size).

I have noticed that some of the random SHP's i have use 0 but the original's don't so they are likely saved by a 3rd party editor like SHP Builder. I guess the test would be to use a SHP saved with compression 0 in TS/RA2 and see if it looks as it should. If so 0 is the same as 1.


------------------------------------------------------------------



P.S SHPToolz is turning out to be the most impressive looking SHP Editor with the least amount of editing ability (Currently only Radar Color #Tongue)
_________________
Free Map Editor - Game Requirements - Stucuk.Net

Back to top
View user's profile Send private message Visit poster's website
tomsons26lv
Scorpion Sniper


Joined: 30 Dec 2009
Location: Latvia

PostPosted: Sun Aug 21, 2016 6:51 pm    Post subject: Reply with quote

stucuk wrote:
tomsons26lv wrote:
If possible add a path box, it will be actually usable then and will work as a temp until you get the proper one working


It is actually usable, you just have to double click the folders to change (And it then updates).

That's the point, that is annoying as hell when you have many folders that are in folders, being able to paste in the path to the dir you are working with makes it so much more usable. Navigating folders using that tree view was annoying as hell on Windows 3.1 already.(not saying replace tree view, just this in addition to it)

Ha found something in YRpp
https://github.com/Ares-Developers/YRpp/blob/master/ConvertClass.h#L47
So that's that, confirms the game decides what blitter to use via those flags, no idea how that works tho but seems the flags are known https://github.com/Ares-Developers/YRpp/blob/master/GeneralDefinitions.h#L1307, well some of them, but i think those are internal values, not what the shps have.
_________________
Tiberian Dawn, Red Alert, Tiberian Sun ,Red Alert 2,Renegade, Command & Conquer 3,Tiberium and Tiberium Wars and Westwood related image & video archive
https://picasaweb.google.com/113361105083292812413?noredirect=1

Skype live:tomsons26
Don't forget to state who are you otherwise i'll ignore the invite

Back to top
View user's profile Send private message Visit poster's website
stucuk
Geek


Joined: 27 Aug 2002

PostPosted: Mon Aug 22, 2016 1:50 am    Post subject: Reply with quote

Doesn't prove SHP's have flags (Not saying they don't as the 3 bytes after the compression byte sometimes has a value). From what i can tell that code is "linking" directly to the memory of the game and using its procedures/functions (Or overriding).

In re-guards to the combobox, i have found the issue with the ShellComboBox and its actually an issue with the ComboBoxEx screwing up when Insert is used.... The dropdown has a different set of items(Its overwriting values instead of inserting) due to it. No clue why.

I have fixed it by storing all the items in a list, clearing the items from the combo, inserting into list and then building the combo's items from the list. Its an over the top solution but it works and wasn't hard to implement.

Note that the ShellComboBox doesn't do exactly what you want. May be possible to implement what you want into it without wasting alot of time on it.
_________________
Free Map Editor - Game Requirements - Stucuk.Net

Back to top
View user's profile Send private message Visit poster's website
tomsons26lv
Scorpion Sniper


Joined: 30 Dec 2009
Location: Latvia

PostPosted: Mon Aug 22, 2016 2:45 am    Post subject: Reply with quote

We (at CnCNet) were debugging fog of war crashes which happen in Blitting functions, thanks to Bittah we have a test map that consistently crashes, in the process i asked him to test the known Flags with a building that was seemingly causing or facilitating in some way to the crash results are here
https://dl.dropboxusercontent.com/u/58357192/RApp/compressionsandstuff.txt
Welp that proves its Flags, and also says it's a lot more complicated what those bytes are then when i grasped
_________________
Tiberian Dawn, Red Alert, Tiberian Sun ,Red Alert 2,Renegade, Command & Conquer 3,Tiberium and Tiberium Wars and Westwood related image & video archive
https://picasaweb.google.com/113361105083292812413?noredirect=1

Skype live:tomsons26
Don't forget to state who are you otherwise i'll ignore the invite

Back to top
View user's profile Send private message Visit poster's website
stucuk
Geek


Joined: 27 Aug 2002

PostPosted: Mon Aug 22, 2016 5:08 am    Post subject: Reply with quote

Then tell me how to detect which mode to use? Cos its not a flag. Flags store each value on DIFFERENT bits. Its the only way flags work. 1+2=3=Not Flag. The whole point of flags is to store multiple boolean values inside a number(each one being a single bit).

Also 01 and 02 exist in official ww shp's (As in ones with actual data) so i don't see how they can claim Format1 and Format2 don't exist.

-----------------------------


_________________
Free Map Editor - Game Requirements - Stucuk.Net

Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic Page 1 of 3 [149 Posts] Goto page: 1, 2, 3 Next
View previous topic :: View next topic
 Forum index » Featured Tools » OS SHP Builder
Jump to:  
Quick Reply
Username:


If you are visually impaired or cannot otherwise play the game below please contact the Administrator for help.


 
You can post new topics in this forum
You can reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © phpBB Group

Wildcard SSL Certificates
[ Time: 0.4281s ][ Queries: 14 (0.0143s) ][ Debug on ]