:: Home :: Get Hosted :: PPM FAQ :: Forum FAQ :: Privacy Policy :: Search :: Memberlist :: Usergroups :: Register :: Profile :: Log in to check your private messages :: Log in ::
Do you want to advertise at Project Perfect Mod. Find out how to do it HERE.

The time now is Sat Dec 15, 2018 7:38 pm
All times are UTC + 0
 Forum index » Featured Tools » Community Tools
Engie File converter + Game formats documentation
Moderators: Community Tools Developpers
Post new topic   Reply to topic Page 1 of 1 [20 Posts] View previous topic :: View next topic
Author Message
Nyerguds
General


Joined: 24 May 2004
Location: Flanders (Be) Posts:300000001

PostPosted: Wed Aug 15, 2018 2:23 pm    Post subject:  Engie File converter + Game formats documentation
Subject description: New tool and up-to-date format specs
Reply with quote

Hello, PPM!

For the past year or so, I've been working on adding new game file formats into a tool I originally created to convert files extracted from the Nintendo 64 ROM of Command & Conquer.

In the past few months I finally added the classic C&C formats into the tool, and with the help of the people of the RedAlert++ project, I managed to make my tool create versions of these files that are virtually indistinguishable from the original Westwood-created ones, and, with some optimisations of my own, usually even better compressed.

At a certain point in the development I stumbled on the Shikadi.net DOS Games Modding Wiki, and decided to document a bunch of the formats I had implemented. All existing documents on these formats are ancient, and many of them are full of inaccuracies, so since we gained so much knowledge from disassembling the exes in recent years, this was long overdue.

Well, this documenting process is finally finished, so, with some pride, I present to you:


(image can differ slightly from current version; this is my WIP of the next version)

Download:

Extra modding features:
  • Combine / split shadows (for TS SHP files)
  • Convert frames into a single image
  • Split single image into frames
  • Paste image onto a range of frames, with transparency support (mostly meant for modding cutscenes in other games)
  • Height map generation tools for N64 maps (not that I think many people here will care much about that Razz)

Wiki pages on the Shikadi.net DOS Games Modding Wiki:

The supported (Westwood related) formats:
The font formats are probably better edited with the Westwood Font Editor, though. They are included because this tool has the ability to convert them to sprite sheets.

Westwood Studios' compression formats:

_________________

Last edited by Nyerguds on Thu Dec 06, 2018 2:51 pm; edited 8 times in total

Back to top
View user's profile Send private message Visit poster's website Skype Account
E1 Elite
Commander


Joined: 28 May 2013

PostPosted: Wed Aug 15, 2018 2:53 pm    Post subject: Reply with quote

How to load unittem.pal for a TS SHP?

Back to top
View user's profile Send private message
Nyerguds
General


Joined: 24 May 2004
Location: Flanders (Be) Posts:300000001

PostPosted: Wed Aug 15, 2018 2:56 pm    Post subject: Reply with quote

You just put the palette in the tool's folder and it'll add it to the list.

There's a feature to save a palette from a file into the list directly, which also puts it in the tool's folder, but there seem to be some bugs in the refresh logic at the moment, and palettes added that way don't seem to show up unless the list is somehow reloaded Confused

That said, any image can be saved as palette, so if you got some converted image of a TS SHP you can just open it, go to File-> Save, select "6-bit palette", and save it in the tool's folder that way. But you'll likely need a program restart to load it anyway.

There are also still some bugs in the transparency handling of the palettes... at the moment it's forced by file type, rather than set to an initial state per file type, meaning it often can't be changed despite the palette editor having features to edit it. I still need to dig into that and clean it all up.
_________________

Back to top
View user's profile Send private message Visit poster's website Skype Account
E1 Elite
Commander


Joined: 28 May 2013

PostPosted: Wed Aug 15, 2018 3:48 pm    Post subject: Reply with quote

The details from the SHP format is shown framewise, that is nice.

Some thoughts:
- Could scan for palettes in any subfolder (or use a Palettes subfolder to start with)
- Could show the first frame of SHP instead of -1th frame, when opening
- Open dialog could use last used folder
- frame counter could goto first frame after the last frame in a loop
- background color for TS SHP could be automatically be the first palette color

Back to top
View user's profile Send private message
Nyerguds
General


Joined: 24 May 2004
Location: Flanders (Be) Posts:300000001

PostPosted: Wed Aug 15, 2018 4:07 pm    Post subject: Reply with quote

  • A palettes sub-folder... yea, that could work. I will need to give this thing an options dialog eventually I guess.
  • The reason the view is set to -1 by default is because the tool is a converter, more than it is a viewer, and switching to a frame will change all processing to be relevant to that frame rather than to the whole loaded sequence.
    So, if you select frame 0, and then go to File -> Save, you will save that one frame only, and will not get the options to save as frame-containing types.
    Changing it to frame 0 automatically would get especially tedious if you load a frames sequence into the tool to convert it (Open a file from a sequence and, like RAD Video Tools, it'll detect that it's part of a sequence and will ask to load it all), since you'd need to go manually back to index -1 to save it as the intended frames-containing file type.
  • Uh, the Open dialog does use the last-used folder. In what situation didn't it do that for you? Do note that if you press "Cancel" on an "Open file" dialog, there is no way to retrieve what folder you were last in.
  • Frame counter looping is impossible; it's is a standard Windows control. Also, not sure if it should go to frame 0 or frame -1 then anyway.
  • Background for TS SHP is automatically the first palette colour. What I said before was that the rule gets enforced on every palette edit so you can't change it even if you want to.
    Do note that that any transformation you do on the frames, like combining the shadows, will make it stop being a "Tiberian Sun SHP file" and will turn it into a general "listing of frames". These take over some properties of the original, but will no longer truly be treated as TS SHP files.

_________________

Last edited by Nyerguds on Wed Aug 15, 2018 4:19 pm; edited 2 times in total

Back to top
View user's profile Send private message Visit poster's website Skype Account
E1 Elite
Commander


Joined: 28 May 2013

PostPosted: Wed Aug 15, 2018 4:18 pm    Post subject: Reply with quote

When closing and restarting the tool, the open dialog is on the tool folder, not last used.

When I open a SHP file, then select the palette, the Background: for the image shown is still pink, was expecting the blue color from unittem.pal.

Back to top
View user's profile Send private message
Nyerguds
General


Joined: 24 May 2004
Location: Flanders (Be) Posts:300000001

PostPosted: Wed Aug 15, 2018 4:22 pm    Post subject: Reply with quote

Nothing gets saved if you restart it; the tool doesn't have any kind of settings file. If that bothers you, use drag-and-drop to load your files; it's faster anyway.

The #FF00FF fuchsia is the default transparency colour; it can be modified by clicking the little coloured square at the bottom next to the word "Background:". The default 'background' colour on the TS units palette is black (as you can see on the palette viewer in the image I posted), which is not a very good colour for indicating transparency.

Obviously, if I add a settings file, the selected background colour would be one of the things saved in it.
_________________

Last edited by Nyerguds on Wed Aug 15, 2018 4:50 pm; edited 1 time in total

Back to top
View user's profile Send private message Visit poster's website Skype Account
E1 Elite
Commander


Joined: 28 May 2013

PostPosted: Wed Aug 15, 2018 4:34 pm    Post subject: Reply with quote

Blue color I mentioned was from RA2's unittem.pal. I typically use it with TS's customised unittem.pal as well.

Back to top
View user's profile Send private message
Nyerguds
General


Joined: 24 May 2004
Location: Flanders (Be) Posts:300000001

PostPosted: Wed Aug 15, 2018 4:49 pm    Post subject: Reply with quote

Yeah, most palettes I work with from Dune II, C&C1 and RA1 have black as well, hence why I went with "nope, automatic override" Razz
_________________

Back to top
View user's profile Send private message Visit poster's website Skype Account
Nyerguds
General


Joined: 24 May 2004
Location: Flanders (Be) Posts:300000001

PostPosted: Sun Aug 19, 2018 3:59 pm    Post subject: Reply with quote

I released v1.3.1, which fully supports sprite sheets even if the frames in them have differing sizes, like they do in Dune II.

It also has a palette matching feature for high colour content.



And I fixed the forced transparency bugs; file type based transparency is now applied to all palettes once at the moment a file is loaded, but can be freely adapted afterwards. It'll be reset when a new file is loaded, of course.
_________________

Back to top
View user's profile Send private message Visit poster's website Skype Account
OmegaBolt
President


Joined: 21 Mar 2005
Location: York, England

PostPosted: Mon Aug 20, 2018 1:59 pm    Post subject: Reply with quote

Just so you know, I get an unhandled exception when zooming in 3x on a spritesheet thats 2913x3270. Here's the log: https://pastebin.com/uGCEKkc0

Would be nice if there were zoom out options for spritesheets too.

Also after covering it to frames, it seems to upscale each image within the set frame-size, cutting off about 3rd of it.
_________________
TAK02 wrote:
Screw AttachEffect. Can't think of any use for it.

Back to top
View user's profile Send private message Send e-mail Skype Account
Nyerguds
General


Joined: 24 May 2004
Location: Flanders (Be) Posts:300000001

PostPosted: Mon Aug 20, 2018 7:51 pm    Post subject: Reply with quote

The Windows GDI+ system that's used by the .Net framework for graphics operations has certain upper limits, yea... it starts bugging out when you exceed a maximum possible image buffer size. And if you start zooming in on 3kx3k images I guess you'll get there pretty fast, yes.

I've never seen it crash on that, though, usually it just bugs out a bit. Problem is, this is pure UI code, so nothing I can catch in a function. The closest I can do is check what these buffer limits are and restrict the zoom to stay below it.

I've mostly been working with graphics from 320x200 games on this, so uh, a zoom out feature honestly never crossed my mind. I might look into that though, yea Confused

The second issue is an odd one, though. Could you give me the sprite sheet? Or at least tell me its bit depth and the exact settings you used? Note that in the latest versions the preview you see in the "image to frames" window is generated by exactly the same function as the one doing the final conversion, so you should be able to spot this by just going through the frames there...
_________________

Back to top
View user's profile Send private message Visit poster's website Skype Account
OmegaBolt
President


Joined: 21 Mar 2005
Location: York, England

PostPosted: Mon Aug 20, 2018 8:30 pm    Post subject: Reply with quote

Yeah just limiting the zoom if it would bug out would be nice. I guess honestly I'm probably trying to do things with this program that it wasn't really designed for.

I attached the spritesheet and some images showing the issue in a zip.


engi_bug.zip
 Description:

Download
 Filename:  engi_bug.zip
 Filesize:  2.12 MB
 Downloaded:  1 Time(s)


_________________
TAK02 wrote:
Screw AttachEffect. Can't think of any use for it.

Back to top
View user's profile Send private message Send e-mail Skype Account
Nyerguds
General


Joined: 24 May 2004
Location: Flanders (Be) Posts:300000001

PostPosted: Tue Aug 21, 2018 7:17 am    Post subject: Reply with quote

Thanks for the feedback!

On the first one... did you get that problem at 3x zoom, or when exceeding 3x zoom? I've been implementing a solution that maxes the control on an average of 10.000 * 10.000 pixels, but I'm not sure if this suffices. I may need to max out both dimensions at 10.000 instead.

Sadly though, the control (a standard .Net PictureBox) does seem to mess up consistently when zooming too far, though exactly how far seems rather system-dependent. It's a side effect of it actually reserving the full amount of memory for the zoomed image. I've looked for alternatives online, but while there are optimised solutions of the "cut out the actual shown part" kind, those would come without scrollbars, making it quite hard to select the piece to look at. And implementing my own scrollbars... ehhh, not really something I'm too keen on.


On the second thing... seems the dpi set in the image is 72, whereas the default in the .Net framework is 96. This indeed leads to oddities when copying pieces out, since the new frames I make are of course set to that default of 96. I thought the method I used avoided that by setting its measurement unit to "pixels" specifically, but it turned out that was only the measurement unit for the selected rectangle, not for the target... so the DPI difference problem remained Embarassed

The reason this didn't show up in the preview was because the whole image-to-frames window works with a cloned version of the image, something I did for the sake of having the it in the most optimal format for repeated calls to to the cutting-up logic.

Anyway, that one's an easy fix.


On the negative zoom one, I do have a bit of a problem... I can set it to convert negative values to fractions, but... default zoom is "1", meaning 1x zoom. I can't really make it jump from 1 to -2...

Unless I go for percentages, that is, but that'd mean a complete overhaul of the zoom logic, since there are currently some logics in there that count on exact-multiples zoom.

[edit]

Released v1.3.3 Smile

Solved the zoom issue by... indeed making it jump from 1 to -2 Razz
_________________

Last edited by Nyerguds on Tue Aug 21, 2018 2:22 pm; edited 1 time in total

Back to top
View user's profile Send private message Visit poster's website Skype Account
OmegaBolt
President


Joined: 21 Mar 2005
Location: York, England

PostPosted: Tue Aug 21, 2018 12:30 pm    Post subject: Reply with quote

Pretty sure it did crash at 3x zoom, but now it seems to be working in the latest version. Didn't get any exceptions by zooming. The zoom out is nice too. Splitting that sheet into frames appears to work properly now too. Thanks! Smile
_________________
TAK02 wrote:
Screw AttachEffect. Can't think of any use for it.

Back to top
View user's profile Send private message Send e-mail Skype Account
OmegaBolt
President


Joined: 21 Mar 2005
Location: York, England

PostPosted: Thu Aug 23, 2018 3:02 pm    Post subject: Reply with quote

Well, after splitting another spritesheet into frames, got another strange result. #Tongue


f.jpg
 Description:
 Filesize:  109.41 KB
 Viewed:  1064 Time(s)

f.jpg



_________________
TAK02 wrote:
Screw AttachEffect. Can't think of any use for it.

Back to top
View user's profile Send private message Send e-mail Skype Account
ApolloTD
Commander


Joined: 19 Nov 2003

PostPosted: Thu Aug 23, 2018 3:56 pm    Post subject: Reply with quote

Pity this is of no help for centering frames of different sizes when separated given many sheets out there are sometimes hand made and thus are jumpy :/

and yeah it causes the odd vertical lines when give it 24bit image using the split but 32bit source is fine...go figure.

Back to top
View user's profile Send private message
Nyerguds
General


Joined: 24 May 2004
Location: Flanders (Be) Posts:300000001

PostPosted: Fri Aug 24, 2018 5:01 am    Post subject: Reply with quote

Oh wow, that's a silly bug... especially since I specifically designed the whole system to retain the original colour depth. And then right at the end I forget to pass that original format to a function at one crucial step in the conversion Wall bash!
_________________

Back to top
View user's profile Send private message Visit poster's website Skype Account
Nyerguds
General


Joined: 24 May 2004
Location: Flanders (Be) Posts:300000001

PostPosted: Fri Aug 24, 2018 5:41 am    Post subject: Reply with quote

Mind you, unless you're modding some game that accepts 32bpp/24bpp frames, you should be matching that thing straight to some kind of anim.pal anyway #Tongue

But yeah. That was literally one single forgotten word in the code. Derp. And it's literally the one line that should do the conversion...

I released a new version; the problem should be fixed now.

ApolloTD wrote:
Pity this is of no help for centering frames of different sizes when separated given many sheets out there are sometimes hand made and thus are jumpy :/

Well, I can't really help with that. I can't just magically make this into a full editor with image shifting controls... that'd take loads of extra development. As it stands, this tool is only a converter. You'll need to use something like good ol' Gimp to solve that. I mean, you can always use this to convert it to frames, then dump those in gimp as layers to line them up, and re-save the frames you realign.

The only somewhat related thing to that here would be the trimming code, but that was really just meant for handling file types with unequal frame sizes, like Dune II, and it works in tandem with the equivalent "fill around frames" option in the frames-to-image conversion. The TS SHP saving would reject frames of unequal sizes anyway.


By the way... dunno if anyone cares, but I rewrote the Westwood CPS Format article on the shikadi modding wiki... which also means I dug into all that, and the tool now supports all types of CPS, including the Amiga types. Was a bit worried that they'd be hard to distinguish, but given the fact Amiga uses 5-bit image instead of 8-bit, and CPS images are always 320x200, one check on the uncompressed size value in the header was all it needed Smile
_________________

Back to top
View user's profile Send private message Visit poster's website Skype Account
Nyerguds
General


Joined: 24 May 2004
Location: Flanders (Be) Posts:300000001

PostPosted: Thu Dec 06, 2018 1:59 pm    Post subject: Reply with quote

ApolloTD wrote:
Pity this is of no help for centering frames of different sizes when separated given many sheets out there are sometimes hand made and thus are jumpy :/

Mind you, I could put a simple "enlarge frames to same size" option... with the side options to select a fill colour, and to center instead of just filling the lower right. That's actually not hard at all.

But that'd only work with cropped frames anyway, and if they jump around they'd still need extra work...
_________________

Back to top
View user's profile Send private message Visit poster's website Skype Account
Display posts from previous:   
Post new topic   Reply to topic Page 1 of 1 [20 Posts] View previous topic :: View next topic
 Forum index » Featured Tools » Community Tools
Jump to:  
Share on TwitterShare on FacebookShare on Google+Share on DiggShare on RedditShare on PInterestShare on Del.icio.usShare on Stumble Upon
Quick Reply
Username:


If you are visually impaired or cannot otherwise answer the challenges below please contact the Administrator for help.


Write only one of the following words: Brotherhood, unity, peace! 

 
You can post new topics in this forum
You can reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © phpBB Group

Wildcard SSL Certificates
[ Time: 0.1535s ][ Queries: 14 (0.0111s) ][ Debug on ]