:: Home :: Get Hosted :: PPM FAQ :: Forum FAQ :: Search :: Memberlist :: Usergroups :: Register :: Profile :: Log in to check your private messages :: Log in ::
Do you want to advertise at Project Perfect Mod. Find out how to do it HERE.

The time now is Sun Jun 25, 2017 12:09 am
All times are UTC + 0
 Forum index » Featured Tools » CnC Editing Tools
ccmix, a command line mix creator
Moderators: CnC Editing Tools Developpers
Post new topic   Reply to topic Page 1 of 2 [56 Posts] View previous topic :: View next topic
Goto page: 1, 2 Next
Author Message
Blade
Cyborg Commando


Joined: 23 Dec 2003

PostPosted: Fri Apr 25, 2014 9:57 pm    Post subject:   ccmix, a command line mix creator Reply with quote

I've forked a C++ project called tsunmix off of googlecode, rewritten its interface and added the ability to create mix files, not just extract. Currently is only builds on linux, but I hope to be able to build on windows soon. You can find it up on https://github.com/OmniBlade/ccmix. Criticisms and suggestions welcome.

The program can create mix files with and without encrypted headers for ra and ts style mixes and should be able to extract files from protected mixes provided you know the name of the file you want to extract (untested). It also can optionally add the local mix database.dat file that XCC uses to get filenames for custom mix files.

Back to top
View user's profile Send private message
Blade
Cyborg Commando


Joined: 23 Dec 2003

PostPosted: Fri Apr 25, 2014 10:46 pm    Post subject: Reply with quote

I've done a slap dash cross compile from linux, try the ccmix.exe in the zip file and let me know if it works on windows.


ccmix.zip
 Description:

Download
 Filename:  ccmix.zip
 Filesize:  474.64 KB
 Downloaded:  142 Time(s)


Back to top
View user's profile Send private message
gistop
Stealth Laser Trooper


Joined: 19 Feb 2014
Location: Canada

PostPosted: Fri Apr 25, 2014 11:19 pm    Post subject: Reply with quote

hmmm... seems interesting. but I'm not to sure about how it works... let me try this.

Back to top
View user's profile Send private message
Banshee
Supreme Banshee


Joined: 15 Aug 2002
Location: Brazil

PostPosted: Fri Apr 25, 2014 11:37 pm    Post subject: Reply with quote

Quote:
Overview

ccmix is a command line tool to create and extract from archive files in the Westwood Studios .mix format. The tool can create any of the 3 main varients used in classic Westwood games. These are Tiberian Dawn versions, Red Alert versions with both encrypted and unencrypted file headers and Tiberian Sun versions with encrypted and unencrypted file headers. It also optionally adds the XCC extension of including a local filenames database to allow recovery of filenames from the one way hash they are stored as.


Useage

Generally the useage is of the form ccmix --mode --mix /path/to/file.mix with various options available that alter the progams behaviour when creating or extracting files. The currently supported modes are as follows:

--extract

This will extract a file or all files from a mix. Requires a --mix option.

--create

This will create a mix file from the contents of a directory. Currently requires a minimum of the --dir and --mix options.

--list

This will list the contents of a mix file. Requires the --mix options.

The following options can also be used.

--file /path/to/file.name

Specifies a specific file to extract.

--dir /path/to/dir

Specifies a folder to extract to, or the contents of which to make a .mix from.

--encrypt

Used when creating a file, encrypts the header with blowfish. Key source is generated from rand().

--lmd

Specifies if a local mix database.dat file following the XCC format should be generated and included in the mix.


Acknowledgements

ivosh-l author of tsunmix on which ccmix is based. Olaf van der Spek for his work reverse engineering the C&C files formats and the tools he has developed over the years to allow modding these classic games.

Back to top
View user's profile Send private message Visit poster's website MSN Messenger
Graion Dilach
Defense Minister


Joined: 22 Nov 2010
Location: Iszkaszentgyörgy, Hungary

PostPosted: Sat Apr 26, 2014 2:47 am    Post subject: Reply with quote

Did you tested if your Blowfish-encrypted mix gets read by RA2? Cool project but tbh that's the sole thing I'm interested in. #Tongue
_________________



AS Discord server: https://discord.gg/7aM7Hm2

Back to top
View user's profile Send private message MSN Messenger
ApolloTD
Commander


Joined: 19 Nov 2003

PostPosted: Sat Apr 26, 2014 6:11 am    Post subject: Reply with quote

It doesn't get read apparently so something wrong.

Back to top
View user's profile Send private message
E1 Elite
Railgun Soldier


Joined: 28 May 2013

PostPosted: Sat Apr 26, 2014 9:17 am    Post subject: Reply with quote

Doesn't work on windows. Used dat files from XCC Utilities, as it was not provided. Doesn't even list properly or extract tibsun.mix.

Back to top
View user's profile Send private message
Parasite03
Cyborg Artillery


Joined: 23 Jul 2013
Location: Russia

PostPosted: Sat Apr 26, 2014 10:14 am    Post subject: Reply with quote

Would be a nice tool if it had worked properly on windows and had a UI IMO. XCC Mixer is not always convenient to use.
_________________
DarkVen9109 wrote:
What in the name of insanity is this? I FRICKING LOVE THIS LOGICCCC!!!!!!!!!!!!OOOOOOOOHEEAWWWWWWWWWWWYAAAAAAAAAAAAAAAAAWWWWWW PEW PEW PEW PEW BOOM BOOM BOOM!! Nice I love this!!!! Ferriswheel bomb, Dive bomb. New Logic discovered thanks to Kenosis Very Happy

_________________

Back to top
View user's profile Send private message
Blade
Cyborg Commando


Joined: 23 Dec 2003

PostPosted: Sat Apr 26, 2014 10:49 am    Post subject: Reply with quote

True I hadn't tested it with a game yet which is the ultimate test as I didn't have copies installed and I was relying on the known working reading code from tsunmix to verify the mix files, I've built a version that works on windows correctly now, but the problem still remains that the game doesn't like the mixes it makes though annoyingly XCC does. It doesn't looke like its an encryption issue (un)fortunately since the game(s) don't like the unencrypted headers either. Anyone know what the game does differently to XCC when reading files that mine might be falling foul of? I'm sure I've just made a silly oversight somewhere.

Parasite03... A UI? Seriously? The whole point of the tool is that it doesn't so it can be invoked from a batch file or an installer.

I've attached a new build for windows though it still doesn't make conforming mix files. Looks like ccmix is failing to write something since extracting a working mix and repacking it results in a smaller file size  Confused


ccmix.zip
 Description:

Download
 Filename:  ccmix.zip
 Filesize:  201.71 KB
 Downloaded:  91 Time(s)


Back to top
View user's profile Send private message
E1 Elite
Railgun Soldier


Joined: 28 May 2013

PostPosted: Sat Apr 26, 2014 11:58 am    Post subject: Reply with quote

Doesn't even run now - says libgcc_s_dw2-1.dll missing.

Back to top
View user's profile Send private message
Blade
Cyborg Commando


Joined: 23 Dec 2003

PostPosted: Sat Apr 26, 2014 12:37 pm    Post subject: Reply with quote

I obviously suck at setting up a mingw build environment and its linking against files in the build env rather than static linking. I'll hopefully get that fixed.
I also think I know why the game doesn't read my mix files, it seems that the header entries have to be sorted by ID (presumably the game does a binary search to decide if a file is present or not rather than using a map or something similar). I'll update the code and hopefully create a working build too.

Back to top
View user's profile Send private message
Blade
Cyborg Commando


Joined: 23 Dec 2003

PostPosted: Sat Apr 26, 2014 10:39 pm    Post subject: Reply with quote

Okay, I've attached a new build that sorts the index so games like it. I've tested the windows build myself, it can't list TS mixes with names for some reason which I'll fix later, but seems to build them fine. Don't forget to put --game ts if you are building for TS so it uses the right id gen algorithm.

As for encryption, it looks like my encryption code is good, but that the games are more particular with the key source or the key it generates. I've included a key.source file that I pulled for RedAlerts main or redalert mix (don't recall which off hand). If it is in the working directory where you run ccmix from and use the --encrypt option it will use the key.source instead of randomly generating one which seems to work. Red alert (irans portablera version) at least will use an encrypted mix made with key.source but will crash otherwise. XCC can decrypt mix files with random sources, and the sources do differ between the mix files bundled with game so if any exe hackers have any insight into how one might generate valid key sources for the game I'd like to hear it.

For completness sake I'd also be interested in how mix checksums are calculated, even though they appear to be irrelevant to the game.


ccmix.zip
 Description:

Download
 Filename:  ccmix.zip
 Filesize:  604.37 KB
 Downloaded:  85 Time(s)


Back to top
View user's profile Send private message
Blade
Cyborg Commando


Joined: 23 Dec 2003

PostPosted: Mon Apr 28, 2014 10:44 am    Post subject: Reply with quote

If anyone is interested, the checksum appears to be a sha1 digest of the body of the mix file (not including the header and index). I intend to add the ability to generate checksums in a future version now.

Back to top
View user's profile Send private message
Banshee
Supreme Banshee


Joined: 15 Aug 2002
Location: Brazil

PostPosted: Mon Apr 28, 2014 12:07 pm    Post subject: Reply with quote

I plan to make OS BIG Editor use it as a plugin in a near future. I just have a bunch of RL problems at the moment, which is why I'm not giving the attention this tool deserves at the moment. I'm sorry.

Back to top
View user's profile Send private message Visit poster's website MSN Messenger
Blade
Cyborg Commando


Joined: 23 Dec 2003

PostPosted: Wed Apr 30, 2014 2:01 pm    Post subject: Reply with quote

Edit: Nevermind, looks like it just broke things more and was on a wild goose chase for the reading problem :/

Back to top
View user's profile Send private message
Blade
Cyborg Commando


Joined: 23 Dec 2003

PostPosted: Fri May 02, 2014 10:58 pm    Post subject: New Build Reply with quote

Here is a new build complete with global mix database from XCC and a key source taken from a random encrypted mix that the program will use to generate encrypted mix files that will work with the game. If anyone knows what the crypto algorithm is that was used to encrypt the key source I'd like to know so I can add the ability to just randomly generate one rather than reusing an existing one, but it makes no difference as far as the game is concerned. This version also supports adding the checksum with the options --add --checksum --mix or adding --checksum to the options used when creating a mix. I think I've really solved the issues with reading and extracting on windows, but again this is a cross compile from ubuntu that I haven't been able to test on a real windows system yet so I would like feedback. There is also an --info option that will print info about a mix other than its file listing.
So in summary this version should let you create and extract mix files from any of the games up to YR with or without encryption and checksum when the mix format allows and with or without a local mix database. Selective add and remove options for files are not yet available as they require the file to be rewritten completely or there abouts.


ccmix.zip
 Description:

Download
 Filename:  ccmix.zip
 Filesize:  524.75 KB
 Downloaded:  85 Time(s)


Back to top
View user's profile Send private message
E1 Elite
Railgun Soldier


Joined: 28 May 2013

PostPosted: Sat May 03, 2014 6:52 am    Post subject: Reply with quote

It is able to list, extract and create now, checked for TS with text files only on Win7. Didn't give problem when
running TS with a mix file with ini files created with ccmix. Didn't use any param for encryption/checksum.

Some comments:
- TS and YR should be separated
- lmd file has a few bytes changed when compared to XCC
- should be able to use -g instead of --game
- could use a bare listing of just filenames
- use backslash n instead of forward in --help (typo on first few chars)

Must have recursive funtionality. If I have to extract all hundreds of files of tibsun.mix then don't expect the
user to give all the filenames as parameter. While creation, subdirectories could be packed as mix files taking
the folder name as its mix filename. Right now it ignores folders as it should.

Can you list the features that are added/changed over tsunmix?

Back to top
View user's profile Send private message
Blade
Cyborg Commando


Joined: 23 Dec 2003

PostPosted: Sat May 03, 2014 11:26 am    Post subject: Reply with quote

1. Why do YR and TS need to be seperate options? Internally they would do the same thing as the formats are the same as far as I understand.
2. I'll look into that as XCC might use it to give it hints as to what game it is.
3. I'll look at aliases for some of the options once the program is feature complete.
4. I wrote it on linux where the folder seperator is / not \. I suppose I could put a define in that will change them in the help, but I'll need to rewrite the help at some point anyway as its no longer up to date.

I don't agree on the recursive option, that just complicates things, better to do that with multiple calls to this program from a batch script instead IMO but I might reconsider once I have implemented add, remove and replace options. It can already extract all files in a mix file if you don't give it a filename to extract.

Main feature is that it can create mix files, not just read and extract from them. Also, tsunmix only supports files with the TS style id's although internally the code was already there to support the other formats. This works with TD upwards.
There are internal differences too, the way the info from global mix database.dat is handled is different and much quicker than the original tsunmix code for example so it lists large mixes quicker and doesn't need the "fast" option to only extract id's to speed it up.

Back to top
View user's profile Send private message
E1 Elite
Railgun Soldier


Joined: 28 May 2013

PostPosted: Sat May 03, 2014 12:39 pm    Post subject: Reply with quote

1. I don't know the TS/YR mix formats, but atleast the games are different, their game ID might be different 2/5.
When you create empty mix files with XCC mixer, there is change. Files have different extensions like mmx/yro.
My guess is, there must be differences when it comes to encryption. If it was not, why would XCC make a difference.

3. Wherever there is a comparison of command line arguments, an OR would do, to use -g instead of --game etc.

5. On typo it is only once at start of your string - /n***ccmix program usage***. Not a linux issue.

On extracting all files in a mix - without filename on extract it says - No file or ID to extract given. Can you give
an example for the exact command line for it?

For a command line tool, it should give advantage over XCC. On recursive create, these games only use 1 level deep,
so, implementation shouldn't be that hard. But no compulsions.

Back to top
View user's profile Send private message
RP
Commander


Joined: 12 Jul 2012
Location: Mapping God Heaven

PostPosted: Sat May 03, 2014 1:28 pm    Post subject: Reply with quote

E1 Elite wrote:
why would XCC make a difference.


Where does it make that difference? ID's are the same type as normal MIX files, they can be encrypted and have a checksum like normal MIX files.
The only difference between all those files is the extension, the game definitely makes a difference when it comes to those files, yes. I'm not so sure about XCC though, could have something to do with the global database...


E1 Elite wrote:
these games only use 1 level deep


IIRC TS has a deeper level, there's ecache01.mix inside FS' expand01.mix. TS/FS does a loop for ecache99-00.mix besides the ecache*.mix / elocal*.mix loop through the game's directory.
Yuri's Revenge doesn't have this though.
Nevertheless for expand(md)XX.mix, my search goes deeper than 1 level. For example expandmd01 can exist in expandmd02 which could exist in expandmd03.
When you load expandmd03 and after that look for expandmd02 in the game's root or any loaded mix file, it will also look for it in expandmd03, thus going deeper than 1 level.
_________________


Mental Omega 3.0 Mission creator - Creator of FinalOmega: APYR 3.0 Map Editor

/ppm/'s stupidity

Back to top
View user's profile Send private message
E1 Elite
Railgun Soldier


Joined: 28 May 2013

PostPosted: Sat May 03, 2014 4:04 pm    Post subject: Reply with quote

So, when you create empty mix files from XCC Mixer for TS and YR, why the files are different? Why don't we have yro
files in common use for say map packs?

When you create an expand*.mix in TS, it can contain other mix files which themselves do not contain more mix files
inside them. This is not about having say multiple ecache*.mix files inside say one expand02.mix. Are you saying
expand02.mix can contain ecache02.mix and ecache02.mix can contain ecache03.mix. And game will recognize
the content of ecache03.mix? It doesn't.

It also doesn't do that in RA2. I think YR 1.001 update had these kind of changes to mix file parsing. Wonder why it
doesn't work for me, that is expandmd inside another expandmd with YR1.001 or Ares 0.4/0.6. Are you sure?
Anyhow I don't put mix files in that fashion.

Back to top
View user's profile Send private message
RP
Commander


Joined: 12 Jul 2012
Location: Mapping God Heaven

PostPosted: Sat May 03, 2014 9:20 pm    Post subject: Reply with quote

I never said YR does it that way, I said I do it that way in my application.

Don't ask me about YRO files, Westwood made them, I just tried to explain to you what purpose they have.
You wouldn't load a fucking .MIX file if it shoud only contain a single map (which is probably what Westwood intented),
instead Westwood create MMX/YRO files for some reason.
expand*.mix can only exist in the game's root, how would you check for ecacheSuperDerpMIXFILE:DVery Happy.MIX in other MIX files?
As said before (hence the IIRC), TS has more than 1 level deep: ecache01.mix exists in expand01.mix, I never said the game would read 100 levels deep.

Nevertheless, thanks for telling me the game doesn't load expandXX from expandXX+1, saves me time.
YR is shallow when it comes to MIX loading, TS has atleast 1 level deeper (as stated before).
_________________


Mental Omega 3.0 Mission creator - Creator of FinalOmega: APYR 3.0 Map Editor

/ppm/'s stupidity

Back to top
View user's profile Send private message
Blade
Cyborg Commando


Joined: 23 Dec 2003

PostPosted: Sat May 03, 2014 9:44 pm    Post subject: Reply with quote

You are right, extract all doesn't work in this version, I messed up the logic when I implemented extract by id. I've attached an updated version that should though. Just ommit --file and --id and it should extract everything.

I don't think there are many modding use cases that would require recursive builds and extracts really, the only one is distributing a patch to a total conversion that has its files built exactly like the original game, but in such a case, the modder can go to the trouble of scripting the mix filles. A mod build system should be scripted similarly.

I'd be interested in you sending me a couple of lmd files that differ from the ones ccmix creates so I can figure out what flags XCC is setting in them. I had read that they only worry about total file size and count of entries.


ccmix.zip
 Description:

Download
 Filename:  ccmix.zip
 Filesize:  524.23 KB
 Downloaded:  82 Time(s)


Back to top
View user's profile Send private message
E1 Elite
Railgun Soldier


Joined: 28 May 2013

PostPosted: Sun May 04, 2014 5:59 am    Post subject: Reply with quote

@RP

Now you say it is in your application, that confirms expandmd inside expandmd doesn't work in-game.

And this tool takes a folder name for mix creation, so it has to pack only 1 level of subfolders if recursion is used.
That would result in the likes of ecache01.mix inside expand01.mix.

@Blade

Now the extract all works.

With this build, if I use --lmd, ccmix creates mix files with encoded filenames for the content, both in the presence or
absence of key.source. Even in the earlier case with more than one file there were always differences on the order
of file listing inside lmd files created by XCC and ccmix.

Attached lmds with this build and of XCC for a mix with only one file.


LMDs.zip
 Description:

Download
 Filename:  LMDs.zip
 Filesize:  761 Bytes
 Downloaded:  80 Time(s)


Back to top
View user's profile Send private message
Blade
Cyborg Commando


Joined: 23 Dec 2003

PostPosted: Mon May 05, 2014 11:15 am    Post subject: Reply with quote

Looks like XCC does flag the lmd with which game it belongs to, I based my code on the discussion at http://forums.cncnz.com/topic/17313-a-command-line-mix-editor/ which suggested that only the size and number of entries was recorded. For the record, according the structs that define the format in XCC, the format is as follows:

constant char[32] "XCC by Olaf van der Spek\x1a\x04\x17\x27\x10\x19\x80" which acts as an ID or signature of sorts.

int32 size - size of the file
int32 type - always 0 in lmd
int32 version - aways 0 in lmd
int32 game - flags which game it is for, 0 is td, 1 is ra, 2 is ts, 5 is ra2, 6 is yr (there are more, but we don't need to worry about them).
int32 c_fnames - count of how many file names we record in the file.
char[] series of 0 terminated strings that are the file names.

For this program, the only games we really need to care about should be the first 3, maybe even only 0 and 2 since they should make XCC read the files we make correctly. As far as the games care, there are only 3 mix formats, original TD that has a simple header, RA that has complex header supporting encryption and checksums and TS onward that has a different ID generation algorithm but is otherwise the same as RA.

Back to top
View user's profile Send private message
E1 Elite
Railgun Soldier


Joined: 28 May 2013

PostPosted: Mon May 05, 2014 1:33 pm    Post subject: Reply with quote

That explains the 1 byte (2/5) difference in the new empty mix files created by XCC for TS/RA2.

With the last build, file name encoding has got mixed up as I mentioned in the last post.

cmd> ccmix --list --game ts --mix 2.mix
Input mix is 2.mix
020000001e020000
2 files, total size 542 according to header
    FILENAME |       ID |    ADDRESS |      SIZE
battlefs.ini | bfe53b8c |        316 |        260
   <unknown> | 4d4bae21 |         34 |        282

This was created with ccmix without --lmd and the second filename 1.mix not present in global mix db.
Unknown is for the encoded filename for 1.mix.

Back to top
View user's profile Send private message
Blade
Cyborg Commando


Joined: 23 Dec 2003

PostPosted: Mon May 05, 2014 10:00 pm    Post subject: Reply with quote

Without lmd, that is expected behaviour for a filename that isn't in global mix database, the filenames will always be lost if you don't create an lmd. The game should still happily use it provided it knows to look for the files you have added.

Back to top
View user's profile Send private message
E1 Elite
Railgun Soldier


Joined: 28 May 2013

PostPosted: Tue May 06, 2014 5:05 am    Post subject: Reply with quote

Don't you think it should be with lmd by default and --nolmd should be an option or disable mix files
without lmd altogether? Leave alone protectionism, if I have to recover my own mix file with custom
content created without lmd, is there a way?

There should be a warning in the readme, if the tool allows to pack a mix file and is unable to
unpack it to its original state.

Back to top
View user's profile Send private message
Blade
Cyborg Commando


Joined: 23 Dec 2003

PostPosted: Tue May 06, 2014 8:25 am    Post subject: Reply with quote

The reason it packs without lmd by default is that I wanted it to build files that are the same as ship with the game by default without the lmd extension.

I'll consider changing the default lmd, but you shouldn't be relying on a packed mix file to be your backup in case of loss, you should backup your data in an unpacked format or in an archive that does keep names. This tool should be for final release packaging or generating internal builds of a mod, in which case it having names only helps other people who want to recover the files.

At any rate this tools is alpha, doesn't have a complete feature set and comes with no warranty of any kind, use at your own risk and such.

Back to top
View user's profile Send private message
Blade
Cyborg Commando


Joined: 23 Dec 2003

PostPosted: Tue May 06, 2014 3:43 pm    Post subject: Reply with quote

Well, I've got ccmix writing an lmd that is identical to that written by XCC as far as my hex editor can tell, but XCC still won't use it as an lmd, go figure. Also, XCC mix editor does something very strange, even after compact it has the remnents of old lmd entries inbetween the files, bloating the filesize slightly, but has nothing pointing to it.

Back to top
View user's profile Send private message
zsu
Vehicle Drone


Joined: 29 Sep 2012

PostPosted: Fri May 16, 2014 1:39 am    Post subject: Reply with quote

Blade wrote:
I've attached an updated version

Thank you very much. This is useful tool.

Back to top
View user's profile Send private message
Blade
Cyborg Commando


Joined: 23 Dec 2003

PostPosted: Tue May 20, 2014 3:45 pm    Post subject: Reply with quote

Thanks, at some point i will add the ability to add and remove files from an existing mix, but I have limited time and many hobbies so this will get done as I get chance as I want to rewrite parts of it before I start tacking the add and remove code and hopefully fix the lmd not being used properly by XCC.

Back to top
View user's profile Send private message
zsu
Vehicle Drone


Joined: 29 Sep 2012

PostPosted: Mon May 26, 2014 7:34 am    Post subject: Reply with quote

Got a lot of game crashes with mix file generated by ccmix.
Yuris revenge.

Back to top
View user's profile Send private message
Blade
Cyborg Commando


Joined: 23 Dec 2003

PostPosted: Thu May 29, 2014 8:48 am    Post subject: Reply with quote

More information would be helpful. Have you tried generating a mix with XCC mix editor instead using exactly the same files and testing again (to rule out it being a file in the mix rather than the mix itself)? What options did you use when making the mix file?

Back to top
View user's profile Send private message
zsu
Vehicle Drone


Joined: 29 Sep 2012

PostPosted: Thu May 29, 2014 10:07 am    Post subject: Reply with quote

I have no crashes since switch back to XCC mixer.

ccmix.exe --create (with and without) --encrypt --game game_ts --directory mod --mix ra2yr\expandmd98.mix

Back to top
View user's profile Send private message
Blade
Cyborg Commando


Joined: 23 Dec 2003

PostPosted: Fri May 30, 2014 1:52 pm    Post subject: Reply with quote

You may have to wait until I'm done rewriting some parts of the code since its not in a buildable state at the moment for me to troubleshoot. I'll do more extensive testing once the rewrite is done. Have you tried building it unencrypted and seeing if that makes a difference?

Back to top
View user's profile Send private message
zsu
Vehicle Drone


Joined: 29 Sep 2012

PostPosted: Sat May 31, 2014 1:50 pm    Post subject: Reply with quote

One attempt without difference.

Back to top
View user's profile Send private message
Blade
Cyborg Commando


Joined: 23 Dec 2003

PostPosted: Fri Jun 20, 2014 3:52 pm    Post subject: Reply with quote

I've pretty much finished the rewite to the point the program works as well as it did before. Looks like XCC even likes my local mix database.dat files now. Bad news is the machine I used to build the windows version is waiting for me to attempt a reflow of its graphics chip so it might be a while until a windows binary shows up.
I've pushed the new code to github if anyone wants to try building it. I've abstracted out handling of the mix databases along with some untested code for writing "global mix database.dat" that someone could base an editor around. A lot of the header handling it seperated out now too which should make adding add and delete functions for single files a good bit easier.

Back to top
View user's profile Send private message
Blade
Cyborg Commando


Joined: 23 Dec 2003

PostPosted: Wed Aug 27, 2014 2:53 pm    Post subject: Reply with quote

Okay, here is a windows binary that I cross-compiled from linux, let me know if it works. I've implemented add and remove functionality that completely rewrites the file. Unlike XCC, it won't leave inaccessible copies of old local mix databases littering the file either. You can now use "ra2" as a game option, but the only reason to do so is when including a local mix database so XCC reads the game correctly.

I've also implemented handling for files that aren't in global mix database, when extracted they are named [id]A123456F. The program recognises the [id] tag at the start and will add files named this way using the following 8 chars as the id number. The first character not a valid hexdecimal char will cause the id number to be based on everything up to that point and then front padded with 0's. Anything after the first 8 chars is ignored. This does mean that you cannot add a file named [id]filename.ext with this program and have the ID represent that name, you will just have to make do without that flexibility. This does mean however that you can extract an existing file where the names are not known and then repack it and it will still work.

I haven't tested generated file in game so feedback on that front would be appreciated and I haven't tested on windows.

edit:
Fixed a few issues before anyone downloaded, sweet. Also, looks like cross compiling it from ubuntu produced a smaller binary than centos... go figure?


ccmix_win32.zip
 Description:

Download
 Filename:  ccmix_win32.zip
 Filesize:  709.45 KB
 Downloaded:  66 Time(s)


Back to top
View user's profile Send private message
Glukv48
Cyborg Firebomber


Joined: 11 Nov 2012
Location: Russia, Krasnodar.

PostPosted: Sat Oct 04, 2014 2:48 am    Post subject:   Reply with quote

It seems it works well.


I just copy the documentation here
Quote:
Usage: ccmix.exe [--mode] (--file FILE) (--directory DIR) (--game [td|ra|ts|ra2]) [--mix MIXFILE]

Modes:

--extract
Extracts the contents of the specified mix file to the current directory.
--file specifies a single file to extract.
--directory specifies an alternative directory to extract to.
--game specified the game the mix is from, td covers the
orignal C&C and Sole Survivor. ra covers Redalert and its
expansions. ts covers Tiberian Sun and ra2 covers Red Alert 2/Yuri's Revenge.

--create
Creates a new mix file from the contents of the current folder.
--file specifies a single file as the initial file to add to the
new mix.
--directory specifies an alternative directory to create mix from.
--checksum specifies the mix should have a checksum.
--encrypted specified the mix header should be encrypted.
--game specified the game the mix is from, td covers the
orignal C&C and Sole Survivor. ra covers Redalert and its
expansions. ts covers Tiberian Sun and ra2 covers Red Alert 2/Yuri's Revenge.

--list
Lists the contents of the specified mix file.
--game specified the game the mix is from, td covers the
orignal C&C and Sole Survivor. ra covers Redalert and its
expansions. ts covers Tiberian Sun and Red Alert 2/Yuri's Revenge.

--add
Adds the specified file or mix feature.
--file specifies a single file to add.
--checksum specifies the mix should have a checksum.
--game specified the game the mix is from, td covers the
orignal C&C and Sole Survivor. ra covers Redalert and its
expansions. ts covers Tiberian Sun and ra2 covers Red Alert 2/Yuri's Revenge.

--remove
Removes the specified file or mix feature.
--file specifies a single file to remove.
--checksum specifies the mix should not have a checksum.
--game specified the game the mix is from, td covers the
orignal C&C and Sole Survivor. ra covers Redalert and its
expansions. ts covers Tiberian Sun and ra2 covers Red Alert 2/Yuri's Revenge.

Back to top
View user's profile Send private message
Blade
Cyborg Commando


Joined: 23 Dec 2003

PostPosted: Mon Oct 06, 2014 4:00 pm    Post subject: Reply with quote

Thanks for the feedback, do the games seem to like the mix files it generates? Particularly encrypted ones?

Back to top
View user's profile Send private message
thexshadow
Vehicle Driver


Joined: 28 Sep 2014
Location: Canada

PostPosted: Fri Oct 10, 2014 12:41 am    Post subject: Reply with quote

This works really good, but it has a problem unpacking WAV files. They come out corrupt.
Also, any way you could make a command to remove the encryption of mix files, it would be so handy.

Back to top
View user's profile Send private message
Matthias M.
Missile Trooper


Joined: 15 Jun 2012
Location: Germany

PostPosted: Fri Oct 10, 2014 4:15 am    Post subject: Reply with quote

See also https://github.com/OpenRA/OpenRA/pull/6713 for a cross-platform variant in C#.

Back to top
View user's profile Send private message Visit poster's website
Blade
Cyborg Commando


Joined: 23 Dec 2003

PostPosted: Fri Oct 10, 2014 8:23 am    Post subject: Reply with quote

thexshadow, can you provide me with a couple of small files both loose and in a mix that ccmix will break so I can compare packing and extracting results against? I've tested with some simple text files and the files are packed and extracted perfectly with the create and extract options. Did you use the add and/or remove options or just create and extract.

Matthias, the OpenRA tool only packs a directory from the looks of it, this can edit them to some degree (or at least it will when it's bug free) and can maintain the header encryption which is needed for the RA1 3rd party map editor. C# also requires an additional runtime which this tool doesn't.

Back to top
View user's profile Send private message
thexshadow
Vehicle Driver


Joined: 28 Sep 2014
Location: Canada

PostPosted: Fri Oct 10, 2014 10:47 pm    Post subject: Reply with quote

Blade, I have 7z'pd them for you, the smallest I could get it was 29MB, if I removed the unneeded files from the mix (using xcc), it would make it so I could extract the files with ccmix. I have included the original mix, an extracted wav from ccmix and one from xcc.

If I add the xcc file back into a new mix (using ccmix), the wav turns out to be [id]9eb77510 in the mix. When I extract it, if I add .wav to the end of it, it still plays fine.

Download
http://www.mediafire.com/download/d0ww59khep74wn7/audio.7z

Back to top
View user's profile Send private message
Blade
Cyborg Commando


Joined: 23 Dec 2003

PostPosted: Sat Oct 11, 2014 11:17 am    Post subject: Reply with quote

If I extract a wav using ccmix on my system, I get a different wav to what you have provided, I get a file with a reading frame off by 4 bytes which is because of some poor assumptions my header handling code made.

What program did you use to make audio.mix originally? Did you use ccmix without specifying which game it was for with --game? The header format is the old TD style which you will get by default with ccmix, if you used --game ts or --game ra2 the mix would probably work fine. Does the game read this mix?

At any rate I've fixed my header code so ccmix will be able to correctly extract this mix file now.


ccmix_win32.zip
 Description:

Download
 Filename:  ccmix_win32.zip
 Filesize:  688.36 KB
 Downloaded:  47 Time(s)


Back to top
View user's profile Send private message
thexshadow
Vehicle Driver


Joined: 28 Sep 2014
Location: Canada

PostPosted: Sat Oct 11, 2014 10:31 pm    Post subject: Reply with quote

I didn't make audio.mix, I just extracted it from the game (it was inside another mix). I always have the game parameter set to ra2, and the game reads it fine because I extracted it from the game ;P

Anyways, I'll test it now.

Edit: Works great now. Thanks!

Back to top
View user's profile Send private message
Blade
Cyborg Commando


Joined: 23 Dec 2003

PostPosted: Sat Oct 11, 2014 11:44 pm    Post subject: Reply with quote

I didn't realise that RA2 still read the old style mix headers provided the ids were generated using the correct algorithm, but I guess it does.

Back to top
View user's profile Send private message
Blade
Cyborg Commando


Joined: 23 Dec 2003

PostPosted: Wed Dec 10, 2014 12:00 pm    Post subject: Reply with quote

I've pushed a new branch to the github repo (crypto) that supports encryption without needing to read a key from an existing mix. I need to set up a new build environment for windows to generate a new windows build, but the new branch uses cmake so it might be easier for anyone with visual studio to build their own.

It now depends on crypto++ to provide the cryptographic functions for the header and checksum since I figured out how the header was structured and encrypted properly rather than just using the undocumented XCC code.

Back to top
View user's profile Send private message
Blade
Cyborg Commando


Joined: 23 Dec 2003

PostPosted: Wed Dec 17, 2014 10:48 am    Post subject: Reply with quote

Okay, this is new release built with MSVC. It no longer needs the keysource to create encrypted mix files, it will randomly generate a key as needed. I've also fixed a few bugs, it will gracefully fail on renegade mix files, and will successfully extract and list mix files with fake local mix database entries in their header (this is used to break XCC mix reading along with fake sizes for the body of the mix which this tool always ignored). Any mix the game can read this tool should handle and be able to repack correctly.

Edit... arrgh, looks like this suffered from the bug that messed up extraction again. I've fixed it much better this time and I'll get another build out soon.

Edit2... Fixed version now attached.


ccmix_win32.zip
 Description:

Download
 Filename:  ccmix_win32.zip
 Filesize:  386.7 KB
 Downloaded:  235 Time(s)


Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic Page 1 of 2 [56 Posts] Goto page: 1, 2 Next
View previous topic :: View next topic
 Forum index » Featured Tools » CnC Editing Tools
Jump to:  
Quick Reply
Username:


If you are visually impaired or cannot otherwise play the game below please contact the Administrator for help.


 
You can post new topics in this forum
You can reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © phpBB Group

Wildcard SSL Certificates
[ Time: 0.4825s ][ Queries: 15 (0.0207s) ][ Debug on ]