OS BIG Editor :: Information and Download

Banshee · Last edited by Banshee on Tue Apr 02, 2024 10:33 pm; edited 11 times in total

OS BIG Editor is an open source .BIG, .MEG and .ZIP file editor. The .BIG files are packages of files used on games made by Electronic Arts Los Angeles, like Command & Conquer Renegade, Command & Conquer: Generals, Lord of the Rings: Battle For Middle Earth I, Lord of the Rings: Battle For Middle Earth II and Command & Conquer 3: Tiberium Wars. The .MEG files are packages of files used on games made by Petroglyph Games, such as Star Wars: Empire at War, Universe at War, GreyGoo, Mytheon, End of Nations, Forged Battalions, 8-Bit Armies, 9-Bit Armies, Command & Conquer: Remastered Collection, among others.

OS BIG Editor is a freeware, ad-free, and it has the following features:

-> Browse and save .BIG files from Renegade, Generals, Battle For Middle Earth I and II, Command and Conquer 3, Red Alert 3 and Command & Conquer 4 including the compressed files inside them.
-> Browse and save .MEG files from Star Wars: Empire at War, Universe at War, Grey Goo, 8 Bits Armies, Conan Unconquered, Command & Conquer: Remastered Collection, and 9-Bit Armies.
-> Browse and save .ZIP files.
-> A quick and friendly interface that allows the user to quick extract the files, including through drag and dropping them to Windows Explorer.
-> Preview all TGA and text files without extracting them.
-> Associate your .big files with the program to open them straight from the Windows Explorer.
-> Online support and help at Project Perfect Mod Forums.

This program counts the (direct or indirect) contributions from Jonwil (Refpack decompression code), Danny van Loon (Drag and Drop support), Zlatko Minev (TGA Viewer), and Davie Reed (TGA support).

The latest version can be downloaded at the links below:

-> Download OS BIG Editor 0.588 Beta (win64)
-> Download OS BIG Editor 0.588 Beta (win32)

This tool is freeware, but if you enjoy it and want to motivate and contribute to its development, feel free to donate money with PayPal, credit cards, and bank transfers by clicking on the button below:

Make sure you mention your forum nickname. This is crucial. And also, one feature that you'd need. You'll receive a response, and if that feature is not viable, you can request another one.

Banshee · Posted: Sat Oct 04, 2008 9:16 pm Post subject:

Update: The alpha version 0.58 is now mentioned above. This version has a lot of new features, saving .BIG files with and without refpack, loads and saves .MEG files (for EAW and UAW), display files organized by directories, a lot of drag and drop support with windows explorer, it previews a lot of files including binaries and more image file types, you can add, rename files and folders, etc. It's definitely an editor, but somethings are kinda bugged.

I've also added a button for donations. Every donation submited throught this button will speed up the development of the program.

Donators may get priorities on getting specific features done as long as it is a viable and worth feature and I'm aware of it. To make me aware of it, use the field that should have instructions to the seller to say your nickname in the forums and the feature that you want.

Also, if you wanna know if a feature is worth and viable, you can PM me first.

Finally, before I get insulted, you can request features without giving a single cent and I'll probably work on them, but I'll certainly priorize the paid ones.

zocom7 · Soldier **Joined**: 31 Dec 2010 **Location**: USA

It's been about a decade you first released this, Banshee. Is there going to be an update where I can create a BIG file that works for RA3 and C&C4, for the files at least 800 MB and above? Last time I tried your 0.58 alpha, the files I tried to add to create a BIG file doesn't work.

_________________
RIP PurpleScrin (2007-2010)

Banshee · Posted: Tue Jun 28, 2016 10:04 pm Post subject:

This is one of the problems that killed my motivation to work on the OS BIG Editor. The problem is actually that 800mb or bigger files generates a huge RAM memory block (over 2gb) and, since the program is written for 32 bits, it generates an access violation. I've started coding a solution for it, but I've never finished it. It's too much complicated and boring to code. Right now I have other priorities in my life, such as my doctorate thesis, so I won't work on it in this year. But the source code is online and if anyone manages to fix it, send me the source and we can bring the solution to 0.58 Alpha.

zocom7 · Soldier **Joined**: 31 Dec 2010 **Location**: USA

Well it was already possible to use a 4 GB RAM tool (notably the 4GB patch found here: http://www.ntcore.com/4gb_patch.php) to make all C&C Worldbuilders, C&C3/KW, RA3/Uprising and even C&C4 to use up to 4 GB of RAM.

_________________
RIP PurpleScrin (2007-2010)

Banshee · Posted: Wed Jun 29, 2016 7:18 am Post subject:

I think that OS BIG Editor's RAM limit is actually 3GB. But due to the way the refpack compression is done, the limit of the size of the file that it can generate is less than half of it. It actually uses the same thing that this tool does to do 'this kind of magic'.

Ju-Jin · Posted: Thu Jun 30, 2016 7:05 pm Post subject:

I wrote a class in C# (you can adapt that easily for C++) for WrathEd 2.0 which reads a refpack compressed file only using a 8 MB pre buffer and a 256KB decomressed buffer (you need only 128 KB but that and the 8 MB pre buffer gave me the best speeds without taking up much space).
Everything is done in memory so there are also no slow downs from writing temp data to the disk.
The code is quite lengthy as the functions for read and seek are quite big.

Spoiler (click here to read it):

Code:

using System.IO;

namespace WrathEd.FileSystem
{
   public class RefPackStream : Stream
   {
   private enum State
   {
   COMPLETE,
   RUN_LITERAL,
   RUN_REF,
   RUN_LITERALONLY,
   RUN_EOF
   }

   // tested as a good value for speed 8388608 Bytes (8192 KB (8 MB))
   private const int _preBufferLength = 0x00800000;
   // 0x00020000 actually 131072 Bytes (128 KB), 256 KB so it doesn't wrap around as much
   private const int _refPackBufferLength = 0x00040000;
   private const int _refPackBufferLengthMinusOne = 0x0003FFFF;

   private readonly Stream _baseStream;
   private readonly byte _flags;
   private readonly byte _highestNonStopValue;
   private readonly long _length;
   private readonly int _dataOffset;
   private readonly byte[] _preBuffer;
   private readonly byte[] _refPackBuffer;

   private long _preBufferPosition;
   private long _internalPosition;
   private long _position;
   private byte _ref0;
   private byte _ref1;
   private byte _ref2;
   private byte _ref3;
   private long _refPosition;
   private int _run;
   private int _runRef;
   private State _state;

   public unsafe RefPackStream(Stream baseStream)
   {
   _baseStream = baseStream;
   byte[] buffer = new byte[6];
   baseStream.Read(buffer, 0, 6);
   fixed (byte* pBufferF = &buffer[0])
   {
   byte* pBuffer = pBufferF;
   _flags = *pBuffer++;
   _highestNonStopValue = *pBuffer++;
   _length = (((*pBuffer++ << 8) | *pBuffer++) << 8) | *pBuffer++;
   if ((_flags & 0x80) != 0)
   {
   _length = (_length << 8) | *pBuffer++;
   _dataOffset = 4;
   }
   else
   {
   _dataOffset = 3;
   }
   if ((_flags & 0x01) != 0)
   {
   _dataOffset *= 2;
   }
   _dataOffset += 2;
   baseStream.Position = _dataOffset;
   }
   _position = 0;
   _preBuffer = new byte[_preBufferLength];
   _refPackBuffer = new byte[_refPackBufferLength];
   _baseStream.Read(_preBuffer, 0, _preBufferLength);
   _state = State.COMPLETE;
   }

   public static bool IsCompressed(Stream stream)
   {
   if (stream is RefPackStream)
   {
   return true;
   }
   byte[] buffer = new byte[2];
   stream.Read(buffer, 0, 2);
   stream.Position -= 2;
   return (buffer[0] & 0x3E) == 0x10 && buffer[1] == 0xFB;
   }

   private unsafe void InternalSeek(long position)
   {
   fixed (byte* pPreBufferF = &_preBuffer[0])
   {
   byte* pPreBuffer = pPreBufferF;
   switch (_state)
   {
   case State.RUN_LITERAL:
   pPreBuffer += _preBufferPosition;
   goto SeekLiteral;
   case State.RUN_REF:
   pPreBuffer += _preBufferPosition;
   goto SeekRef;
   case State.RUN_LITERALONLY:
   pPreBuffer += _preBufferPosition;
   goto SeekLiteralOnly;
   case State.RUN_EOF:
   pPreBuffer += _preBufferPosition;
   goto SeekEof;
   default:
   pPreBuffer += _preBufferPosition;
   SeekStart:
   if (_position == position)
   {
   goto SeekEnd;
   }
   // Prebuffer, 0x70 equals highest literal copy run
   if (pPreBuffer - pPreBufferF > _preBufferLength - 0x70)
   {
   _internalPosition += pPreBuffer - pPreBufferF;
   _baseStream.Position = _dataOffset + _internalPosition;
   _baseStream.Read(_preBuffer, 0, _preBufferLength);
   pPreBuffer = pPreBufferF;
   }
   _ref0 = *pPreBuffer++;
   // ref0 = xyyzzzww where x = 0; y = offset decompressed bytes; z = count decompressed bytes; w = count new bytes
   // ref1 = yyyyyyyy
   if ((_ref0 & 0x80) == 0)
   {
   _ref1 = *pPreBuffer++;
   // count range: 0 - 0x03 (0 - 3)
   _run = _ref0 & 0x03;
   // set refPosition to last decompressed byte minus range 0 - 0x03FF (0 - 1023)
   _refPosition = _position + _run - 1 - (((_ref0 & 0x60) << 3) | _ref1);
   // count range 0x03 - 0x0A (3 - 10)
   _runRef = ((_ref0 & 0x1C) >> 2) + 3;
   _state = State.RUN_LITERAL;
   goto SeekLiteral;
   }
   // ref0 = xxzzzzzz where x = 10; y = offset decompressed bytes; z = count decompressed bytes; w = count new bytes
   // ref1 = wwyyyyyy
   // ref2 = yyyyyyyy
   if ((_ref0 & 0x40) == 0)
   {
   _ref1 = *pPreBuffer++;
   _ref2 = *pPreBuffer++;
   // count range 0 - 0x03 (0 - 3)
   _run = _ref1 >> 6;
   // set refPosition to last decompressed byte minus range 0 - 0x3FFF (0 - 16383)
   _refPosition = _position + _run - 1 - (((_ref1 & 0x3F) << 8) | _ref2);
   // count range 0x04 - 0x43 (4 - 67)
   _runRef = (_ref0 & 0x3F) + 4;
   _state = State.RUN_LITERAL;
   goto SeekLiteral;
   }
   // ref0 = xxxyzzww where x = 110; y = offset decompressed bytes; z = count decompressed bytes; w = count new bytes
   // ref1 = yyyyyyyy
   // ref2 = yyyyyyyy
   // ref3 = zzzzzzzz
   if ((_ref0 & 0x20) == 0)
   {
   _ref1 = *pPreBuffer++;
   _ref2 = *pPreBuffer++;
   _ref3 = *pPreBuffer++;
   // count range 0 - 0x03 (0 - 3)
   _run = _ref0 & 0x03;
   // set refPosition to last decompressed byte minus range 0 - 0x01FFFF (0 - 131071)
   _refPosition = _position + _run - 1 - ((((_ref0 & 0x10) >> 4) << 16) | (_ref1 << 8) | _ref2);
   // count range 0x05 - 0x03FF (5 - 1028)
   _runRef = ((((_ref0 & 0x0C) >> 2) << 8) | _ref3) + 5;
   _state = State.RUN_LITERAL;
   goto SeekLiteral;
   }
   // ref0 = xxxwwwww where x = 111; w = count new bytes;
   // 0xFB Highest Non Stop Value; equals if (count <= 0x70)
   else if (_ref0 <= _highestNonStopValue)
   {
   // count range 4 - 0x80 (4 - 128)
   // actual range 4 - 0x70 (4 - 112), always multiple of 4
   _run = ((_ref0 & 0x1F) + 1) << 2;
   _state = State.RUN_LITERALONLY;
   goto SeekLiteralOnly;
   }
   // end of file
   // count range 0 - 0x03 (0 - 3)
   else
   {
   _run = _ref0 & 0x03;
   _state = State.RUN_EOF;
   goto SeekEof;
   }
   SeekLiteral:
   while (_run-- > 0)
   {
   _refPackBuffer[_position++ & _refPackBufferLengthMinusOne] = *pPreBuffer++;
   if (_position == position)
   {
   goto SeekEnd;
   }
   }
   _state = State.RUN_REF;
   SeekRef:
   while (_runRef-- > 0)
   {
   _refPackBuffer[_position++ & _refPackBufferLengthMinusOne] = _refPackBuffer[_refPosition++ & _refPackBufferLengthMinusOne];
   if (_position == position)
   {
   goto SeekEnd;
   }
   }
   _state = State.COMPLETE;
   goto SeekStart;
   SeekLiteralOnly:
   while (_run-- > 0)
   {
   _refPackBuffer[_position++ & _refPackBufferLengthMinusOne] = *pPreBuffer++;
   if (_position == position)
   {
   goto SeekEnd;
   }
   }
   _state = State.COMPLETE;
   goto SeekStart;
   SeekEof:
   while (_run-- > 0)
   {
   _refPackBuffer[_position++ & _refPackBufferLengthMinusOne] = *pPreBuffer++;
   if (_position == position)
   {
   goto SeekEnd;
   }
   }
   _state = State.COMPLETE;
   SeekEnd:
   _preBufferPosition = pPreBuffer - pPreBufferF;
   _internalPosition += _preBufferPosition;
   _baseStream.Position = _dataOffset + _internalPosition;
   break;
   }
   }
   }

   // Stream interface
   public override bool CanRead
   {
   get { return _baseStream.CanRead; }
   }

   public override bool CanSeek
   {
   get { return true; }
   }

   public override bool CanWrite
   {
   get { return false; }
   }

   public override long Length
   {
   get { return _length; }
   }

   public override long Position
   {
   get { return _position; }
   set
   {
   if (value < _position)
   {
   _baseStream.Position = _dataOffset;
   _internalPosition = 0;
   _position = 0;
   _baseStream.Read(_preBuffer, 0, _preBufferLength);
   _state = State.COMPLETE;
   InternalSeek(value);
   }
   else if (value > _position)
   {
   InternalSeek(value);
   }
   }
   }

   public override void Flush()
   {
   }

   public unsafe override int Read(byte[] buffer, int offset, int count)
   {
   if (count == 0)
   {
   return 0;
   }
   long readEnd = _position + count;
   int bufferPosition = offset;
   fixed (byte* pPreBufferF = &_preBuffer[0])
   {
   byte* pPreBuffer = pPreBufferF;
   switch (_state)
   {
   case State.RUN_LITERAL:
   pPreBuffer += _preBufferPosition;
   goto ReadLiteral;
   case State.RUN_REF:
   pPreBuffer += _preBufferPosition;
   goto ReadRef;
   case State.RUN_LITERALONLY:
   pPreBuffer += _preBufferPosition;
   goto ReadLiteralOnly;
   case State.RUN_EOF:
   pPreBuffer += _preBufferPosition;
   goto ReadEof;
   default:
   pPreBuffer += _preBufferPosition;
   ReadStart:
   if (_position == readEnd)
   {
   goto ReadEnd;
   }
   // Prebuffer, 0x70 equals highest literal copy run
   if (pPreBuffer - pPreBufferF > _preBufferLength - 0x70)
   {
   _internalPosition += pPreBuffer - pPreBufferF;
   _baseStream.Position = _dataOffset + _internalPosition;
   _baseStream.Read(_preBuffer, 0, _preBufferLength);
   pPreBuffer = pPreBufferF;
   }
   _ref0 = *pPreBuffer++;
   // ref0 = xyyzzzww where x = 0; y = offset decompressed bytes; z = count decompressed bytes; w = count new bytes
   // ref1 = yyyyyyyy
   if ((_ref0 & 0x80) == 0)
   {
   _ref1 = *pPreBuffer++;
   // count range: 0 - 0x03 (0 - 3)
   _run = _ref0 & 0x03;
   // set refPosition to last decompressed byte minus range 0 - 0x03FF (0 - 1023)
   _refPosition = _position + _run - 1 - (((_ref0 & 0x60) << 3) | _ref1);
   // count range 0x03 - 0x0A (3 - 10)
   _runRef = ((_ref0 & 0x1C) >> 2) + 3;
   _state = State.RUN_LITERAL;
   goto ReadLiteral;
   }
   // ref0 = xxzzzzzz where x = 10; y = offset decompressed bytes; z = count decompressed bytes; w = count new bytes
   // ref1 = wwyyyyyy
   // ref2 = yyyyyyyy
   if ((_ref0 & 0x40) == 0)
   {
   _ref1 = *pPreBuffer++;
   _ref2 = *pPreBuffer++;
   // count range 0 - 0x03 (0 - 3)
   _run = _ref1 >> 6;
   // set refPosition to last decompressed byte minus range 0 - 0x3FFF (0 - 16383)
   _refPosition = _position + _run - 1 - (((_ref1 & 0x3F) << 8) | _ref2);
   // count range 0x04 - 0x43 (4 - 67)
   _runRef = (_ref0 & 0x3F) + 4;
   _state = State.RUN_LITERAL;
   goto ReadLiteral;
   }
   // ref0 = xxxyzzww where x = 110; y = offset decompressed bytes; z = count decompressed bytes; w = count new bytes
   // ref1 = yyyyyyyy
   // ref2 = yyyyyyyy
   // ref3 = zzzzzzzz
   if ((_ref0 & 0x20) == 0)
   {
   _ref1 = *pPreBuffer++;
   _ref2 = *pPreBuffer++;
   _ref3 = *pPreBuffer++;
   // count range 0 - 0x03 (0 - 3)
   _run = _ref0 & 0x03;
   // set refPosition to last decompressed byte minus range 0 - 0x01FFFF (0 - 131071)
   _refPosition = _position + _run - 1 - ((((_ref0 & 0x10) >> 4) << 16) | (_ref1 << 8) | _ref2);
   // count range 0x05 - 0x03FF (5 - 1028)
   _runRef = ((((_ref0 & 0x0C) >> 2) << 8) | _ref3) + 5;
   _state = State.RUN_LITERAL;
   goto ReadLiteral;
   }
   // ref0 = xxxwwwww where x = 111; w = count new bytes;
   // 0xFB Highest Non Stop Value; equals if (count <= 0x70)
   else if (_ref0 <= _highestNonStopValue)
   {
   // count range 4 - 0x80 (4 - 128)
   // actual range 4 - 0x70 (4 - 112), always multiple of 4
   _run = ((_ref0 & 0x1F) + 1) << 2;
   _state = State.RUN_LITERALONLY;
   goto ReadLiteralOnly;
   }
   // end of file
   // count range 0 - 0x03 (0 - 3)
   else
   {
   _run = _ref0 & 0x03;
   _state = State.RUN_EOF;
   goto ReadEof;
   }
   ReadLiteral:
   while (_run-- > 0)
   {
   buffer[bufferPosition++] = *pPreBuffer;
   _refPackBuffer[_position++ & _refPackBufferLengthMinusOne] = *pPreBuffer++;
   if (_position == readEnd)
   {
   goto ReadEnd;
   }
   }
   _state = State.RUN_REF;
   ReadRef:
   while (_runRef-- > 0)
   {
   buffer[bufferPosition++] = _refPackBuffer[_refPosition & _refPackBufferLengthMinusOne];
   _refPackBuffer[_position++ & _refPackBufferLengthMinusOne] = _refPackBuffer[_refPosition++ & _refPackBufferLengthMinusOne];
   if (_position == readEnd)
   {
   goto ReadEnd;
   }
   }
   _state = State.COMPLETE;
   goto ReadStart;
   ReadLiteralOnly:
   while (_run-- > 0)
   {
   buffer[bufferPosition++] = *pPreBuffer;
   _refPackBuffer[_position++ & _refPackBufferLengthMinusOne] = *pPreBuffer++;
   if (_position == readEnd)
   {
   goto ReadEnd;
   }
   }
   _state = State.COMPLETE;
   goto ReadStart;
   ReadEof:
   while (_run-- > 0)
   {
   buffer[bufferPosition++] = *pPreBuffer;
   _refPackBuffer[_position++ & _refPackBufferLengthMinusOne] = *pPreBuffer++;
   if (_position == readEnd)
   {
   goto ReadEnd;
   }
   }
   _state = State.COMPLETE;
   ReadEnd:
   _preBufferPosition = pPreBuffer - pPreBufferF;
   _internalPosition += _preBufferPosition;
   _baseStream.Position = _dataOffset + _internalPosition;
   break;
   }
   }
   return bufferPosition - offset;
   }

   public override long Seek(long offset, SeekOrigin origin)
   {
   switch (origin)
   {
   case SeekOrigin.Begin:
   Position = offset;
   break;
   case SeekOrigin.Current:
   Position += offset;
   break;
   case SeekOrigin.End:
   Position = _length - offset;
   break;
   }
   return Position;
   }

   public override void SetLength(long value)
   {
   }

   public override void Write(byte[] buffer, int offset, int count)
   {
   }
   // End Stream interface
   }
}

_________________

Ju-Jin · Posted: Thu Jun 30, 2016 7:16 pm Post subject:

Ah sorry I missed the point.
But I don't get why its so hard for you. Just write a stream that gets the data on demand instead of pumping everything at once into memory.

_________________

Banshee · Posted: Fri Jul 01, 2016 2:52 am Post subject:

What made it hard is that the whole program is designed to pump everything into the memory. Changing that is the main cause of the hassle #Tongue

. But now I'm really in trouble with my thesis. I'll resume working on OS BIG Editor once things calm down on my side.

Ju-Jin · Posted: Fri Jul 01, 2016 6:12 am Post subject:

Btw what do you think of my solution to stop/continue reading? ^^
(I know I could have used breaks and returns instead of gotos, but they were simply the faster solution in this case)

_________________

Banshee · Posted: Sat Jul 02, 2016 7:53 am Post subject:

It seems to be easy to read and understand, despite the use of goto. But you are right, goto is a faster solution in this case.

Banshee · Posted: Sun May 24, 2020 4:30 am Post subject:

And here's a desired update for the upcoming Command & Conquer: Remastered Collection: a 64 bits version of Open Source .BIG Editor ready to roll! Note that this version does load and save .MEG files, which will be used there, according to undisclosed sources inside the C&C Community Council. The same source claims that OS .BIG Editor and other .meg editors are already compatible with it.

Armen123_dnej · Civilian **Joined**: 26 Nov 2020

Hello, I have a question.
You see, I wanted to edit one of the .big mod files I've already assembled and was even able to extract files from it using the .BIG Extractor. The problem is that the extracted files are not .xml but .bin, .imp, .manifest, .relo and .version. I wonder if it is possible to edit and/or extract files from them?

Banshee · Posted: Fri Nov 27, 2020 4:50 pm Post subject:

Nope. I know programs that can extract content from .bin, .imp, .manifest, .relo files, but not edit them.

Banshee · Posted: Tue Apr 02, 2024 12:47 am Post subject:

Update: 0.588 loads and save ZIP files. It also loads most, if not all, config.meg files from the games from Petroglyph, including the recently released 9-Bit Armies.