Protected: Endianness – not about Native Americans

Many of you are familiar with the concept of byte ordering, or you’ve heard the terms Big-endian and Little-Endian. Various utilities are used to converts one to the other. Let’s take a look at this topic so we know where and how we need to be concerned about byte ordering, regardless of the DBMS we use.

The endianness of a platform is often related to the chipset, whether Intel or Motorola, but it’s more complex than that, the only thing you really need to know is which mode your system uses. In simplified terms, some systems generate and require data to look like 11110000 and others represent the same data as 00001111. As another example, let’s look at the name of the "fnuxi" utility, a command in Universe to convert from one format to another. The command name itself is a sort of anagram for Format UNIX, where the bytes of the word UNIX have been reordered into NUXI in their internal storage. Some of you have seen frames in Advanced Pick and D3, initialized with the stamp _IPKC_ … that’s "endianness" at work.

In typical geek tradition, the terms Big-endian and Little-endian were derived from a completely non-technical book, Gulliver’s Travels. You’ll find references to this and a lot of other info about this topic at Wikipedia.

As far as how this relates to MultiValue development – our code is tested on all possible supported platforms but the final releases are mostly generated from Windows-based systems. Both Linux and Windows use Little-Endian encoding so the code generated on our Windows systems should work with no change on your Linux systems. For AIX, which uses Big-Endian format, a conversion is required for DBMS platforms that store data in the host OS. But rather than provide a separate package for UV/AIX, UV/HP-UX, and UV/Windows, for example, we provide a single Universe package for any given product and leave it to our clients to do the conversion if required. As examples of which DBMS platforms require a conversion and why, let’s consider D3, Universe, and Unidata.

Since D3 object code is inside a virtual machine, all D3 systems generate object in the same way and no compensation for "endianness" is required. So you can compile your BASIC code in D3NT, move the object code to D3AIX, and it should run. However, if you Flash-compile your D3 programs, you do need to re-flash the object modules on the target platform. Flash-object in D3 is stored next to the traditional cross-platform, tokenized, intermediate object code in D3 dictionaries, but it’s generated with OS-specific C routines. For this reason and others, it’s generally a good idea to re-Flash all object code on a target system whether there is an issue with byte ordering or not. In this case, there is no utility to convert the endianness, because D3 can regenerate the platform-specific object code using the original object code as the source.

Universe and Unidata can make use of both hashed files and OS-level files (DIR, type 19, etc) for data, source programs, and object code. Byte ordering is a consideration for our products because we generate object code into these OS files on Windows and send it to our clients who run AIX and other operating systems. So when a client gets our package, they may need to convert the object code (in file NEBULA.BP.O in Universe or in _items in NEBULA.BP in Unidata).

The fnuxi utility in Universe is an OS-level utility. You can get help on this command with HELP FORMAT.CONV, which provides equivalent functionality. That help page tells you that a command like "FORMAT.CONV -6 NEBULA.BP.O" will convert the object code in that file to the proper endian format for the current system. You can use something like " fnuxi -6 * " to do the same function for all files, but the asterisk is not recursive, so if you don’t get the results you expect, you may need to reconvert some files individually. Unidata provides the !CONVCODE command, and related help, to get similar results. CONVCODE is recursive and alternately switches endianness without command-line options.

For details on fnuxi, CONVCODE, and the utilities available for your DBMS/OS/chipset environment, consult your own documentation and with your software providers.