Let’s clean up our dictionaries

For better or worse, there are no standards for MV / Pick dictionary (schema) definitions. Over the years people have created names like AMOUNT, AMT, AMT$, PCT, PERCENT, %PCT, etc. There are funky characters and mis-spellings and inconsistencies even within each individual application. Shall we try to clean some of this up?

Note: I wrote this about a year ago and never published it because I wanted to refine it, but having read over it a few times it still looks OK to me, so here we go…

Maybe the first question is Why? Outside of establishing consistency and keeping things nice for the next person who comes along, there’s really no reason. If it works, don’t fix it – right? Well, if you’re doing a demo or trying to teach someone how to use your system, and the syntax for your "English" queries looks more computerese than English, then maybe there is a problem that needs to be fixed. If nothing else, elegance is a nice thing. Consistency in an industry or even a single app is a nice thing. The "why" isn’t so important at the moment, I’m focusing on the "how".

It’s a shame there isn’t a standard technique for versioning on dict items so that we can clean up a system. I’ll propose one now and because this is completely off the top of my head and I’m typing fast I’m sure there is something wrong with it. But at least we have a place to start. The approach I’m proposing here allows an app to continue functioning without interruption, and all changes can be made over a long period of time. There’s no need to do everything all at once or have a major cut-over event.

Let’s clean up the CUSTS file. Maybe I don’t like CUSTS but prefer to standardize on full words like CUSTOMERS. I also want to fix a bunch of dict items.

You’ll note that references to D-pointers and Q-pointers are D3-specific but the concepts should apply to all MV platforms.

Create two new dictionary-only files, CUSTOMERS.V1 and CUSTOMERS.V2. Add a Q-pointer in v1 and to CUSTS, so when you say LIST CUSTOMERS.V1 you get the data file of CUSTS, same with v2. Copy all dict items from CUSTS to v1 and v2, they should all work as-is.

Create a basic Q-pointer called CUSTOMERS to CUSTS. This will eventually be a d-pointer with a full dict. One by one, when you have a statement in a proc or program that says "SELECT CUSTS WITH …", change it to "SELECT CUSTOMERS.V1 WITH …". This should not break functionality. If someone uses CUSTOMERS or CUSTOMERS.V1, they’ll get the current dict and data. If they use CUSTOMERS.V1 they’ll absolutely get the version 1 dict items.

Take a look at one proc/program and just one statement. Target just the dict items for that one statement – not the whole program or the whole system. Remember, this is supposed to be a very progressive process. If you see BAL$, change it to something like ACCOUNT.BALANCE. Then go to the dict of CUSTOMERS.V2, copy BAL$ to ACCOUNT.BALANCE. Don’t make changes to the dict item because it has functioned well in the past and you do not want to change functionality here, only names. Put a note in BAL$, maybe in atb 18, that this definition is being superseded by ACCOUNT.BALANCE.

Do that operation for all definitions in the one statement. Then Change the file reference for that one statement from CUSTOMERS.V1 to CUSTOMERS.V2. Test it, and if it works, leave it there for a while to make sure it continues to work properly.

Continue this pattern, copying dict items in v2, standardizing the definitions where possible, re-using definitions that are exactly the same as others, and pointing existing statements from CUSTS to CUSTOMERS.V2. Eventually you’ll have the entire application using CUSTOMERS.V2 rather than CUSTS.

Point the CUSTOMERS q-pointer to CUSTOMERS.V2 and start using that. Run with it for a few weeks or months – make sure it survives a year-end close and subsequent adjustments. Everything still working OK?

Now do a global search in programs and procs for CUSTOMERS.V2 and zap them all to CUSTOMERS. Everything still working OK?

Finally, delete the CUSTOMERS q-pointer. Rename CUSTOMERS.V2 to CUSTOMERS. Remove the q-pointer to CUSTS in the dict of CUSTOMERS and move the d-pointer for the data section of CUSTS into CUSTOMERS.

At this point the CUSTS and CUSTOMERS.V2 files should be completely unused. All references should be to CUSTOMERS and the clean dict items.

Potential issues, suggestions, and considerations

If you use Common to provide the names of key files then you can’t change the file names in individual statements or programs, you’ll need to change it for the entire application. This issue needs to be resolved on an application-specific basis. If you do have this real-world scenario I’d like to know about it and maybe we can figure out a way to work with it.

Backup before all changes!

Start with small files. Get some practice with this technique. Refine it – and please let me know where it needs improvement.

While you can do different files at the same time, don’t do too much at once. If there is an issue it should be small, temporary, and easy to fix because only minor changes have been made. You should not need to do massive reversals back to v1 or the original file.