mv.NET .Attributes and .DictionaryList
It’s been a while since I’ve published an entry here. That doesn’t mean I haven’t been writing – I have over 30 unpublished blog entries. I just need to clean them up one of these days when I have some time.
LOL, like that day will ever come… 
Anyway, I decided to write something up here for our clients and colleagues who ask this FAQ about mv.NET:
"When using mvSelect, what do I use for the .DictionaryList and .Attributes?"
This is an important topic because it relates to the performance of your application – and new mv.NET developers will get turned off if their first experience with this tool results in very slow data transfers of large volumes of unused data.
Definition of .Attributes and .DictionaryList
The .Attributes property of mvSelect tells the server which attributes to return for each item. It helps to reduce bandwidth by requesting only the data that is required. Use this if you are not using a DictionaryList.
Example 1:
sel.Attributes = "0 2 4 13 16";
sel.DictionaryList = "";
… get items…
string val2 = item[2];
string val4 = item[4]; …
The .DictionaryList property is used to pre-load the list of dict definitions that will be used to retrieve data. When using the .DictionaryList, set the .Attributes to only retrieve the item ID.
Example 2:
sel.DictionaryList = "ID FIRSTNAME SURNAME";
sel.Attributes = "0";
… get items…
string ID = item["ID"];
string First = item["FIRSTNAME"]; …
Focus on the .Attributes property
Setting .Attributes to empty string "" has the same effect as not using it at all, and the entire contents of every item will be retrieved from the server, even if only some fields will actually be used.
Looking at the Connection Monitor, here is what will be seen when retrieving two items from the server using the code from Example 2:
SETREAD ITEM^SOP.DEV $661 2469296 1 3^0^1]2
00000000550000015346^~5346^Lee^Anderson|
000002939^~939^Leo^Andrews<EOM>
Here we see the ID only "5346" retrieved, followed again by the ID, the Firstname, and the Surname.
(Connection Monitor data includes unprintable boxes and "254" and "253" markers that represent attribute and value marks. These markers and other housekeeping data are passed on the pipe from the DBMS to the client. The data has been simplified a little here for readability. Also, since we’re here, the data on my system will be different than that on your system, depending on how many records you generated for the CONTACTS file. And just to keep the data unique, I had a few instances of Leo Andrews, so I manually modified one to Lee Anderson – so you won’t have a Lee Anderson no matter how many records you generate, unless you zap a record too.)
The Connection Monitor displays whenever a connection is made. It displays all data flowing across the wire.
To see the Connection Monitor:
1) Open Data Manager application.
2) Select menu > Session Monitor > Open.
3) Edit a Server Profile
4) Check the box "Display connection monitor on startup"
5) Save the profile and connect to an account using a profile for that server.
Don’t forget to uncheck the box when not debugging. The Session Monitor must always be open for Connection Monitor to launch – I forget about this all the time and wonder why it’s not starting…
Assume for a moment a perfectly logical guess, setting .Attributes to the data that is desired and .DictionaryList to load the dict items:
sel.Attributes = "0 2 3";
sel.DictionaryList = "ID FIRSTNAME SURNAME";
Here is the data that is returned from the server:
SETREAD ITEM^SOP.DEV $661 2469314 1 4^0]2]3^1]2
00000000790000015346^^Lee^Anderson~5346^Lee^Anderson|
000002939^^Leo^Andrews~939^Leo^Andrews<EOM>
The read from the DBMS has the ID 5346, and the first and last name as directed by .Attributes. Then the .DictionaryList requests the exact same data again. Bandwidth and download time are doubled!
Let’s see what happens if a different list of .Attributes is requested than the .DictionaryList:
sel.Attributes = "0 2 1 4";
sel.DictionaryList = "ID FIRSTNAME SURNAME";
SETREAD ITEM^SOP.DEV $661 2469326 1 7^0]2]1]4^1]2
00000001000000015346^^Lee^1797^Customer Services~
5346^Lee^Anderson|
000002939^^Leo^315^Secretary~939^Leo^Andrews<EOM>
The .Attributes are returned but the .DictionaryList fields are also returned.
Focus on the .DictionaryList property
A great deal of data is transferred if dictionary definitions are used to extract data, but they are not specified in the .DictionaryList. For example:
sel.Attributes = "0 2 3";
sel.DictionaryList = "";
When the first mvItem is retrieved from the mvItemList, only the .Attributes data is retrieved. Here is code that will extract mvItem data and insert it into a defined hashtable:
foreach ( mvItem item in list )
{
hash.Add(item["ID"] , item[ "SURNAME" ] +
", " + item[ "FIRSTNAME" ]);
}
As expected, the request for the first item causes this response from the server:
SETREAD ITEM^SOP.DEV $663 2469352 1 1^0]2]3^1]2
00000000460000015346^^Lee^Anderson|
000002939^^Leo^Andrews<EOM>
But the first request for dictionary-defined fields inside the foreach block also causes the entire dictionary to be loaded from the server:
DICTLOAD DICT CONTACT
000000134609:59:15 01 DEC 2001
DICT CONTACT^ID^A^0^ID^^^^MR^^R^10~
0^ID^RightAlign^10^DataInteger^Singular^^^
False^ID^^^^None^^False^False^^^|
^253ORGANIZATION^A^1^Organization^^^^^TORGANIZATION;X;;1^L^10~
1^Organization^LeftAlign^10^DataAlphaNumeric^Singular^^^
False^Organization^^^^None^^False^False^^^^ORGID|
ORGID^A^1^Organization ID^^^^MR^^R^10~
1^Organization ID^RightAlign^10^DataInteger^Singular^^^
False^Organization ID^^^^None^^False^False^^^^^
ORGANIZATION^NUMBER^NAME|
FIRSTNAME^A^2^First Name^^^^^^L^10~
2^First Name^LeftAlign^10^DataAlphaNumeric^Singular^^^
True^First name^^^^None^^False^False^^^|
FULLNAME^A^2^Full Name^^^^^F;2;" ";3;:;:^L^30~
-1^Full Name^LeftAlign^35^DataAlphaNumeric^Singular^^^
False^Full name^^^^None^^False^False^^^^FIRSTNAME^253SURNAME|
SURNAME^A^3^Surname^^^^^^L^10~
3^Surname^LeftAlign^10^DataAlphaNumeric^Singular^^^
True^Surname^^^^None^^False^False^^^|
POSITION^A^4^Position^^^^^^L^10~
4^Position^LeftAlign^10^DataAlphaNumeric^Singular^^^
False^Position^^^^None^^False^False^^^^^^^^^^False^<EOM>
Why is the entire dictionary retrieved? The first statement may request just one or a few dict references, but the code has no way of knowing how many more will be required. So it loads the entire dictionary once and keeps that data in cache in case someone else needs anything from that file during the current logon session.
Combining .Attributes and .DictionaryList
By design, these fields are intended to work in collaboration – that is, they are additive. Take another look back at Example 2. The .Attributes property is set to "0" so we retrieve the item ID, and .DictionaryList has two different fields which are retrieved by name. An important function of .DictionaryList is to allow code to make use of fields that are not in the data, but calculated – in other words, synonym dict items. Example:
sel.DictionaryList = "FULLNAME";
sel.Attributes = "0";
… get items…
string ID = item.ID;
string FullName = item["FULLNAME"]; …
The data the comes from the system will be the item ID, as specified in .Attributes, and the calculated full name as defined by the dict item FULLNAME ( FIRSTNAME:" ":SURNAME ). As you see, item["FULLNAME"] doesn’t just refer to an element where FULLNAME is an attribute number. The mvItem has a place allocated to contain calculated data.
mvItem – Not Just a Dynamic Array
That "place" is not just another attribute. We see that .Attributes and .DictionaryList combine to create an mvItem which contains both attributes and calculated fields – right there we know this isn’t the average dynamic array, and it would be an error to try, for example, to reference some attribute number to get the FULLNAME data.
Execute some code like that last example which only returns a couple fields. After the data is read the mvItem object will only contain the data that you expect – data that you see coming across the wire in the Connection Monitor. If you right click on the mvItem while in Debug mode, and take a look at it manually, you will see that the act of looking a this data causes a re-connect to the server and a read of the entire item from the DBMS. The same thing happens if you put a Watch on an mvItem. The data you see is not what was retrieved originally, and unless you’re watching the Connection Manager you probably wouldn’t realize that.
Remember that I said that FULLNAME and other calculated fields aren’t just added to some high attribute in an array within the mvItem. You may want to find the number of fields returned in a given mvItem using:
int count = item.DCount(); // assumes dcount of attributes for whole item
Remember that special space was allocated in the mvItem for the calculated data – the attributes are only part of the data in the mvItem. But you asked for a DCount, so once again a connection is made to the server to get all of the attributes so that a good DCount can be done.
The rule of thumb is to be sure the code only accesses attributes that were pulled back via one of the properties discussed here, and only request calculated fields defined in .DictionaryList. And if you are using mvSelect to generate an mvItemList, and then reading the items one at a time, don’t treat the mvItems like they’re just any other dynamic array.
The Item ID
It’s worth a note that since the ID dict item wasn’t used in this last example, you don’t want to refer to the ID there as item[ "ID" ]. That will cause the entire dictionary to be read – and again you wouldn’t notice this unless you were watching Connection Monitor (think it’s time to add that to your bag of tricks?).
To get the ID, you can simply use the ID property of the mvItem object: item.ID. You can also refer to attribute 0 with syntax that is familiar to .NET developers, but not to MV people: item[0].
Summary
Imagine a query that is only supposed to connect, return a small list of items, maybe just one, and then disconnect. At worst, the entire dictionary will be loaded for every query, and entire items will be retrieved from the server even if only a few attributes are needed. This will consume a great deal of bandwidth (and time) when only a few bytes are required.
The solution to this problem is first to use .Attributes to limit the data coming from the server, and to use .DictionaryList if you want to refer to fields by name rather than by attribute number, or if you want to return a calculated field.
You can also avoid the use of dictionary names altogether, and this eliminates all dictionary processing. To minimize the burden of using hardcoded attribute references in code, consider using constants. For example:
Rather than this:
foreach ( mvItem item in list )
{
hash.Add(item["ID"] , item[ "SURNAME" ] +
", " + item[ "FIRSTNAME" ]);
}
Try this:
const int ID = 0;
const int FIRSTNAME = 2;
const int SURNAME = 3;
foreach ( mvItem item in list )
{
hash.Add(item[ID] , item[ SURNAME ] +
", " + item[ FIRSTNAME ]);
}
To the MV developer, using attribute numbers is natural and using dict names is helpful (available in BASIC in some MV DBMS platforms), but to anyone not familiar with MV, referencing fields by number could be confusing. Consider who will be maintaining the code when deciding how fields will be retrieved into mvItems. (Yes, anyone using the Core Objects library should have familiarity with MV, there are no absolutes…)
I hope this was helpful. Comments are welcome here. Extensive discussion should be moved to the Nebula R&D forum.