Connecting with MV

It’s a common question: "How do I connect something from outside of my MV DBMS to my application software?" I’ve written about all of these points in various forms before but it looks like it’s time for a recap.

The topic usually starts with "how do I serve up web pages", and almost always gets muddled with acryonyms, product names, frameworks, etc. Let’s get away from the client side for a while and focus just on how the connection is made from anything outside the MV DBMS to the business application code – the last connection point to the last tier of an n-tier architecture. For better or worse there aren’t too many ways to do this, though there is enough material here to warrant breaking this into a multi-part posting. So for now I’ll just focus on…

Telnet

All MV platforms support a telnet server process. In short, a client (like a terminal emulator)connects to a socket that is expecting exchanges with the Telnet protocol (RFC 854 et al). Characters are sent from the client to the server, and the server further passes these characters to an MV input prompt, whether TCL or a BASIC Input statement. The MV environment then sends back through the pipe a stream of characters, using for example the CRT/PRINT statements.

Many people confuse the Telnet data transfer protocol with the user interface agent. All telnet clients that have a UI have 2 components, the socket interface mentioned above, and a UI component that interprets the emulation-specific character-streams and renders them in a format that makes sense to a human being. This is where we break off when we’re doing web comms.

A web interface obviously does not do 80×24 plotting of the data it receives. For this reason, when the Telnet protocol is used as a data transport medium, we try to eliminate any sort of UI character sequences like screen positioning, clearscreen, etc. All data coming from the MV environment up that pipe and to the client should be pure data.

But what sort of data? This is where the developer decides how to wrap their data, whether simple delimited, XML, JSON, etc. The data wrapper is really a higher-level custom protocol which rides over the lower level transport protocol.

So on the MV side, you can do this:

CRT "<custid>1234</custid>"

and on the client side you will be expecting XML, which you will parse, and then it’s up to you to figure out what the data is and what you want to do with it.

Where does one get code that conforms to the Telnet protocol? You’ll find free code and for-fee components in every language for every OS, all with varying levels of stability and ease of use for the developer. People who understand the code and protocols may want to roll their own.

Where does all of that code fit? Obviously the MV DBMS is in the office, behind some firewall from the internet. Telnet connections used with web interfaces are almost exclusively used between the local DBMS and a local web server. Let’s walk through the entire sequence from a browser to a server. A browser client connects using HTTP to a web server, which then executes CGI code, ISAPI, or some other server-oriented middleware. This code then executes site-specific code that’s not server-oriented, and it’s that code that initiates the communication with the MV DBMS.

It’s completely irrelevant to your communications code and back-end code what language or framework is being used by the client. This is why it’s possible to front-end an MV app with Java, Flash, Silverlight, plain DHTML and JavaScript, etc. What may required a bit of thought is how you execute your telnet-enabled code from the CGI/ISAPI code in the web server. In short, CGI is slower (*sigh* let’s ignore FastCGI) because it executes your code from scratch on every new connection. ISAPI and related servers maintain a reference to components that they use, so there is only once instance of initialization pain, and from then on all connections go through the existing instance of your code.

All kinds of questions come up at this point:

What if you’re on Linux vs Windows? It doesn’t matter. Your Apache web server can execute code that you provide which executes a telnet connection into your MV DBMS.

Does the connection always stay open? Not with CGI. The connection will be closed as soon as your "client" process in the web server dies, and that happens right after a response is returned to the client browser. However, there are techniques for having your telnet connection open as a persistent process, and then getting the web server to attach to that process rather than starting up telnet on every browser connection. I’ll discuss this later.

What if the connection dies? If your MV process goes into the debugger, what would a terminal emulator see? Probably an asterisk or other prompt as the debugger waits for someone to type "G" or "END" or some other command. But your client code isn’t expecting a debugger prompt, it’s expecting data. So the DBMS waits, your code waits, and the browser waits until it goes through a timeout wrapup. In short – don’t let your server code fall into debug.

What if the server process executes a long Select? Again, you can’t let the client wait too long or their connection will timeout. You need to do long processes "asynchronously". Which means the connection from the client is simply a message telling the server to start a process in the background, and subsequent connections are just asking "is it done yet?"

Those questions and others need to be answered for every connectivity method. Let’s move on to other connectivity methods, and then compare and contrast them with Telnet.