Actian Community Forums - Blogs

Want to learn more about OpenROAD and see what the team is up to?

teresa — Fri, 17 May 2013 15:13:11 GMT

Several members of the OpenROAD Engineering team will be running a code Sprint at the Thistle City Barbican hotel in London on June 8th, 9th, and half day on the 10th followed by the UK AUA 1-daytechnical conference on June 11th, at which there will be several OpenROAD Specific presentations.

Join the OpenROAD Sprint June 8th - 10th in London
Sprint projects tend to be 4GL projects as they can be implemented quickly. How do we determine the projects implemented at a sprint? We propose items to the attendees based on the following:

Review enhancements submitted via the Idea Center (previously Service Desk) for ideas that can be implemented in a reasonable timeframe
Other Projects we’ve documented at OpenROAD Sprint Projects - Ingres Community Wiki
Reported low priority bugs that are good training candidates for the sprint attendees
Enhancements that are important to the Sprint attendees, for example, Ingres OpenROAD Projects/Add ToolTextTip attribute to BitmapObject class - Ingres Community Wiki
Improving information on using OpenROAD
- Composing “How To” articles that should be added to the documentation or written up as a blog entry (Note, we will gladly accept any contributions that you may want to write up and submit to us prefer not to go through the hassle of putting together a blog entry yourself)
- Improving areas of the documentation that are unclear to the attendees
Other items raised by the Sprint attendees

Attending and participating in an OpenROAD sprint provides an opportunity to get a feature implemented that you’ve long wanted or fix a particularly annoying bug that is more of an annoyance than a showstopper. Past events have seen a number of features either released as part of OpenROAD 5.1 or 6.0 or targeted for the upcoming release. For more information on the sprint projects targeted for the next major OpenROAD release see http://community.actian.com/forum/bl...11-uk-iua.html and http://community.actian.com/forum/bl...-iua-giua.html.

Attend the UK AUA June 11th in London
After the Sprint, join the OpenROAD team at the UK AUA on Tues June 11th at the Thistle City Barbican hotel in London. The UK AUA is a great opportunity to learn more about OpenROAD as well as Ingres, Vectorwise, and other products newly added to the Actian portfolio in addition to networking with other customers using the technology. The AUA event is focused on providing great technical information to the attendees.

There will be four OpenROAD Specific presentations on Tues afternoon including:

Sneak Peek at OpenROAD 6.2's Sprites - Can it be that easy?
Sprites! What are they trying to do, make OpenROAD into a game development platform? The answer to that question is no, but sprites make it possible to create displays previously considered impossible with OpenROAD. Join us for an in-depth look at some new user classes that allow you work with small actionable graphic elements. See for yourself how this can revolutionize your existing applications, and imagine the possibilities for new initiatives. After this sneak peek at OpenROAD Sprites you will be able to answer for yourself this question: Can it be that easy?
Cats and Dogs – How to domesticate list-handling to obey your application’s every whim
To handle sets of business items, OpenROAD provides you with two representations: customized and type specific Arrays (cats); and dynamic, optimized, generically structured ChoiceLists (dogs). Cats and dogs are complementary: cats provide focused browsing and editing via TableFields; dogs provide at-a-glance selection via five different types of ChoiceField; between them they cover the range of multi-item handling that most applications require.

To get the best results, dogs in particular must be approached in the right way. This presentation begins with a common requirement – how to provide row-specific dropdown lists in TableFields – to illustrate some key techniques; then follows with advanced techniques for building lists from a range of source types (database, files, datastrings, arrays, master-lists). The session will also give you the opportunity to raise and discuss other list-handling issues you may have, so come prepared!
Don’t call us, we’ll call you
Revealing the mysteries of the XMLParserCallbacks class

This session provides an in-depth view of the new feature of the upcoming OpenROAD release.
It will answer a series of questions, e.g.:
- What is a callback?
- What are XML parser callbacks useful for?
- When does the XML parser call what?
- How to define a callback procedure?
- What does a callback procedure cannot do?
- How to tell the parser which procedures should be called when?
The session will also provide several usage examples, e.g.:
- Abort parsing after getting the information required.
- Prevent memory overflow by discarding parsed elements.
- Loading very big XML files into a database.
OpenROAD Status and Roadmap
It is an exciting time for OpenROAD developers, with major new features available in the last two releases; OpenROAD 6.0 is seeing a lot of uptake due to the single-byte, multi-byte and Unicode support provided. We are interested in any challenges you may be facing or have faced with the migration to this release so we can continue to improve your OpenROAD experience.

The next release of OpenROAD provides graphical features you can use to enhance the appearance of your applications and improves deployment options. The feature set of this release is completely defined and nearing completion, but to polish the features you actually use, OpenROAD development must now focus on how to make these powerful features easy to use. This is a perfect time discuss challenges you face-using OpenROAD, and join us for a lively discussion about product improvements that could help alleviate some of these challenges. Come prepared to participate with your challenges, questions, and OpenROAD needs!

Highlights of the General Sessions:

Keynote from Mike Hoskins
Product Roadmap Overview from Mark Milani
Ingres and OpenROAD Update from Emma McGrattan
And several others

In addition, there will be presentations specific to Ingres, Vectorwise, and new additions to the Actian Portfolio. See Join the UK AUA and Attend the 2013 Conference | UK Actian Users Association for the provisional agenda. It’s not too late to register, see http://www.regonline.co.uk/ukiua2013.

On Terminate event block in field scripts

teresa — Fri, 12 Apr 2013 17:57:47 GMT

On Terminate event is currently valid only in frame scripts. An enhancement request, SIR 122582, for this event to be valid in field scripts is under review for possible inclusion in a future release. In the meantime, an alternative solution is to write a facility in 4GL code that provides equivalent results to this proposed feature, one that even offers a more flexible and complete solution. This article describes alternative ways to implement this solution.

Problem

OpenROAD allows initialize blocks in field scripts. In these initialize blocks developers can declare variables that are visible (in scope) only for the field’s field script.

When the frame terminates, there is no terminate block available in field scripts that is equivalent to a frame script terminate block, yet you may need to access the field-local variables to perform local cleanup, state checking, or value finalization of some kind. For example, if the variable type is a userclass, you may need to call some methods when a frame closes.

This is a useful feature, and provision of such a capability is under review for a future release. However, two additional considerations to look at are:

Frame open and frame close are not the only critical state-changes that may require local access and local response, particularly if the application is using generically coded field templates. State changes such as field resize, open/switch/save business object, change focus (between panels), change access mode, and call-return may all benefit from local access and local response.
Until the “local terminate blocks” feature is implemented, there is a solution already available that gives equivalent results and offers a more flexible and complete response (see Resolution).

Resolution

Field-local procedures can always be invoked from anywhere in the frame (including from a terminate event block) by their prochandles. The following two statements are equivalent if the prochandle and procedurename in this example both refer to the same procedure:

returnvalue = prochandle.Call(…parameters…);
returnvalue = callproc procedurename(…parameters…);

There are three possible 4GL solutions to this problem:

Option 1: Put the identical code that you planned to use in a field-local terminate block into a field-local onclose procedure instead, with a frame-local variable referencing that procedure's prochandle. The frame-local terminate code can then invoke it by a single-line call before continuing its own processing. It can also inspect and respond to a return value—something not possible in event block processing without awkward status-variable management. The Notes section below offers additional considerations to this first option.

Option 2: Have a frame-local variable reference an array that contains prochandles for each field's onclose procedure. This approach has the advantage that it does not require that the frame-local terminate code be aware of which fields have this terminate code. Then it can run as follows:

for i = 1 to oncloses.LastRow do
prochandle = oncloses[i];
returnvalue = prochandle.Call();
endfor;

The Implementation section describes a straightforward example of this second option.

Option 3: Use a StringHashTable that a frame-local variable references to store the prochandles, keying each entry with a text string that identifies which state-change invokes it. You can store each prochandle as many times as there are state-changes that invoke it. Then you can handle all the frame's state-changes—not just the onclose state-change—through a single simple mechanism. That mechanism uses the StringHashTable’s FindAll() method to collect all prochandles with the same state-change key, ready to have the option 2 code applied to them:

prochashtable.FindAll(key='onclose', results=oncloses);

This third option is the most generic approach to handling all field script code that you want to execute when a frame terminates. It is a bit more complex because it builds on the second option techniques.

What these approaches do not offer is the ability to avoid a frame-close that a resume from a local terminate event block offers. However, if the onclose procedures are invoked from the windowclose event instead of from the terminate event, a resume statement within the procedures will work as desired.

Implementation

This implementation confines itself to option 2, since it directly addresses the particular topic of this blog article. Note that option 3, although more complex, offers more power and flexibility across a wider range of requirements.

For each field script that requires "on terminate" processing:

Write the field script "on terminate" event block exactly as if the local terminates feature exists.
Convert the "on terminate =" text to "procedure onclose ="
Adjust any frame-return or openframe invocations in the procedure (see Notes).
In the field-local initialize block, add the following line of code, which inserts each procedure in turn at row 1, so that last initialized is first closed:

oncloses.InsertRow(
rowobject=curscriptscope.GetProcHandle(name='onclo se'));

In the frame initialize block, add the following declaration:

oncloses = array of ProcHandle;

In the frame terminate (or windowclose) block, declare the prochandle and returnvalue variables and write the following code, adjusting the response to the returnvalue to suit your needs:

for i = 1 to oncloses.LastRow do
prochandle = oncloses[i];
returnvalue = prochandle.Call();
if returnvalue = then
resume;
endif;
endfor;

Notes

There are two minor caveats with this solution, both unlikely to cause problems in the context of a frame close:

If the procedure issues an openframe statement, it must explicitly include “with parentframe=curframe” in the statement.
If the procedure wants to force an immediate frame close, it cannot simply issue return, as a terminate block would. In the context under discussion, returning a "close now" returncode for the calling event block to act on would work.

This article was contributed by Sean Thrower from OpenROAD Engineering.

Surrogate Keys: Part 5, Pitfalls, Booby-Traps and the 1970s Resurrected

rhann — Fri, 05 Apr 2013 11:04:51 GMT

I'm almost finished ranting. For now.

So far I've shown that being scared of natural keys is about as grown-up and professional as being scared of balloon animals. I still need to show why properly diligent database designers must actively shun surrogate keys until they are absolutely unavoidable.

If you skipped Part 2 of this series of postings to get to the "juicy bits" you should go back and read it now.

I've already established the very limited cases where a surrogate key is justified. Now I want to look at the booby-trap waiting for those who mechanically use a single surrogate key even when the only natural key is actually composite—and who ever heard of anyone deliberately introducing a composite surrogate key?

Imagine an application to study air transport service quality. Amongst other things we are interested in routes operated by airlines between airports; the aircraft used on those routes, and (completed) flights. (We are not interested in superfluous details of code shared flights.)

One obvious design using only natural keys would be this:

Code:

CREATE TABLE airlines 
( 
   iata_designator VARCHAR(3) NOT NULL UNIQUE,
   ... 
);

Code:

CREATE TABLE routes 
( 
   iata_designator VARCHAR(3) NOT NULL, 
   flight_nr VARCHAR(5) NOT NULL,
   CONSTRAINT distinct_routes 
      UNIQUE (iata_designator, flight_nr), 
   CONSTRAINT route_airline 
      FOREIGN KEY (iata_designator) 
      REFERENCES airlines (iata_designator)
      ON UPDATE CASCADE,
   ... 
);

Code:

CREATE TABLE aircraft 
( 
   iata_designator VARCHAR(3) NOT NULL,
   tail_nr VARCHAR(7) NOT NULL,
   CONSTRAINT distinct_aircraft 
      UNIQUE (iata_designator, tail_nr),
   CONSTRAINT aircraft_airline 
      FOREIGN KEY (iata_designator) 
      REFERENCES airlines (iata_designator)
      ON UPDATE CASCADE, 
   ... 
);

Code:

CREATE TABLE flights
( 
   iata_designator VARCHAR(3) NOT NULL,
   flight_nr VARCHAR(5) NOT NULL, 
   tail_nr VARCHAR(7) NOT NULL,
   scheduled_date DATE NOT NULL, 
   CONSTRAINT distinct_flights 
      UNIQUE (iata_designator,flight_nr,tail_nr,scheduled_date ),
   CONSTRAINT flight_airline 
      FOREIGN KEY (iata_designator) 
      REFERENCES airlines (iata_designator) 
      ON UPDATE CASCADE, 
   CONSTRAINT flight_route 
      FOREIGN KEY (iata_designator, flight_nr)
      REFERENCES routes (iata_designator, flight_nr)
      ON UPDATE CASCADE,
   CONSTRAINT flight_aircraft FOREIGN KEY (iata_designator, tail_nr)
      REFERENCES aircraft (iata_designator, tail_nr)
      ON UPDATE CASCADE,
   ...
)

Though I say so myself this is a pretty good schema. It implements a surprising amount of business logic implicitly, eliminating the need to implement it (and repeatedly re-implement it) explicitly in application code. It lends itself to writing simple queries to address any conceivable information need.

Now, what do we get if we follow the advice to use surrogate keys everywhere? It's hard for me to guess what a person who thinks like that might do but I'm pretty sure this is one good possibility:

Code:

CREATE TABLE airlines 
( 
   airline_id INTEGER PRIMARY KEY,
   iata_designator VARCHAR(3) NOT NULL UNIQUE,
   ... 
);

Code:

CREATE TABLE routes 
( 
   route_id INTEGER PRIMARY KEY,
   airline_id INTEGER NOT NULL, 
   flight_nr VARCHAR(5) NOT NULL,
   CONSTRAINT distinct_routes 
      UNIQUE (airline_id, flight_nr), 
   CONSTRAINT route_airline 
      FOREIGN KEY (airline_id) 
      REFERENCES airlines,
   ... 
);

Code:

CREATE TABLE aircraft 
( 
   aircraft_id INTEGER PRIMARY KEY,
   airline_id INTEGER NOT NULL, 
   tail_nr VARCHAR(7) NOT NULL,
   CONSTRAINT distinct_aircraft 
      UNIQUE (airline_id, tail_nr),
   CONSTRAINT aircraft_airline 
      FOREIGN_KEY (airline_id) 
      REFERENCES airlines, 
   ... 
);

Code:

CREATE TABLE flights
( 
   flight_id INTEGER PRIMARY KEY,
   route_id VARCHAR(3) NOT NULL,
   aircraft_id INTEGER NOT NULL,
   scheduled_date DATE NOT NULL, 
   CONSTRAINT distinct_flights 
      UNIQUE (route_id, aircraft_id, scheduled_date ),
   CONSTRAINT flight_route 
      FOREIGN KEY (route_id) 
      REFERENCES routes,
   CONSTRAINT flight_aircraft 
      FOREIGN_KEY (aircraft_id) 
      REFERENCES aircraft,
   ...
)

Ignore the constraints if they bother you. They're not relevant to my argument.

I certainly see lots of designs like this so I'm confident this isn't just a straw man I'm setting up here.

Unfortunately this design does not behave like the real world nor like my proposed design above. It throws away important knowledge that the aircraft and the route flown belong to the same airline. Worse still, the absence of that information compels the computer and the programmer to assume it is not so—even though it really is.

Perhaps the designer very optimistically hopes "something else" takes care of that, always and forever; or maybe the designer doesn't even see the problem. I would worry that having set out with the "managing the containers" mindset which underlies indiscriminate use of surrogate keys the designer would be unlikely to spot the problem. It would then be up to the application programs (and all future application programs) to enforce the real world logic—assuming the programmers somehow become aware of it. With the design that uses only natural keys it is simply impossible to say a flight was served by the wrong airline's plane.

So much for simpler programming. So much for data quality.

Furthermore a lot of very reasonable queries on the surrogate key design will involve joining tables that weren't previously needed.

So much for improved performance.

Remind me, why are we supposed to use surrogate keys everywhere?

Sure one could cooper up the design with constraints and horrible database procedures to enforce the real world rules, but we'd end up with a way more complex, non-portable and unweildy design, just to allow us to use the surrogates I say aren't needed in the first place.

If this example seems contrived I can give you others. But it doesn't matter. The use of surrogate keys here concealed a logical error and that error can happen in any real design. It does happen in real designs.

All the other arguments against using surrogate keys: the complexity; the futility, the obfuscation, all build up to a compelling case for avoiding unnecessary surrogate keys. But this is the knock-out blow.

Surrogate keys as they are usually applied subvert the logical integrity of the database.

I have done a lot of ranting now and I'm feeling quite worn out, but I have just one final mini-rant to get off my chest. The surrogate key disease is reaching whole new levels of stupid.

I have lately been seeing database "designs" in which natural keys are not just replaced by spurious surrogate keys, but even the surrogates are being replaced by yet-more-spurious surrogates. I'm not going to go into this in depth but typically what you see is table A has a surrogate key A.id. The related table B has its own surrogate key B.id and a foreign key referencing A that would have been called id in any sane universe can't be because of B.id so usually it gets renamed something like A_id. And another related table C has yet another surrogate key C.id and contains a foreign key B_id that references B. And so on and so on.

Even if the facts in tables A, B and C depend on the same natural key you can't get at the facts in C even though you know the natural key. You are forced to trudge from A to B to C.

This is just implementing an old-fashioned 1970s network database (only much, much worse because it uses the lumbering great SQL engine instead of a lightweight disk access API).

Two things about that:

surrogate keys are claimed to improve performance and here they are degrading it by requiring unnecessary joins, and
actively concealing the essential logic of the database (by concealing the identity of the facts) forces the programmer to be the optimizer and hard-code the query strategy.

It's bonkers! We did that stuff 40 years ago and hated it then.

Don't get me started on the very similar fetish for encoding everything...

Conclusions

1. When you have a usable natural key there is no automatic justification for introducing a surrogate key. Forget the limitations of other products, Ingres likes natural keys.

2. If you have a natural key but it's genuinely not usable it is OK to introduce a surrogate key, but take care not to accidentally trample over the constraints of the real world.

3. If you don't have a natural key you can't use a surrogate to hide the problem.

The obvious and straightforward way to design a database is the right way to do it. There is no automatic justification for doing anything else, whether it is called "best practice" or "future-proofing" or whatever. And when it makes things worse, well that is unprofessional!

There, I feel better now.

Surrogate Keys: Part 4, In Search of a Niche

rhann — Thu, 04 Apr 2013 21:35:00 GMT

In the previous section I showed that natural keys—including composite natural keys—can perform fine. In my extensive experience they usually do.

There are exceptional cases when they don't do so well though, and so a surrogate key is justified.

A big VARCHAR key can be a problem when we try to use it as a physical key. For instance the biggest VARCHAR that can be used as a physical key in an Ingres B-tree is 1,000 characters. If your key is longer than that you are forced to introduce a surrogate.

OK, fine, so what about big VARCHARs smaller than that?

A fully-qualified file pathname uniquely identifies a file so it's a logical key, but they can be quite long with very little variation in the left-most portion of the pathname. It is reasonable to wonder how well it would perform as a physical key.

Staying with the pathname example, there are two cases to consider: tables where the pathname is a logical key (i.e. unique values in the table) and tables where it is a foreign key (i.e. potentially many copies of the same value in the table).

If the pathname is used in only one table, as a logical key, and not as a foreign key in any other table, there is no benefit to having a surrogate for it. You can't know the surrogate without looking it up (scanning for it) using the natural key, and once you've done that you already have the row you want! If scanning the table is unacceptable then the solution is to look for a more tractable natural key in the conceptual model. A surrogate doesn't help.

Looking at the second case where the pathname is used as a foreign key, possibly recurring many times, a surrogate may be helpful, but it's not clear-cut.

Two issues seem to make a surrogate look attractive here: wasted space and slow execution.

So what about wasted space? If you have related tables and the pathname is the foreign key in a one-to-many relationship, the very long string will be repeated in every row where it is a foreign key—and that could be a lot of rows.

And what if the key is vastly bigger than the non-key value it identifies, which might be as small as just a single integer (representing, say, the number of times the file has been requested)?

Such a "waste" of space offends some kind of aesthetic. It goes against intuitions of efficiency.

Tough. Disk space is one thing we really can't afford to spend too much time or complexity to conserve. Space is practically free today and only getting cheaper.
Source: http://www.mkomo.com/assets/hd-cost-graph.png

The longest VARCHAR column Ingres supports as a B-tree key is less than 1kb. Let's say we use it all. At $0.075 per gigabyte for HDD storage today that works out to about .0000075c/key at most.

So forget about the cost of disk space; even if we take into account the cost of extra space for multiple "needlessly" bloated checkpoints it is still close enough to free.

Of course if you want to use SSD or you want to design a database to fit comfortably into RAM, the cost of space might justify the use of surrogates. But you still need to do the sums to be sure.

If space isn't a concern, how about execution time? How slow is it to compare two big VARCHAR keys when joining tables or doing a lookup with Ingres? The following graph has the answers.

This graph shows the result of a series of benchmarks done on a very, very low-end machine, using worst-case conditions.

Two tables were used. One was a 1 million row table with a big VARCHAR column (the natural key), an integer surrogate key column, and an integer non-key column. The other was heap of 10,000 randomly selected and shuffled rows from the big table (with both the natural and the surrogate key).

A series of four tests were done in each trial: a join between the two tables on the natural key; a join on the surrogate key; 10,000 singleton selects using the natural key, and 10,000 singleton selects using the surrogate key. The big table was physically MODIFY'd on the key in question for each test.

Three trials were done, structuring the big table as a hash table, an ISAM table, and then a B-tree.

There's lots of interesting features in this graph but I'll point out just a few.

First of all the performance of the hash table (middle graph) was completely insensitive to whether the physical key was the big natural key or the integer surrogate. On that basis there is just no point ever introducing a surrogate physical key if you intend a table will always be a hash table. Furthermore, the hash table performed far better than any other structure when using a natural key for random look-ups, with a worst-case access time of about 8.7msec even on the junk hardware I was using. If your use case predominantly involves look-ups and there is no compelling reason to choose another structure, natural keys in a hash table are the way to go.

So far I've done a pretty good job of not doing what the title of this posting promised. This was supposed to be about finding cases where surrogate keys are a net benefit. I better get on with it.

The diamond-shaped red dots in the other two graphs show the performance when doing random lookups using a big VARCHAR physical key in an ISAM or B-tree table. The triangular red dots in the same two graphs show the performance doing random lookups with a surrogate key. Clearly, as the natural key gets bigger the performance of the surrogate key that replaces it degrades more slowly. When the natural key is much bigger than about 400 characters I will concede there may be some benefit to using a surrogate—although the difference in response time between a 50 character key and 400 character key is only a footling 4msec, even using junk hardware. On decent hardware with a key that wasn't designed specifically to perform badly, the difference might not even exist.

Surrogate keys look most beneficial when joining tables. The blue dots in the B-tree and ISAM graphs show the response time when joining to an ISAM table or a B-tree table. The diamond dots show the response time when using a big VARCHAR key and the triangular dots show the response time for the surrogate key. Surrogates start to pull away from big natural keys when they get to about 600 characters in an ISAM table, and above about 300 characters in a B-tree.

Glossing over a lot of detail and special cases, boiling it down to a simple one-size-fits-all rule of thumb, physical keys bigger than 300 characters can justifiably be replaced with a surrogate key. Frankly though, I wouldn't bat an eye if you showed me a table with a key twice that size.

Another rule of thumb is to assume you can use a natural key until you find out you can't.

Surrogates are not wholly evil, they are just minimally useful—how many business applications have a 300 character natural key?—and surrogates can lead the unwary designer into making some serious logical errors. Nor are they free; I've not said much about how they obfuscate the meaning of the database nor how they can easily subvert the integrity of the database. This posting wasn't meant to be a licence to use them willy-nilly. It was just acknowledging that sometimes even I need to use one. In my next posting I'll show how surrogates can lead us very badly astray.

Surrogate Keys: Part 3, Surrogates for Composites

rhann — Wed, 03 Apr 2013 15:04:00 GMT

If you made the effort to wade through part 2 of this series you have seen my claim that keys should be simple. Who could argue?

A simple key is one which is easy to use. It is easy to use if it is easy to think about and it performs well in the physical database. If the conceptual model leads naturally to a certain physical key in the database, and if the end-users who own the data are familiar and comfortable with it, and if it performs adequately, there is no reason not to use it. In fact I would say anyone who wants to throw it out and replace it with a newfangled surrogate (which means nothing to anyone) has a very steep hill to climb.

This argument still holds if the physical key happens to be composite. Compactness does not automatically trump familiarity when we are trying to decide what is simple. But speed might.

It is widely held that composite physical keys perform badly. We need to look at the performance of composite physical keys so we can decide when it might be sensible to introduce a surrogate. A composite key consisting of a couple or three columns is probably pretty manageable, but just how many columns is too many?

I set up a benchmark involving a million row table. As well as non-key (data) columns it also had a 10-column logical key, and a single column surrogate key. From this table I sampled 10,000 keys at random and shuffled them in a heap. The trial consisted of two parts: joining the two tables, and doing lookups on the big table using the sample keys. This simulated batch processing and on-line transaction processing. The trial was done using a surrogate physical key of one column, then using a two-column composite, then a three-column composite, and so on up to 10 columns. The same trial was repeated structuring the big table as a B-tree, a hash table, and an ISAM table. The cache was cleared between tests. The benchmark was run on a terrible, ancient low-spec machine.

Pervasive folk-wisdom says the performance should get dramatically worse going from a one-column physical key to an N-column physical key. Here is what happened:
Wow.

Evidently the belief that composite keys perform poorly is just flat-out wrong! Possibly it is true of other products but it certainly isn't true of Ingres. (I am not certain it is true of other products either, but I don't care about other products. I am an Ingres bigot to the bone.)

How many columns is too many in a composite key? Beats me. My guess is way more than ten. (I've never seen even a ten attribute logical key. They must be quite rare.) If a ten-column key makes sense to the business it makes sense to me—and if I don't understand it the burden is all on me to learn.

So what can we conclude from this? Well one thing we certainly can't conclude is that composite keys perform so badly they should be avoided just on principle.

To introduce a spurious surrogate key merely on the assumption of a performance advantage is emphatically not "best practice" nor "professional". Quite the opposite.

Now I don't want to leave you with the mistaken idea that surrogate keys are never beneficial. In Part 4 I'll look for cases where they do help.

Surrogate Keys: Part 2, The Boring Bit

rhann — Tue, 02 Apr 2013 14:42:00 GMT

There's a lot of poorly-chosen terminology and badly explained theory behind keys so I want take a shot at clearing things up so that my next fews rants can be more ranty. If you're really not keen on this you can skip straight to the third posting now, but the last posting isn't going to make much sense if you don't come back and read this first. (I'll put a reminder in the last posting.)

As well as being a bit of a conversational tinderbox the term "key" is grossly overloaded. In these postings I will rant about both logical and physical keys:

a logical key is a column or an irreducible set of columns, the value of which is unique to a row in a table
a physical key is a column or set of columns, the value of which can be used by the DBMS to rapidly locate the row container(s) in the underlying storage.

To put it crudely: the value of a logical key is the "name" of a row, so you can say things about that specific row; the physical key is used to cluster records for rapid access.

UPDATE: Following an offline conversation I realize that it's not even clear what a row is. A row is not a container—it is the content. I've slightly amended the wording of the paragraph above with that in mind. Perhaps it would be better if we spoke of "facts" and "records" instead of calling them both "rows", but I don't think that would catch on.

A physical key need not consist wholly (nor even in part) of columns from a logical key. On the other hand it is quite rare that the columns of a logical key aren't wholly or in part used in a physical key too.

Every table has to have a logical key and it may have many, and they may overlap. The same table can also have zero or more physical keys. (A heap has no physical key but it has to have a logical key.)

A good key is usually said to be:

unique
stable
simple
familiar

The only objective criterion is uniqueness—that's what makes it a logical key. The others are entirely subjective and follow from the fact that we will probably also want to use the columns of a logical key for a physical key.

Stable does not mean immutable; it means "changes only in exceptional circumstances". Changing a couple or ten times over the lifetime of the row is definitely tolerable. This is important because one of the arguments put forward for introducing a surrogate key is that it is not vulnerable to data-entry errors and therefore never needs to be corrected. If you use a credit card number as a key and you have references to it in lots of other tables, and you need to correct it, you have to correct it everywhere it appears.

I say "so what"? That's what ON UPDATE CASCADE is for.

Simple does not exclude composite keys; it excludes unwieldy keys. I think we can agree that a ten-column composite key is well on the way to being a nuisance. Similarly we can probably agree that two- or even three-column composites are pretty manageable. Just where you draw the line is down to your personal psychology and I'm willing to be fairly flexible on where you draw it as long as it's more than three columns and less than too many.

I think we can also agree that using an excessively long string for a key can be a problem. But how long is excessively long? Again, I'm willing to be flexible but I'd want some demonstrable gain from ditching a familiar natural key, so it's going to be have to be very long before I'd agree there's much of a case for using a surrogate instead.

Familiarity is hugely important for usability and it's tough to benchmark, but I say having to look up the surrogate first when you already know the natural value is a titanic pain in the you-know-what. More so when you have to navigate via three or four tables each with a new surrogate only because some dogmatic clown had the idea it was "professional" to eliminate natural keys.

That's enough theory for now. Time to get rough. In the next section I'll expose the claim that surrogate keys improve performance for the nonsense it is.

Digression 1: I think we DBAs tend to over-specify physical keys. The physical key needs to be just good enough to find the right page, not the specific row. A lot of unnecessary cost is incurred by creating over-specific physical keys. (That's yet another rant for another day.)

Digression 2: You will know the term "primary key". It is just a logical key. The only thing remotely special about it is the table designer gave it a gold star and a box of chocolates (for some reason that I'm sure seemed to make sense at the time).

When one declares that a logical key is the PRIMARY KEY Ingres automatically also creates a physical key (secondary index) using the same column(s). That's not a law of nature, it's just what Ingres does and it's not the intended reason for declaring a key primary. I don't know what the intended reason is. It's probably some kind of New Age desire to share one's feelings, no matter how flakey or ill-judged.

I won't distract you by starting yet another rant just now, but if you detect a note of disdain when I write about primary keys you are not imagining it. I won't mention the term again—everything I have to say about logical keys applies to so-called primary keys too.

Benchmarking

rhann — Tue, 02 Apr 2013 11:31:40 GMT

I am a big fan of benchmarking and load-testing. Intuition and common-sense are truly terrible guides to what is efficient. I am frequently surprised at what I discover when I start looking under the rocks.

Recently I've been involved in a discussion about the benefits of surrogate keys. I've been doing some benchmarking to understand them better and I'll be posting some results soon.

When I do this kind of benchmarking I like to create worst-case scenarios. Knowing the worst possible outcome and designing for it, I can be pretty confident that when I deploy live, in production, the real performance can only be better.

For the benchmarking I've just been doing I have used the worst machine I own. It's a single-core, 1.3GHz workstation with 1Gb of RAM and a single low-spec hard drive. The only good thing about it is that it's running CentOS Linux.

To ensure the worst possible Ingres performance I have configured a modest DMF cache. I clear the DMF cache between trials by setting trace point DM421.

The file system cache is highly effective so it also has to be cleared. I was reminded of just how effective it is when I forgot to clear it. The difference was night and day. When I noticed the error I fixed the program to clear it by disconnecting all Ingres sessions from the test database (so the Ingres server process fully closes all its files) then executing sync; echo 3 > /proc/sys/vm/drop_caches as root.

BTW, if anyone knows a way to clear the file system cache on a Windows machine I'd be very pleased to hear about it.

When I generate test data I try to make it as challenging as possible. For instance in my surrogate key benchmarks I did some tests involving very long strings of characters that varied only slightly, and only in the right-most positions—somewhat like what you would expect a list of fully qualified file pathnames to look like, only much, much worse.

When generating surrogate key values I used unordered sequences to minimize the benefits of clustering and cacheing. And when I ran the tests, I made sure I shuffled all the key values to randomize access, maximizing disk head-motion and minimizing cache effectiveness.

I also used quite big tables, with millions of rows in hundreds of thousands of pages.

I always do error-checking and row-count checking in my benchmarks. Any time I get blindingly good performance I have to suspect I've broken a query so that it finds no rows.

Finally, any time I think I see an effect in a benchmark, I try to make it go away. I try changing the order of trials in the benchmark. I check whether defragmenting the disk has any impact. I reboot and try again. If possible I try a different version of Ingres too. Finally, when I run out of ideas I show a friend and ask if they can replicate the effect. It is very, very easy to fool oneself. I distrust my results always. Nine times out of ten when I see an "effect" it turns out to be an error on my part.

The reward is that once in a while I discover something genuine and surprising and maybe counter-intuitive—like the new trick I found for improving the performance of queries on large hash tables.... But that is another story for another day.

Surrogate Keys: Part 1, Best Practice or Wrong-Headed Dogma?

rhann — Mon, 01 Apr 2013 17:15:00 GMT

Recently I've been discussing surrogate keys over on comp.databases.theory.

Everyone seems to have passionate views on the virtues and desirability (or otherwise) of surrogate logical keys. Me too, so this is going to be an unapologetic rant from start to finish.

Surrogate keys are used in lieu of natural keys. Natural keys are identifying values that come from outside the application domain—they appear naturally in the data we receive. If I send you my credit card number, that is a natural key that identifies me.

For years people have been trying to tell me surrogate keys (or system-generated row IDs) are "best practice". This view has become so entrenched that people are lately trying to tell me that it is "unprofessional" to reject them.

I dislike the damn things for very good reasons so I am stung by the implicit accusation that my designs are "unprofessional".

Mostly when people tell me such stuff it turns out to be second-hand dogma and they have no idea why they should believe it. Other times they do manage an argument and it's usually either surrogate keys used as physical keys improve performance, or surrogate keys simplify programming.

Actually there is a third argument that I am seeing increasingly often. I am told "the ORM framework insists on surrogate keys". Now even I know that's not true, but people say it. I am not going to address that issue head-on. It might be the subject of another rant some other time. For now I'll just say that if someone wants to use Hibernate or similar, very good luck to them. I wonder if their customer knows?

As far as Ingres is concerned I make three counter-claims:

natural keys perform about as well as surrogates (if not better)
natural keys lead to far more usable database designs
natural keys prevent serious logical design errors creeping in.

This series of five blog postings will explain all. In the next posting I'll cover some of the terminology and distinguish different kinds of keys. You can skip straight to the third posting if you don't want the build-up.

Serialized Objects in the Database

rhann — Sat, 09 Mar 2013 17:31:46 GMT

I have recently been working with a colleague on an application to manage Java objects. It's an interesting project because it highlights some things that are often hidden when we design databases and write applications.

Long, long ago (1995) in a place far, far away (New Orleans) I attended a wonderful presentation by Chris Date in which he pointed out a fundamental error then being made by the eager proponents of the new object databases that were starting to pop up. Specifically, the OODB enthusiasts talked about classes as if they were analagous to table definitions—when in fact a class is a data type, if it's anything.¹

If classes are data types then SQL tables can perfectly well have columns with instances of a specified class as their values. That is, if I have a class Pizza I can have a table with a column of Pizza instances and regardless of what objects might be nested inside a Pizza.

In less than an hour one can create a Java program that instantiates and serializes objects and inserts rows containing the serialized objects into the database. With a tiny amount more work the program can select and deserialize the objects. It all works fine and none of it is done at the expense of relational theory (RT).

RT explicitly ignores the internal structure of values—all values are presumed to be atomic and are treated as such. (Obviously SQL provides functions to do things like extract substrings from supposedly "atomic" strings, but the essential operation of SQL doesn't depend on being able to do that. Such functions are for convenience alone.)

At first glance then, serializing objects looks like a slick answer to the often-heard grumble from OO programmers about having to laboriously unpack the content of their objects into rows in bazillions of tables when they want to "persist" them. As long as the data management task doesn't depend on looking at the internals of the objects we can just serialize them and park them in the database till they're needed again, like putting a letter in an envelope and putting the envelope in the post. The post office doesn't need to know anything about the content of the envelope to do its job perfect effectively—in fact we expect and insist that the post office will never know anything about the content of the letter.

So is this a slick answer? Well maybe, but most likely not.

First off, doing this is sensible only if your business problem is the actual mangement of serialized objects. Note I am talking about the business problem, not your solution to the business problem—there is a world of difference between those two and if it's not obvious stop and think about it till it is.

Serialized objects are the values. The contents (or the state) of those objects are not the values. To the DBMS those values (the serialized objects) have no values within them; they are atomic. Only the objects are accessible, like the envelopes to the post office.

In almost thirty years I have encountered only one application that was fundamentally about transmitting and arbitrating access to objects.

Databases are for managing shared access to values in a way that ensures logical consistency. To do that the DBMS has to be able to see the values. Values concealed inside other values are invisible and inaccessible. Values encoded inside objects belonging to a particular application or package, based on a particular language implementation, are accesssible only to that application or package. In general, users of other tools or applications can never see the encoded values so shared access is severely limited. The utility of the information is severely limited. In fact it is limited to just the operations the programmers coded into the application.

It is extremely unlikely that a database would serve any other useful purpose when the data is encoded in that way.

Incidentally, I should point out that serialization is a considerable can of worms whether you're interested in databases or anything else. It's a problem for the OO people to address. If you know about the issues you might have thought of a way of dealing with them, and if you don't, well, you'll find out soon enough...

¹ Those of you who were at Chris Date's presentation will remember the impotent fury of the next speaker, who wanted to promote his hideous product which was entirely based on the error! You couldn't have made it up. It was glorious.

Customizing Keyboard Mapping in OpenROAD

howle01 — Fri, 04 Jan 2013 15:40:46 GMT

Every key on your keyboard is identified by a virtual key code. Virtual keys provide a mechanism to represent any key or key combination that could be issued from the keyboard. Virtual keys—also called logical keys—mapped to physical keyboard keys. This article reveals how you can customize keyboard mappings.

The Default Keyboard Map File

When OpenROAD starts, it performs a default mapping of logical key names to physical keyboard keys. The file %II_SYSTEM%\ingres\files\default.kbd reflects the internal default keyboard mapping (if the file is unchanged since OpenROAD installation). All entries in the default keyboard map are tabulated in Keyboard Map File Example.

You can customize or override the default keyboard map file to map an alternative scheme of logical names to physical keyboard keys that may be more appropriate for your needs than the default mapping.

Specifying a Different Keyboard Map File

To change keyboard mapping, we recommend that you follow these basic steps:

Copy the default.kbd file and rename it.
Point OpenROAD to the new file through one of the following environment variables.
Modify the copy.

Copy the Default.kbd File

Copy the default.kbd file in %II_SYSTEM%\ingres\files\. You can store the copy in the same directory or move it elsewhere. You may name it whatever you want, but where you store the file and what you name it determines which OpenROAD environment variable you must set.

Specify an Environment Variable to Point OpenROAD to the New Mapping File

There are two environment variables you can use to specify a different keyboard map file. When OpenROAD is first installed, neither of the variables is set. (OpenROAD uses its own internal version, represented in the installed default.kbd, until one of these variables is defined.)

II_KEYBOARD

Specifies the prefix name of a text file containing the custom keyboard mapping. OpenROAD makes two assumptions about the file specified with II_KEYBOARD:
The file has an extension of ".kbd" (for example, custom.kbd).

The file resides in the "files" directory of the OpenROAD installation: %II_SYSTEM%\ingres\files\prefix_name.kbd

II_KEYBOARD_FILE

Specifies the full path, file name, and extension of a keyboard mapping file that either does not have the .kbd suffix or does not reside in the "files" directory of your OpenROAD installation, for example: C:\MyDirectory\custom.keyboard. Specifying this variable overrides the II_KEYBOARD variable.

The file you specify will be used the next time you start OpenROAD. But you must first modify the mappings in the file.

For more information about how to set environment variables in OpenROAD, see Environment Variables.

Edit Your Custom Keyboard Map File

After copying the default.kbd file to a custom file and pointing OpenROAD to it, you must customize the mappings in the file.

The OpenROAD keyboard map file is a text file with a specific format. The file describes 24 OpenROAD system-defined keys and 36 user-defined keys. The file contains one line for each key with four entries on each line. The entries are arranged in columns separated by tabs. All four columns are mandatory. For example:

Code:

-24    ""              VK_PAUSE           SK_BREAK
-23    Shift+Ctrl+S    Shift-Ctrl-VK_S    SK_SAVE_AS

Because the contents of some columns require specific values, it's best to copy an existing keyboard map file as a template if you want to customize the mapping. See the previous section.

The following table describes the columns, left to right:

Column 1

Index that OpenROAD uses to map the key. It is read into memory and always must match the text label in column 4. Do not change the entries in this column.

Column 2

Text that appears in the pull-down menu for the speed key.

Column 3

Virtual key name for the keystroke that occurs (see Keyboard Mapping).

Column 4

Internal symbol name used by OpenROAD, which must match the text label in column 1 (see SpeedKey Settings). Do not change the entries in this column.

For example:

Index: -23
Pull-down Menu Text: Shift+Ctrl+S
Virtual Key Name: Shift-Ctrl-VK_S
Internal Symbol Name: SK_SAVE_AS

Index: -22
Pull-down Menu Text: Shift+Ctrl+R
Virtual Key Name: Shift-Ctrl-VK_R
Internal Symbol Name: SK_REPLACE_FIND

All 104 virtual key definitions are listed in Keyboard Mapping.

The changes you make to your keyboard mapping file will be used the next time you start OpenROAD.
For a discussion of how to assign speed keys in OpenROAD, see Defining Speed Keys.

Defining Speed Keys

howle01 — Wed, 02 Jan 2013 16:11:37 GMT

A speed key is a keyboard shortcut that invokes a menu option or toggle state typically invoked by mouse clicking. Assigning a speed key to a menu button or menu toggle lets the user press Alt plus a speed key rather than selecting the menu item. For a menu button, pressing the speed key combination executes the command. For a menu toggle, pressing the speed key combination switches the option between the on and off states.

OpenROAD lets you assign speed keys to menu buttons and toggles in your applications by selecting a logical key name for the menu item from a list of supplied constants. The logical key assigned to the menu item may map to the keys on physical keyboards differently.

Keyboard Map Files

If a key is defined in the mapping file, it will activate if assigned to an actual menu item. For example, if the VK_RETURN key is mapped to SK_USER1 and SK_USER1 is not used, it does not change any behavior. On the other hand, if it is assigned to a menu item attached to the frame, it overrides the default behavior and raises that event block.

The same mapping is used for all platforms. The file %II_SYSTEM%\ingres\files\default.kbd reflects the internal default keyboard mapping (if the file is unchanged since OpenROAD installation). All entries in the default keyboard map are tabulated in Keyboard Map File Example.

There are 104 different key definitions available. OpenROAD has a limit of 59 slots available for speed key definitions that are active for the life of a session.

How You Can Assign Speed Keys

In the Workbench Menu Editor, you can associate a predefined speed key with a menu item (button or toggle) by using the Property Inspector to set the SpeedKey property. (System constant settings are listed in SpeedKey Settings.) The physical key mappings for these logical keys can be different for each system. To assign your own logical key, select one of the SK_USER# options, where # is 1–36.

How You Can Define an Alt Speed Key

OpenROAD can set Alt speed keys for Button, Toggle, and Radio fields.

To position an underscore character, indicating which Alt speed key is desired, place an ampersand (&) prior to the character to be underscored when you create the label for the field, for example, "&Save" to underscore the S. You can create the label in the Frame Editor's Property Inspector for the field, or it may be created or changed dynamically by your 4GL code.

OpenROAD maintains the dynamic speed key table and associates the speed key with the field. This is done automatically for three types of fields: buttons, toggles, and radio fields.

Considerations When Defining Alt Speed Keys

When defining Alt speed keys, consider the following issues:

Control Bias

Alt speed keys are activated only if the bias of the field with which they are associated is FB_CHANGEABLE or FB_LANDABLE. When the Alt key and the underscored character are pressed, it is as if the mouse button is clicked while the cursor is positioned over the field.

Printable Characters

The characters that can be used for Alt speed keys are all the printable characters in the current character set, except space (" ") and ampersand ("&"). Characters whose internal value is greater than 127 (0x7F) are supported so long as they can be typed without using the Alt key and the numeric keypad. Alt+function keys are not supported (except in the user-defined speed key tables.

Case Significance

The case of letters used as Alt speed keys is not significant. For example, both of the following button labels have Alt+K as the speed key: "O&K" and "O&k." (Therefore, Alt+K and Alt+Shift+K perform the same function.)

Including the "&" Character in a Label

Use double ampersands ("&&") in the label of a field if you want to display a single ampersand as part of the label. For example, "This && That" displays as "This & That" with no underscore (and no Alt speed key).

Button Speed Keys

If the label for a button contains an ampersand ("&"), when the Alt key and the underscored character are pressed, the button is pressed as if the primary mouse button were clicked while the cursor is over the button. Focus behaviors, events, and so on, are the same as if the mouse were used.

Toggle Speed Keys

Both the ON-Label and the OFF-Label can have Alt speed keys. They can be the same (for example, "&On" and "&Off") or different (such as "O&n" and "O&ff").

If they are different, only the one currently displayed is active. In the same manner as for buttons, using the Alt speed key is identical to clicking with the primary mouse button.

Radio Button Speed Keys

Any, all, or none of the buttons in a radio field can have Alt speed keys associated with them. Pressing the Alt speed key associated with a radio button selects that button. In the same manner as for button fields, using the Alt speed key is identical to clicking with the primary mouse button.

Order of Precedence for Speed Keys

Speed keys using the Alt key can be defined in the following ways:

Alt speed keys
Defined by placing an ampersand in the label of a button, toggle, or radio button
User speed keys
Defined by creating a user-defined speed key file
Menu speed keys
Defined by creating a menu item with an ampersand in its label
Only one action is performed when a speed key using the Alt key is pressed, even if that speed key is defined in more than one of the previous ways at the same time. The order of precedence for speed keys is Alt, User, then Menu.

Limitations

The following limitations apply to speed keys:
Only one Alt speed key is supported per label.

If more than one ampersand is included in the text for a label, only the character following the last ampersand will be the Alt speed key. Prior ones are ignored. This does not apply to double ampersands used as an escape to display a single ampersand.

Duplicate Alt speed keys are not supported.

Using the ampersand to define more than one field to have the same Alt speed key is not supported. If you use the ampersand in this way, only one of the fields will actually be associated with that Alt speed key. The field chosen cannot be predicted and is subject to random fluctuations while the frame is running. (Toggle fields may use the same Alt speed key for their ON-Label and OFF-Label, since only one of these will be active at a given time.) Take care when assigning Alt speed keys dynamically.

MenuButton.SpeedKey Attribute

Data Type: integer
4GL Access: RW

The SpeedKey attribute provides a numbered speed key that the end user can press to select the MenuButton.

Pressing the speed key triggers any click events specified for the MenuButton.

You can set user-defined speed keys 1–36 by using those integer values directly. OpenROAD also provides a special set of constants for the standard speed keys used in an application (for example, SK_SAVE and SK_UNDO). The keys associated with these constants vary for each system.

Descriptions of system constant values and their numeric equivalents are listed in SpeedKey Settings. For the physical key mappings for these constants, see the appendix Speed Key Mapping in the Workbench User Guide.

How to Construct a File Menu Group That Contains Speed Keys

The following discussion steps through the construction of the File menu group.

Create the MenuGroup object:

Code:

top_menu = MenuGroup.Create(); /* File Menu */

Define the TextLabel and internal name for the File menu group:
Code:
```
top_menu.TextLabel = 'File';
top_menu.Name = 'file_menu';
```
The text assigned to the TextLabel attribute is displayed by the frame in its menu bar. The preceding code, therefore, causes the word "File" to appear as one of the selections available in the frame's initial menu. The value assigned to the Name attribute is the name of the reference variable pointing to that MenuGroup object.
Define the two submenu options that appear when the user selects the File option from the menu bar: "Commit and Close" and "Rollback and Close." Defining these operations follows the same procedures as creating the MenuGroup:
- Create the object for each item (in this case a MenuButton object).
- Define the TextLabel and Name for each menu item.
The following code creates the first submenu item:
Code:
```
test_menu = MenuButton.Create();
test_menu.Name = 'commit_menu';
test_menu.TextLabel = 'Commit and Close';
```
In addition to letting the user commit the transaction and close the frame by selecting the Commit menu button, the dynamic frame provides a second way to commit and close. The example frame provides alternative access to the Commit operation by defining a speed key, which lets the user select the Close operation from the keyboard rather than using the mouse.

The following code defines the speed key:
Code:
```
test_menu.SpeedKey = SK_CLOSE;
```
After each submenu option is defined, the following code attaches it to the field MenuGroup object:
Code:
```
test_menu.ParentMenuGroup = top_menu;
```
The preceding statement makes the MenuGroup object represented by top_menu to be the parent of the MenuButton object represented by test_menu.

After defining all of the individual items for the MenuGroup, the following code attaches the MenuGroup itself to the MenuBar object:

Code:

top_menu.ParenTMenuGroup = test_frame.StartMenu;

The following is the complete code that constructs both menu groups that belong to the menu bar:

Code:

top_menu = MenuGroup.Create();   /* File Menu */
top_menu.TextLabel = 'File';
top_menu.Name = 'file_menu';

test_menu = MenuButton.Create();   
    /* Commit and close menu item */
test_menu.Name = 'commit_menu';
test_menu.TextLabel = 'Commit and Close';
test_menu.SpeedKey = SK_CLOSE;
test_menu.ParentMenuGroup = top_menu; 
    /* Attach to File menu */

test_menu = MenuButton.Create();  
    /* Rollback and close menu item */
test_menu.Name = 'rollback_menu';
test_menu.TextLabel = 'Rollback and Close';
test_menu.ParentMenuGroup = top_menu;  
    /* Attach to File menu */
top_menu.ParentMenuGroup = test_frame.StartMenu;
/* Now, create the Edit menu */
top_menu = MenuGroup.Create();   /* Edit Menu */
top_menu.TextLabel = 'Edit';
top_menu.Name = 'edit_menu';

test_menu = MenuButton.Create();      
    /* Delete menu item */
test_menu.Name = 'delete_menu';
test_menu.TextLabel = 'Delete and Next';
test_menu.SpeedKey = SK_DELETE;
test_menu.ParentMenuGroup = top_menu; 
    /* Attach to Edit menu */

test_menu = MenuButton.Create();     
    /* Update menu item */
test_menu.Name = 'update_menu';
test_menu.TextLabel = 'Update and Next';
test_menu.ParentMenuGroup = top_menu;   
    /* Attach to Edit menu */

test_menu = MenuButton.Create(); 
    /* Next menu item */
test_menu.Name = 'next_menu';
test_menu.SpeedKey = SK_NEXT;
test_menu.TextLabel = 'Next';
test_menu.ParentMenuGroup = top_menu;     
    /* Attach to Edit menu */

/* Attach to Frame */
top_menu.ParentMenuGroup = test_frame.StartMenu;

For more information about the various types of menu objects, see the Language Reference Guide.

For more information on keyboard mapping, see Customizing Keyboard Mapping.

Understanding Database Connection Profiles in OpenROAD 6.0

howle01 — Fri, 30 Nov 2012 21:13:48 GMT

A database connection profile is a collection of settings that Workbench uses to establish a connection with a local or remote database. To use any of the Workbench tabs other than the Connect tab, you first must connect to a database through a database connection profile. This means you must create at least one connection profile before you can connect to a database to do application development using Workbench.

Connection parameters include:

Profile name
Description of the profile
Database management system (Ingres, Oracle, Microsoft SQL Server, or IBM DB2 UDB)
Connection type (local, vnode, or dynamic)
Database name
Connect as username
Group identifier for the session
Role identifier for the application image
Optional flags to use on the connect statement
Profile icon

The Connection Profile portlet provides other connection parameters, depending on the value you choose for the Type parameter. The above example shows the connections parameters available if Type is set to Local. If Type is set to Vnode, you must define the additional Node parameter with the remote virtual node name. If you set Type to Dynamic, you must define additional parameters for the dynamic connection definition (server, installation, and protocol).

For more information about fields and parameters used in the OpenROAD Startup Assistant, see OpenROAD Startup Assistant and Connection Details Properties.

Ways to Define a Database Connection Profile

The first time you start OpenROAD 6.0, the OpenROAD Startup Assistant prompts you to create a new database connection profile. Subsequently, you can create profiles to connect to existing databases or, if OpenROAD is installed in an Ingres installation, you can create a new database along with the connection.

You create database connection profiles from the Connect tab using the Connection Profile Assistant, a handy wizard that guides you through the creation process. To start the wizard, click File, New, Profile.

New Location for Database Connection Profiles in OpenROAD 6.0

When you upgrade from OpenROAD 5.1 to OpenROAD 6.0 on newer 64-bit Windows platforms, the path to where database connection profiles are stored changes. To continue using profiles you created in OpenROAD 5.1, you must copy the 5.1 profile files to the OpenROAD 6.0 directory or recreate the profiles in Workbench. Here's how.

Finding the Location of Your OpenROAD Profiles

You can find the location of your profiles by issuing the following command at the OpenROAD command prompt:

openroad /profiles

The output will look something like the following, depending on your version of Windows:

Code:

=========================================================
openroad: The following options were selected by openroad
---------------------------------------------------------
  OPTION_DBNAME                 = qtdemodb
  OPTION_PROFILES               = TRUE
---------------------------------------------------------
OR_CONFIG  = "C:\Documents and Settings\All Users\Application Data\OpenROAD\Profiles\"
    Profile  = "%OR_CONFIG%\user01.prc"
    Profile  = "%OR_CONFIG%\user01.prf"
    Profile  = "%OR_CONFIG%\user01.qp"
    Profile  = "%OR_CONFIG%\user01.tld"
    Profile  = "%OR_CONFIG%\user01kbd.prc"

OR_CONFIG is an environment variable that specifies the location of the OpenROAD profiles. This is set to a location that the user has write access. For more information about OR_CONFIG, see Environment Variables for All Platforms and openroad command—Check OpenROAD Status.

Copying Profiles to Connect to the Same Ingres Instance

If you're accessing the same Ingres instance in OpenROAD 6.0 that you used with OpenROAD 5.1, you can copy the database connection profiles from the OpenROAD 5.1 storage location to the new OpenROAD 6.0 location. Be sure to close OpenROAD before copying the files.

Windows Server 2008, Windows Vista, Windows 7

Copy files from 5.1 location:

%ALLUSERSPROFILE%\OpenROAD\Profiles

Paste files to 6.0 location:

%USERPROFILE%\AppData\Local\OpenROAD\Profiles\inst allation_ID

Windows Server 2003, Windows XP

Copy files from 5.1 location:

%ALLUSERSPROFILE%\Application Data\OpenROAD\Profiles

Paste files to 6.0 location:

%USERPROFILE%\Local Settings\Application Data\OpenROAD\Profiles\installation_ID

Recreating Profiles to Connect to a New Ingres Instance

If you want to connect to a new Ingres instance, at the end of installation or the first time you start Workbench, the OpenROAD 6.0 Startup Assistant will guide you through the process of creating a default database connection profile.

For more information about creating new database connection profiles, see Define a Database Connection Profile.

Starting Workbench from the Command Line to Connect with a Specific Connection Profile

You can start Workbench 6.0 with a command line flag to specify a connection profile and optionally skip the splash screen:

w4gldev runimage workbnch.img -/appflags profile=profilename nosplash

When you start Workbench from the command line, profilename cannot contain embedded white space.

What You Can Do from the Workbench Connect Tab

On the Connect tab, you can perform the following tasks concerning connection profiles:

Define new connection profiles (see Define a Database Connection Profile)
Modify existing connection profiles (see Modify a Database Connection Profile)
Delete connection profiles (see Delete a Database Connection Profile)
Set a connection profile as the default (see Set a Default Database Connection Profile)

In OpenROAD Workbench 6.0, when you double-click an existing connection profile in the Default Profile, Recent Profiles, or Connection Profiles portlet of the Connect tab, the connection is established and the Develop tab automatically opens using that connection. There, you can begin application development using the data in the connected database.

For more information, see Connect to a Database Through a Connection Profile.

Attached Images

	Connection_Profile_Assistant.jpg (18.5 KB, 114 views)
	Edit_Connection_Profile.jpg (13.4 KB, 191 views)

Understanding the Port Keyword in the .NET Data Provider/JDBC Connection String

teresa — Fri, 30 Nov 2012 16:33:56 GMT

A C# application using the .NET Data Provider or a Java application using the JDBC Driver communicates to the Ingres or Vectorwise data sources through the Data Access Server (DAS). The application must specify in its connection string the hostname and the listen port for the DAS. The DAS listens on a network protocol port which is configured when the Ingres or Vectorwise instance is first installed. The port can be referenced by either its TCP/IP numeric port number or by an equivalent symbolic port identifier. By default the symbolic network protocol port identifier for the DAS is the two-letter instance ID followed by the number 7. For example, when installing Ingres or Vectorwise, if installing a default instance, the default symbolic port identifier for an Ingres instance is II7 and for a Vectorwise instance it is VW7. These equate to TCP/IP numeric ports 21071 and 27839 respectively.

If you do not know the port value for your instance, you can search the errlog.log in the target instance for the strings, “DASVR” and “E_GC4803_NTWK_OPEN”. This line will contain the following information, a reference to the protocol TCP_IP and the port xxx information, where xxx is the symbolic port identifier followed by the numeric port number in parentheses. It is important to ensure you are accessing the line that contains both strings, as other servers such as the communications server, COMSVR, will also open network ports. For more information about Data Access Server listen ports, see the section Data Access Server Parameters—Configure DAS.

More than one DAS can be configured on the DBMS server instance to provide a level of application load balancing. For example, within CBF you could set a startup count of 2 for the DAS and set the port parameter for the DAS to VW7+ which tells the system to start the first DAS on the symbolic port of VW7 and the second DAS on the symbolic port of VW8 . You could then configure your application connection string to allow application A to communicate to DAS VW7 and application B to communicate with DAS VW8, splitting up the application traffic between two different DASs. You could also configure the application connection string to allow Application A to use either DAS since multiple ports can be specified. Other more complex application load-balancing scenarios could also be configured. Frequent readers of my blog may remember a recent article about Scalable Load Balancing “On a Dime” with .NET Connection Pooling that detailed a complex load balancing scenario.

For more information on configuring and using multiple DAS, see the Connectivity Guide.

Reading Ingres Database Configuration Files

rhann — Wed, 24 Oct 2012 08:51:29 GMT

In recent weeks I've been asked twice (kinda indirectly) about the Ingres database configuration file.

Each Ingres database has a file in its data area called aaaaaaaa.cnf which has information needed to open or recover the database. There are several backup versions in other places too, for instance there's a version in the dump area.

It's a machine-readable binary file. Normally the best tool to get most of the information from it is the infodb utility, run from the command line or from Actian Director.

If for some reason you want the information not shown in the infodb report, you need to dump the configuration file and intepret the fields yourself. The structure of the file is not trivial and it has changed many times over the years, so the way you interpret it will depend on which Ingres version you are using.

You could write your own configuration file dump utility using C or something, but probably the quickest way to do it is to use a hex editor such as HexEdit.

HexEdit treats all files as binary files. The most useful feature of HexEdit is the ability to define a template to show the location and extent of the fields in the file. It also allows you to display field values in whatever way is convenient. For example flag fields can be displayed in hex, count fields can be displayed as integers, and strings as...well, as strings.

Crucially, templates can include some logic, so they can adapt to the actual content of the file. This is important to us because the number and location of fields in an Ingres configuration file has changed from release to release, and we'd like to have one template that can figure out what it's looking at all by itself.

I've created a HexEdit 4.0 template file that (I believe) will work for all configuration file formats from Ingres 10s back to Ingres 6.x. I am confident it works for the current file format but I admit I've not gone to the trouble of resurrecting an old 6.x configuration file to test it. Some minor fixes may be required.

Note HexEdit will allow you to actually edit the configuration file. Editing the configuration file risks making your database inaccessible and possibly unrecoverable. I strongly advise you not to edit it. (In any case, the most tempting changes—such as editing the location names—will probably not work.)

If you want to write your own configuration file dumper or fix my template, you should refer to
src/back/dmf/hdr/dm0c.h and src/back/dmf/dmp/dm0c.c.

Accomplishments in OpenROAD Sprint at the 2012 German IUA (GIUA)

teresa — Tue, 23 Oct 2012 17:57:23 GMT

Members of the OpenROAD development team and customers participated in an OpenROAD Sprint October 9–11 in Langen, Germany. It was Actian's first ever Sprint in Germany, and from all reports it was very successful. Here's what we accomplished.

Joe Kronk, Teresa King, and Bodo Bergmann thoroughly reviewed the OpenROAD Enhancement queue prior to the event, looking for potential projects that could be completed in our time together. Participants also suggested projects that would be beneficial to them. Although many projects were considered, the issues addressed were all OpenROAD enhancements that customers requested. Smaller enhancements will be considered for implementation at future Sprints so it is important that all of our customers log feature requests via the Support Mechanism.

Continuing our recent trend, Bodo and Joe decided to focus on 4GL projects because 3GL projects are usually too large to tackle in the timeframe available, and the majority of attendees are 4GL developers.

Considering that half the customers were participating in their first Sprint, they accomplished a significant amount of work.

4GL Projects Tackled

Stefan Massopust, Kim Ginnerup, Andreas Gießel, and Bodo completed the following 4GL enhancements:

Keep fields selected in the Frame Editor after saving the frame.
Provide search capabilities in the Field Tree Window those in the Script Editor.
Enhance the Connection Profile Assistant:
Allow any FormField to be inserted into an OpenROAD Toolbar.
- While testing this feature a 3GL bug was discovered when a tabfolder was inserted into the toolbar. This bug was resolved at the Sprint by Joe and Thomas Christlieb to complete the implementation of the feature.

3GL Project Accomplished

Although the plan was to focus on 4GL projects, Joe and Thomas completed one 3GL project previously logged as a customer enhancement request since it turned out to be a very simple change:

Enable FilePopup dialogs to be resizable.

Participants considered other projects that at first glance appeared fairly easy to implement. However, it quickly became clear this wasn’t always the case because, if implemented incorrectly, there could be serious side effects on other parts of the system. Another specific request was researched but abandoned not because the idea wasn’t good, but because the proposed implementation would adversely impact the end user application.

Customer Challenges Resolved

A Sprint is a good time to get the OpenROAD development team to look at your hard to reproduce problems.

Thomas's customers had been seeing some abnormal terminations, but Thomas was having trouble providing reproducible test cases and sufficient data to expose the underlying problem. Joe and Thomas spent some time analyzing traces and crash dumps and managed to get to the heart of one problem. Another one was resolved earlier this week.

More Than Just a Sprint

Since all the slots at the German IUA were filled, Consolidate Systems took the opportunity to share some of the work that they are doing with OpenROAD. They delivered a presentation, "OpenROAD: A LEADER IN MOBILE REVOLUTION," along with a live demo of two OpenROAD applications that run on the web and on mobile devices. It was very impressive.

Because the attendees were unable to attend the German IUA and hear Emma McGrattan’s presentation about OpenROAD 6.2, Joe and Bodo delivered an informal presentation using content from the OpenROAD 6.2 Roadmap page to show the attendees where we are headed with the next major OpenROAD release. They demonstrated a chess application created to showcase some of the upcoming functionality. General consensus was that we are headed in the right direction.

What Happens Next

Bodo and Joe have returned from the Sprint and are already busy creating wiki pages on the OpenROAD project page that describe the projects. This ensures that the code changes are submitted into our development codeline.

The features discussed in this article will be available in the OpenROAD 6.2 release. Projects for which a project page exists are linked to the project page. The completed Sprint projects also will be included in the upcoming 6.2 Workbench Express preview.

Summary

Congratulations to all the participants for a job well done! The attendees were hopeful we would do another Sprint in Germany and, based on feedback, they're already looking forward to it.