The QueryResultsProcessor – part Three

The previous two posts provided a walk through the API components in the QueryResultsProcessor object. Now I want to move into what you can, and arguably should do with the feature.

QRP views and the HCL Notes® Client (and others)

The most visible means of publishing Domino data is via the UIs provided by HCL that people are familiar with from everyday use. And to answer the question: “Is Domino a mail product or an application platform?” the answer is “Yes” to those who know it. And it’s “Yes” for the QRP feature also.
But QRP views are something new to those UIs. A traditional Domino view is backed with documents in the database in which the view was created. So, any operation that uses the view to open, search or inspect those documents has a direct association to data in the NSF file. QRP views have no such local association. They are created locally then populated with data from external databases. Each entry in the view can be from one of any of several source NSFs. QRP views are index-only data.

To make it possible to access the underlying documents, QRP views are populated with additional hidden columns at the end of each entry. Those columns are filled with the database path and NoteID of the source document. You can reference them as $DBPath and $NoteID in Java or Lotusscript to process the source documents programmatically. And to support the double-click behavior people are so used to, we implemented Lotusscript code in the QueryOpenDocument event to fetch those columns and open the source document. That means users can open the documents as usual.

Since there is no single database context behind QRP views, things like FT searching and right-click document properties cannot work. Hot search (typing values into the view window) and all navigation operations (traversal, expand/collapse, etc.) work fine. We manufactured non-existent NoteIDs for use within QRP views for programmatic subsetting or NIFLocateNote processing, but they are not valid outside a particular QRP view context.

Using HCL Designer®, QRP view colors and fonts are fully editable. However, if columns are added or removed, or formula data is updated, the results are unpredictable. Most such updates will fail altogether, as QRP view updating is designed as a single-pass population operation only.

Administration and QRP views

The temporary nature of results views opens the door to their propagation. Like all views, they consume disk and memory. They cannot be updated using updall, dbmt or update tasks, since they are query results – snapshots of the data at the time they were created and populated.

To view what QRP views exist, and their expiration times, the server console show database command can be used with the “e” option:

show database e

To force the removal of a QRP view – that is, to override the normal expiration time – use updall -Tx

load updall <database path> -tx <viewname>

To add to the expiration time, in days, of a QRP view, use updall -T-

load updall <database path> -t- <viewname> <days>

Also, like all views, the remove() method in the (Notes)View back-end class can be used to delete them.

Performance

QRP views are views that get created and built once. The speed at which this happens is relative to the resources available and the amount of data in the view. You can certainly create a very large view using QRP with lots of data from lots of databases. But that’s not necessarily what the feature at its best.
The three parts of creating and populating a QRP view are:

a. Creating the input document collections – using FTSearch, db.search, getAllDocumentsByKey or DQL, the time to do so varies by the method used and the amount of data found

b. Creating the view – done automatically under executeToView – should not be a major factor in the performance but neither is it free.

c. Populating the view – fetching the document data from each of the input collections and computing values as specified for the view. Includes sorting and creating indexes. This will take the bulk of the time.

As I say, times vary, and it may take too long for users to wait for the results from the beginning. But the results are persistent and if built asynchronously, no one will have to wait for anything.

Once QRP views are built, traversal and other view operations will be on par with all other views.

Ideas for QRP views

By no means do I claim to ability to come up with an exhaustive list of QRP view ideas. Indeed, as the feature has been deployed internally, new groupings of related data keep getting revealed. QRP views can be big time-savers, aggregations of data that was difficult to see together before and analytic tools for business operations and flow. What I would suggest for you to do if you’re thinking of deployment:

a. Consider the Domino data stores you or your customers currently have – all of them, including mail and even documentation databases. What relationship exist between them? Or what relationships COULD exist?
b. Track the work it commonly takes to navigate between those stores – what data do you need to copy/paste? What databases must be opened? What kind of searching goes on?
c. What kind of cyclical or repeated data gathering goes on? I know of several use cases where monthly or annual reporting is a very consumptive process. And even if it’s not on a cycle but on a repeated basis, identify that work. It’s often laborious.

With that, here are some concrete ideas:

1. Cross-partition report view – gather trends and data facts from data partitioned in any way.
2. Drill down view – using a particular data value, trace its mention and appearance across any set of databases, including mail, workflow, discussion and teamroom databases. Supply input data using FTSearch or field value lookup.
3. Ultimate inbox – created every day, a selective dashboard view of the morning’s e-mail and data from critical Domino databases all in one place. The time-sensitive @functions fit very well with the one-time build nature of QRP views. Or combine inboxes from multiple source e-mail databases.
4. Snapshot QRP view – to capture the point-in-time state of data for future and ongoing reference, create a QRP view with column data that may changes over time. You will need to specify ample hours before expiration, but business flow and progress can be recorded across time.
   
   Forward – QRP Futures
   
   As I’ve said before, we at HCL always want to hear your ideas and suggestions, problems and requests. We have worked hard to provide this feature (which has taken three blog entries to fully describe). But we know we’re not done. There are many low-level enhancements, but some of the larger ones are:
   
   I. Refreshable QRP views (probably using DQL only)
   II. Joins/Lookups across QRP input collections; data from different sources in the same view entry
   III. Create QRP views from another view’s design
   IV. Incorporate data from outside Domino into QRP views